Lazy Loading & Basic Interaction

You will scroll pages, wait for images, and trigger dynamic content with small JavaScript snippets.

scan_full_page wait_for_images js_code wait_for

01/concept

Concept

Modern pages defer content until scroll or interaction. Crawl4AI handles this with scan_full_page=True to scroll from top to bottom, wait_for_images=True to wait until image requests finish, and js_code to click buttons or evaluate custom scripts. Use wait_for to pause until a selector is present or a JavaScript predicate returns true.

Lazy loading works best with CacheMode.BYPASS during testing, because cached pages may skip network fetches and leave images unloaded.

02/use-cases

Use cases

Capture all images in a gallery that loads on scroll.
Click a Load More button and wait for new items before extraction.
Wait for a dynamic chart or map to render before taking a screenshot.

03/watch

Video walkthrough

Official Crawl4AI tutorialyoutube · loads on click

04/run

Code example

python Scroll and wait for images

import asyncio
from crawl4ai import AsyncWebCrawler, BrowserConfig, CrawlerRunConfig, CacheMode

async def main():
    run_cfg = CrawlerRunConfig(
        cache_mode=CacheMode.BYPASS,
        scan_full_page=True,
        scroll_delay=0.5,
        wait_for_images=True,
        exclude_external_images=True,
    )

    async with AsyncWebCrawler(config=BrowserConfig(headless=True)) as crawler:
        result = await crawler.arun(
            url="https://example.com/gallery",
            config=run_cfg,
        )

        if result.success:
            images = result.media.get("images", [])
            print(f"Images found: {len(images)}")
            for img in images[:5]:
                print(img.get("src"))

asyncio.run(main())

python Click and wait for dynamic content

run_cfg = CrawlerRunConfig(
    cache_mode=CacheMode.BYPASS,
    js_code="document.querySelector('.load-more')?.click();",
    wait_for="css:.new-item",
    delay_before_return_html=1.0,
)

async with AsyncWebCrawler(config=BrowserConfig(headless=True)) as crawler:
    result = await crawler.arun(url="https://example.com/list", config=run_cfg)

05/build

Practice lab

Lab 2.5

Practice lab

0/3

Objective: Crawl a page that lazy-loads images and verify every image is captured.

Steps

Find or create a test page with images below the fold.
Crawl it once without scan_full_page and count images.
Crawl it again with scan_full_page=True, scroll_delay=0.5, and wait_for_images=True.
Compare the image counts.

Success criteria

The scrolled crawl returns more images than the default crawl.
Image sources are printed without duplicates from external domains (if excluded).
Cache is bypassed during the comparison.

Expected output

Two image counts showing that scrolling captured additional lazy-loaded images.

Hints

Use exclude_external_images=True to avoid third-party tracker images.
Increase scroll_delay if the page loads images in batches.

Solution

python Complete solution

import asyncio
from crawl4ai import AsyncWebCrawler, BrowserConfig, CrawlerRunConfig, CacheMode

async def crawl(with_scroll: bool):
    cfg = CrawlerRunConfig(
        cache_mode=CacheMode.BYPASS,
        scan_full_page=with_scroll,
        scroll_delay=0.5,
        wait_for_images=True,
        exclude_external_images=True,
    )
    async with AsyncWebCrawler(config=BrowserConfig(headless=True)) as crawler:
        result = await crawler.arun(url="https://example.com/gallery", config=cfg)
        return len(result.media.get("images", [])) if result.success else 0

async def main():
    print("Without scroll:", await crawl(False))
    print("With scroll:", await crawl(True))

asyncio.run(main())

06/check

Common mistakes

TRAP 01

Crawling a lazy-loaded page with the cache enabled, then wondering why images are missing.

TRAP 02

Setting scan_full_page=True without a scroll_delay for pages that load in bursts.

TRAP 03

Using js_code to click an element before wait_for confirms it exists.