Schema-Based Extraction

CSS schemas turn repeating HTML into JSON without an API token. They are the fastest extraction option when the DOM is stable.

JsonCssExtractionStrategy JsonXPathExtractionStrategy schemas sibling source

01/concept

Concept

When the HTML structure is reliable, schema-based extraction is the fastest and cheapest strategy. JsonCssExtractionStrategy and JsonXPathExtractionStrategy take a schema with a baseSelector, optional baseFields, and a list of fields. Each field has a name, selector, type, and optional transform, default, or attribute.

The source key is the escape hatch for sibling layouts like Hacker News, where the title and score live in adjacent rows. Pass the strategy to CrawlerRunConfig(extraction_strategy=...), then parse result.extracted_content with json.loads().

02/use-cases

Use cases

Extract product name, price, and image URL from an e-commerce grid.
Pull article titles and links from a blog archive.
Parse table-like layouts where related data sits in sibling elements.

03/watch

Video walkthrough

Official Crawl4AI tutorialyoutube · loads on click

04/run

Code example

python CSS schema extraction

import asyncio
import json
from crawl4ai import AsyncWebCrawler, CrawlerRunConfig, CacheMode
from crawl4ai import JsonCssExtractionStrategy

async def main():
    schema = {
        "name": "Articles",
        "baseSelector": "article.post",
        "fields": [
            {"name": "title", "selector": "h2", "type": "text"},
            {"name": "url", "selector": "a", "type": "attribute", "attribute": "href"},
            {"name": "summary", "selector": "p.summary", "type": "text", "default": ""},
        ],
    }

    run_cfg = CrawlerRunConfig(
        cache_mode=CacheMode.BYPASS,
        extraction_strategy=JsonCssExtractionStrategy(schema, verbose=True),
    )

    async with AsyncWebCrawler() as crawler:
        result = await crawler.arun(
            url="https://example.com/blog",
            config=run_cfg,
        )

        if result.success and result.extracted_content:
            data = json.loads(result.extracted_content)
            print(json.dumps(data, indent=2))
        else:
            print("Failed:", result.error_message)

asyncio.run(main())

python Sibling extraction with source

schema = {
    "name": "HN Submissions",
    "baseSelector": "tr.athing.submission",
    "fields": [
        {"name": "rank", "selector": "span.rank", "type": "text"},
        {"name": "title", "selector": "span.titleline a", "type": "text"},
        {"name": "url", "selector": "span.titleline a", "type": "attribute", "attribute": "href"},
        {"name": "score", "selector": "span.score", "type": "text", "source": "+ tr"},
        {"name": "author", "selector": "a.hnuser", "type": "text", "source": "+ tr"},
    ],
}

strategy = JsonCssExtractionStrategy(schema)

05/build

Practice lab

Lab 2.1

Practice lab

0/3

Objective: Extract a structured list from a page with repeated elements using a CSS schema.

Steps

Choose a page with repeated items (e.g., a blog archive or documentation list).
Inspect the HTML and identify a stable container selector and child selectors.
Write a schema with at least three fields, including one attribute field.
Crawl the page with JsonCssExtractionStrategy and parse the JSON output.
Print the number of items extracted and the first two items.

Success criteria

The schema extracts at least three fields per item.
The output is valid JSON parsed with json.loads().
At least two items are extracted successfully.

Expected output

A JSON array of objects, each with title, URL, and summary keys.

Hints

Use browser dev tools to test selectors before writing the schema.
Add default values for optional fields so missing data does not break the extraction.

Solution

python Complete solution

import asyncio
import json
from crawl4ai import AsyncWebCrawler, CrawlerRunConfig, CacheMode
from crawl4ai import JsonCssExtractionStrategy

async def main():
    schema = {
        "name": "Blog Posts",
        "baseSelector": "article.post",
        "fields": [
            {"name": "title", "selector": "h2 a", "type": "text"},
            {"name": "url", "selector": "h2 a", "type": "attribute", "attribute": "href"},
            {"name": "summary", "selector": "p", "type": "text", "default": ""},
        ],
    }

    run_cfg = CrawlerRunConfig(
        cache_mode=CacheMode.BYPASS,
        extraction_strategy=JsonCssExtractionStrategy(schema),
    )

    async with AsyncWebCrawler() as crawler:
        result = await crawler.arun(
            url="https://example.com/blog",
            config=run_cfg,
        )
        if result.success and result.extracted_content:
            posts = json.loads(result.extracted_content)
            print(f"Extracted {len(posts)} posts")
            print(json.dumps(posts[:2], indent=2))

asyncio.run(main())

06/check

Common mistakes

TRAP 01

Using brittle nth-child selectors that break when the site adds one item.

TRAP 02

Forgetting json.loads() on result.extracted_content.

TRAP 03

Using sibling source when a nested child selector would work, or vice versa.