Syllabus
The full module map for the Crawl4AI Mastery Course.
01/map
All 19 modules
| Module | Title | Focus |
|---|---|---|
| 1.1 | Installation & First Crawl | AsyncWebCrawler, BrowserConfig, CrawlerRunConfig, context manager, result.success |
| 1.2 | Browser vs. Run Configuration | Where global browser settings live vs. per-crawl settings; CacheMode |
| 1.3 | Reading the CrawlResult | html, cleaned_html, markdown, media, links, metadata, error_message |
| 1.4 | Errors, Status Codes & Cache Discipline | Failure handling, redirects, CacheMode.BYPASS vs. ENABLED |
| 2.1 | Schema-Based Extraction | JsonCssExtractionStrategy, JsonXPathExtractionStrategy, schemas, sibling source |
| 2.2 | Regex Extraction | RegexExtractionStrategy, built-in patterns, custom patterns, one-shot LLM pattern generation |
| 2.3 | Markdown Generation & Content Filtering | DefaultMarkdownGenerator, PruningContentFilter, raw vs. fit markdown |
| 2.4 | Batch Crawling & Rate Limiting | arun_many, MemoryAdaptiveDispatcher, RateLimiter, streaming |
| 2.5 | Lazy Loading & Basic Interaction | scan_full_page, wait_for_images, js_code, wait_for |
| 3.1 | Sessions, Pagination & Stateful Crawling | session_id, js_only, sequential workflows, kill_session |
| 3.2 | Proxies, SSL & Identity-Based Crawling | ProxyConfig, rotation, SSLCertificate, BrowserProfiler, geolocation |
| 3.3 | Anti-Bot, Stealth & Fallback Escalation | enable_stealth, UndetectedAdapter, proxy lists, fallback_fetch_function |
| 3.4 | Virtual Scroll, PDFs & Downloads | VirtualScrollConfig, PDFContentScrapingStrategy, file downloads |
| 3.5 | Network Capture, Hooks & C4A-Script | lifecycle hooks, network/console capture, declarative browser automation |
| 4.1 | LLM-Powered Extraction & Chunking | LLMExtractionStrategy, LLMConfig, Pydantic schemas, chunking, token tracking |
| 4.2 | Semantic Clustering & Hybrid Pipelines | CosineStrategy, combining clustering with downstream LLM extraction |
| 4.3 | Adaptive & Deep Crawling | AdaptiveCrawler, BFSDeepCrawlStrategy, stop conditions, state persistence |
| 4.4 | Observable Production Pipelines | logging, retry policy, cost tracking, result validation, dispatch metrics |
| 4.5 | Custom Strategies & Deployment | custom ExtractionStrategy, Docker/self-hosting, CLI, scheduled jobs |
02/outcome
Expert competency statement
You can design, build, and operate production web-data pipelines with Crawl4AI. You select deterministic extraction first, escalate to LLMs only when structure is unreliable, manage sessions and persistent identities, route and rotate proxies ethically, respect robots.txt and rate limits, chunk and route long content through LLMs with cost tracking, and extend the framework when off-the-shelf tools fall short.