unclecode / crawl4ai
git clone https://github.com/unclecode/crawl4ai.git
v0.9.5 · 15d ago
Open-source web crawler and scraper that converts any website into clean, LLM-ready Markdown, with an optional hosted cloud API.
Why you might care
Aimed at RAG pipelines and AI agents that need page content as Markdown, JSON extraction, or MCP access. Runs self-hosted via Python library, CLI, or Docker, or use the hosted Crawl4AI Cloud to skip managing browsers and proxies.
+60
stars, 24h · #715
+482
stars, 7 days · #585
+~1.9k
stars, 30 days · #543
86k
★ total · +1% this week
9.0k
forks · 289 open issues
Star history (as recorded here)
LanguagePython
LicenseApache-2.0
Created2024-05-09 (2y ago)
Last pushtoday
Watchers425
OwnerUser
GitHub topics
aiai-agentscrawlerdata-extractionllmmarkdownmcpopen-sourceplaywrightpythonragscraperweb-crawlerweb-crawlingweb-scraping
Links
- GitHub: github.com/unclecode/crawl4ai
- Homepage: https://crawl4ai.com
- Releases · Issues · Hacker News mentions
README
Loading README…
Blurb generated by claude-haiku-5-5 3d ago. Star deltas marked ~ are extrapolated from partial history.