Crawl4AI
unclecode/crawl4aiWeb crawler that turns sites into LLM-ready Markdown
PersonalEnterpriseRAG & vector DBsDocker / self-hostLibrary / SDKPermissive
About
An open-source crawler and scraper built for AI: clean Markdown output, structured extraction and browser control.
From the repository: “Open-source web crawler and scraper for LLMs and AI agents: any website into clean, LLM-ready Markdown. Run it yourself, or use Crawl4AI Cloud with one key.”
Alternatives
All alternatives →LightRAGSimple and fast graph-based RAGDocker / self-hostLibrary / SDK86
gpt-researcherAutonomous deep research agent for web and local documentsWebDocker / self-host85
ChromaOpen-source embedding database for AI appsDocker / self-hostLibrary / SDK77
FirecrawlWeb data API that turns sites into LLM-ready dataWebDocker / self-host77