promptfoopromptfoo/promptfoo
Test and red-team LLM appsEnterpriseEvals & observabilityCLILibrary / SDK★ 26kMITcommit 9 hours ago85worth
Open-source alternatives to DeepEval
24 projects in the catalog share a category with DeepEval (Evals & observability). They are ranked by how closely their categories, platforms and GitHub topics match, then by worth score. Of these, 10 run with Docker, 19 have a permissive license and 24 had a commit in the last 90 days.
Compare at a glance
| Project | Runs on | Self-host with Docker | License | Stars | Last commit |
|---|---|---|---|---|---|
| DeepEval (this project) | Library / SDK | Not listed | Apache-2.0 | 19k | |
| promptfoo | CLI, Library / SDK | Not listed | MIT | 26k | |
| MLflow | Docker / self-host, Library / SDK | Yes | Apache-2.0 | 28k | |
| Opik | Docker / self-host, Library / SDK | Yes | Apache-2.0 | 22k | |
| Phoenix | Docker / self-host, Library / SDK | Yes | Custom (read before use) | 12k | |
| Giskard | Library / SDK | Not listed | Apache-2.0 | 5.9k | |
| SkillOpt | Library / SDK | Not listed | MIT | 18k | |
| Langfuse | Web, Docker / self-host | Yes | Custom (read before use) | 35k | |
| garak | CLI | Not listed | Apache-2.0 | 9.4k | |
| SigNoz | Docker / self-host | Yes | Custom (read before use) | 32k | |
| iFixAi | CLI | Not listed | Apache-2.0 | 19k | |
| Sentry | Docker / self-host | Yes | Custom (read before use) | 45k | |
| Helicone | Docker / self-host | Yes | Apache-2.0 | 6.2k | |
| Browser Use | Library / SDK | Not listed | MIT | 117k | |
| Crawl4AI | Docker / self-host, Library / SDK | Yes | Apache-2.0 | 85k | |
| MarkItDown | CLI, Library / SDK | Not listed | MIT | 188k | |
| Headroom | Library / SDK | Not listed | Apache-2.0 | 74k | |
| Docling | CLI, Library / SDK | Not listed | MIT | 68k | |
| MemPalace | Library / SDK | Not listed | MIT | 59k | |
| llama.cpp | CLI, Library / SDK | Not listed | MIT | 130k | |
| vLLM | Docker / self-host, Library / SDK | Yes | Apache-2.0 | 93k | |
| Transformers | Library / SDK | Not listed | Apache-2.0 | 167k | |
| LightRAG | Docker / self-host, Library / SDK | Yes | MIT | 40k | |
| OpenViking | Library / SDK | Not listed | AGPL-3.0 | 39k | |
| LlamaIndex | Library / SDK | Not listed | MIT | 52k |
Libraries & SDKs (18)
MLflowmlflow/mlflow
Open-source AI engineering platformEnterpriseEvals & observabilityDocker / self-hostLibrary / SDK★ 28kApache-2.0commit 5 hours ago83worth
Opikcomet-ml/opik
Debug, evaluate and monitor LLM appsPersonalEnterpriseEvals & observabilityDocker / self-hostLibrary / SDK★ 22kApache-2.0commit 17 hours ago80worth
PhoenixArize-ai/phoenix
AI observability and evaluationPersonalEnterpriseEvals & observabilityDocker / self-hostLibrary / SDK★ 12kcommit 8 hours ago77worth
GiskardGiskard-AI/giskard-oss
Evaluation and testing for LLM agentsPersonalEnterpriseEvals & observabilitySecurity & identityLibrary / SDK★ 5.9kApache-2.0commit yesterday75worth
SkillOptmicrosoft/SkillOpt
Optimizer that trains reusable agent skillsPersonalEnterpriseAgent skillsEvals & observabilityLibrary / SDK★ 18kMITcommit 2 days ago74worth
Browser Usebrowser-use/browser-use
Let AI agents use a web browserPersonalEnterpriseAgents & frameworksAutomationLibrary / SDK★ 117kMITcommit 8 hours ago89worth
Crawl4AIunclecode/crawl4ai
Web crawler that turns sites into LLM-ready MarkdownPersonalEnterpriseRAG & vector DBsDocker / self-hostLibrary / SDK★ 85kApache-2.0commit last week88worth
MarkItDownmicrosoft/markitdown
Convert Office files and PDFs to Markdown for LLMsPersonalEnterpriseRAG & vector DBsCLILibrary / SDK★ 188kMITcommit 6 hours ago87worth
Headroomheadroomlabs-ai/headroom
Compress tool output before it reaches the LLMPersonalEnterpriseCoding assistantsLibrary / SDK★ 74kApache-2.0commit 7 hours ago87worth
Doclingdocling-project/docling
Get your documents ready for generative AIPersonalEnterpriseRAG & vector DBsCLILibrary / SDK★ 68kMITcommit 17 hours ago87worth
MemPalaceMemPalace/mempalace
Open-source AI memory systemPersonalAI & LLM toolingMCP serversLibrary / SDK★ 59kMITcommit 7 hours ago87worth
llama.cppggml-org/llama.cpp
LLM inference in C/C++ on CPUs and GPUsPersonalLocal inferenceCLILibrary / SDK★ 130kMITcommit 8 hours ago86worth
vLLMvllm-project/vllm
High-throughput LLM serving engineEnterpriseLocal inferenceDocker / self-hostLibrary / SDK★ 93kApache-2.0commit 7 hours ago86worth
Transformershuggingface/transformers
Model definitions for state-of-the-art ML modelsPersonalEnterpriseAI & LLM toolingLibrary / SDK★ 167kApache-2.0commit 12 hours ago86worth
LightRAGHKUDS/LightRAG
Simple and fast graph-based RAGPersonalEnterpriseRAG & vector DBsDocker / self-hostLibrary / SDK★ 40kMITcommit 7 days ago86worth
OpenVikingvolcengine/OpenViking
Context database for AI agentsPersonalEnterpriseRAG & vector DBsLibrary / SDK★ 39kAGPL-3.0commit 14 hours ago86worth
LlamaIndexrun-llama/llama_index
Data framework for LLM applications and agentsEnterpriseAgents & frameworksRAG & vector DBsLibrary / SDK★ 52kMITcommit 2 days ago85worth
Web apps (1)
Langfuselangfuse/langfuse
LLM engineering platform: tracing, evals and prompt managementEnterpriseEvals & observabilityWebDocker / self-host★ 35kcommit 11 hours ago81worth
Self-hosted Docker apps (3)
SigNozSigNoz/signoz
OpenTelemetry-native observabilityEnterpriseEvals & observabilityInfra & DevOpsDocker / self-host★ 32kcommit 20 hours ago76worth
Sentrygetsentry/sentry
Error tracking and performance monitoringEnterpriseDeveloper toolsEvals & observabilityDocker / self-host★ 45kcommit 7 hours ago75worth
HeliconeHelicone/helicone
Open-source LLM observabilityPersonalEnterpriseEvals & observabilityDocker / self-host★ 6.2kApache-2.0commit 2 weeks ago68worth
Command-line tools (2)
garakNVIDIA/garak
LLM vulnerability scannerPersonalEnterpriseEvals & observabilitySecurity & identityCLI★ 9.4kApache-2.0commit 15 hours ago77worth
iFixAiifixai-ai/iFixAi
Independent auditing of AI agentsPersonalEnterpriseEvals & observabilityCLI★ 19kApache-2.0commit 13 hours ago76worth