promptfoopromptfoo/promptfoo
Test and red-team LLM appsEnterpriseEvals & observabilityCLILibrary / SDK★ 26kMITcommit yesterday85worth
Open-source alternatives to garak
24 projects in the catalog share a category with garak (Evals & observability). They are ranked by how closely their categories, platforms and GitHub topics match, then by worth score. Of these, 13 run with Docker, 16 have a permissive license and 24 had a commit in the last 90 days.
Compare at a glance
| Project | Runs on | Self-host with Docker | License | Stars | Last commit |
|---|---|---|---|---|---|
| garak (this project) | CLI | Not listed | Apache-2.0 | 9.4k | |
| Giskard | Library / SDK | Not listed | Apache-2.0 | 5.9k | |
| promptfoo | CLI, Library / SDK | Not listed | MIT | 26k | |
| iFixAi | CLI | Not listed | Apache-2.0 | 19k | |
| MLflow | Docker / self-host, Library / SDK | Yes | Apache-2.0 | 28k | |
| Langfuse | Web, Docker / self-host | Yes | Custom (read before use) | 35k | |
| Shannon | Docker / self-host, CLI | Yes | AGPL-3.0 | 49k | |
| Opik | Docker / self-host, Library / SDK | Yes | Apache-2.0 | 22k | |
| Strix | Docker / self-host, CLI | Yes | Apache-2.0 | 66k | |
| Phoenix | Docker / self-host, Library / SDK | Yes | Custom (read before use) | 12k | |
| SigNoz | Docker / self-host | Yes | Custom (read before use) | 32k | |
| DeepEval | Library / SDK | Not listed | Apache-2.0 | 19k | |
| Sentry | Docker / self-host | Yes | Custom (read before use) | 45k | |
| SkillOpt | Library / SDK | Not listed | MIT | 18k | |
| Helicone | Docker / self-host | Yes | Apache-2.0 | 6.2k | |
| NemoClaw | Docker / self-host | Yes | Apache-2.0 | 23k | |
| Ollama | Docker / self-host, Desktop, CLI | Yes | MIT | 182k | |
| Gemini CLI | CLI | Not listed | Apache-2.0 | 107k | |
| OpenClaw | Docker / self-host, Desktop, CLI | Yes | MIT | 391k | |
| Career-Ops | CLI | Not listed | MIT | 73k | |
| Geth | CLI | Not listed | LGPL-3.0 | 51k | |
| goose | Desktop, CLI | Not listed | Apache-2.0 | 55k | |
| nanobot | Web, Docker / self-host, CLI | Yes | MIT | 49k | |
| MarkItDown | CLI, Library / SDK | Not listed | MIT | 188k | |
| Caveman | CLI, Extension / plugin | Not listed | Apache-2.0 | 109k |
Command-line tools (13)
iFixAiifixai-ai/iFixAi
Independent auditing of AI agentsPersonalEnterpriseEvals & observabilityCLI★ 19kApache-2.0commit yesterday76worth
ShannonKeygraphHQ/shannon
AI pentester for your web apps and APIsEnterpriseAgents & frameworksSecurity & identityDocker / self-hostCLI★ 49kAGPL-3.0commit 3 days ago80worth
Strixusestrix/strix
AI agents that pentest your own appsPersonalEnterpriseAgents & frameworksSecurity & identityDocker / self-hostCLI★ 66kApache-2.0commit yesterday79worth
Ollamaollama/ollama
Run open-weight LLMs locally with one commandPersonalEnterpriseLocal inferenceDocker / self-hostDesktopCLI★ 182kMITcommit yesterday90worth
Gemini CLIgoogle-gemini/gemini-cli
Gemini as an agent in your terminalPersonalEnterpriseCoding assistantsCLI★ 107kApache-2.0commit 2 days ago90worth
OpenClawopenclaw/openclaw
Personal AI assistant that acts on your behalf across appsPersonalEnterpriseAgents & frameworksDocker / self-hostDesktopCLI★ 391kMITcommit 21 hours ago90worth
Career-Opscareer-ops-hq/career-ops
AI-assisted job search pipelinePersonalAgents & frameworksAutomationCLI★ 73kMITcommit yesterday89worth
Gethethereum/go-ethereum
Go implementation of EthereumPersonalEnterpriseSecurity & identityCLI★ 51kLGPL-3.0commit 3 days ago89worth
gooseaaif-goose/goose
Extensible on-machine AI agent from BlockPersonalEnterpriseAgents & frameworksCoding assistantsDesktopCLI★ 55kApache-2.0commit yesterday88worth
nanobotHKUDS/nanobot
Ultra-light self-hosted personal AI agentPersonalAgents & frameworksWebDocker / self-hostCLI★ 49kMITcommit yesterday87worth
MarkItDownmicrosoft/markitdown
Convert Office files and PDFs to Markdown for LLMsPersonalEnterpriseRAG & vector DBsCLILibrary / SDK★ 188kMITcommit 22 hours ago87worth
CavemanJuliusBrussee/caveman
Cut coding agent tokens by making it talk lessPersonalAgent skillsCoding assistantsCLIExtension / plugin★ 109kApache-2.0commit 23 hours ago87worth
Web apps (1)
Langfuselangfuse/langfuse
LLM engineering platform: tracing, evals and prompt managementEnterpriseEvals & observabilityWebDocker / self-host★ 35kcommit yesterday81worth
Self-hosted Docker apps (7)
MLflowmlflow/mlflow
Open-source AI engineering platformEnterpriseEvals & observabilityDocker / self-hostLibrary / SDK★ 28kApache-2.0commit 21 hours ago83worth
Opikcomet-ml/opik
Debug, evaluate and monitor LLM appsPersonalEnterpriseEvals & observabilityDocker / self-hostLibrary / SDK★ 22kApache-2.0commit yesterday80worth
PhoenixArize-ai/phoenix
AI observability and evaluationPersonalEnterpriseEvals & observabilityDocker / self-hostLibrary / SDK★ 12kcommit yesterday77worth
SigNozSigNoz/signoz
OpenTelemetry-native observabilityEnterpriseEvals & observabilityInfra & DevOpsDocker / self-host★ 32kcommit yesterday76worth
Sentrygetsentry/sentry
Error tracking and performance monitoringEnterpriseDeveloper toolsEvals & observabilityDocker / self-host★ 45kcommit 23 hours ago75worth
HeliconeHelicone/helicone
Open-source LLM observabilityPersonalEnterpriseEvals & observabilityDocker / self-host★ 6.2kApache-2.0commit 2 weeks ago68worth
NemoClawNVIDIA/NemoClaw
Run agents more securely in NVIDIA OpenShellEnterpriseAgents & frameworksSecurity & identityDocker / self-host★ 23kApache-2.0commit yesterday75worth
Libraries & SDKs (3)
GiskardGiskard-AI/giskard-oss
Evaluation and testing for LLM agentsPersonalEnterpriseEvals & observabilitySecurity & identityLibrary / SDK★ 5.9kApache-2.0commit 2 days ago75worth
DeepEvalconfident-ai/deepeval
The LLM evaluation frameworkPersonalEnterpriseEvals & observabilityLibrary / SDK★ 19kApache-2.0commit yesterday76worth
SkillOptmicrosoft/SkillOpt
Optimizer that trains reusable agent skillsPersonalEnterpriseAgent skillsEvals & observabilityLibrary / SDK★ 18kMITcommit 3 days ago74worth