promptfoopromptfoo/promptfoo
Test and red-team LLM appsEnterpriseEvals & observabilityCLILibrary / SDK★ 26kMITcommit 15 hours ago85worth
Best open-source LLM eval & observability tools
Open-source tools for testing, evaluating and tracing LLM applications.
14 projects
MLflowmlflow/mlflow
Open-source AI engineering platformEnterpriseEvals & observabilityDocker / self-hostLibrary / SDK★ 28kApache-2.0commit 10 hours ago83worth
Langfuselangfuse/langfuse
LLM engineering platform: tracing, evals and prompt managementEnterpriseEvals & observabilityWebDocker / self-host★ 35kcommit 2 days ago81worth
Opikcomet-ml/opik
Debug, evaluate and monitor LLM appsPersonalEnterpriseEvals & observabilityDocker / self-hostLibrary / SDK★ 22kApache-2.0commit 2 days ago80worth
iFixAiifixai-ai/iFixAi
Independent auditing of AI agentsPersonalEnterpriseEvals & observabilityCLI★ 20kApache-2.0commit 2 days ago77worth
PhoenixArize-ai/phoenix
AI observability and evaluationPersonalEnterpriseEvals & observabilityDocker / self-hostLibrary / SDK★ 12kcommit yesterday77worth
garakNVIDIA/garak
LLM vulnerability scannerPersonalEnterpriseEvals & observabilitySecurity & identityCLI★ 9.4kApache-2.0commit 2 days ago77worth
SigNozSigNoz/signoz
OpenTelemetry-native observabilityEnterpriseEvals & observabilityInfra & DevOpsDocker / self-host★ 32kcommit 2 days ago76worth
DeepEvalconfident-ai/deepeval
The LLM evaluation frameworkPersonalEnterpriseEvals & observabilityLibrary / SDK★ 19kApache-2.0commit 2 days ago76worth
Sentrygetsentry/sentry
Error tracking and performance monitoringEnterpriseDeveloper toolsEvals & observabilityDocker / self-host★ 45kcommit 15 hours ago75worth
ouroborosQ00/ouroboros
Agent OS and specification runtime for AI coding workflowsPersonalMCP serversCoding assistantsEvals & observabilityCLIExtension / plugin★ 6.2kMITcommit 18 hours ago75worth
GiskardGiskard-AI/giskard-oss
Evaluation and testing for LLM agentsPersonalEnterpriseEvals & observabilitySecurity & identityLibrary / SDK★ 5.9kApache-2.0commit 2 days ago75worth
SkillOptmicrosoft/SkillOpt
Optimizer that trains reusable agent skillsPersonalEnterpriseAgent skillsEvals & observabilityLibrary / SDK★ 18kMITcommit 3 days ago74worth
HeliconeHelicone/helicone
Open-source LLM observabilityPersonalEnterpriseEvals & observabilityDocker / self-host★ 6.2kApache-2.0commit 3 weeks ago68worth