Best open-source LLM eval & observability tools for enterprise

Open-source tools for testing, evaluating and tracing LLM applications.

Filters · 2 Enterprise, Evals & observability

For

Personal8Enterprise13

Purpose

AI & LLM tooling195Chat UIs5Agents & frameworks66Local inference11RAG & vector DBs41Coding assistants19Evals & observability13Agent skills15MCP servers16Developer tools285Web frameworks115Databases & backends49Editors & IDEs6API tools21Self-hosted apps45Notes & docs17Automation46Data & analytics47Infra & DevOps70Security & identity44Communication25Media & photos19Design & creative43

Runs on

Web1Docker / self-host7Desktop0Mobile0CLI3Library / SDK7Extension / plugin0

License

Permissive9Copyleft0Source-available0Other / custom4

More

Has live demo0Commit in last 90 days13

13 projects

SortWorth scoreTrendingMost starsRecently updatedNewest in catalog

p

promptfoopromptfoo/promptfoo

Test and red-team LLM appsEnterpriseEvals & observabilityCLILibrary / SDK★ 26kMITcommit 18 hours ago

85worth

☆ Save

M

MLflowmlflow/mlflow

Open-source AI engineering platformEnterpriseEvals & observabilityDocker / self-hostLibrary / SDK★ 28kApache-2.0commit 15 hours ago

83worth

☆ Save

L

Langfuselangfuse/langfuse

LLM engineering platform: tracing, evals and prompt managementEnterpriseEvals & observabilityWebDocker / self-host★ 35kcommit 21 hours ago

81worth

☆ Save

O

Opikcomet-ml/opik

Debug, evaluate and monitor LLM appsPersonalEnterpriseEvals & observabilityDocker / self-hostLibrary / SDK★ 22kApache-2.0commit yesterday

80worth

☆ Save

P

PhoenixArize-ai/phoenix

AI observability and evaluationPersonalEnterpriseEvals & observabilityDocker / self-hostLibrary / SDK★ 12kcommit 18 hours ago

77worth

☆ Save

g

garakNVIDIA/garak

LLM vulnerability scannerPersonalEnterpriseEvals & observabilitySecurity & identityCLI★ 9.4kApache-2.0commit yesterday

77worth

☆ Save

S

SigNozSigNoz/signoz

OpenTelemetry-native observabilityEnterpriseEvals & observabilityInfra & DevOpsDocker / self-host★ 32kcommit yesterday

76worth

☆ Save

i

iFixAiifixai-ai/iFixAi

Independent auditing of AI agentsPersonalEnterpriseEvals & observabilityCLI★ 19kApache-2.0commit 22 hours ago

76worth

☆ Save

D

DeepEvalconfident-ai/deepeval

The LLM evaluation frameworkPersonalEnterpriseEvals & observabilityLibrary / SDK★ 19kApache-2.0commit yesterday

76worth

☆ Save

S

Sentrygetsentry/sentry

Error tracking and performance monitoringEnterpriseDeveloper toolsEvals & observabilityDocker / self-host★ 45kcommit 17 hours ago

75worth

☆ Save

G

GiskardGiskard-AI/giskard-oss

Evaluation and testing for LLM agentsPersonalEnterpriseEvals & observabilitySecurity & identityLibrary / SDK★ 5.9kApache-2.0commit 2 days ago

75worth

☆ Save

S

SkillOptmicrosoft/SkillOpt

Optimizer that trains reusable agent skillsPersonalEnterpriseAgent skillsEvals & observabilityLibrary / SDK★ 18kMITcommit 3 days ago

74worth

☆ Save

H

HeliconeHelicone/helicone

Open-source LLM observabilityPersonalEnterpriseEvals & observabilityDocker / self-host★ 6.2kApache-2.0commit 2 weeks ago

68worth

☆ Save

Narrow down

Runs on

Web · 1Docker / self-host · 7CLI · 3Library / SDK · 7