AI Observability & Evaluation AI Tools
Open-source tools for monitoring, logging, tracing, evaluating, and debugging AI model outputs, prompts, and pipelines.
Language model applications fail in ways a stack trace never captures: the request returned 200, the output read fluently, and the answer was still wrong. This tooling makes that class of failure visible and countable, and it gets picked up by whoever owns a system whose behavior shifts when a model version, prompt, or corpus changes underneath it. Three lineages converge here. Experiment trackers carried over from MLOps and still anchor training work: MLflow, Weights and Biases, ClearML, Aim. Trace-first observability grew up around OpenTelemetry and records what production did, span by span: Langfuse, Arize Phoenix, OpenLLMetry, OpenLIT, Helicone, Laminar, AgentOps. Offline evaluation scores fixed datasets instead: promptfoo, DeepEval, LM Evaluation Harness, LightEval, OpenCompass, Inspect AI, with Giskard and PyRIT covering adversarial passes. The split is honesty against repeatability, since traces show real traffic but arrive unlabeled and cost money to score, while offline suites gate CI cheaply and drift from what users send. A reasonable first stack is Langfuse for tracing plus promptfoo as a pull request gate, adding DeepEval once regressions need defined metrics rather than eyeballing. The trap to check before self hosting is open core packaging: several of these repositories ship a permissive core next to an enterprise directory under a separate commercial license, and single sign on, role based access, and retention controls usually live there. Storage is the other cost, since full trace capture on a chatty agent fills a ClickHouse volume fast, so sampling policy belongs in the design, not the postmortem.