Evals#
An eval is a test for a model-driven system. Without evals, prompt changes are vibes-driven and regressions ship silently. The discipline borrows from software testing but the assertions are fuzzier and the inputs are noisier.
Levels#
Unit-level eval. A single prompt or tool call against a single input, with a deterministic check (regex, JSON structure, exact-match, contains).
Scenario eval. A multi-turn agent trajectory against a scripted environment. Asserts on the final answer and on trajectory properties (no forbidden tools called, no PII leaked, budget respected).
Production eval. Sampled real traffic, scored offline by a judge model or by humans. The only true measure; everything else approximates it.
Assertion styles#
Exact match / contains. The cheapest. Works for extraction tasks (“is this CVE in the output?”).
JSON schema. Validate structure first, content second.
LLM-as-judge. A second model scores the answer against a rubric. Powerful and noisy; calibrate against human judgment before trusting.
Pairwise preference. Show the judge two candidates, pick the better. More reliable than absolute scores.
Functional correctness. For code-generating systems: did the generated code pass tests? Bypasses the judge problem entirely.
Property tests. Invariants that must hold (no PII in output, cited chunks exist, tool calls type-check).
Building the eval set#
Start with logs. Real production failures become eval cases. Each bug ticket adds at least one regression test.
Cover the boring cases. Ninety percent of traffic is similar; eval most of it cheaply. Reserve the long tail for the tricky set.
Adversarial set. Prompt injection, jailbreaks, malformed inputs, partial context. See AI Systems as Targets for the attack catalog.
Hold out a private set. Models drift; vendors update. A private set the model has never seen is the only honest measure.
Operational practice#
Run on every prompt change. CI runs the eval suite the same way it runs unit tests.
Track scores over time. A dashboard with score-per-prompt- version is the most useful single artifact.
Sample production traffic for offline eval. Five percent is usually enough.
Cost the eval. Eval suites get expensive. Cache results against (prompt-version, input) pairs.
Beware Goodhart. When a score becomes a target, it stops measuring quality. Mix the metrics; rotate the set.
Tools#
promptfoo, declarative YAML eval suites; CI-friendly.
inspect (UK AI Safety Institute), production eval framework.
DeepEval, pytest-style assertions for LLM output.
Braintrust / Langfuse / Weights & Biases, hosted eval dashboards.