Evaluating LLM Features Before Your Users Do

Traditional code has tests; a function either returns 42 or it does not. LLM features return prose, and prose has no assertEqual. So most teams ship on vibes: try five prompts, nod, deploy. Then the model updates, quality drifts, and nobody notices until users complain. Evaluation is how you replace vibes with engineering.

Start with a golden set, not a framework

Collect 30 to 50 real inputs your feature will face — from support tickets, logs, or your own dogfooding. For each, write down what a good output must contain and what it must never contain. This file is worth more than any evaluation library, because it encodes your definition of quality.

GOLDEN_SET = [
    {
        "input": "How do I rotate my API key?",
        "must_include": ["Settings", "Regenerate"],
        "must_not_include": ["I don't have access", "as an AI"],
        "must_cite": "docs/authentication.md",
    },
    # ... 40 more, drawn from real usage
]

Three layers of checks

1. Deterministic assertions: response parses as JSON, citations point at real documents, length within bounds, no banned phrases. Cheap, fast, zero false positives — run on every commit.
2. Model-graded rubrics: a second LLM scores each response against explicit criteria — accuracy, groundedness, tone. Noisy per-sample, reliable in aggregate across 50 cases.
3. Human spot checks: ten random production samples a week, read by a person. This is how you discover the failure categories your rubric never imagined.

Make it a regression gate

The payoff comes when evaluation runs in CI. Score every prompt change and model upgrade against the golden set, and block the merge when the aggregate drops. Suddenly prompt engineering stops being guesswork — you can refactor a prompt with the same confidence you refactor typed code.

def test_no_regression(baseline: float):
    score = run_eval(GOLDEN_SET, model=CURRENT_MODEL, prompt=CURRENT_PROMPT)
    assert score >= baseline - 0.02, (
        f"Quality dropped: {score:.2f} vs baseline {baseline:.2f}"
    )

If you cannot measure your LLM feature, you cannot improve it — you can only change it and hope.

The LLM Engineering learning path dedicates a full module to building evaluation suites, including LLM-as-judge calibration and cost-aware sampling strategies.