Build the evaluation harness before the feature

There is a shape to AI projects that is easy to recognise once you have seen it twice. The first month is fast. The second is fast. Around the third, changes start taking longer, and by the fourth nobody wants to touch the prompt that works, because touching it might break something and there is no way to find out except by shipping.

The project has not run into a hard problem. It has run out of the ability to tell whether a change made things better.

Why conventional testing does not apply

The same input does not produce the same output, so equality assertions are useless. The output is usually text, so there is no clean pass or fail. And the failures that matter are not crashes: they are answers that look fine and are wrong in a way that requires domain knowledge to notice.

So teams fall back on judgement. Somebody tries a few examples after each change and forms an impression. That works for about six weeks and then quietly stops working, because impressions do not accumulate and nobody can remember what the system used to do.

What a harness actually is

Less than it sounds. In its useful minimum:

A set of real cases. Fifty to two hundred inputs drawn from actual usage, including the awkward ones — the ambiguous request, the badly scanned document, the customer with two accounts.

A known good answer for each. Or, where there is no single right answer, a rubric a scorer can apply consistently.

Automatic scoring. Exact match where it fits, structured field comparison where the output is structured, and a model-based judge with a written rubric where it is prose. Each has limits; having any is the difference.

A number that moves. One overall score plus per-category breakdowns, run on every change and recorded, so a regression is a fact rather than a feeling.

The discipline that makes it work

Build it before the feature. Once a system is live, assembling a clean evaluation set is much harder and always deprioritised in favour of the next thing.

Keep the awkward cases. Every incident becomes a case. This is how the harness grows into the most valuable artefact in the codebase, and it costs nothing to maintain that way.

Score retrieval separately. In retrieval-augmented systems most bad answers are retrieval failures, and a single end-to-end score hides which half of the system is at fault.

Never let the harness be written by the same prompt. If the evaluation and the implementation share a model, a prompt or an assumption, the harness will approve of whatever the system happens to do.

Publish the number. Including when it goes down. A quality metric that only appears when it flatters the team is not a metric.

What it buys

Changes become safe. Model upgrades become a measurable decision rather than a leap. Cost optimisation becomes possible, because you can move work to a smaller model and see exactly what it cost you in quality. And an incident becomes a case rather than an argument.

It is a week of work at the start of a project and it is the difference between a codebase that speeds up over time and one that seizes. It is the first thing we build in an AI-native apps engineering engagement — including, and especially, when we are taking over somebody else’s system.