The AI feature nobody dares to change

Somewhere in most organisations that adopted AI early there is a feature like this. It went live a year or so ago, people use it, and it mostly works. The person who built it has moved on. The prompt has grown by a sentence after every complaint. Nobody knows how good it is, so nobody wants to change it, so it does not change.

That is fine until something forces a change. Then it turns out to be the most expensive kind of system to own: one you depend on and cannot safely touch.

What forces the change

The model is retired. Providers deprecate models on their own schedule. A feature built against one specific model will have to move whether or not anybody is ready, and without a way to measure quality the move is a guess made under a deadline.

The bill grows. Usage rises, the cost rises with it, and the obvious levers — a smaller model, shorter prompts, caching — all carry the same question: what will this do to quality? Without an answer, nobody pulls them.

The process changes. A new product line, a new regulation, a new document template. The feature has to learn something new, and every edit risks breaking something it already did.

An incident. A wrong answer reaches a customer. Somebody asks how often that happens. Nobody can say.

None of these is unusual. Every live AI feature meets one of them sooner or later, and the only question is whether the effect of the change can be measured when it does.

Why patching does not get you out

The natural response to each of these is a patch: a sentence added to the prompt, a special case in the code. Each patch is reasonable, and each makes the system harder to reason about, because nobody can see any longer what the earlier patches were protecting against.

A feature in this state does not need more patches. It needs a way to tell whether a change made it better or worse. Until that exists, every change — including the one you are being forced to make — is a gamble.

What taking it over looks like

Infographic

Taking over a live AI feature

1. 01Read before touchingLogs, prompt history, complaints and the cases people remember.
2. 02Build the evaluation setReal production inputs with good answers agreed with the experts.
3. 03Measure the baselineThe baseline, and often the first honest measure of quality.
4. 04Make the forced changeModel, cost or requirement, with the effect on quality measured.
5. 05Hand it backEvaluation on every change, reasons written down, your team in charge.

Reading first. Logs, the prompt history, the complaints, the cases people remember. The first days are mostly listening.

An evaluation set built from what the feature actually sees. Real inputs from production, awkward ones included, with known good answers agreed with somebody who does the work. This is usually the first week or so, and it usually reveals the actual problem — which is often not the one that prompted the call.

A score for the system as it stands. Before anything changes. That number is the baseline for every later decision, and it is frequently the first time anybody has known how good the feature is.

One change at a time. The forced one first — the model migration, the cost reduction — with its effect on quality measured rather than hoped for.

A system that can be changed. With the evaluation set running on every change, the reasons behind each patch written down, and your team able to make the next change on its own.

What it costs you in time and attention

Access to production logs and to the people who complain about the feature. A few hours from a domain expert to agree what good answers look like for the evaluation cases; without that, the work stalls. And a decision-maker prepared to hear that the feature is worse than assumed, or better, or good at half its job and poor at the other half.

What a good outcome looks like

The model migration, the cost reduction or the new requirement is done, with a quality number from before and after to show for it. The feature has a measured quality that somebody reviews. And the next forced change is a routine piece of work rather than a crisis.

When to call someone in

Ideally before the deadline. A retirement notice with a few months to run is enough time to build the evaluation set, score the current system and migrate calmly. The same notice with a few weeks left turns all of it into the gamble described above.

In AI-native apps engineering, taking over an existing codebase starts with the evaluation harness, because without it any change to an inherited AI system is a gamble. It is typically the first week, and it usually shows what the real problem is.