Ask an agent supplier for the decision log, not the demo

Agent demonstrations are good now. The supplier picks a task, the agent reads an email, looks something up, drafts a reply and updates a record, and the room is impressed. None of that is evidence of anything except that the agent can do the task once, on a case somebody chose. The demonstration is not dishonest. It simply answers a different question from the one you are paying to have answered.

What a buyer needs to know is how the system behaves on its bad days, and whether anybody will be able to tell what happened. Those questions can be answered before signing, and the way a supplier answers them is most of the evaluation.

Questions worth asking

Can we see a decision log from a real run? Not a dashboard — the record of what the agent saw, what it concluded, what it did and who approved it, for an individual case. If somebody outside the project cannot read it and reconstruct a decision, nobody will be able to after an incident either.

What is the error rate, and on what? A supplier with an evaluation set of real cases, awkward ones included, can tell you how often the agent is wrong and in which way. A supplier who answers with an accuracy figure from a demonstration dataset, or with reassurance, has not built one.

What exactly may it do? The boundaries — the data it may see, the actions it may take, the value limits, the cases that always go to a person — should exist in writing and be enforced by the system rather than by instructions to the model. Ask where they are enforced.

What happens on the day it is wrong? There should be a defined way to narrow what the agent may do unattended, a runbook, and a named person who gets called. A promise to investigate is not a reduction path.

How long does an approval take? Approval gates on irreversible actions are not negotiable. Whether they work depends on whether a person can check an action in seconds rather than minutes, because slow review turns into rubber-stamping. Ask to see the approval screen, and if it does not exist yet, ask who will design it and when.

What happens when the model changes? Model providers change quickly. The model should sit behind an interface, so that your process is not rebuilt around one vendor’s release schedule. Ask what had to change the last time they switched.

What stays with us? Code, tool definitions, the evaluation set, logs and the runbook, in your repositories and your infrastructure, with exit terms agreed at the start, while you still have leverage.

Infographic

Before you sign for an agent

• A real decision logFrom a real run, readable by someone outside the project.
• A published error rateOn real cases, awkward ones included, not a smoothed average.
• Written boundariesWhat it may read, change and must escalate, enforced in code.
• A reduction pathHow autonomy is narrowed on the day something goes wrong.
• Approval designWhere people confirm, and how long one confirmation takes.
• A replaceable modelYour process survives a change of model provider.
• Exit termsCode, tools, evaluation set and logs stay with you.

Reading the answers

The pattern matters more than any single answer. A supplier who builds for production will answer most of these with an artefact: a log, a number, a document, a screen. A supplier who builds for demonstrations will answer with a description of what the model can do.

Specific beats impressive. An error rate on a named set of cases, with the worst examples shown, is worth more than a higher figure with no examples at all.

Notice also who answers. If every question goes to the person who ran the demonstration and none to the person who would run the system, that is information too.

What it costs you to ask

An afternoon, and some discomfort. These questions can make a sales meeting awkward, which is part of their use. A supplier who is comfortable with them has usually been through a real incident and built for the next one.

They also cost the supplier something, which is the point. Producing a real decision log or an honest error rate takes work that only exists if the system was built to produce it. A supplier who needs a few weeks to put those together is telling you how the system was built.

Questions to ask yourselves first

Some of these questions turn back on the buyer. Who in your organisation will read the decision logs? Who will approve actions, and is that time in their week? Who owns the process the agent will work in, and will they read the escalations? A supplier can build all of this well and the agent can still fail if nobody on your side is there to receive it.

How we approach it

These are the questions AI agents engineering is designed to answer with something you can look at: a task and boundary definition, an evaluation set built before the agent, approval and escalation design, decision logs, and a runbook written before it is needed.