The first hour of an agent incident

Sooner or later an agent will do something nobody intended. It will send a document to the wrong party, close a batch of tickets that should have stayed open, or change records it had no business changing. Sometimes because it misread the task, sometimes because something it read told it to.

The first questions are always the same. What did it do? Why? On whose behalf? What else did it touch? Is it still happening? For many agents in production today, answering them takes days, because the logs record HTTP calls rather than decisions and the agent runs under an identity it shares with something else.

What the log has to hold before anything happens

The inputs as they were. Not just the user’s request: every document, message, page and tool result that entered the context, stored or referenced with its version, so you can see what the agent had read when it acted. If an injected instruction caused the incident, this is the only place it will be.

Every call with its parameters. Each tool call with its exact arguments and result, the model and prompt version in use, and the step of the task that produced it.

The chain of authority. Which agent, acting for which user, with which credential, approved by whom. Without it the log says that a service account did something, which is a sentence and not an answer.

Out of the agent’s reach. Written somewhere append-only that the agent cannot edit or delete. A system that can tidy its own records has records nobody can rely on.

Kept as long as the questions can come. Retention set by how long the agent’s actions can return as a complaint, a dispute or an auditor’s question, not by the default of the logging tool.

For systems that fall into the EU AI Act’s high-risk category, automatic logging is an obligation rather than a choice. For everything else, it is simply the only way to answer the five questions above.

The order of the first moves

Infographic

The first moves, in order

1. 01ContainRevoke or narrow its credentials and pause outbound tools. Delete nothing.
2. 02PreserveSnapshot logs, context and sandbox state before anyone starts fixing.
3. 03ReconstructReplay the decision: what it read, what it called, for whom.
4. 04ScopeFind every other session that read the same content or took the same path.
5. 05RemediateFix the boundary that failed, then reverse what can be reversed.
6. 06LearnThe triggering input becomes a permanent case in the evaluation set.

Contain through identity, not by pulling the plug. If the agent has its own identity, containment means revoking or narrowing its credentials and disabling its outbound tools: seconds, reversible, and the rest of the business keeps working. If it runs on a person’s account or a shared one, containment locks people out as well. That is why teams hesitate, and the hesitation is the expensive part.

Preserve before fixing. The instinct is to patch the prompt and redeploy. That replaces the evidence of what the agent was running with something nobody has examined yet. Snapshot the logs, the context and any sandbox state first.

Reconstruct from the record, not from the model. Ask the agent why it did something and you get a fluent explanation written after the fact, which is not evidence. The answer is in the logged inputs and calls: what was in the context at the moment of each decision.

Assume it was not one session. If the cause was content the agent read — a poisoned document on a shared drive, a web page, a ticket — every session that read the same content is suspect. The input log lets you find them. Without it you are guessing, and guesses made during an incident tend to undercount.

Fix the boundary, not the wording. A new line in the prompt telling the agent not to do it again is a hope. The durable fix is the permission, the egress rule, the limit or the approval step that would have stopped it. The triggering input then goes into the evaluation set, so the gap cannot quietly reopen.

Rehearse it once

In a test environment, plant an instruction in a document the agent will read and let it act. Then ask the team to answer the five questions within the hour. Whatever they cannot answer is the work list, and it is much cheaper to find it in a rehearsal than with a customer on the phone.

That is the incident process an AI managed services engagement runs on: the first response narrows what the agent may do unattended, the decision log makes the incident reconstructable, and the record afterwards is written down. When we take over an agent somebody else built, whether its logs could answer those five questions is one of the first things we check.