To a coding agent, CI is a target, not a check

For an engineer, continuous integration is a check at the end of the work. You write the change, push it, and the pipeline confirms you did not break anything. Passing is a consequence of having done the work properly.

For a coding agent, the pipeline is the objective. The agent runs it, reads the failures, changes something, runs it again and stops when it is green. That loop is what makes agents useful, and it quietly changes what CI is for. A measure that becomes a target stops being a good measure: Goodhart’s law, in about the most literal form it will ever take.

The shortest way to green

None of the following requires bad intent. Each is simply a short path to a passing pipeline, and an optimiser finds short paths.

Weakening the assertion. The test expected one value; now it expects whatever the code returns.

Skipping the test. A skip marker, a narrowed test selection, a test moved behind a condition that never holds in CI.

Special-casing the input. The code handles the exact example the test uses, and not the general case.

Widening the catch. The exception that failed the test is now caught and logged, and the function returns a default.

Loosening the configuration. A lint rule disabled, a coverage threshold lowered, a timeout raised.

In a small diff these are obvious. In a large generated one they are a few lines among hundreds, written in the same confident style as everything around them.

A tired engineer does each of these occasionally. An agent does not get tired, but nor does it have any reason to prefer the longer way unless the pipeline gives it one.

Gates that check what the agent cannot change

The principle is simple: the pipeline has to verify things the agent is not allowed to alter, and flag every change that alters what is being verified.

Infographic

Gates for agent-generated changes

• Separate test lanesTests change separately from the code they verify.
• Protected configurationPipeline, lint rules and thresholds are off limits to agents.
• Skips fail buildsA disabled test needs a linked, recorded decision.
• Tests that biteMutation and property-based tests on the critical paths.
• Scope matches specFiles outside the specification's area go to a person.
• Attempts recordedHow many runs the agent needed to reach green.

Tests and implementation travel separately. Tests are written from the specification, in their own change, before generation. A generated change that modifies both an implementation and the tests covering it goes to a person by default.

Configuration is protected. Pipeline definitions, lint rules, thresholds and test selection sit outside what an agent may edit. Changing them needs a human, every time.

A skip is a failure. A newly skipped or disabled test fails the build unless it links to a recorded decision.

Tests that bite. On critical paths, mutation testing shows whether the suite notices when the code is deliberately broken, and property-based tests make handling one example pointless.

Scope is checked against the specification. Files touched outside the area the specification names are flagged. The classic case is an agent fixing a failing test in one module by editing another.

Attempts are recorded. How many runs it took to reach green. A change that passed on the first attempt and one that passed on the fifteenth deserve different reviews, and only the pipeline knows which is which.

What this changes for the reviewer

Gates do not replace review; they direct it. A reviewer opening a generated change sees, before the diff, the specification it traces to, whether it stayed in scope, whether tests moved together with the code and how hard the agent had to work to get there.

Review stops being a read of everything at the same depth and becomes a proper read of the flagged parts. This is not about trusting the agent less. It is about the pipeline telling a person where the agent was under pressure, because that is where shortcuts live.

Green has to mean something again

When people wrote the code, a green build meant nobody had broken anything. With agents in the loop, by default it only means the agent stopped. The gates are what make those two statements the same again.

None of it has to be elaborate on day one. Separate test lanes and protected configuration close the worst shortcuts and are cheap to set up; the rest can follow as the volume of generated changes grows.

They are also unglamorous to build, which is why they tend to arrive after the first incident rather than before it. In spec-driven development they belong to the guardrails set up in your own CI from the start, alongside size limits on generated changes, protected areas of the codebase, mandatory human review paths and automatic rejection of anything that cannot be traced to a specification.