How We Check AI-Written Code Before It Goes Live

AI agents write our code, for client projects as well as for our own tools. A second agent reads every line before a change is merged, and the change goes live only when a person has approved exactly that version. This is the path a change takes with us, and how we check the reviewer itself.

Every task starts with a written brief

A change starts with us as a task with a short brief: what to do, why, and how to tell that it is done. The agent that implements it works in its own copy of the code, never directly on the shared version. Several agents can therefore work at the same time without overwriting each other.

Before anyone reviews the change, the agent runs the project's checks: formatting, type checking and the affected tests. Type checking verifies that every value in the code has the expected shape. Another tool looks for code that the change leaves unused; as long as it finds any, the change goes no further. Why tests and types are the foundation for AI-written code is explained in AI Software Development: Build Faster Without Losing Control.

An independent agent reviews

Then a second agent reads the finished change. It runs in a fresh session and may not change anything. Normally it comes from the other provider: if Claude from Anthropic wrote the change, Codex from OpenAI reviews it, and the other way round. If the other provider is unavailable, a fresh agent from the same provider reviews, and the review report records the substitution. The reviewer sees the brief and the change, but not the conversation in which the change was made. Why an author is a poor reviewer of their own work is the subject of the four-eyes principle for AI agents.

The reviewer reports only two kinds of findings. A defect needs an exact place in the code and a concrete trigger: which input leads to which wrong result. While a defect is open, the change is not merged. A decision is a trade-off a person has to make, such as whether a special case is deliberately left out. It does not hold up the change and later appears on the overview a person uses to approve the release. The reviewer does not report style or naming; a separate review every two weeks covers those.

Two rules make the review stricter. If a change touches sign-in, permissions, payments, saving data or a database migration, a missing test for the visible behavior counts as a defect in itself. And before the reviewer declares a change free of defects, it must also read the code that calls the changed parts, and the related tests.

Two review rounds, then a new plan

When one agent writes and another reviews, the review needs a limit. A person waves off a minor point in seconds. An agent fixes it, submits a new version and so triggers the next review round. Without a limit, author and reviewer can go back and forth indefinitely.

That is why every change gets one full review and at most one follow-up review, which reads only the fixes made since the first review. If the follow-up still finds a defect, the change is narrowed and the part without defects is reviewed afresh. Or it waits until a person has decided on the approach. We assume that a defect which survives two fixes lies in the approach rather than in a single line.

What gets merged must work with the current version

A reviewed change goes into a queue. There it is combined with the latest shared version, and the checks run again on exactly that result. Two changes can each be free of defects and still break something together. That is why the checks run on the combination before the shared version changes.

Only an approved version goes live

The shared version is not yet the running application. With us, no change waits for a person before it is merged. In return, nothing goes live before a person has approved exactly that version. The approval overview lists the merged changes, the decisions from the reviews and, for visible changes, a short video showing the new feature on a computer and on a phone.

So the machine reads every line, and a person stays responsible for what goes live. Why the depth of review should follow the risk is explained in How Much Review Does AI Code Need?

How we check the reviewer

Our process has one gap: we do not yet know exactly how good our reviewer is. That is why we introduced three changes on 20 October 2026.

• We record how each finding ended: fixed, disproved or still open. That shows how often the reviewer raises a false alarm.
• Defects that got through despite review become a test set for the reviewer. Before that, our tests of the code reviewer mainly checked whether its answer had the required form. When we change the reviewer or its model, we run it against this set again.
• The reviewer names the type of each defect and lists every place it occurs. If the author fixes only the one line named while the same defect sits in two other places, the follow-up review fails, and the change has to be narrowed or set aside.

The second point matters most to us. As long as we do not know what the reviewer misses, we do not know whether a review without findings means a good change or a weak reviewer.

More review does not automatically mean more quality. From 11 to 13 October 2026, our agents spent days on tools meant to safeguard our checks while finished changes waited to be merged. Since then, speed has been an explicit goal for us. That is why we first measure what our reviewer misses, and only then decide whether more review effort is justified.

What you can ask your provider

If AI agents write software for you, a few questions are enough to judge the quality assurance. How many review rounds does a change get at most, and what happens after that? How does your provider know which defects its reviewer misses? And who approves which exact version before it goes live for you? We do not yet have a complete answer to the second question ourselves; the test set of real defects is meant to provide it.

In the development stage we build software that you own afterwards, and every change to it takes the path described here. Discuss development