Software Quality: What "All Checks Passed" Is Worth

At a project meeting, your supplier reports that the change is ready and all checks passed. You want to know whether your team can use it.

A green tick beside a check means it found no fault. On our own website, a passed check missed a fault.

The button looked right, but its animation was broken

On 1 September 2026 we changed a button on our website. We then checked its font size, colour and spacing in the browser. Everything was correct. The button was supposed to fade into view as you scrolled. To make it easier to measure, we had switched off that fade for the check. Our change, however, had broken precisely that behaviour. Under the conditions of our check, the fault was therefore invisible.

A second AI agent subsequently found the fault by reading the finished code. Our measurement confirmed the button's appearance but left its behaviour during scrolling unchecked.

Ask to see the workflow as your team will use it.

For our button, that would have included scrolling through the page and watching it fade into view.

A check must be able to detect the fault it is meant to catch

On the same day we found another gap. We had written 13 automated checks for our own tool that combines completed software changes. They were meant to protect an important sequence: first combine the changes, then check the result. In the wrong order, a change would have gone through that was never checked.

The reviewing agent deliberately reversed that sequence. The checks still reported success. They could not detect the very fault they were meant to prevent. Only after we added the missing check did they detect the wrong sequence.

Since those two incidents, we recreate the fault a check is meant to catch and see whether it detects it. We also make sure that preparing a check does not disable the behaviour being checked.

What to ask at your next project meeting

You do not need to read software tests to do this. Your supplier should be able to use an example to explain what is covered and where a gap remains.

• "Show me the workflow as our team will use it." Look for anything that is simplified or switched off.
• "Which fault would this check detect?" The answer should name a specific fault. For our button, it would have been the broken fade.
• "Have you tried whether the check detects that fault?" Your supplier can demonstrate this in a separate test environment.
• "What has not been checked yet?" The answer helps you judge whether you want to start using the change.

We would hold back approval if an important workflow came with only an "all checks passed" report and nobody could explain which fault those checks would detect. A list of passing checks alone is not enough for that decision.

Who checks and what they check belong together

In both cases, a second AI agent found the gap. In the first, it read the code; in the second, it recreated the fault. A second reviewer for AI work can therefore also help spot an unsuitable check.

How thoroughly a change should be checked depends on the risk. Ask to see whether the check covers the workflow your business depends on and what uncertainty remains before approval.