Acceptance Criteria for AI Tasks: Reviewing What You Cannot Test
In a trade-fair builder's warehouse, someone photographs a bin of lights. The app suggests a name, description, color, category and unit. The suggestion looks good. Only someone who knows the material can tell whether the category is right. The software checks whether the field is filled. It cannot check whether the category matches how your company classifies the item.
For tasks whose results cannot be tested automatically, the standard must therefore be set beforehand: how will you know in everyday use that the solution works correctly?
This is a review bottleneck in miniature. What a machine cannot decide needs criteria that exist before the first run.
Where automated checks end
In the automated DATEV export for MAFU-SHERPA, "right" is a question of format. The debit/credit indicator, decimal comma and column order have to match. DATEV then either accepts the file or it does not. There is no equivalent external check for the warehouse photo. Nobody outside the company knows whether "light" or "stage technology" is the correct category.
Acceptance criteria first
The most important step happens before the AI starts. Write down how you will recognize a finished result. Good criteria are concrete, checkable and include the cases that can go wrong. The test is strict: two people should reach the same judgment independently.
In XPO Inventory for trade-fair builder NEO Expo, those boundaries are built into the data model. A photo may suggest a name, description, color, category and unit. Dimensions and quantity stay manual because those values are too critical for inventory workflows. The form also works without AI.
A requirement such as "the mapping should be correct" becomes checkable only when it is stated precisely: each supplier item maps to exactly one article number, and ambiguous cases are flagged instead of guessed.
You can ask these questions when you speak with a provider:
- Which entries may the AI suggest, and which does a person type?
- What happens in an unclear case: guess, flag or report?
- How does the reviewer see within a minute that something is wrong?
- Who signs off before anything reaches customers?
- What keeps working when the AI fails?
Reference examples as the standard
A rule from our own work: no pilot without real cases from the company. Take between 3 and 10, including a normal case, an exception and one that went wrong. Walking through them together reveals what "right" means in the company.
This collection becomes the standard. "The demo runs" and "the client likes it" do not amount to a review. A proper review needs fixed examples, an error threshold and documented corrections.
Sampling and evals instead of full review
When an agent maps 100 records or generates 50 texts, reviewing every result individually is rarely economical. You check a random selection, estimate the error rate and decide on that basis. If the task repeats, an eval pays off: fixed examples with expected results for every new run. Sampling is the ceiling only where "correct" cannot be derived from raw data, as with tone or judgment calls. Where a second, independent computation path exists, an agent checks all 100 cases instead of ten; exhaustive review was always possible and never economical, and exactly that has changed.
Dry run, rollout, sign-off
Actions with real consequences start with a dry run. It previews what the agent would do without carrying out the action. A small rollout follows, and expansion comes only after that.
Results that require human judgment also need a named sign-off. On the warehouse floor, that means the photo suggestion stays a draft until someone confirms it in the form. One specific person checks it against the criteria and approves it. That person carries the responsibility.
When the effort is worth it
Not every task needs the full apparatus. The higher the cost of error and the more often the task repeats, the more structure pays off. The generative AI profile of the NIST AI risk framework records the same principle in its July 2024 guidance: different applications deserve different levels of oversight and review. A one-off internal note needs no eval. A recurring data transfer into the accounting system does.
How do I know that an AI task is really finished?
By having written it down beforehand. Define which entries the AI may suggest, which ones a person enters and what happens in an unclear case. In XPO Inventory, dimensions and quantity therefore stay manual: the photo suggestion fills the uncritical fields, and a person keeps entering the critical values.
What goes into good acceptance criteria for an AI task?
A concrete, checkable result, the important edge cases and a rule for ambiguity. That includes who signs off on the result and what the sign-off depends on. The test is whether two people can reach the same judgment independently. "Should be correct" does not pass it, because everyone reads something different into it.
How do I review 100 results without reading all of them?
You check a random sample, estimate the error rate and use that to decide about the whole batch. If the task repeats, this becomes an eval with fixed examples that runs on every pass. Actions with an effect outside the company run as a dry run first, until samples stay stable across several runs.