TL;DR
Coding agents report a task as done when the work looks done, and a vague task lets a six-line placeholder pass. Define done as observable behaviour: the main path, failure and empty states, and explicit non-goals, written so a placeholder would fail them. Trace each criterion to a change and a check you can read, let a hook or a reviewer close the task instead of the agent, and record who waived any criterion. You end up with a completion claim you can verify rather than trust.
A heading, a paragraph and a ticked box
A Spec Kit user posted in the project's GitHub discussions in April 2026 with a short complaint, written in English and Chinese. Task T017 in their tasks.md said to implement the frontend item list page and create ItemListView.vue. Codex created a six-line file holding a heading and a paragraph, then marked the task [X]. The task never said the page needed list rendering, filtering or pagination, so the agent delivered what the user called "the minimum possible interpretation". The one reply suggested being more specific in the project's constitution and specification.
A Cursor forum thread from August 2026 asked whether a coding agent should be allowed to mark its own task complete. Its author listed what a confident final message can hide, such as a narrow test passing while the required suite never ran, or a commit that exists only locally, and argued that an agent may submit a completion claim but should not be the one that accepts it.
The first report is about what done means; the second is about who gets to say it. A product owner has to settle both.
Why the shortcut counted as finished
Nothing in the Spec Kit case required the agent to lie. Spec Kit's implement command tells the agent to mark each completed task off as [X] (implement template), and "create ItemListView.vue" was, literally, done. Claude Code's best-practices guide names the mechanism: Claude stops when the work looks done, and without a check it can run, looking done is the only signal available (best practices). The task wording set the bar, and the bar was a file existing.
The checks need protecting as well. METR reported in June 2025 that recent frontier models, given tasks with automated scoring, sometimes raised their score by modifying the tests or scoring code, or by finding the answer used to check their work, instead of solving the task (METR). Those were research tasks, not product features, but the lesson carries over: the criterion and the check that proves it should not be editable by the agent being judged.
Write criteria that a placeholder would fail
A practical test for any acceptance criterion: could a stub that only resembles the requested screen or endpoint satisfy it? If it could, the criterion describes the shape of the work, not its outcome. Criteria that resist placeholders tend to share these properties:
- They name real data and a visible result. "With 25 items stored, newest first, page 2 shows the 5 oldest", not "pagination works". An order the check depends on is stated, not assumed.
- They cover the failure path. What the user sees when a request fails, times out or is refused.
- They cover empty and boundary states. No items, one item, a filter that matches nothing, the last page.
- They say who may do it. Which roles can see or change the thing, and what everyone else gets.
- They survive a fresh read. Anything that should persist is checked by changing the input, saving it and reading it back in a new page load or session. A plain reload can still show a hard-coded value; changed data read back from storage cannot be faked that way.
- They list non-goals. What the feature deliberately leaves out, so the agent neither skips needed work as out of scope nor adds work nobody asked for.
Here is T017 rewritten in that style. The criteria are illustrative, written for this article, not taken from the thread:
The six-line page fails AC-1 the moment anyone opens it, because it renders nothing from storage. That is the point of writing criteria against data: a shortcut becomes obvious to the first person who checks.
Trace each criterion to a change and a check
A list of criteria still lets an agent declare victory on the ones it got to. Give every criterion its own row, with the change that implements it and the check whose result proves it:
Comparison
| Criterion | Change expected | Check | Evidence kept |
|---|---|---|---|
| AC-2 page 2 | List query accepts page and size; page control in the view | Test that stores 25 items and opens page 2 | Test name and the output of the run |
| AC-4 empty state | Empty-state branch in the list view | Component test with no stored items | Test output |
| AC-5 request fails | Error branch and retry action | Test with the items request forced to fail | Test output |
| Non-goal: user sorting | None | Review that the diff adds no sort controls | Reviewer's note on the pull request |
Three rules make the table useful. A criterion with no check is unverified, not passed. A test that mentions a criterion's ID is a pointer, not proof; someone still has to confirm that it asserts the behaviour. And if no change in the diff serves a criterion, treat it as missing, whatever the summary says, unless existing code already meets it and a check run on this change confirms that.
Spec Kit itself includes a step built on the same idea. Its converge command reads the spec, plan and tasks, includes every task regardless of checkbox state because "completion claims are not evidence", and classifies each gap as missing, partial, contradicts or unrequested before appending the remaining work as new tasks (converge template). It is still a model reading code, so its findings need review too, but its four categories make a good vocabulary for your own review. The article on stopping a coding agent changing unrelated code covers the unrequested category in more depth.
Decide who can waive a criterion
Sometimes a criterion proves wrong, too expensive or blocked by something outside the feature. The agent can discover that; it should not settle it.
Decision guide
- If a criterion cannot be met as written, the agent stops and reports it with the evidence, and the task stays open.
- If meeting it differently would change what a user sees or can do, the product owner decides.
- If the proposal is to drop or defer a criterion, the product owner decides, and a deferred criterion becomes its own task.
- If the change is technical and every criterion's observable result stays the same, the developer reviewing the work can approve it.
- If the agent says existing code already meets a criterion, it names the check that proves it, and that check runs.
Each waiver or change keeps a few facts next to the specification: the criterion ID, the old text, the new text or "deferred", who decided, when, why, and the evidence that prompted it. Keep the original wording as history rather than overwriting it, because an acceptance given against the old criteria does not cover the new ones. The article on updating a specification halfway through AI-assisted development applies the same rule to a changed requirement.
Let something other than the agent close the task
Hooks can turn "I am done" into a request that a check has to grant. In Claude Code, a TaskCompleted hook runs when a task is being marked as completed; exit code 2 keeps it open and sends your message back to the model, and the documentation suggests it for completion criteria such as passing tests or lint checks (Claude Code hooks). A Stop hook can refuse the end of a turn, up to a cap of eight consecutive continuations. Codex's Stop hook works in a similar way: a block makes Codex continue, using your reason as the next prompt (Codex hooks). Claude Code's /goal has a separate small model judge after each turn whether your stated condition holds, so the agent doing the work is not the one grading it.
Know the limits. A hook runs the checks you wrote, so it is only as good as the criteria table behind it. A model-evaluated condition is still a judgment. None of these decides whether a waiver was acceptable. And tests and scoring scripts belong under review, so the agent cannot quietly edit what judges it.
Paqad builds the definition step into its workflow when its optional specification pipeline is turned on; it is off by default. The pipeline writes acceptance criteria in given, when, then form with a proof method for each, and on the graduated and full lanes the freeze refuses a specification while any criterion lacks one. A specification that changes after the freeze raises a decision for a person to answer. The sign-off name is recorded rather than authenticated, and a frozen specification does not prove the requirements are complete. Its compliance commands match each criterion ID to the tests that mention it, and Paqad's compliance command reference explains why that shows which requirements have tests, not whether the tests are right.
Frequently Asked Questions
How do I define done so an AI agent cannot mistake a partial feature for a finished one?
Use two layers. A short project-wide definition of done covers what applies to every task: the required test suite ran, no placeholder or TODO code remains in shipped paths, and documentation is updated. Feature criteria then say what this particular feature must do. Put both where the agent reads them before it starts, in the task itself, not only in a separate document.
Which observable behaviours and failure cases belong in the acceptance criteria?
Walk the user's path through the feature: how they arrive, the main action, and every way it can end. Each outcome a user would report as a bug if it went wrong deserves a criterion. Speed or accessibility belong there only when stated as something measurable, such as a time limit or a named standard.
How do I separate a real user outcome from placeholders that only resemble the requested screen or API?
Review the diff for the usual signs: hard-coded arrays, mock data in production code, functions that return empty results, and TODO comments. Then try the feature yourself against stored data. It also helps to ask the agent to list which criteria it did not meet, which is easier to answer honestly than "is it done?".
How can every acceptance criterion be connected to an implementation change and a meaningful check?
Give each criterion a stable ID and use it in the test name or an annotation, so the mapping can be searched. Paqad's compliance check, for example, reads @obligation annotations, but only in JavaScript and TypeScript tests under tests/. To confirm a test is meaningful, break the behaviour on purpose and make sure the test fails.
Who can approve an unmet criterion or a changed requirement, and what evidence must remain in the record?
The product owner, for anything that changes what users get. Keep the record in the repository beside the specification and in the same pull request as the code, so the reviewer sees every waiver before merging. A waiver without a name, a reason and the evidence that prompted it is an unrecorded scope change.