paqad

Try Paqad

How do I get an AI coding agent to follow a real test-first workflow?

  • Written by Haider Lasani
  • Published
  • 10 min read
Mira holds a clipboard with a printed checklist steady while Noor, standing close beside her, writes on it with a pen.

TL;DR

Telling a coding agent to work test-first does not reliably change what it does, because a rule in its instructions is advisory and a workflow template can treat tests as optional. Make each step something you can see instead: expected behaviour written down before any implementation, one failing test whose failure you have read, the tests committed and locked while the code is written, and a record of both runs. Then a passing suite tells you something about the requirement, not only that the agent stopped.

What counts as test-first when an agent writes the code

Test-first is a sequence, not a preference. In Kent Beck's description of the method (Canon TDD, December 2023) you list the behaviours the change needs, turn exactly one of them into a runnable test, watch it fail, make it pass, optionally tidy up, and repeat. With an agent, each of those states has to be visible to you afterwards, because the agent's summary of what it did is not the same as a record of it. Three facts should be checkable for any change you accept: the test existed and failed before the production code changed, it failed because the behaviour was missing, and it was not altered on the way to passing.

Two public questions show what happens when that sequence lives only in writing. In March 2026 a developer asked in the GitHub spec-kit discussions how the test-first flow is supposed to work in practice: they had followed the full specify, plan, tasks and implement workflow and got an implementation with no tests at all. Spec-kit's own documentation describes test-first as a core principle, but its task-generation prompt says tests are optional unless the specification or the user explicitly asks for them; that line was still in the template on 1 October 2026. A project collaborator replied that the documentation probably needed updating to say so.

In January 2026 a Cursor user asked on the Cursor forum what they were doing wrong: the agent never did test-driven development and rarely checked its work between steps, although an always-applied rule told it to write tests first. A Cursor staff member described a known instability with always-applied rules and suggested mentioning the rule explicitly in each chat and spelling the steps out, including running the tests and expecting them to fail.

Why a rule in the instructions file is not enough

Both cases have the same shape: test-first was stated, and nothing in the agent's path depended on it. Claude Code's documentation puts the distinction plainly: instructions files are advisory, while hooks are deterministic (Claude Code best practices). An instruction competes with everything else in the context, including a template that says the opposite. The article on a coding agent ignoring project rules covers how to tell whether a rule was loaded at all; here the more useful question is which parts of test-first a check can see.

Most of it can be seen. Whether a test file changed before the production file, whether a run failed, and whether the test diff changed after that run are all facts in version control and test output. Whether the test expresses the requirement is a judgement, and it stays with you. The workflow below puts each checkable fact where you can check it.

Write the expected behaviour before there is code to copy

An agent that writes the implementation first and the tests second can end up with tests that describe what the code does, which is the opposite of what you need. Keep the expected behaviour ahead of the code:

  1. Write the test list yourself, or approve the agent's. Each line is one behaviour in the requirement's terms, with a concrete input and the expected result, plus any existing behaviour that must not break.
  2. Give values, not descriptions. Claude Code's documentation suggests example test cases with their expected results rather than a description of the function. Values from the requirement or from someone who knows the domain cannot be borrowed from an implementation.
  3. Write tests in a session that has no implementation to look at. The same documentation suggests having one session write tests and another write code to pass them. If existing code is nearby, point the test session at the interface it must call, not at the code behind it.
  4. Turn one item into a test at a time. Beck lists writing every test up front and then making them pass as a mistake, because the first test you make pass often changes what the others should say.
  5. If your workflow generates tasks, ask for tests in the specification. In spec-kit's current template, that is what makes test tasks appear.

Read the red run before any production code is written

A failing test proves nothing until you know why it failed. The agent should run only the new test, then show you the command and the output, and you should read the failure itself rather than the agent's description of it.

  • The test ran. The runner names it in the output and the count of tests collected went up. A test that was never collected cannot fail.
  • It failed on an assertion. The failure compares an expected value with an actual one, and the expected value is the one from your test list.
  • It did not fail for a side reason. An import error, a missing module, a syntax error, a missing fixture or environment variable, a timeout or a different test failing all mean the behaviour was never tested.
  • The fix is not already there. Production files are unchanged at this point, apart from the stub described below; if the behaviour under test is already implemented, the red run tells you nothing.

If the code under test does not exist yet, a bare function signature or stub that returns a neutral value turns a missing-module error into an assertion failure. That is the only production change worth allowing before red: scaffolding the test can call, not the behaviour it checks, small enough to read in one glance.

Illustration of Ivo alone on a high stool at a narrow wooden counter, hand at his chin as he looks at his laptop, his pen pressed to an open notebook beside it.
A red run is only useful once someone has read why it failed.

Lock the tests while the implementation is written

Once the red test is right, commit it on its own and note the commit. From then on the requirement lives in that commit, and the implementation phase should not be able to change it quietly. Beck's list of mistakes includes deleting assertions so the test only pretends to pass; an agent pushed to finish can make the same move in quieter forms.

Make the lock real where your host allows it:

  • Claude Code. A deny rule such as Edit(tests/**) in the project's permission settings applies to the built-in file tools, to file commands Claude Code recognises in Bash and to shell redirections, but not to a script that opens files itself; the documentation points to the sandbox for OS-level enforcement (permissions).
  • Codex and Cursor. Both run a pre-tool hook that can deny a file edit: Codex's PreToolUse matches apply_patch (Codex hooks), and Cursor's preToolUse matches Write (Cursor hooks). Test what happens when your hook script errors, because Cursor documents that a failed hook lets the action proceed by default.

Whatever the host, check afterwards. git diff <red-commit> -- tests/ should be empty when the implementation is done. If it is not, look for changed expected values, removed or loosened assertions, new skip or expected-failure markers, regenerated snapshots, and mocks that replace the unit under test. The article on telling whether AI-generated tests catch real bugs or just raise coverage covers how to judge the assertions that remain.

Sometimes the test really was wrong. Then the requirement changes, and that is your decision: stop the implementation, change the test list, write the corrected test, read its red run again and commit it before the code continues.

Illustration of Noor, Mira and Ivo at a kitchen island, Noor holding her laptop toward the others while Mira points at the screen and Ivo, a mug at his lips, leans in to look.
The test diff since the red commit should be empty; when it is not, the change needs a reason.

The record a reviewer should ask for

A reviewer accepting the change should not have to take the agent's word for any step. Ask for a short record in the pull request. The example below is an illustrative format, not output from any tool:

The revert check is the cheapest way to show the test exercises the change: if the new test still passes with the production change removed, it was not testing that change. Read both diffs as well. The test diff should read like the requirement, and the production diff should do no more than the tests ask.

Paqad covers the passing end of this record. Its checks stage runs your project's own format, test and build commands, parses the test output into per-test results with failed assertions kept apart from crashes and runner errors, and saves the report with the change; on Claude Code and Codex CLI the end-of-turn verdict reads that report rather than the chat, and a change with no report reads as inconclusive, not passed. On the graduated and full lanes, each acceptance criterion in a frozen specification needs a proof method, and a specification that changes mid-build raises a decision for you. It does not impose a test-first order on your project, fix a broken test environment, or show that a passing suite has meaningful assertions or complete coverage. Paqad's structured test output page explains what the checks read and when they report inconclusive instead of a pass.

Frequently Asked Questions

Make the phases separate requests that you start, rather than one instruction the agent has to remember: write the test list, write one test and show its red run, commit, then implement. Each phase ends with something you can look at, and the next one does not start until you have.

Write the expected values by working them out from the requirement, never by running code and copying what it returns. Beck calls copying computed values into expected values a mistake because it removes the double check that gives the test its value.

Run the red commit yourself once, outside the agent's session, or let continuous integration run it. A failing pipeline on a tests-only commit is a record the agent did not write, and it confirms the failure is not specific to the agent's environment.

Make test changes need an owner's approval. A code-owner rule on the test directories, with code-owner review required on the branch, means a change to tests needs approval from a named owner of those tests. That is not a separate approval by itself: one reviewer who owns both tests and code can approve both in one review. If expected values must be approved apart from the code, keep test owners distinct from implementers, or merge the red commit through its own pull request first.

Runs captured by a command or by continuous integration, not pasted summaries from the agent. Reject the change if the red commit already contains the behaviour under test rather than a stub, if the green run used a different command from the red one, or if the test diff since red has no stated reason.