Structured test output

What structured test output is

When Paqad runs your tests, it turns the runner's output into one small, fixed-shape record: how many tests passed, failed, were skipped or errored, and the details of each failure. This is called structured test output. It lets Paqad's checks decide pass or fail from real test results, whatever test runner you use, and keeps a long test log from filling the AI's context.

The reading is done by a script, not by the AI.

Why this matters to you

Raw test output mixes the useful part (which test failed and why) with colour codes, progress spinners, start-up logs and long lists of passing tests. A plain "the command exited with 0" tells you nothing about which tests ran. The structured record keeps the failures and the counts, drops the noise, and says honestly when it could not read the output.

Where the results come from

The command paqad-ai checks run runs the format, test and build commands mapped for your project and reads the test output. It saves a report in the current change's evidence folder, or in .paqad/checks/last-run.json when no change is active. The feature-development workflow tells the AI to run it during the checks stage. On Claude Code and Codex CLI the end-of-turn hook then reads the saved report, so a change whose checks never ran is noticed; on other tools nothing checks this automatically.

For this to work, your stack pack declares its test runners in pack.yaml under test_runners: the runner's name, the output format, any extra command-line options that make the runner print that format, and whether the result goes to the screen or to a file. During onboarding Paqad adds those options to the test command in your project profile. For a React project using Vitest, for example, the test command becomes pnpm test -- --reporter=tap.

Supported formats

Format	Used by (in the shipped packs)	How it is read
tap	Vitest	The TAP text format printed to the screen
jest-json	Jest (--json)	JSON printed to the screen
junit-xml	Pest and PHPUnit, JUnit for Spring Boot, Playwright	One or more XML result files; several files are merged
pytest-json	pytest with --json-report	A JSON result file. This needs the pytest-json-report plugin installed in your project.
go-json	go test -json	Go's JSON event stream
rspec-json	RSpec (--format json)	JSON printed to the screen
none	Angular, Flutter and Rust tests	No structured reading; Paqad reads the plain text as best it can

In 1.91.1 the dotnet, flask, kotlin-android, nestjs and nextjs packs declare no test runner. For those stacks paqad-ai doctor warns "Structured test output ready", and test results are judged from plain text and exit codes.

Three ways reading can end

Every record says how its output was read, in parse_metadata.parse_strategy:

• structured: the output was read in its declared format.
• plain-text-fallback: the format was none, or the structured reading failed, so Paqad read the plain text instead. A warning explains why.
• degraded: no output was found (for example, the result file was never written), or even the plain-text reading failed.

What the checks do with the results

Two of Paqad's verification checks, code-tests-lint and behavioral-correctness, use structured results when they exist:

• If code changed and no test evidence covers the changed files or modules, code-tests-lint fails and behavioral-correctness is marked inconclusive.
• If any result was degraded, the check is inconclusive: Paqad does not report a pass it cannot prove.
• If any result reports a failed or errored test, the check fails.
• Otherwise it passes and reports how many checks passed out of the total.

Projects without declared test runners still work: the checks fall back to the simple passed or failed result of the commands.

What a record contains

• summary: total, passed, failed, skipped and errored counts, duration, time, and the runner's ID.
• failures and errors: kept apart, so a failed assertion is not confused with a crash or a runner error. Each has the test ID, message, stack trace, file and line where known, and a category (assertion, error, timeout or unknown).
• warnings: notices that did not stop the reading.
• parse_metadata: how the output was read, its size before and after compacting, and any reading warnings.
• evidence_scope: which changed files or modules the result covers, used to judge whether the tests relate to the change.

How doctor tests the readers

paqad-ai doctor feeds a small built-in sample through the reader for each test runner your stack declares. If a reader cannot handle its own sample, doctor fails "Structured test output ready". This tests Paqad's readers, not your tests. See Diagnosing problems with doctor.