compliance: check tests against a spec

paqad-ai compliance is a group of commands that compare a written feature specification with your tests. A specification (spec) here is a Markdown file that lists what a feature must do. Paqad turns each requirement in it into an obligation, a single numbered promise such as "AC-3: the user sees an error when the card is declined", and then looks through your test files for evidence that each obligation is tested.

Every step is a script. The commands match text in your spec and tests; they do not ask an AI model to judge anything. That makes the results repeatable, and it also sets their limit: a test that mentions an obligation counts as evidence, whether or not the test really proves the behaviour. Deciding that is still a review job for a person or the AI.

The usual order

1. compliance review: check the spec itself for common writing problems.
2. compliance extract: turn the spec into an obligation index (a JSON list of obligations).
3. compliance check or compliance report: see which obligations your tests cover.
4. compliance skeleton: create failing test stubs for the gaps, then fill them in.

All subcommands accept --project-root <path> (default: the current folder). Results are stored per spec under .paqad/compliance/<spec-slug>/, where the slug is the spec's file name in lower case, so docs/features/checkout-flow.md becomes checkout-flow.

Review the spec

paqad-ai compliance review docs/features/checkout-flow.md [--json]

Runs a set of fixed checks for spec problems, such as a rule with no stated behaviour for the "does not hold" case, a formula that can divide by zero, conflicting goals, contradictions and broken references. The report is saved to .paqad/compliance/<spec-slug>/spec-review.json. On the next run, earlier findings are carried forward and marked resolved when fixed. The spec must be inside the project folder. Add --json for the full report instead of the summary.

Extract obligations

paqad-ai compliance extract --spec docs/features/checkout-flow.md [--index-path <path>]

--spec is required. Paqad picks up obligations from three places:

• Requirement tables: tables whose header has an ID column, a requirement or condition column, and an expected result or pass criteria column. Each row becomes one obligation.
• Lines that start with an ID in the forms FR-1, NFR-2, AC-3 or EC-4 (also as a heading or bullet). The ID becomes the obligation ID.
• Numbered list items under an "Acceptance criteria" or "Definition of done" heading. These have no ID of their own, so they get a generated one that starts with GEN-, based on the item's position in its section, so it stays stable when unrelated sections change.

Each obligation is sorted into a category from its heading: acceptance criteria, edge cases, functional requirements, non-functional requirements, or unclassified. The index is written to .paqad/compliance/<spec-slug>/obligations.json unless you pass --index-path.

Check test coverage

paqad-ai compliance check --spec docs/features/checkout-flow.md

Scans test files that match tests/**/*.{test,spec}.{ts,tsx,js,jsx} and gives each obligation one of four states:

State	What Paqad found
covered	A test file carries an @obligation <ID> annotation, or a test name in it(...), test(...) or describe(...) contains the ID.
partial	The ID appears somewhere in a test file, but not as an annotation or in a test name.
uncovered	No mention of the ID in any test file.
indeterminate	A generated GEN- obligation with no mention at all. It cannot be matched by ID, so it is left out of the ratio.

The compliance ratio is covered divided by all obligations that are not indeterminate. The command prints JSON, saves it to .paqad/compliance/<spec-slug>/report.json, and exits with 1 if anything is uncovered.

To use it as a pass or fail gate in CI, add --gate. It fails when the ratio is below --min-ratio (default 0.9) or when more than --max-uncovered-critical (default 0) acceptance or edge-case obligations are uncovered.

Test files outside tests/, or in languages other than JavaScript and TypeScript, are not scanned by this command in this release.

Read a plain report

paqad-ai compliance report --spec docs/features/checkout-flow.md

Prints the same results as readable text: totals per state, the ratio as a percentage, and the uncovered and partial obligations with their descriptions. It uses the same exit rule as check.

Generate test skeletons

paqad-ai compliance skeleton --spec docs/features/checkout-flow.md [--out tests/compliance-skeletons] [--all]

Writes one failing Vitest test file per uncovered or partial obligation (every obligation with --all), named after the obligation ID. Each file already contains the @obligation annotation, so once you write a real test in it, check sees it as covered. The default folder is tests/compliance-skeletons. The generated files use Vitest syntax only.

Validate the stored files

paqad-ai compliance doctor --spec docs/features/checkout-flow.md

Checks the obligation index, the spec review and the boundary report. It reports errors (an unsupported schema version, duplicate obligation IDs) and warnings (a missing file, a spec edited after its last review, unhandled boundary states). It prints {"ok": ..., "issues": [...]} and exits with 1 only when there is an error.

Check shared state boundaries

paqad-ai compliance boundary [--generate] [--json]

For TypeScript projects. It scans src/**/*.ts for comments like // @boundary OrderStatus producer:checkout consumer:billing states:paid,refunded placed above a type, and reports states that consuming features do not handle. When no states are listed, they are read from the enum or union type below the comment. The report goes to .paqad/compliance/boundary-report.json; --generate also writes test stubs under .paqad/compliance/boundary-tests/.

List recurring defect patterns

paqad-ai compliance patterns [--min-frequency 1] [--prune [days]] [--export <path> --format json|markdown]

Shows spec defects that keep coming back, stored in .paqad/defect-patterns/. --prune removes entries older than the given number of days (365 if you give none). --export writes them to a file. This store is separate from the cross-project library that paqad-ai patterns manages.