The security review workflow

What the security review workflow is

The pentest workflow is a step-by-step security self-review that your AI tool runs when you ask for one. It follows a fixed written procedure shipped with Paqad, structured around the OWASP Web Security Testing Guide (WSTG), a public catalogue of web security tests. It produces a report with numbered findings and a playbook of commands you can run to confirm them. It helps you harden code before a professional penetration test; it does not replace one.

Most of the work is AI judgment: the AI reads your documentation, tests and code and decides what looks risky. Helper scripts gather some evidence, and the AI interprets it.

Starting a run

Ask your AI tool in plain words, for example "run a pentest" or "security audit". Paqad's router sends the request to the pentest workflow. Progress is saved in .paqad/pentest/runs/<run_id>/progress.json, so an interrupted run can resume: a step already completed with the same inputs is skipped.

The steps, in order

1. Collect context. Load module documentation and tests, then apply the review skills in a fixed order. stride-threat-model runs first and lists possible threats by category (STRIDE: spoofing, tampering, repudiation, information disclosure, denial of service, elevation of privilege). Then come permission-boundary-review, business-logic-abuse-review, input-validation-review, auth-mechanism-review, cryptographic-review, logging-monitoring-review and rate-limiting-review.
2. Run project scripts. Dependency audits, a search for secret-like strings, a listing of routes and documented surfaces, and a database configuration check. Findings from dependencies go through dependency-advisory-triage.
3. Write the local attack playbook. docs/pentest/<timestamp>-local-playbook.md holds a test command for each suspected problem, with the safe response to expect. It is never run automatically.
4. Live validation. If a target address is available, check the running app with read-only requests (see below). Otherwise record the step as blocked.
5. Combine findings. Turn all evidence into structured findings with stable IDs such as PT-0001, and write finding-index.json to the run folder.
6. Write the report. A Markdown report at docs/pentest/<timestamp>.md and a JSON copy at docs/pentest/<timestamp>.json.

The procedure's own rules: never skip a step (record it as blocked instead), never run destructive operations such as writes to a live database, and always write both the report and the JSON file, because retests depend on it.

The helper scripts

The workflow expects these scripts in your project's scripts/ folder. The paqad-ai package ships templates for them.

Script	What it actually does
pentest-audit-dependencies.sh	Runs npm audit, pnpm audit or composer audit where available and saves a package inventory.
pentest-scan-secrets.sh	Searches tracked files for patterns such as private key headers, api_key, token= and password=.
pentest-enumerate-surface.sh	Lists modules and their API, integration and error documentation, exports Laravel routes when artisan is present, and searches for debug and admin surfaces.
pentest-db-readonly.sh	Reads the database connection settings from .env for a follow-up read-only check.
pentest-runtime-checks.sh	Sends GET requests to paths such as /, /admin, /debug and /.env on the target and records status codes.

Each review skill also ships its own small search scripts (for example, looking for risky authentication or cryptography patterns) that the AI runs and interprets.

What live validation sends

The target address comes from your request, from APP_URL in .env, or from the project profile. Besides the status-code script, the procedure asks the AI to check security headers, CORS, error pages that leak details, open redirects, exposed files such as /.git/config, GraphQL introspection, and login rate limiting with about ten quick requests. Only point it at a local or test instance.

What the report contains

• A stack summary and capability profile.
• A WSTG coverage matrix listing each test ID as covered (the check ran) or blocked (it could not run). "Covered" describes the review, not the safety of the app.
• Findings ordered by severity, blocked checks with reasons, remediation priorities, method notes and a link to the playbook.

Each finding in the JSON file records a title, description, impact (high, medium or low), effort, reproduction steps, evidence, affected modules and packages, a confidence value, and whether runtime access or manual follow-up is needed.

Retesting after fixes

Ask for a retest and the request goes to pentest-retest. It reads the JSON file of an earlier report, gathers fresh evidence for each finding ID, and gives each one a status:

• fixed: fresh evidence no longer supports the finding.
• still-open: equivalent evidence still supports it.
• needs-manual-verification: the evidence needed could not be checked, for example because the app was not running.

A checking script fails the retest output if it contains an ID that was not in the original report. New problems go in a separate report. The instructions forbid marking a finding fixed just because evidence is missing.

Limits to keep in mind

• Findings do not block commits, merges or further work. There is no --allow-security-warnings option because nothing needs overriding.
• Results depend on the AI tool and model, the quality of your documentation, and which scripts and tools are installed.
• A report with few findings is not evidence that the app is secure.