TL;DR
When agents write most of the code and AI reviewers comment on every pull request, it is tempting to let the tools decide what merges. Sort each review question by what it needs to know. Let checks and AI reviewers cover what the diff can show, and keep intent, product consequences, architecture fit, risk acceptance and the merge itself with a named person. That person approves on evidence they can state, not on agreement between AI reviews.
One change, three kinds of reviewer
Take an illustrative change, not a real incident: a coding agent opens a pull request that adds an endpoint letting support staff export a customer's orders as a CSV file. The tests pass and an AI reviewer has left three comments. Ask what each kind of reviewer could tell you about it.
The first two lists are about the code. The third is about the organisation it runs in, and none of it is in the diff. Automate the review questions the diff can answer; keep the ones that need knowledge it does not contain.
What practitioners are asking, and what the evidence shows
In August 2026 someone asked on Hacker News whether a human still reviews code. Their team's review was largely automated, and they asked whether others who use AI for most of their code keep a human in the loop for more than spot checks. Replies ranged from fully automated review to a second human on every change. In April, a company rolling out Cursor asked on the Cursor forum how to prevent critical mistakes without blocking people: "Or do you rely on developers and users to manually verify everything every time?" Replies pointed to protected branches and human approval of every pull request.
A large study compares 278,790 inline review conversations across 300 open-source GitHub projects (Zhong et al., March 2026). AI agents made far more code suggestions than human reviewers, but 16.6% of theirs were adopted against 56.5% of the humans', and over half of the unadopted AI suggestions were either incorrect or addressed with a different fix. Human comments also covered understanding, testing and knowledge transfer. The authors conclude that AI agents can scale defect screening while human oversight remains critical. It is observational data from open-source projects, not a measurement of your team.
The vendors say the same about their own reviewers. GitHub's documentation states that Copilot is not guaranteed to spot all problems and advises supplementing its feedback with a human review (Copilot code review). OpenAI's says that review rules guide Codex but do not replace tests, branch protections or required approvals, and that in GitHub it flags only its two highest priority levels by default (Codex in GitHub).
Sort each review question by what it needs to know
Ask of each review question what a reviewer would need to know to answer it. If the answer is in the diff and the repository, a tool can usually establish it. If it is in a product decision, a contract or someone's appetite for risk, it cannot.
Comparison
| Reviewer | What it can establish | What it needs | How it fails |
|---|---|---|---|
| Mechanical checks | Builds, types, tests that exist, lint, size and file limits; scanner hits for known vulnerable dependencies and supported secret patterns | A defined rule and a command that exits non-zero | Silent where no rule, test or scanner pattern exists |
| AI reviewer | Likely bugs, missing edge cases, inconsistency with nearby code, unclear naming, suspicious patterns | The diff plus whatever repository context it can read | Plausible but wrong comments; nothing to say about what it was never told |
| Human approver | Whether the change is the right one, what it costs to own, which risk is acceptable, whether to merge | Product, architecture, operational and organisational knowledge | Rubber-stamping when the other two look green |
Much of the gap is context. Put the ticket, design notes and agreed constraints where an AI reviewer can read them, such as review rules in AGENTS.md for Codex or .github/copilot-instructions.md for Copilot. That moves some questions from the third row to the second. It does not move the decisions.
The questions only the approver can answer
Some review questions are judgements an AI reviewer can raise but should not settle, because settling them commits the organisation:
- Intent. Does the change do what was asked, and nothing that was not? The agent's description of its own work is not the ticket.
- Product consequences. What users see and can now do.
- Data and exposure. Who can reach which data through this path, and whether that is allowed.
- Architecture fit. Whether this belongs here and what it adds to maintain.
- Irreversible steps. Migrations, deletions, changed public contracts and anything a revert would not undo.
- Risk acceptance. Waiving a failing check, accepting a known limitation, shipping behind a flag instead of fixing first.
- The merge. Someone has to be the person who said yes.
How much of this a change needs depends on where it lands. Code owners for sensitive directories make that explicit: a change there needs their approval whatever wrote it.
Keep author, AI reviewer and approver separate
When one person asks an agent for a change, has another agent review it and then approves it, the only human judgement in the chain is the author's. Separate the roles:
- The author owns the change. Whoever asked the agent for it can explain it, and resolves or dismisses each AI review comment with a reason.
- The AI reviewer is a screen, not a vote. Its comments are findings to resolve, and its approval stays out of the required count.
- The approver is someone else. They answer the questions above and accept the risk by name.
GitHub can enforce part of this. If you ask Copilot's coding agent to create a pull request, your approval does not count toward required approvals, so someone else must approve it (reviewing Copilot pull requests). Branch protection can require that the most recent push be approved by someone other than the person who pushed it, and can dismiss approvals when new commits change the diff (protected branches). The article on keeping up with code review when agents open pull requests faster than people can review them covers the queue side of the same problem.
What should convince the approver, even when every AI review agrees
Two AI reviewers that agree are weaker evidence than they look: they read the same diff with the same missing context, so they tend to miss the same things. The approver needs evidence they can state in their own words:
Decision guide
- Approve when you can say what the change is for and confirm it matches the request, the required checks passed on this commit (a signal, not proof the intended behaviour is covered), every AI finding is fixed or dismissed with a reason, each question on the list above that applies has an answer from someone entitled to give it, and there is a way back if it goes wrong.
- Send back when any of those is missing, however clean the AI reviews look.
- Escalate when it touches a decision that is not yours, such as data exposure.
If you cannot explain the change without the agent's summary open, you are not ready to approve it; the article on using AI coding agents without losing understanding of the codebase lists what to be able to explain.
Paqad prepares a change for that approval rather than giving it. A deterministic classifier sizes each change into a fast, graduated or full lane, and a file in a module marked highly sensitive forces the full lane. Review starts from a machine-built digest of what scripts already found, including blind spots no machine covers, and each recorded finding must cite a digest row or be marked as judgement only. The change ends with a receipt: Safe to merge, Needs your attention or Inconclusive. Safe to merge attests that the gates passed, not that the code is correct; the recorded review is still a model's judgement, and the receipt does not authorise a merge. Paqad's introduction has a table of what a script decides, what the AI decides and what stays with you.
Frequently Asked Questions
Which parts of AI-generated code review should stay with a human?
The decisions, not the reading. A person should own intent, product consequences, data exposure, architecture fit, irreversible steps, risk acceptance and the merge. Write that list into the pull request template so the approver ticks named questions rather than a single box.
What can automated checks and an AI reviewer establish from the diff?
Anything a rule can test and much of what a careful reader would notice in the code itself. A coverage report can show a branch no test executes, but executing a line is not checking it: whether the tests assert the behaviour the change was meant to have is a judgement, so compare what the change adds with what its tests assert.
Which product, architecture and risk decisions require knowledge those checks may lack?
Any decision that depends on facts outside the repository: a customer commitment, a regulation, an incident nobody wrote up, the team's capacity to operate a new component. Writing those facts down where an AI reviewer can read them helps it raise the question, but someone still has to answer it.
How should author, AI reviewer and human approver responsibilities be separated?
Make the separation visible in the record: the author's own comments resolving each AI finding, and a different person's approval on the latest commit. If a team is too small for a second approver, a delay before merging and a written self-review are weaker substitutes; say which one you use.
What would convince the accountable reviewer that a change is acceptable, including when the AI reviews agree?
Evidence that does not come from the same source as the code. A check the author did not write, a person who knows the product, and a rollback that has been thought through count for more than a third AI review saying the same as the first two.