TL;DR
Ask an AI assistant why something is failing and it will often answer with the most plausible story, stated as fact. A diagnosis you can act on needs a reproduction, observations kept apart from inferences, and at least two competing explanations, each with an experiment that could rule it out, before any code is edited. Call it the root cause when removing it makes the fault disappear, restoring it brings the fault back, and a regression test fails without the fix.
An answer that arrived before the evidence
"Based on nothing. I made it up to explain why you said it wasn't retrying." According to a Claude Code bug report filed in May 2026, that was the assistant's reply when its user asked what a claim about a stalled retry queue rested on. The report lists other moments from the same long session, in which production logs were being pasted in: code behaviour described before the code was read, an assurance that an item was still queued when it had been dropped, and a confident root cause for a failure after a restart. The real cause surfaced only after pushback: an existing unit test had checked for the wrong behaviour since the code was written. The user's project memory already said never to reason from assumptions.
Others describe it from the receiving end. A Codex user on the OpenAI Developer Community described asking it to analyse some bugs in June 2026: it quickly claimed to have found the issue, the approved fix had no effect, and it took repeated requests before it would generate a log. That month a Cursor forum thread asked how to stop an AI's guess from quietly becoming "fact", once unchecked guesses sat beside confirmed facts with the same weight.
The instruction to verify was already there, and it did not hold. What helps is asking for a different deliverable: an investigation whose output is evidence you can check, delivered before anyone discusses a fix.
How each claim is known
An assistant's fluent prose makes statements of very different strength sound alike. Before asking why, ask for the facts sorted by source, as in these examples from the pricing case below:
Comparison
| Kind of statement | Example | Before you rely on it |
|---|---|---|
| Reproduced | Euros, then dollars a second later, returns the euro price both times | Nothing: anyone can rerun it |
| Reported | Customers say some prices look wrong | Turn it into a reproduction |
| Read in the code | The cache key is the product ID | Cite file and line; confirm that code is what runs |
| Assumed | Exchange rates change once a day | Find the record, or keep it out of the reasoning |
Ask the assistant to label each sentence of its explanation this way, citing the output, file and line, or log entry behind it; anything it cannot cite is a hypothesis. The article on handing unfinished work to another coding agent applies similar labels to a handoff note.
Start from a reproduction. Claude Code's best-practice guide suggests giving the symptom, the likely location and what "fixed" looks like, and asking for a failing test that reproduces the issue. Take the test's expected value from the specification or the behaviour's owner, not from existing tests or current output: in the bug report, an existing test encoded the defect.
Ask for explanations that could be proved wrong
One explanation is a story; two give you something to test. Ask for at least two candidate causes and, for each, the observation it predicts that the others do not. In Andreas Zeller's account of scientific debugging, a prediction should ideally "distinguish the hypothesis from likely alternatives" (The Debugging Book). An experiment every candidate agrees on tells you nothing.
- Work read-only. Plan mode or your tool's equivalent: the assistant reads, runs and reports, and edits no production code.
- Reproduce. One command that shows the failure, with its output pasted, not summarised.
- List the candidates. At least two, each with a distinguishing prediction and the cheapest experiment that could rule it out.
- Keep the losers. A short table of candidate, prediction, experiment, result and status, with each ruled-out cause left next to the result that ruled it out.
- Report before fixing. The table and the surviving explanation with its evidence. A person decides whether that is enough.
Keep the losers visible: the plausible one tends to return in the next session. The article on stopping an AI coding agent getting stuck in a fix-break-fix loop covers keeping that record across sessions.
Two explanations for the wrong prices
A small investigation was run for this article on 6 October 2026, in Python 3.9.6. A hand-written price service converts euro base prices at the current exchange rate and caches each converted price for five minutes. A synthetic log of 6,000 requests over three hours, in euros, dollars and pounds, was replayed while rates moved by up to 0.4% every fifteen minutes, and every served price was compared with an uncached calculation. No assistant took part: code, explanations and experiments were written by hand.
Explanation A, the obvious suspect behind a cache, is stale exchange rates. It predicts small errors that vanish when rates stop changing, and suggests a shorter cache lifetime. Explanation B, a cache key that ignores the currency, predicts large errors, about the gap between two currencies, that vanish when every request uses one currency.
First the reproduction: a product requested in euros, then in dollars a second later, came back as 1,144 cents both times; the correct dollar price was 1,236. Reversing the order reversed the error. Then the replays:
Explanation A was not false, which made it dangerous. Freezing the rates removed the 91 small errors and left all 3,265 large ones. A's patch halved the large errors, a drop that reads as progress on a dashboard, without touching their cause. B's prediction held: one currency, no large errors. Temporary logging then recorded which currency had filled each cache entry served: of 5,346 cache hits, 3,265 came from an entry filled for another currency, exactly the number of large errors. The service's two existing tests passed on the defective code. A new regression test, euros then dollars, failed on it with 1144 != 1236 and passed after the fix.
When you can write "resolved"
Zeller asks a diagnosis to show causality, how the failure came about, and incorrectness, why that code is wrong. A change that makes the failure fade without explaining why may only treat the symptom, as the 30-second cache did. Agree the evidence before the fix lands:
- The cause switches the fault on and off. With the cause present the reproduction fails; removed, it passes, and nothing else changed between the runs.
- It accounts for every observation. Including why single-currency traffic was fine and why a shorter cache helped a little.
- The regression test fails without the fix. Run it against the old code and keep the output. A test only ever seen passing shows that it agrees with the new code, nothing more.
- The symptom is gone where it was seen. In the replay, support queue or production metric, not only the unit test.
- The losing explanations are written down. Each with the result that ruled it out, and any remaining effect, such as the price lag, given to a named person to decide.
An assistant can do most of this work; grading its own diagnosis belongs to someone else.
Paqad does not make a diagnosis correct, and incident policy stays with your team. Its router sends a question about the codebase that asks for no change to its project-question workflow, where the AI is told to read the documentation, then the code, and to change nothing; on Claude Code and Codex CLI, an edit to a code file still meets the pre-edit checks for feature development. For an incident after the fact, its root-cause-analysis workflow writes one document under docs/rca/, with instructions to gather symptom and timeline evidence before concluding and never to present a guess as a confirmed cause. A script checks only that the sections are present and in order, and the template has no place for ruled-out explanations. Its answer builder labels evidence as observed, inferred or missing, but in paqad-ai 1.91.1 no Paqad command or hook calls it. Its documentation explains how Paqad handles a question about your project and what those evidence labels cover.
Frequently Asked Questions
How do I get an AI assistant to investigate the root cause instead of inventing a plausible explanation?
Change the request from "why" to "show me". Ask it to reproduce the failure and paste the output, to list what it has not checked yet, and to name the result that would prove its leading explanation wrong. If it cannot name one, the explanation is not ready to act on. Running the fix in a separate session from the investigation also stops a half-checked story turning into a patch on the way.
Which observations can we reproduce, and which statements are still assumptions?
Anything you have rerun yourself and seen fail is reproduced, but only for the environment you ran it in. A failure reproduced on a laptop is still an assumption about production until the versions, configuration and data are shown to match. Treat pasted logs as reported: they show what happened, not why, and a missing log line is not evidence that something did not happen.
What code path, logs or experiment could disprove the leading explanation?
Look for a case that works next to one that fails, and change one thing at a time between them. In the pricing example, single-currency traffic was the working case, and running it was the cheapest disproof available. Trace the code path to the narrowest point where the two cases diverge and log there, rather than adding logging everywhere and reading whatever comes back.
How do we keep competing hypotheses and failed experiments visible before choosing a fix?
Put the hypothesis table in the ticket or the pull request description, with the exact command behind each result, so the reviewer sees what was ruled out as well as what was chosen. Save the output of temporary instrumentation before deleting it. A ruled-out explanation without its evidence will be proposed again by the next person, or the next session.
What causal evidence and regression check would justify saying the root cause is resolved?
The fault appears and disappears with the cause, the regression test fails on the old code and passes on the new, and the symptom is gone where it was reported. Then watch long enough to cover the conditions that triggered it, such as mixed currencies or a rate change. If the symptom returns, reopen the investigation instead of adding another patch.