15.5 Before the pull request: evidence, not "done"
- Approved by
- Panaversity ·
In everyday life. A mechanic who says "the brakes are fixed" gives you a claim. A mechanic who hands you the test-drive printout, the measured stopping distance and the old brake pads gives you evidence. You can check evidence yourself.
What you will learn. What to ask for before a change goes to anyone else: evidence, a second reader, and a short brief when the risk needs one.
Why it matters. Anyone can check evidence, and nobody can check a claim.
What counts as evidence?
Evidence for a change is the commands that check it, run on this version of the code, with their real output. "Done" is a claim. So ask for evidence before a change becomes a pull request. For Brightline that means npm test, npm run typecheck, npm run test:py, npm run check:py, and npm run due -- 2027-03-05, which prints the payment list. The agent can run all of them. Ask it to show the output, not to describe it.
Evidence is only as good as its expected results. "The tests pass" says nothing if the agent changed the tests. So the evidence for any change also includes git diff main...weekend-rule -- tests/ py/tests/, and the answer should be "no lines removed or changed", unless a test change was agreed.
Who should read it first?
A second reader who has not seen the work. The agent that wrote the change cannot easily see its own mistakes. Before the pull request exists, ask a separate reviewer with fresh context to read the diff. Fresh context means it has not seen the session that built the change. The reviewer can be a subagent, a new session, or a different agent. It should know the request and the acceptance criteria, the conditions the change must meet, but not the builder's reasoning. It should also not be able to edit the files it judges. A reviewer that can change what it reviews is no longer independent. In each desktop app, a reviewer with fresh context is a new session in the same folder, or a review command set to run in its own context. It is not a question in the chat that built the change (15.11). Then run your app's review command too, and compare the two reviews.
What is a change brief?
A change brief is a short file the agent keeps in the repository, such as changes/weekend-rule.md. Ask for one when a change moves money, touches data or changes a rule. It records five things:
| Part | What it records |
|---|---|
| Intent | The outcome, what is not part of the job, the acceptance criteria, and decisions still open, with who decides them |
| Investigation | How the behavior was reproduced, confirmed findings, guesses still unproven, what else the change touches |
| Evidence | Each claim, where its expected result came from, the command, the result, and the commit it ran on |
| Review | Each finding and how it was settled, what was not tested, what is still uncertain |
| Delivery | What merging does and does not do, who accepts it, how to undo it |
Match the brief to the risk. A typo fix needs a one-line pull request description and no brief. The agent keeps the brief up to date. When a commit changes the code, it reruns the checks and marks which evidence went stale. You make the decisions and judge the evidence. The brief serves one change, and nobody updates it after the merge. What lasts is the pull request's history and the tests. The real risk is not that nobody writes it. The real risk is a brief so well written that reviewers stop checking it. So every claim points to a command anyone can rerun. Never paste a key or private data into it.
Who owns the change?
The person who directed the agent owns the change. They answer the review, fix what it finds, and are the first to act if it breaks after the merge.
In the case study. The team asks Claude to show the output instead of saying it is done. A subagent reviews the diff with fresh context before any pull request exists. A finding that keeps coming back is fixed at the source, with a test, a lint rule or a line in a rules file, so it does not come back again.1
At work. Ana asks the agent for the evidence on weekend-rule. It shows four passing checks, 8 invoices, $19,229.53 with the flag off, and 10 invoices, $33,109.53 with PAY_WEEKEND_RULE=on. The test diff lists only two new files, so no existing test changed. She then opens a new session as the reviewer: "Here is the request and the acceptance criteria. Review the diff between main and weekend-rule. Report findings with a file and line. Change nothing." Its first finding is in a file the request never mentioned.
The operating rule: accept output you can run again, not a claim, and keep a short brief for any change that moves money.
Figure 15.10. A claim and evidence. Only the right side can be checked by someone else, and only for the commit it names.
Practice it. Practice: evidence and a fresh reader, in your own tool, fifteen minutes.
Check yourself
Question 1 / 8
0 answered
An agent reports "Done. All tests pass." What do you ask for next?
Sources
- Owner
- Panaversity