13.11 Judgment
- Approved by
- Panaversity ·
In everyday life. A good bookkeeper does not double-check every receipt the same way. A $6 coffee gets a quick look. A $60,000 wire transfer gets a call to the vendor on a phone number the bookkeeper already knows. And an automatic payment, a rule that pays a vendor every month, gets the most care of all, even when each payment is small, because a mistake in it is paid again every month. The skill is knowing which is which.
What you will learn. When your check is enough, when to read the rule lines, and how to turn a vague goal into tests.
Why it matters. The more a mistake would cost, and the more often the code will run, the harder you check. Code that runs again next week meets rows nobody has checked.
TDG and PRIMM-AI+ are the default for every change. This concept is about the cases where you check harder, check differently, or do the work yourself.
Which lines carry a rule?
The few lines that decide what happens to every row, the kinds that Concepts 13.3 to 13.7 teach. They are text converted to a number where it is read, a decision, a lookup, a sort, and the date math. Before you accept a change, write each of those lines as a sentence in NOTES.md, and compare the sentence with the business rule. When the two disagree, the line is a bug, whatever the check says. The output you predict from the sentence shows it, as Ana's sort line did in 13.6.
When is my check enough, and when do I read?
Your check is enough when a mistake would cost little and the code runs once, on the rows you checked. Read the rule lines when a mistake would cost a lot, or when the code will run again on rows nobody has checked. The table asks both questions, and a few more.
| Situation | What you do |
|---|---|
| Someone asks the AI for a number, such as a total, a due date or a tax | Have it write or run a tested function that computes it. Never accept arithmetic done in a reply |
| The data is private, such as real vendor or employee records | Give the agent only the column names, the types and a few made-up rows, and test on those. The real data goes only to the finished, tested code, never to the agent1 |
| The code moves money, such as making a payment, or handles passwords, API keys or personal data | Read it line by line, and ask someone who knows the area to review it. If you cannot verify it, do not release it |
| The code is too long or complex to read in full | Rely more on tests, not on trust. Add more cases and more failure cases. Read at least the lines that carry a rule |
| The code will run again on rows nobody has checked, such as next week's export | Write a sentence for each rule line before you accept it, even when your check matches |
| The change is tiny and obvious, such as one number | It may be faster to make it yourself. Then have the agent run the tests and the type checker anyway |
| The goal is vague | Break it into small pieces you can each test, before any code |
| The tests pass | Red-team them: try the inputs that should fail |
Review the diff before you accept it. Before every commit, read the diff and ask these questions. Which files changed? Did any test change? Did it add a package, and is that package real, not a made-up or look-alike name, and needed? Did it touch anything sensitive? Do the tests and the type checker pass? Can you say in one sentence what the main function does? The last question catches the most.
How do I red-team my tests?
Try the inputs that should fail. When the finance team tests a control, it does not only check that good invoices get through. It puts through one that should be stopped, and sees whether it is. A weak test is dangerous because its green result makes you feel safe when you are not. So know what each test does not cover. Inputs to try: an empty file, a zero amount, a missing column, text where a number belongs, terms nobody has seen. In TypeScript, Number("") gives 0 with no error, so an empty cell becomes a zero amount. Ask the agent "what input would break this?", and then you decide which answers become tests.
How do I turn a vague goal into tests?
"Make the script smarter" cannot be tested. Turn it into pieces you can test, in five steps:
- Say the goal in one sentence. If you cannot, ask the person who set it what they mean, as Ana does below, until each part is a rule.
- List the steps you would do by hand, as verbs: load, check, flag, report.
- Give each piece a typed signature, with no body yet.
- List the cases for each piece, have the agent write them as tests, and read each test against your list. Then run TDG on it.
- Join the pieces and add one test of the whole.
If a piece is hard to test, it is doing too much. Split it.
At work. Brightline's controller asks Ana for a script that will "flag anything suspicious in the invoice export". Ana cannot test "suspicious", so she asks the controller three questions and gets three rules. Each one becomes a function with a signature and a test before the agent writes any code.
| Rule | Signature | One test |
|---|---|---|
| The same invoice number appears twice | duplicateNumbers(invoices): string[] | Two rows numbered INV-8301 give ["INV-8301"] |
| An amount over $10,000.00 needs a second approver | needsSecondApprover(cents: number): boolean | 1290000 gives true, 1000000 gives false |
| A stated due date before the invoice date | dueBeforeInvoice(invoice): boolean | Invoice date 2026-10-05, due 2026-10-01, gives true |
The invoice inputs have no type label here, because the invoice record gets its own type in Chapter 14.
The second rule had a hidden question: is exactly $10,000.00 "over"? The controller said no. Without the test, the agent would have guessed. That was an edge case, a case at the limit of a rule. Each one you find, by asking or by reading a rule line, becomes a test.
The operating rule: let tested code, never a model's reply, produce every number. Match the depth of your check to what a mistake would cost. Read the rule lines of code that will run again on rows nobody has checked. Test the failures as well as the normal case. And turn every vague goal into pieces you can test before any code is written.
Figure 13.13. From a vague goal to testable pieces. If you cannot write a test for a piece, it is not specified yet.
Practice it. Practice: judgment, in your own tool, fifteen minutes.
Check yourself
Question 1 / 10
0 answered
The code touches API keys. What does the chapter say to do?
Sources
- Owner
- Panaversity