The AI Agent Factory

11.3 Where a failure lives: five causes, cheapest first

Status
stable
9 min read
Owner
Panaversity
Approved
Panaversity ·

In everyday life. The car will not start. You check the fuel and the lights before you replace the engine.

What you will learn. How to find where a failure comes from among five common causes, checking the quickest first, and why one failure can have two causes. You practice it in steps 2 and 4 of the lab.

Why it matters. The most expensive change, such as a stronger model or a full rewrite, often fixes none of the real causes. A one-minute check in the right place can find them.

In this chapter's story, Brightline Wholesale Supply, the book's example company, had four failures in one week in January 2027. They came from its AP Worker, the AI Worker that handles the bills the company owes: AP means accounts payable. Maria, the office manager, runs the worker day to day, and Dave Kowalski, the controller, approves its work.

Bad outputs look alike. The causes behind them do not. Five causes explain most failures:

CauseWhat it meansWhen it usually showsCheap check
The briefThe written instructions never included what the work neededWrong from the first outputCheck it against its four parts: outcome, format, inputs and autonomy (Chapter 5)
The inputsA source is wrong, renamed or out of date"It used to work"Open the source that the task record names. Check its name and date.
Tools and permissionsA connector to another app failed, or access is too wide or too narrowA step with an error in the record, then the worker guessingFind the first error in the step list
CapacityToo much context, a usage limit, or the wrong feature or modelGets worse over time, stops at a limit, or one repeated kind of errorStart a fresh session, check usage, or try the right feature once
Task fitThe task asks for something the tool cannot doNever right, however you askOnly after the other four look unlikely

These five checks are for a wrong output. For a missing output, such as the duplicate check's below, run the 11.2 checks first. One of those checks, a usage limit, is a capacity cause. The time a failure appears tells you where to look first. It does not prove the cause.1 The first four checks each take about a minute, so start where the timing points. A negative check, one that finds nothing wrong, makes that place less likely. It is not proof. Run the other checks too, and leave the most expensive change for last. Dave did the opposite: he asked first for the most expensive model and for new briefs. Do not stop at the first cause, because one failure can live in two places.

January failureCauseEvidence in the record
Duplicate check did nothingCapacity: usage limit. Also the brief.Every run on Tuesday, January 12: "Not started. Usage limit reached." 7781-R was invoice 7781, sent again. The brief says nothing about a suffix like "-R", so the check could not be trusted to match the two.
Wrong terms in dispute repliesThe brief"Reply to these three vendors about their short payments." No source for the terms is named.
"No invoices due" (the Thursday note)The inputsOpened "AP Register," the closed 2026 file
"Attached" with nothing attachedTools and permissions. Also the brief.Step 2: access denied. Step 3: drafting replies. The brief had no rule for a missing attachment.

In two of the four, a person also missed the problem. Maria forwarded the Thursday note to Dave without checking its first line, and she sent the Friday replies without opening an attachment. A Review Contract (Chapter 6) lists the checks you agree before the work. One that said "check the first line" or "confirm each message carries its own attachment" would have found both. So ask where the fault lives, and also which check was skipped.

When one kind of error repeats, that suggests the wrong feature.1 In the lab, the hands-on part of this chapter, the worker's early-payment discounts are wrong by a few cents to $41.30. The rate and the invoice amounts are correct. The math is written in sentences, so prose arithmetic is the likely cause. Run the calculation as code, or in a spreadsheet. That tests the cause and fixes it.

The title reads "Five places a failure can live." The line below it reads "Start with the quickest checks. Let evidence guide the diagnosis." Under "Recommended check order," five numbered rows each give a check. Row 1, "The brief," reads "Check the brief against its four parts." Row 2, "The inputs," reads "Open the named source. Verify its identity, date and contents." Row 3, "Tools and permissions," reads "Inspect the first failed step. Check access and tool errors." Row 4, "Capacity," reads "Check usage limits, context load and feature availability." Row 5, "Task fit," reads "Assess suitability after investigating the earlier checks." A panel on the right asks "Switch the model?" and reads "A common first reaction. Investigate the cause before making the change." A note at the bottom reads "A quick check gives a clue, not proof. Verify the cause before changing the setup."

Figure 11.3. Five places a failure lives. Check the cheapest first.

Check yourself

Question 1 / 8

0 answered

In a fresh, short session, the first output was wrong. Which cause do you check first?

  1. Workflow Design & Diagnosis, Panaversity, first edition.↑↑2

On this page