The AI Agent Factory

11.5 Fix the brief, the inputs or the permissions

Status
stable
9 min read
Owner
Panaversity
Approved
Panaversity ·

In everyday life. You tell a friend, "Dinner was not quite right." That helps nobody. "Less salt next time" changes the next dinner.

What you will learn. How to turn "it was wrong" into one specific change, to the instructions, the source or the access, and how to test that change. You practice it in steps 5 and 6 of the lab.

Why it matters. Most failures need a small change in the right place, not a new worker.

In this chapter's story, Brightline Wholesale Supply, the book's example company, uses an AP Worker, the AI Worker that handles the bills the company owes. In one week in January 2027, four of its outputs went wrong. This page shows how three of them were fixed. The fourth, the duplicate check that never started, is 11.4's.

Once the cause is known, the fix is usually small. Think of the brief, the inputs and the permissions as three levers, like the handles on a machine. Most fixes pull one of them.

The brief. "Wrong" is a reaction, not a fix. A real fix says what must be present and which part of the brief controls it.1 Jordan Ellis, a clerk, had asked the worker to "reply to these three vendors about their short payments." That brief, the worker's written instructions, named no source for the payment terms, so the drafts gave every vendor the same terms. Jordan's brief was rewritten. Now it says "Outcome: a reply to each vendor that explains the short payment. Format: invoice amount, terms, discount or credit, amount paid, difference. Inputs: the vendor's terms from its vendor record, and the invoice and payment lines from the register. Autonomy: draft only. If the terms are not in the vendor record, say so and do not state any terms." If you cannot name which part to change, you are still only reacting.

The inputs. Point to sources by link, not by a name that another file can also use. And check the input itself. The Thursday job writes the pre-run note before each Friday's payment run. It had searched for its file by name, and found the closed 2026 register. The job now opens the 2027 register by its link. It also states the register's last entry date, and stops if that date is more than three business days old. These were two changes, made one at a time. A concept in the KSoR, the Knowledge System of Record, is a short file that holds the rules for one topic (Chapter 8). An out-of-date concept is fixed through its owner and approval. It is never fixed by pasting a correction into a chat, where the fix helps only that one chat.

The permissions. Access can be wrong in two ways: too narrow or too wide. On Friday, January 15, the worker had drafted five replies that said a file was attached. But its link to the shared drive had stopped working. The fix for the Friday replies made two changes, one at a time. First, the worker got back read access to the Remittances folder only, not to the whole drive. That folder holds the remittance notices, the messages that tell a vendor what was paid. Then the brief gained a rule: if an attachment cannot be retrieved, stop and report it. Widening access to make an error go away is a bad fix.

Four rules help you trust a fix:

  • One change at a time. A lever is where a fix lives. A change is one edit you test. If you change two things at once, you will not know which one worked.
  • Re-run on the same input. Test the fixed brief on the case that failed, not on a new and easier one. If the fix is to the input itself, keep everything else the same.
  • Fix what has already gone out. A failure is not one bad output. It is every output since the fault began.1 Brightline had to stop the payment of 7781-R, a copy of invoice 7781, which Brightline had already paid. It also had to find every reply that said "attached" and send corrected replies.
  • Resume with approvals on. One clean run is a first check, not proof. Return to the normal authority in the Role Contract, the worker's one-page job definition, only when the worker's owner agrees the fix has worked. Never go beyond the Role Contract.

The title reads "Turn a reaction into a targeted repair." The line below it reads "Name the missing requirement. Identify what controls it. Test one change." Step 1, "Describe the symptom," shows four reactions: "Wrong," "Too generic," "Used old data" and "Said attached." Lines join them to step 2, "Define the requirement," which asks "What must be true, and which part controls it?" Its example reads "Every draft must include the correct attachment for its recipient." Arrows lead to step 3, "Choose the repair," with three choices. The brief: "Clarify outcome, format, inputs and autonomy." The inputs: "Identify the current source, check it and correct source errors." The permissions: "Use only the access required and authorized." A dark band, "Verify the repair," lists four checks: "Change one thing," "Retest with controlled inputs," "Correct affected outputs" and "Resume with required approvals." The last line reads "When correcting source data, keep other test conditions stable. Recovery stays within the Role Contract."

Figure 11.4. Turn a reaction into a change to one lever.

Check yourself

Question 1 / 8

0 answered

Jordan says the short-payment drafts are 'just wrong.' What turns that into a fix?

  1. Workflow Design & Diagnosis, Panaversity, first edition.↑↑2

On this page