The AI Agent Factory

3.4 The final 10 percent: evidence review, correction, approval

Status
stable
6 min read
Owner
Panaversity
Approved
Panaversity ·

In everyday life. A contractor says the bathroom is done. You turn on every tap and check under the sink before you pay, rather than taking "done" as proof. With the contractor, "done" is the status, and the taps and the sink are the evidence you check.

The final 10 percent starts with one rule: a finished run is not a correct run. "Completed" means the worker stopped without crashing. It says nothing about whether the numbers are right. Dave (the controller who runs Brightline's accounting) ran the reconciliation of Midwest Packaging's statement again at 4 p.m. The run was complete, and it even contained the warning he needed. The worker had flagged one register line with a typing error, which later underpaid the vendor by $270. He read the status and skipped the evidence.

The final 10 percent has three steps.

  1. Review the evidence. Check the output against the review contract you wrote in the first 10 percent, item by item. Read what the worker flagged before anything else, because a flag is the worker telling you where it was unsure. The warning in Dave's run was a flag. It was the first thing he should have read. Then check the evidence by risk. Review every critical item and exception, and sample the rest (check some of them, not all). Open cited lines (the lines the worker points to as its evidence) to confirm they say what the worker claims.
  2. Correct. Decide where each problem came from, and fix it there. If the output has a small error, such as a wrong date in one row, fix it in place. A problem across the output goes back to the worker as a revision. A problem that came from the brief means the first 10 percent was wrong. For example, the brief did not say which month to reconcile, or it had no stop rule. Fix the brief, or the Role Contract if the same line would be missing next time, then rerun.
  3. Approve. Where the agreed authority requires it, a named person decides that the work may leave their hands: be sent, filed, paid or published. In this chapter's reconciliation, every action that changes something needs that approval. The named person is Dave. Elsewhere, an approved Authority Envelope (the written limit on what the worker may do, Chapter 7) may let a worker complete routine actions alone. Even then, exceptions are escalated (handed to a person), and the work is reviewed regularly. Either way, a person owns the intent, the authority and how the work is verified.

The review contract is what keeps this part short. With it, you check evidence that was gathered for you, which takes minutes. Without it, you redo the work to find out whether it was right, which takes as long as doing it. Much of the economic reason for delegating depends on that difference.

Tip

The same correction twice

The final 10 percent also feeds the next first 10. A correction you make twice is a signal to diagnose (to find the cause). Often the cause is a line missing from the brief or the Role Contract. But it can also be bad data, a broken tool or an unreliable worker. Fix the cause, and the next run needs less review.

Check yourself

Question 1 / 8

0 answered

A task shows 'Completed.' What does that tell you?

On this page