The AI Agent Factory

10.1 Appropriate, appropriate with review, or inappropriate

Status
stable
8 min read
Owner
Panaversity
Approved
Panaversity ·

In everyday life. You ask a friend to pick a restaurant for a group dinner. If they choose badly, you eat one average meal. For your wedding, the same friend can shortlist three places for you to visit. But you would not let them choose the place and sign the contract for you. Your friend is just as good at choosing. What changed is how much you could lose, and who is responsible for the choice.

What you will learn. How to decide whether an AI Worker may do a task, may do it only with a person's check, or may not do it at all, and how to name the one reason that decides. You practice it in step 1 of the lab.

Why it matters. A yes or a no for every task is too simple. Most useful work sits in the middle, where the worker helps and a named person checks before anything is used.

Every proposed use of an AI Worker, an AI that holds a role at work, gets one of three answers.1

  • Appropriate. Low consequence, easy to undo, easy to check as you go. A first-pass summary of meeting notes.
  • Appropriate with review. The worker is useful, but a defined human check must happen before the output is used, sent or acted on. A reply to a vendor about a disputed payment.
  • Inappropriate. The task stays human-owned. A review cannot change the consequence, how hard it is to undo, or the fact that a person must make the decision. A decision to end a supplier contract, or the final decision about someone's job.

Four screens, or tests, help you choose the answer for a use.

ScreenThe question
ReversibilityIf it is wrong, can we catch and undo it before harm is done?
Consequence of errorIs a mistake small, expensive, harmful, regulated or permanent?
Human judgmentDoes the task depend on care, a relationship or context a person should own?
AccountabilityWho is responsible for the outcome, and can that person really check it?

Classify the specific activity you delegate, not the subject it is about. Drafting a recommendation to end a supplier contract can be appropriate with review. Making the decision itself is not.

Do not score the screens and add them up. Ask which one decides the case. The deciding factor is the screen that, if its answer changed, would move the use into another category.1 For the early-pay list that Dave, the controller at Brightline, the book's example company, asked the worker for, the deciding factor is consequence of error for the people affected. A wrong list keeps small suppliers waiting for money Brightline promised to pay within 15 days.

Naming the factor turns "this feels risky" into a claim someone can check. For example, for a schedule of the bills Brightline owes, drafted for its auditors: "This is appropriate with review, because Brightline stays accountable for what it tells the auditors."

Two boundaries are outside your own judgment. Your organization's policy may already classify the use (10.4). And both AI vendors' usage policies require a person to review high-stakes uses that affect people.2 These include legal, medical, employment, housing and financial decisions (10.3).3 Those rules set the outer edge. The three answers work inside it.

The title reads "Three answers, four screens." The line below it reads "Classify the activity you delegate, not the whole subject." Step 1, "Apply four screens," shows four boxes. Reversibility: "Can an error be caught or undone before harm?" Consequence of error: "Who could be harmed, and how seriously?" Human judgment: "Does this decision need a person's judgment?" Accountability: "Who answers for the result, and can they check it?" Step 2, "Choose the answer for this activity," shows three columns. Appropriate: "Proceed within agreed limits." Low consequence, easy to check and correct. Brightline example: summarize weekly AP meeting notes. Appropriate with review, outlined in red: "Name who checks what, and when." A defined human check happens before use or release. Brightline example: prepare the early-pay list, and Dave checks excluded small suppliers before approving. Inappropriate: "Keep the decision human-owned." The worker must not make this decision, but it may support the person. Brightline example: decide to end a supplier contract, and the worker may gather facts. A line reads "Human accountability remains in every category." A banner at the bottom reads "Name the deciding factor. Do not add the screens up."

Figure 10.1. Three answers, four screens. The middle answer is the one that makes you name a reviewer, what they check and when.

Check yourself

Question 1 / 8

0 answered

A worker drafts replies to customers who dispute a charge on their account. Which answer fits this use?

  1. Governance, Risk & Responsible Use: A Crash Course, Panaversity, first edition.↑↑2
  2. Usage Policy, Anthropic.↑
  3. Usage policies, OpenAI.↑

On this page