The AI Agent Factory

9.1 Is this an agent problem?

Status
stable
7 min read
Owner
Panaversity
Approved
Panaversity ·

In everyday life. You would not hire an accountant to add up one receipt. And you would not write a recipe card for a meal you will cook once.

What you will learn. How to decide, with three questions, whether work that repeats should go to an AI Worker at all, and when a fixed path is enough. You practice it in step 1 of the lab.

Why it matters. Some repeating work does not need a worker. An ordinary tool, or a chat each time the work comes up, can be cheaper and safer than something you must build and check every week.

Before you hand recurring work, work that repeats, to an AI Worker, ask whether it needs one. Put the work through three gates, three questions asked in order.1 Any gate can stop it.

Gate 1. Does an ordinary tool already do it? A total, a lookup, a sort or a standard report belongs in the tool that was built for it. At Brightline, the book's example company, Maria, the office manager, thought of asking the worker to add up payments by vendor each month, for year-end tax forms. Gate 1 stopped her. The accounting system already prints that report. A worker adds cost and a chance of error to a sum. If a result matters, compute it.2

Gate 2. Once, or every week? Three dials decide, and each one can be up or down. Frequency: does the work come back regularly? Consistency: do the inputs and the finished output have the same shape each time? Payoff: is the time you save, or the error you avoid, worth the cost of building and checking it? If all three are up, it is worth making recurring. Go on to Gate 3. If any one is down, build nothing. Handle the work in a conversation each time it is needed. A useful signal: the third time you do the same task the same way, stop and run Gate 2.

Gate 3. What does finished look like? Write down the input, the output format, and one check you can do in under a minute. For Maria's Thursday list, the check is: is every invoice on it open, and due by the Friday after the run? If you cannot write that check, the work is not ready to run without you.

One more question comes after the three gates: does the work need judgment at each step, or does it follow a fixed path? A fixed path, such as "every Thursday, list invoices due by the Friday after the run," can be a simple scheduled job with a fixed brief. An agent, a model that works in a loop with tools and decides its next step, is worth its cost when the steps depend on what it finds, when the rules have exceptions, or when the inputs are messy.3 Reading scanned invoices that come in many different layouts is an example: each week brings the same kind of input, but each invoice looks different. Choose the simplest approach that works, and add judgment only where the work needs it.4

The title reads "Three gates before recurring work." The line below it reads "Follow the gates in order. Take the exit when it applies." Gate 1 asks "Does an ordinary tool already do it?": a total, lookup, sort, or standard report. Yes leads right to "Use the existing tool," with the example "vendor payment report." No leads down to Gate 2, "Is it worth making recurring?" It has three labels, frequency, consistency and payoff, and the question "Are all three high enough?" No leads right to "Keep it in a conversation. Handle it when it comes up. Build nothing yet." Yes leads down to Gate 3, "Can you define and check finished?": clear input, defined output, one-minute check. No leads right to "Not ready to run unattended. Define the result and its check first." Yes leads down to a gold band, "Ready to design recurring work." Below it, a box reads "Choose the simplest approach that works," with two choices. Fixed path, a calendar icon: "Scheduled job with a fixed brief." Judgment needed, a gear icon: "Agent for changing steps, exceptions, or messy inputs."

Figure 9.1. Three gates before any recurring work.

Check yourself

Question 1 / 7

0 answered

A team wants a worker to add up vendor payments each month. The accounting system already prints that report. What fits best?

  1. Is This an Agent Problem? A Crash Course, Panaversity, first edition.↑
  2. Workflow Design & Diagnosis, Panaversity, first edition.↑
  3. A practical guide to building agents, OpenAI.↑
  4. Building effective agents, Anthropic.↑

On this page