Lab 20Lab 20: the agent loop by hand

Getting ready

The lab has four sittings. Each comes right after the concepts it uses. Each is also short enough for one session: about 2 hours, 1 hour 10 minutes, 1 hour 10 minutes and 1 hour 25 minutes. This section is the setup for all of them. Read it, then do Task 0. When a task says F0 to F6, it means the field guide, in the reference at the end of the chapter.

The agents write, and you explain. This lab follows Chapter 13's PRIMM-AI+ method. In each task, you predict first: you write your prediction in ports/ch20.md. Then the agent runs the code, and you investigate the gap with the agent's help ("explain line 73"). Some tasks also ask you to modify: you tell the agent what to change, predict again, and run again. You must be able to say what every line of it does.

The lab's files already hold the loop. brightline-lab-ch20.zip comes with the finished code and the tests that check it. So Sittings 1 and 2 are about understanding and running the loop, not building it. Your coding agent's job is to set up, run, explain and change. Sitting 4's Task 13 is where the work becomes yours: you review it all, and merge it.

This lab costs a little money, and takes about 6 hours 10 minutes (up to an hour more on route A). Neither API offers a free plan for API calls. Every npm run ask run has a cost cap of 10 cents. The whole lab should cost under $1 on each API (not yet confirmed by a live run). The open-model port is free.

The date in this lab is Tuesday, March 30, 2027, the day after Lab 19. "This week" means up to Sunday, April 4. The worker's instructions, prompts/ap-worker.v1.md, give the model that date.

Not yet run live. No one has run this lab against the real APIs yet. The recorded responses in the tests and the logs in the concepts follow each API's documented shapes. They were checked against the official SDK types, but none of them comes from a live run yet. If a live response differs a little, trust what you see, and record it in ports/ch20.md. One stop rule: Task 0's last step or Task 1 may fail in a way that If something goes wrong does not cover. If that happens, stop, and record what you saw. Every later task depends on reaching the API and the connector.

What counts as a pass. Each task's What good looks like says what to expect. When it lists more than one possible outcome, any of them is a pass. When what you see matches none of them, it is still a pass if two things are true. You can say which step of the loop produced it, and you wrote it in ports/ch20.md. What matters is that you can explain the loop. You do not need to match someone else's run.

Before you start

What you need.

• A computer running macOS, Linux, or Windows with WSL. WSL is the Windows Subsystem for Linux, and you need it because the scripts use Unix paths. The computer needs Node 22.19 or later (the minimum Lab 19 set), Python 3.12 or later and uv 0.11 or later. It also needs git and the GitHub CLI (gh). F0 says where to install each.
• The desktop apps of Claude Code and OpenCode version 2, and Codex in the ChatGPT desktop app, each signed in (Chapter 12).
• A Claude Console account with API credits and an OpenAI API account with credits (F1). Each gets a project or workspace with a small monthly spending limit, which you set by hand in Task 0.
• For the open-model port: Ollama, and several gigabytes of free disk for one small model (F6).
• A connector to call. Either your own Chapter 19 staging connector and a test token (route A), or the local practice connector in the zip (route B). Task 0 helps you choose.

A quick reminder from Chapter 12. Plan mode means the agent writes a plan and waits before it changes anything. Manual mode means it asks you before each command it runs. A second terminal is a new terminal window or tab, opened beside the agent. It is for a program that must keep running, such as the practice connector. Start the program there, leave the window open, and stop it later with Ctrl+C. A branch is a separate line of work in git, and deleting it does not touch the main code (Chapter 15).

How every task runs. Chapter 12's five-step routine: (1) a fresh session with the named agent, in brightline-ap/. (2) Plan mode, with the task's prompt as the brief. Read the plan, and correct it before it starts. (3) Manual mode: approve each command it asks about. In Codex, choose Ask for approval: in that mode, Codex runs commands inside the folder without asking. It asks before any command that needs the network, such as npm run ask. (4) Evidence: the agent shows what proves the task is done, and you check it against What good looks like. (5) A line in ports/ch20.md: the task, your prediction, what happened, anything that surprised you.

What you do by hand, and only this. Make the two API accounts. Make each API key in its own console, and paste it into its file in .secrets/ with your own editor. Set the spending limits, and copy the prices for the pinned models into src/loop/config.json (F5). You also decide when to merge. Agents do everything else. No API key or token passes through a chat.

Rules the agents follow. AGENTS.md says: never print, read aloud or copy anything in .secrets/. Run live calls only through four commands and their Python twins. npm run ask has a cost cap. npm run break and npm run stream:demo make one or two small calls each. npm run stateful runs at most 8 turns. The only other live call is the one embeddings call inside Task 0's load on route A. Ask before any call to a model API. .claude/settings.json also denies reading .secrets/, and opencode.jsonc denies OpenCode's file-reading tool access to it. Codex's settings file cannot deny it, so in Codex, the rule in AGENTS.md is the guard.

Words in the prompts.

Your agents can read this chapter. docs/chapter-20/ holds the concepts and the field guide as text. Each prompt names what to read.

Task 0. Set up from the zip (30 minutes on route B, up to 1 hour 30 minutes on route A, Claude Code)

Goal: both languages installed, and every offline test passing. Also the API keys you made today, in files git ignores, with their prices filled in, and a connector the loop can reach.

By hand first (F1). Unzip brightline-lab-ch20.zip. Then set up the accounts. F1 says where each step is on each provider's site.

• The first API, now. Make the Claude Console account, and add a few dollars of credit. Then make a workspace for this lab with a monthly spending limit of $5. (The Default Workspace cannot have one.) Make one API key in that workspace, and paste it into .secrets/anthropic.env as ANTHROPIC_API_KEY= followed by the key.
• The second API, now or at the start of Sitting 2. Make the OpenAI API account the same way, and paste its key into .secrets/openai.env as OPENAI_API_KEY= followed by the key. If two accounts at once is too much, leave this for Sitting 2, which sets aside 10 minutes for it. Until then, the openai prices stay at zero.

Then open the pricing page for each account you made. Find the pinned model from src/loop/config.json (the models block, also listed in F3). Copy its four prices into that file, with today's date in checked_on (F5). The part you edit looks like this before you start. Change only the zeros for claude, and for openai if you made that account today. The prices are in dollars per million tokens. Keep every quote, comma and brace:

"prices": {
  "checked_on": "",
  "claude": { "input": 0, "output": 0, "cacheWrite": 0, "cacheRead": 0 },
  "openai": { "input": 0, "output": 0, "cacheWrite": 0, "cacheRead": 0 },
  "open": { "input": 0, "output": 0, "cacheWrite": 0, "cacheRead": 0 }
},

checked_on takes the date in quotes, such as "2027-03-30". If a pricing page shows no separate cache-write price, use the input price for cacheWrite. Here is a filled-in row, with made-up numbers: "claude": { "input": 3, "output": 15, "cacheWrite": 3.75, "cacheRead": 0.3 }. It means $3 for each million input tokens, and so on. Copy the real numbers from the page. The open prices stay zero, because the open model is free. The tests do not read this file, so check your edit yourself. Ask your agent to confirm that src/loop/config.json is still valid JSON, without changing it.

Choose a connector.

• Route A, your own connector. If your Chapter 19 staging connector is still deployed, use it. By hand first, copy the files in your Lab 19 folder's .secrets/ into this folder's .secrets/, in Finder or your own terminal: the load and the token need them. Step 4 of the prompt below first loads this lab's Brightline data into it: Lab 19's rows, and three invoices entered since. The loop then calls your real connector, with a test token for Ana at Brightline. You need a way to make that token (F2). If you are not sure you have one, take route B: everything in this chapter works on it.
• Route B, the practice connector. If your Chapter 19 connector is not deployed, or you skipped Labs 17 to 19, use the practice connector in the zip. It offers the same four tools over MCP, on your own computer. It reads this folder's Brightline files, the same rows route A loads. It also checks a bearer token: a sign-in token sent in a request header, as in Chapter 18. It has no sign-in server and no database, so it proves the loop, not the connector.

Predict: before any key is used, which tests can already pass?

In plain words. Your coding agent installs both languages' packages, runs every test that needs no network, and connects the loop to a connector.

Read README.md, AGENTS.md, and docs/chapter-20/field-guide.md, sections F0 to F2.

0. Put this folder under git, with a main branch. Make a first commit of everything here with the message "Start from the end of Chapter 19", and tag that commit lab20-start. If git has no name and email yet, set them for this project only. Show me git log --oneline -1 and git tag.
1. Install with npm ci, then cd py && uv sync --locked for Python. Show me node --version and the two SDK versions from npm ls --depth=0.
2. Run npm run typecheck, npm run typecheck:py, npm run test:loop and npm run test:loop:py. Show me the four results as a table.
3. Check that git ignores .secrets/ (git check-ignore .secrets/anthropic.env should print the path) and that src/loop/config.json has prices and a checked_on date for claude (and for openai, if I made that account today). Do not open the key files.
4. The connector. Do only the route I chose.
   • Route A: ask me first, then run npm run tenant:load -- --company org_brightline --folder data --env staging --replace. It loads this folder's Brightline into staging, Lab 19's rows and three invoices entered since (INV-8316 to INV-8318), and embeds the policies again, one small metered call to the embeddings model from Chapter 19. Then run npm run token:test -- --person ana --company org_brightline, as F2 says. It signs Ana in on staging and writes the connector's staging address and her token into .secrets/connector.env as CONNECTOR_URL= and CONNECTOR_TOKEN=, never printing the token. If it stops, show me why, and I will switch to route B.
   • Route B: write CONNECTOR_URL=http://127.0.0.1:8760/mcp and a random CONNECTOR_TOKEN into .secrets/connector.env, with a short script that never prints it. Then tell me to run npm run connector:practice in a second terminal, in brightline-ap/, and wait until I say it is running. It reads the token from that file itself, so you never need to see it (F2).
5. List the connector's tools with the listTools() function in src/loop/tools.ts (an MCP tools/list call), and show me their four names.

What good looks like: one commit at tag lab20-start. Node 22.19 or later. @anthropic-ai/sdk 1.5.0 and openai 7.51.0. Both type checks clean, 32 TypeScript tests and 22 Python tests passing. .secrets/ ignored, and prices filled in for each account you made (the openai row may stay zero if you left it for Sitting 2). On route A, the load reports invoices 18, purchase_orders 14. Four tool names from the connector. Your prediction: all of them. Every test replays recorded responses and needs no key.

Check it now. Download Task 0's answer key.

If Task 0 fails.

Sitting 1: the loop on the first API (about 2 hours)

Before you start, answer in a line each: What does the model send back when it wants a tool (20.1)? What does a strict schema guarantee, and what does it not (20.2)? After this sitting, you can: explain the loop line by line in either language, predict each request it sends, and say what breaks the next call.

Where you are. The first API (claude.ts, or claude_loop.py in Python), with the connector you set up in Task 0, by route A or route B.

Task 1. One request, one stop reason (15 minutes, Claude Code)

Goal: you read one real request and one real response, and name the stop reason before you see it.

Predict: you ask "What is 2 + 2?", with the four AP tools attached. How many turns will the loop take, and what will the stop reason be?

In plain words. The agent sends one simple question through the loop and shows you exactly what went out and what came back.

Read 20.1 and 20.3 in docs/chapter-20/. Run npm run ask -- --api claude "What is 2 + 2?". It prints the request and the response together, so show me only the request section first, and wait while I write my prediction in ports/ch20.md. Then show me the response section and the last line. Explain each top-level field of the response in one line each.

What good looks like: one request with model, max_tokens, system, messages, tools, tool_choice and thinking, and no temperature (a setting older code used, which 20.9 explains). One response, which may hold a thinking block. (The model decides whether to think. The API may leave the thinking text out and send only a signature, a sealed code that proves the block was not changed.) Then a text block saying 4, stop_reason end_turn, and a usage object. The summary line reads --- stop: end_turn | turns: 1 | cost: … ---. The cost is under a cent. Your prediction: one turn, "finished", because no tool helps with arithmetic.

Check it now. Download Task 1's answer key.

Task 2. The four tools, read three ways (20 minutes, Claude Code)

Goal: you can read a tool definition, and say which part the model reads, which part the schema check reads, and which part only the handler can check.

Predict: for "Which invoices are due this week?", which tool will the model ask for, and with which inputs?

In plain words. The agent shows you the one file that defines the four tools, and the two small functions that turn it into each API's shape.

Read 20.2. Show me tools/ap-tools.json, and compare each tool's description with the connector's own, in servers/ap-ts/src/server.ts. Then show me claudeTools() and openaiTools() in src/loop/tools.ts and claude_tools() and openai_tools() in py/brightline_ap/loop/tools.py, side by side. Explain the two differences between the two tool-definition shapes. Do not run anything yet.

The code. Here is the TypeScript conversion, and the Python one, slightly simplified. (The file adds type hints and comments.)

New in this code. specs.map((s) => ({ … })) in TypeScript and [ … for s in specs] in Python both mean "make a new list, with one entry for each tool in specs". Inside, s stands for one tool at a time. The extra ( ) around { } in TypeScript only says that the braces hold an object, not a block of code. specs = TOOL_SPECS means "use the tools from ap-tools.json unless told otherwise". export lets other files use the function. : Anthropic.Tool[] is a type for the type check, and you can skip it.

export function claudeTools(specs = TOOL_SPECS): Anthropic.Tool[] {
  return specs.map((s) => ({ name: s.name, description: s.description, input_schema: s.input_schema, strict: true }));
}

export function openaiTools(specs = TOOL_SPECS): OpenAI.Responses.FunctionTool[] {
  return specs.map((s) => ({ type: "function", name: s.name, description: s.description, parameters: s.input_schema, strict: true }));
}

def claude_tools(specs=TOOL_SPECS):
    return [{"name": s["name"], "description": s["description"], "input_schema": s["input_schema"], "strict": True}
            for s in specs]

def openai_tools(specs=TOOL_SPECS):
    return [{"type": "function", "name": s["name"], "description": s["description"],
             "parameters": s["input_schema"], "strict": True} for s in specs]

Investigate: the schema is the same object in both. What two things change? (The key name, input_schema against parameters, and the extra "type": "function".)

What good looks like: four tools whose descriptions match the connector's own, word for word (on route B, the practice connector serves the same four). The agent names the two differences in the tool definitions above. (Part B's four differences are about tool calls, a different thing.) Your prediction: list_open_invoices, with due_on_or_before 2027-04-04, vendor_id and page null, and page_size 10 or null (null means five a page, and a second call for the rest). All four inputs are required, so the model must clearly say "no vendor filter".

Check it now. Download Task 2's answer key.

Task 3. Follow the loop, then run it (45 minutes, in two parts, Claude Code)

Goal: you can point to each of the loop's five steps in either language, and match every request and response of a live run to a step.

Predict: when the model answers in words, which line decides to end the loop, and which line actually ends it? And how many entries will the conversation array hold in the second request?

In plain words. Part 1: you go through the whole loop, and the agent explains any line you ask about. Take a break. Part 2: the agent runs Ana's question, and you follow the log step by step.

Part 1. Follow the loop (30 minutes)

Read 20.1 to 20.5. Open src/loop/claude.ts and py/brightline_ap/loop/claude_loop.py side by side. Do not run anything. Wait while I read and write my predictions. Then answer my questions about any line, in one or two sentences each, quoting the line.

Figure 20.9. A map of the loop in both languages. Find a step in your language, then find its twin.

Where the loop starts. Before step 1, both files open the loop and check your own stops first (20.8). In TypeScript, lines 41 to 44:

41  for (let turn = 1; ; turn++) {
42    // Your stops come first: turn limit, cost cap, time limit (20.8).
43    const limitStop = meter.beforeTurn(turn);
44    if (limitStop) return { stop: limitStop, answer, turns: turn - 1, costUsd: meter.spent, messages };

And in Python, lines 44 to 48:

44    while True:
45        turn += 1
46        # Your stops come first: turn limit, cost cap, time limit (20.8).
47        if limit_stop := meter.before_turn(turn):
48            return LoopResult(limit_stop, answer, turn - 1, meter.spent, messages)

Both mean "count the turns, and repeat with no end of its own". (In Python, line 42 sets turn = 0 first, so the first turn is 1.) The loop ends only at a return: one of your stops here, or the stop reason in step 3 below.

Step 1, the model call. Next comes the request, built from the instructions, the whole conversation and the tools. Then comes the call itself. In TypeScript, lines 46 to 58:

46    // Step 1. Call the model with the instructions, the whole conversation and the tools.
47    const request: Anthropic.MessageCreateParamsNonStreaming = {
48      model: o.model,
49      max_tokens: 4096,
50      system: o.instructions,
51      messages,
52      tools: claudeTools(),
53      tool_choice: { type: "auto" }, // the model decides. Forcing a tool is a 400 on current models
54      thinking: { type: "adaptive" }, // the same setting on every turn of the loop
55      ...(o.cache ? { cache_control: { type: "ephemeral" as const } } : {}),
56    };
57    o.log?.("request", request);
58    const response = await tracer.modelCall("anthropic", o.model, () => o.client.messages.create(request),

Lines 59 to 63 (not shown) finish the call and its trace details (20.10). Line 64 logs the response, and lines 65 to 70 add its token counts to the cost meter (20.8). Python's lines 50 to 77 do the same. Its key lines:

50        # Step 1. Call the model with the instructions, the whole conversation and the tools.
51        request: dict[str, Any] = {
52            "model": model,
...
60        if cache:  # ask the API to cache the repeated start of every request (20.12)
61            request["cache_control"] = {"type": "ephemeral"}
...
64        with tracer.model_call("anthropic", model) as attrs:
65            response: Message = await client.messages.create(**request)

In TypeScript. Line 54's adaptive lets the model decide on each turn whether to think. Line 55's ...( … ) adds the caching setting only when caching is on (20.12). Its ephemeral means short-lived: the cache keeps the start of the request for a few minutes. o.log?.( … ) on line 57 means "log it, if a log was set up". The real call to the model is o.client.messages.create(request) on line 58. Its () => … gives that call to the tracer, which runs it and records it. The call continues on lines 59 to 63. Names after : or as, such as Anthropic.MessageCreateParamsNonStreaming on line 47 and as const on line 55, are types for the type check (Chapters 13 and 14). You can skip them.

In Python. Line 65 is the real call. **request passes every field of the request dictionary to it.

The main names in the code. Three names appear everywhere in the loop. o holds the options the loop was given: the model, the instructions, the API client and the tool runner. meter counts turns and cost, and stops the loop at your limits (20.8). tracer records each step for the trace (20.10). If you learn only three pieces of syntax, learn ??, a ? b : c and await, all explained below. Look up the rest when you see them.

The code. The main part of the loop, in TypeScript, with the line numbers from the file. Go through it twice. The first time, look only at the comment lines, Step 2 to Step 5. The second time, look at the code under each comment.

New in this code. A few pieces of syntax you may not have seen before.

• response.stop_reason ?? "unknown": use the stop reason, or the word "unknown" if there is none. ?? means "or, if that is empty".
• response.content.flatMap((b) => (b.type === "text" ? [b.text] : [])): go through every block b, keep the text of the text blocks, and skip the rest. .join("") then joins the pieces into one string. Python's line 80 does the same in one line.
• o.model and o.runTool are the model and the tool runner that o holds, as described above. meter.spent is the run's cost so far. LoopResult( … ) in Python, like the { stop, answer, … } object in TypeScript, is what the loop gives back when it ends.
• In Python, := (line 47 above) stores a value and tests it in one step. So Python's line 47 means "if before_turn gives a reason to stop, keep it in limit_stop".

The rest of the new syntax, in lines 81 to 97, is explained just after the excerpt.

72    // Step 2. Keep everything the model said, exactly as it came back, thinking included (20.5).
73    messages.push({ role: "assistant", content: response.content });
74    answer = response.content.flatMap((b) => (b.type === "text" ? [b.text] : [])).join("");
75
76    // Step 3. Stop on any reason except "wants a tool" (20.3).
77    if (response.stop_reason !== "tool_use") {
78      return { stop: response.stop_reason ?? "unknown", answer, turns: turn, costUsd: meter.spent, messages };
79    }
80
81    // Step 4. Run every tool it asked for, and keep one result per call, matched by id (20.4).
82    const results: Anthropic.ToolResultBlockParam[] = [];
83    for (const block of response.content) {
84      if (block.type !== "tool_use") continue;
85      const outcome = await tracer.toolCall(block.name, block.id, async () => {
86        try {
87          return await o.runTool(block.name, block.input);
88        } catch (error) {
89          // A failure is still a result. The model must hear about it.
90          return { ok: false, text: `The tool failed: ${error instanceof Error ? error.message : String(error)}` };
91        }
92      });
93      results.push({ type: "tool_result", tool_use_id: block.id, content: outcome.text, is_error: !outcome.ok });
94    }
95
96    // Step 5. Send the results back in one user message, results first, then call again.
97    messages.push({ role: "user", content: results });

New in lines 81 to 97.

• tracer.toolCall(block.name, block.id, async () => { … }): run the code inside the braces, and record it as one step in the trace (20.10). In short, it means "run this tool, and note that you did". Python writes the same thing as with tracer.tool_call(…) as span:. Its lines 96 and 97 mark the span as an error when the tool failed, so the trace shows which tool failed. (In TypeScript, the tracer does that marking itself, from outcome.ok.)
• await (Chapter 14): wait for something slow, here the connector, before going on.
• Line 90's error instanceof Error ? error.message : String(error): "if it is a real error, its message, and if not, whatever it is, as text". a ? b : c means "if a, then b, otherwise c".

The same steps in Python. If TypeScript is your language, this block is only for comparison. If Python is, this is the block to study, and the TypeScript above is for comparison.

78        # Step 2. Keep everything the model said, exactly as it came back, thinking included (20.5).
79        messages.append({"role": "assistant", "content": response.content})
80        answer = "".join(b.text for b in response.content if b.type == "text")
81
82        # Step 3. Stop on any reason except "wants a tool" (20.3).
83        if response.stop_reason != "tool_use":
84            return LoopResult(str(response.stop_reason), answer, turn, meter.spent, messages)
85
86        # Step 4. Run every tool it asked for, and keep one result per call, matched by id (20.4).
87        results: list[dict[str, Any]] = []
88        for block in response.content:
89            if block.type != "tool_use":
90                continue
91            with tracer.tool_call(block.name, block.id) as span:
92                try:
93                    outcome = await run_tool(block.name, block.input)
94                except Exception as error:  # a failure is still a result: the model must hear about it
95                    outcome = ToolOutcome(ok=False, text=f"The tool failed: {error}")
96                if not outcome.ok:
97                    span["error.type"] = "tool_error"  # the trace shows which tool failed (20.10)
98            results.append({"type": "tool_result", "tool_use_id": block.id,
99                            "content": outcome.text, "is_error": not outcome.ok})
100
101       # Step 5. Send the results back in one user message, results first, then call again.
102       messages.append({"role": "user", "content": results})

Questions worth asking the agent. Each answer is checked against the code:

• "Line 77 compares with tool_use. What happens to a stop reason this code has never heard of?" (It ends the loop, which is safe.)
• "Where is the tool actually run, and could the model run it without this line?" (Line 87, o.runTool. No.)
• "Line 93 builds a result. Where does its id come from?" (From the tool call's own id, block.id.)
• "Why does TypeScript use for and Python while True?" (Both count turns and repeat until a return. It is the same loop, written differently.)

Where the languages differ. messages.push and messages.append do the same thing. TypeScript's try { … } catch (error) { … } is Python's try: … except Exception as error: …. TypeScript writes !== for "not equal", and Python writes !=. TypeScript also gives the result a type from the SDK, Anthropic.ToolResultBlockParam. Python uses a plain dictionary instead, which the SDK accepts and checks.

End of part 1. Take a short break before part 2.

Part 2. Run it, and match the log (15 minutes)

Before you run it, re-read your two predictions from the top of the task: which line ends the loop, and how many entries the second request will hold.

Run npm run ask -- --api claude "Which invoices are due this week?", and show me each request and response in turn, saying which of the five steps produced it.

Figure 20.10. The two requests of Ana's run. After you run it, compare your own log with it, entry by entry. A1 stands for the long id you will see, such as toolu_….

What good looks like: two requests and two responses. Request 2's messages has three entries: the question, the model's entry (thinking and the tool call), and one result whose tool_use_id matches the call's id. The summary line reads stop: end_turn | turns: 2. If the model left page_size null, it got five invoices and a next_page, and asked once more. That gives three requests and turns: 3, which is also correct. Either way, the answer names ten invoices. Your prediction: line 77 decides (the if), and line 78 ends the loop (the return), or lines 83 and 84 in Python, and three entries.

Check it now. Download Task 3's answer key.

Modify (optional, 10 minutes). Ask the agent to change line 97 on a throwaway branch, a git branch the agent deletes afterwards (Chapter 15). The change puts a text block before the results: content: [{ type: "text", text: "Here you go." }, ...results]. That breaks 20.4's third rule, results first. Predict what the API says. Have the agent run it. Read the 400 and its message, then have the agent delete the branch. Expect a 400 saying that tool results must come first, right after the tool calls.

Task 4. Two calls in one turn, one failing (20 minutes, Codex)

Goal: you see one result per call, matched by id, with the failure returned as a result.

Predict: the model asks for two purchase orders, and one does not exist. How many results will the next request carry, and what will the failed one's error flag be?

In plain words. A question that needs two lookups at once, where one fails, shows how every call gets its answer.

Read 20.4. First run the recorded test only: npx vitest run -t "one result per call" and npm run test:loop:py -- -k one_result_per_call. Show me what each test checks in the second request it captured. Then ask live: npm run ask -- --api claude "Is INV-8301 ready to pay, and does INV-8316 match its PO?". Show me the response with the tool calls, and the next request's last entry.

What good looks like: in the test, two tool_result blocks with ids toolu_01Lab20B1 and toolu_01Lab20B2, the second with is_error: true. Live, the model may ask for both at once or one at a time. Both are correct: record which in ports/ch20.md. The answer says INV-8316 has no purchase order. Your prediction: two results, and true.

Check it now. Download Task 4's answer key.

Task 5. Cut a thinking block's signature (15 minutes, Claude Code)

Goal: you see that the conversation must go back unchanged.

Predict: turn 1 runs normally. Turn 2 sends the model's entry back with its thinking block's signature cut short. What comes back?

In plain words. The agent breaks the one rule the real loop never breaks, so you see the error once.

Read 20.5. Run npm run break -- cut-signature, then npm run break:py -- cut-signature. Show me what each printed. Then show me scripts/break.ts, lines 37 to 50, and explain what it changed, and what line 40 checks first. Last, show me line 73 of claude.ts (or line 79 of claude_loop.py), and explain why the real loop never has this problem.

The code. The part of scripts/break.ts that breaks the rule on purpose:

37  } else if (which === "cut-signature") {
38    // Turn 1 runs normally. Turn 2 sends the model's entry back with its thinking block's signature cut short.
39    const first = await client.messages.create({ ...base, thinking: { type: "adaptive" } });
40    if (!first.content.some((b) => b.type === "thinking")) throw new Error("No thinking block this time, so there is no signature to cut. Run it again.");
41    const call = first.content.find((b) => b.type === "tool_use");
42    if (!call) throw new Error("The model answered without a tool call. Run it again.");
43    const edited = first.content.map((b) => (b.type === "thinking" ? { ...b, signature: b.signature.slice(0, 20) } : b));
44    console.log(`turn 1: stop_reason ${first.stop_reason}, blocks ${first.content.map((b) => b.type).join(", ")}`);
45    console.log(`sending back: ${edited.map((b) => b.type).join(", ")}, with the thinking signature cut short`);
46    await client.messages.create({
47      ...base, thinking: { type: "adaptive" },
48      messages: [question, { role: "assistant", content: edited },
49        { role: "user", content: [{ type: "tool_result", tool_use_id: call.id, content: "[]" }] }],
50    });

Line 40 checks that there is a thinking block whose signature it can cut. .some( … ) means "is there at least one block that is thinking?", and ! turns that into "is there none?". Line 41's .find( … ) picks the first block that is a tool call. Line 43 is the break. .map( … ) makes a new list with one entry for each block. For the thinking block, { ...b, signature: … } copies the block, with its signature replaced by .slice(0, 20), the signature's first 20 characters. Every other block goes back as it was. Lines 44 and 45 only print a list of each block's type, using .map((b) => b.type).join(", "). Line 48 sends edited back where the real loop sends response.content unchanged. Two more pieces. question and base are set near the top of the file. They are the question, and the request settings every call shares. And ...base copies every field of base into the new request, before the code adds to it.

What good looks like: turn 1 shows a thinking block and a tool call. (If the script says there was no thinking block this time, the model chose not to think on that turn. Ask the agent to run it again.) The second call comes back with status 400, type invalid_request_error, a message that the signature in the thinking block is invalid, and a request id. Your prediction: a 400 error. If it came back without an error, record that in ports/ch20.md. The API's documentation says to expect the error, but no one has seen it live yet. Either outcome is a pass once it is recorded.

Check it now. Download Task 5's answer key.

Checkpoint 1: the loop. Before Sitting 2, check that you can do these without looking back.

• Name the loop's five steps, and point to the line in either language that ends it (20.1, 20.3).
• Say what a strict schema guarantees, and give an input it accepts that is still wrong (20.2).
• Say what happens if a tool call gets no result, and how the loop makes sure it never does (20.4).
• Say why the model's entry must go back unchanged, and which line makes sure it does (20.5).

Sitting 2: the same loop on the second API (about 1 hour 10 minutes)

Before you start, answer in a line each: What four things are different about a tool call on the second API (20.6)? What can you no longer see in a stateful run (20.7)? After this sitting, you can: explain the loop on both APIs, name every difference, and say what a stateful mode keeps for you.

Where you are. The second API now (openai.ts, or openai_loop.py in Python), with the same connector, route A or route B, and the same five steps.

If you left the second account for now (10 minutes, by hand, F1). Make the OpenAI API account, add a few dollars of credit, and set a $5 monthly spending limit. Make one key, and paste it into .secrets/openai.env as OPENAI_API_KEY= followed by the key. Then copy the pinned model's four prices into the openai row of src/loop/config.json (F5).

Task 6. The loop on the second API (45 minutes, Codex)

Goal: you can point to the five steps in the second API's loop, name each difference from the first, and run it.

Predict: in the second request, how many items will the input list hold, and in what order?

In plain words. The same work as Task 3, on the second API. Then one change, made on purpose, shows what reasoning items are for.

Read 20.6. Open src/loop/openai.ts and py/brightline_ap/loop/openai_loop.py beside claude.ts. Wait while I read. Then answer my questions, quoting the lines. When I say "run it", run npm run ask -- --api openai "Which invoices are due this week?" and show me each request and response, saying which step produced it.

The code. Steps 2 to 5 on the second API, in TypeScript. These excerpts are shortened, so they carry no line numbers. When you ask about a line, ask the agent to quote it with its number from the file.

New in this code.

• /* … */ in TypeScript and ... in Python mark lines left out here to keep the excerpt short. The full file has them. One of them sets answer, the text of the model's message items joined together. It comes just after step 2 (lines 74 to 77 of the TypeScript file).
• .filter((item) => item.type === "function_call"): keep only the items that are function calls.
• JSON.parse(…) and json.loads(…): turn a JSON string into an object the code can use.
• outcome.ok ? outcome.text : `ERROR: …` means "if the tool worked, its text, and if not, ERROR: and its text". Python writes it as a if outcome.ok else b.
• Step 2 adds every output item to the end of the list, whatever its type. TypeScript writes input.push(...list), where ... spreads the list into separate items, and as InputItem[] is a type for the type check. Python writes input_items += list(response.output).

    // Step 2. Keep every item the model produced, unchanged, reasoning items included.
    input.push(...(response.output as InputItem[])); // every output item goes back, unchanged

    // Step 3. Stop when the model asked for no tool, or when the response was cut short.
    const calls = response.output.filter((item) => item.type === "function_call");
    if (response.status === "incomplete") { /* … return, with the reason … */ }
    if (response.status !== "completed") return { stop: response.status ?? "unknown", /* … */ };
    if (calls.length === 0) return { stop: "done", answer, turns: turn, costUsd: meter.spent, input };

    // Step 4. Run each call. Arguments arrive as a JSON string, so parse them first.
    for (const call of calls) {
      const outcome = await tracer.toolCall(call.name, call.call_id, async () => {
        try {
          return await o.runTool(call.name, JSON.parse(call.arguments));
        } catch (error) { /* … an error result … */ }
      });
      // Step 5. One output per call_id. This API has no error flag, so the text says so.
      input.push({ type: "function_call_output", call_id: call.call_id, output: outcome.ok ? outcome.text : `ERROR: ${outcome.text}` });
    }

And in Python. As in Task 3, study your own language, and use the other for comparison.

        # Step 2. Keep every item the model produced, unchanged, reasoning items included.
        input_items += list(response.output)
        answer = ...  # the text of the message items, joined

        # Step 3. Stop when the model asked for no tool, or when the response was cut short.
        calls = [i for i in response.output if i.type == "function_call"]
        if response.status == "incomplete":
            ...  # return, with the reason
        if response.status != "completed":
            return LoopResult(str(response.status), answer, turn, meter.spent, input_items)
        if not calls:
            return LoopResult("done", answer, turn, meter.spent, input_items)

        # Step 4. Run each call. Arguments arrive as a JSON string, so parse them first.
        for call in calls:
            with tracer.tool_call(call.name, call.call_id):
                try:
                    outcome = await run_tool(call.name, json.loads(call.arguments))
                except Exception as error:
                    outcome = ToolOutcome(ok=False, text=f"The tool failed: {error}")
            # Step 5. One output per call_id. This API has no error flag, so the text says so.
            input_items.append({"type": "function_call_output", "call_id": call.call_id,
                                "output": outcome.text if outcome.ok else f"ERROR: {outcome.text}"})

Investigate: JSON.parse(call.arguments) and json.loads(call.arguments) sit inside the try. What happens when an open model writes broken JSON? (The error becomes an ERROR: output, and the loop goes on. A recorded test on the fixture open-malformed-args proves it: run it with -t malformed and -k malformed. It feeds the loop a function call whose arguments are not valid JSON, and checks that the next request carries an ERROR: output.)

Then modify. Predict first: if Step 2 kept only messages and function calls, leaving out reasoning items, what would the second request do? Then:

On a throwaway branch, change Step 2 in openai.ts so it keeps only message and function_call items, leaving out the reasoning items. Run the same question once. Show me what came back, then delete the branch.

Figure 20.13. The second request on each API. Check it after you run the task. The starred items are where a tool call travels differently, and A1 stands for the real call_… id.

What good looks like: the second request's input holds four items, in this order. They are the question, a reasoning item, a function_call with call_id call_…, and a function_call_output with the same call_id. Both requests carry store: false, the same instructions, and no previous_response_id. The summary line reads stop: done | turns: 2. For the change, record what you saw. The documentation says reasoning items must be passed back with tool outputs, so expect an error or a weaker answer. Which of the two comes has not yet been seen live. Either outcome is a pass once it is recorded. Your prediction: four, in that order.

Check it now. Download Task 6's answer key.

Task 7. The stateful contrast (25 minutes, Codex)

Goal: you see, in your own program's record, what a stateful run leaves out.

Predict: in the stateful run, what will the second call send?

In plain words. The same question with the provider keeping the history, so you can compare the two records.

Read 20.7 and F4. Run npm run stateful -- "Which invoices are due this week?". Show me each turn's line, then the record of what the program sent. Put it beside the second request's input from Task 6, running Task 6's command again if its output is gone. Then wait while I write three notes in ports/ch20.md on what the stateful request no longer shows. Then show me the line where the script deleted both stored responses (F4).

What good looks like: two turns. The second call's input holds one function_call_output, beside a previous_response_id, the instructions and the tools. The input tokens billed on turn 2 are about the same as in Task 6's turn 2, because the whole chain is billed as input. They should be within about a tenth either way, so compare the two input_tokens numbers. Your three notes name at least: the question, the reasoning item, and the function call with its arguments. Your prediction: only the tool's output and an id.

Check it now. Download Task 7's answer key.

Checkpoint 2: two APIs. Before Part C, check that you can do these without looking back.

• Name four differences between the two APIs' shapes, and the one thing they share (20.6).
• Say why the arguments are parsed inside the try (20.6).
• Say what a stateful mode saves, what it does not save, and what it hides (20.7).

Sitting 3: safe to run (about 1 hour 10 minutes)

Before you start, answer in a line each: Why is the model's own stop not enough (20.8)? Which errors should be retried (20.9)? What can a trace not prove (20.10)? After this sitting, you can: stop a loop that keeps going, sort an error and quote its request id, and diagnose a run from its trace.

Where you are. Both APIs, in your chosen language, with your connector, route A or route B. Three tasks, all on the core path.

Figure 20.17. The last two sittings. Each is short enough for one session, with a break and Part D between them.

Task 8. A loop that will not stop by itself (20 minutes, Claude Code)

Goal: you see your own limits stop a loop the model would not stop.

Predict: the turn limit is 6, and a tool always answers "try again". How many requests will the loop send, and what will its stop reason be?

In plain words. A recorded run where a tool keeps saying "try again" shows the turn limit and the cost cap at work, without spending anything.

The task has three short steps. Give the agent one prompt at a time. Steps 8b and 8c use the TypeScript command npm run ask. For Python, ask the agent to use npm run ask:py with the same options.

8a. The recorded tests (free, no live call).

Read 20.8. Run npx vitest run -t "try again", npx vitest run -t "cost cap" and npx vitest run -t "boundar", then the Python twins with npm run test:loop:py -- -k "turn_limit_and_cost_cap or boundary or allow_only". For each test, show me how many requests it expects and the stop reason. Then show me beforeTurn() in src/loop/limits.ts and before_turn() in py/brightline_ap/loop/limits.py, and explain the order of the three checks.

(-t and -k pick tests by part of their name. "boundar" is not a typo: it matches both "boundary" and "boundaries".)

The code. beforeTurn() in src/loop/limits.ts, lines 52 to 57:

52  beforeTurn(turn: number): LimitStop | null {
53    if (turn > this.limits.maxTurns) return "turn_limit";
54    if (this.spentUsd >= this.limits.maxCostUsd) return "cost_cap";
55    if ((this.now() - this.startedAt) / 1000 >= this.limits.maxSeconds) return "time_limit";
56    return null;
57  }

Each line checks one stop and returns its name. return null means "no stop, go on". this is the meter itself, and / 1000 turns milliseconds into seconds. this.spentUsd is the same running total that the loop reads as meter.spent in Task 3: one name inside the meter, one outside. Python's before_turn() has the same three checks in the same order.

You should see: the turn-limit test captures 6 requests and stops with turn_limit. The cost-cap test stops earlier, with cost_cap. The boundary tests show the cap crossed by the last call, and pay_vendor refused with "not allowed for this worker".

8b. A broken connector, live (route B only). On route A, skip this step: your own connector cannot be broken on purpose without changing it.

Give me the command to start a second, broken practice connector: PRACTICE_PORT=8761 PRACTICE_BROKEN=search_policy npm run connector:practice. I will run it in a third terminal and tell you when it is up. Then run CONNECTOR_URL=http://127.0.0.1:8761/mcp npm run ask -- --api claude --max-turns 4 "Is there an early-payment discount?" once. Show me each tool call and the last line. Then I will stop the second connector, with Ctrl+C in its terminal.

You should see: the run calls search_policy, gets "try again", tries again, and stops with stop: turn_limit | turns: 4. A model may instead give up early and answer that the policy tool is not working. That is also a pass: your limit was ready, and the model stopped first.

8c. A one-turn limit, live.

Run npm run ask -- --api claude --max-turns 1 "Which invoices are due this week, and do they match their POs?" and show me the last line.

You should see: stop: turn_limit | turns: 1, because the question needs more than one model call.

What good looks like: 8a, 8b and 8c as above (8a and 8c on route A). Your prediction: 6 requests, turn_limit.

Check it now. Download Task 8's answer key.

Task 9. Three errors, three request ids (20 minutes, Claude Code)

Goal: you can sort an error by whose fault it is, and find its request id.

Predict: for each of temperature, forced-tool and bad-key, write the status code you expect, and whether retrying would help.

In plain words. Three small calls, each wrong on purpose, so you read three real errors.

Read 20.9. Then do these steps one at a time, and wait for me after each.

1. Show me line 33 of scripts/break.ts, and explain why it needs a cast (a note that tells the type check "trust me") to send temperature at all.
2. Run npm run break -- temperature. Show me the status, type, message and request id it printed.
3. Run npm run break -- forced-tool, and show me the same four fields.
4. Run npm run break -- bad-key, and show me the same four fields.
5. Run one of the three again with npm run break:py, and show me that Python reports the same four fields.
6. Write the three request ids into ports/ch20.md.

The code. Line 33 of scripts/break.ts:

33    await client.messages.create({ ...base, temperature: 1 } as unknown as Anthropic.MessageCreateParamsNonStreaming);

The cast is as unknown as …. It tells the type check "trust me, this is a valid request". So the code compiles, and the API, not the type check, gets to reject temperature.

You should see, after each step: step 1, the cast on line 33. Steps 2 to 4, one error each, with a status, a type, a message and an id. Step 5, the same four fields from Python.

What good looks like: for temperature, a 400, invalid_request_error, and a message naming temperature. For forced-tool, a 400, invalid_request_error, and a message about tool choice on this model, rejected before the model runs. For bad-key, a 401, authentication_error. Each has a request id. Your prediction: 400, 400 and 401, and retrying helps none of them, because each is the request's own fault. If temperature or forced-tool does not fail, record it: the API may have changed since this chapter was checked.

Check it now. Download Task 9's answer key.

Task 10. Diagnose a run from its trace (30 minutes, OpenCode)

Goal: you find a missing step from the trace alone.

Predict: if the worker cannot search the policy, what will its trace be missing?

In plain words. Two runs of a policy question, one with the policy tool removed by mistake. You find the mistake by reading the trace before the answer.

Read 20.10. Run npm run ask -- --api claude --trace --quiet "What does our policy say about an invoice with no purchase order? Cite the line." and save trace.jsonl (written in the repository's top folder) as trace-good.jsonl. Show me each span's name and, for tool spans, its gen_ai.tool.call.id. Then make a throwaway branch, remove search_policy from tools/ap-tools.json, and run the same command. Show me only the new trace, not the answer, and wait for my diagnosis. Then show me the answer, and compare its citation with what search_policy returned in the good run. Delete the branch.

What good looks like: the good trace has chat, execute_tool search_policy, then chat, and the answer cites po-matching.md:9. The broken trace has one or more chat spans, and no execute_tool at all. Its answer either says it cannot check the policy, or cites a line it never looked up, perhaps the (po-matching.md:5) from the instructions' own example. Record which. Your prediction: an execute_tool search_policy span.

Check it now. Download Task 10's answer key.

Checkpoint 3: safe to run. Before Part D, check that you can do these without looking back.

• Name the three stops, and say why the cost cap can be crossed by the last call (20.8).
• Sort a 400, a 401 and a 529 by whose fault each is, and say which to retry (20.9).
• Say what a trace shows about a failed tool, and what it can never show (20.10).

Take a break here. Then read Part D, and do Sitting 4.

Sitting 4: the rest of the API, and ship it (about 1 hour 25 minutes)

Before you start, answer in a line each: When can a streamed tool call's input be parsed (20.11)? Which part of a request does caching make cheaper (20.12)? What five things does a loop pin (20.13)? After this sitting, you can: read a stream, see caching in the usage, run the loop on an open model, and ship it.

Where you are. Both APIs and an open model, in your chosen language, with your connector, route A or route B. Task 11 is on the advanced path. Tasks 12 and 13 are on the core path.

Task 11. A stream, and a cached turn (25 minutes, Claude Code, advanced path)

Goal: you see a response arrive in pieces, and the cache show up in the usage.

Predict: in the stream, when can the tool call's input first be parsed? And with caching on, on which turn will tokens first be read from the cache?

In plain words. One answer streamed event by event, then the same question run twice, once with caching, so you can compare the usage.

Read 20.11 and 20.12. Run the recorded streaming test (npx vitest run -t streaming), then npm run stream:demo -- "Which invoices are due this week?". Show me the event types in order. Then show me the lines in readClaudeStream() that handle content_block_stop, and the Python twin in read_claude_stream(), and explain them. Then run npm run ask -- --api claude --max-turns 5 "For each invoice due this week, check its purchase order." twice, once plain and once with --cache. Make a table, one row per turn, of input_tokens, cache_creation_input_tokens and cache_read_input_tokens for each run, and each run's total cost.

The code. In src/loop/stream.ts, lines 42 to 44:

42    } else if (event.type === "content_block_stop" && toolAt.has(event.index)) {
43      const json = partialJson.get(event.index) || "{}";
44      out.toolCalls.push({ ...toolAt.get(event.index)!, input: JSON.parse(json) }); // parse once, at the end

And in py/brightline_ap/loop/stream.py, lines 44 and 45:

44        elif event.type == "content_block_stop" and event.index in tool_at:
45            out.tool_calls.append({**tool_at[event.index], "input": json.loads(partial_json[event.index] or "{}")})

Until the block stops, the pieces only collect in partialJson. Line 44 (45 in Python) parses them once, when the whole input has arrived.

New in this code. toolAt and partialJson are lookup tables, and the block's number (event.index) is their key. toolAt holds each tool call's name and id, and partialJson holds the input pieces joined so far. .has( … ) asks "is this block a tool call?", and .get( … ) gets its entry. || "{}" means "or an empty object, if nothing arrived". The ! after .get( … ) tells TypeScript "this is definitely there". { ...x, input: … } in TypeScript and {**x, "input": …} in Python both mean "a copy of x, plus the input".

What good looks like: the stream shows message_start, then a content_block_start, deltas and a content_block_stop for each block, then message_delta with the stop reason and message_stop. The tool input is parsed at its content_block_stop. With --cache, turn 1 shows tokens written to the cache and turn 2 onward shows tokens read from it, and the total is lower. If no tokens are cached, the repeated start may be shorter than the model's minimum for caching (F5). Record it. Your prediction: at the block's stop, and turn 2.

Check it now. Download Task 11's answer key.

Task 12. Port to an open model (25 minutes, OpenCode)

Goal: the second API's loop runs, unchanged, against an open model on your own computer.

Predict: with a small local model, which is more likely to go wrong, the API shapes or the tool calls themselves?

In plain words. A local model server uses the second API's shape, stateless only. The same loop points at it and runs. You record what a small model does differently.

Read the open-model part of 20.14 (the paragraph on open models and Ollama) and F6. Install Ollama, check it listens on 127.0.0.1 only (the check should show the address 127.0.0.1:11434, not 0.0.0.0:11434), and pull the model pinned in src/loop/config.json as models.open. Run npm run ask -- --api open "Which invoices are due this week?" three times, and npm run ask:py -- --api open "Which invoices are due this week?" once. Show me, for each run, the tool calls it made, whether their arguments parsed, and the last line. Then record in ports/ch20.md what differed from the hosted models: tool choice, argument quality, missing fields, speed. Stop the model server at the end.

What good looks like: the loop runs with no change to the loop code. The only difference is the setting openModel: true, which leaves out the include line that asks for encrypted reasoning (F3). At least one run calls list_open_invoices and answers from its result. Some runs may call the wrong tool, write wrong arguments (which the connector may refuse, giving an ERROR: output) or answer without a tool. The cost is $0.0000, because the open model's prices are zero, and ask checks prices only for the two paid APIs. Your prediction: the tool calls. The shapes are the server's job, and it matches them.

Check it now. Download Task 12's answer key.

Task 13. Review and ship (35 minutes, Claude Code)

Goal: a reviewed pull request, merged by you, tagged ch20.

In plain words. You review everything the sittings used, with two reviewers, and merge it.

1. Show me every pin: the SDK versions in package.json and py/pyproject.toml, the anthropic-version header in a recorded test's request, the model ids and checked_on in src/loop/config.json, and the prompt's file name (20.13).
2. Run npm run typecheck, npm run typecheck:py, npm run test:loop and npm run test:loop:py, and show me the table.
3. Add docs/loop.md: the loop's five steps, each with its line in both languages, the three limits and their values, and the one-line invariant from 20.14 (the one rule that is always true of the loop).
4. Make sure ports/ch20.md has every prediction, every result and the three request ids from Task 9, and move trace-good.jsonl from Task 10 into ports/, which Lab 21 compares with.
5. Commit the work on a new branch lab20. Run /code-review and /security-review on it, and fix or answer every finding. An answer says, in one line, why the finding does not apply. Check especially: no key or token printed or committed, and no temperature, top_p or top_k outside scripts/break.ts and py/brightline_ap/experiments.py.
6. If this repository has no GitHub remote yet, make a private one with gh repo create brightline-ap --private --source . --remote origin, then push main and lab20. Open the pull request from lab20, with a short description of what changed, and show me both reviews. I will merge.
7. After I say it is merged, tag the merge ch20, push the tag, and in a clean clone of the tag, install as Task 0 did (npm ci, then cd py && uv sync --locked) and run the four checks of step 2.

You should see, step by step: Step 1, a short list of pins, one per line. Step 2, a table with four rows, all passing (32 TypeScript tests and 22 Python tests at the start of this lab). Step 3, a new docs/loop.md of about a page. Step 4, ports/ch20.md with no empty prediction. Step 5, two review reports, each finding fixed or answered. Step 6, a pull request link, which shows as merged once you merge it. Step 7, the four checks passing again from the clean clone, and git tag on that clone listing ch20.

What good looks like: both reviews have no open finding. The clean clone passes all four checks, with no key, because every test replays recorded responses.

Check it now. Download Task 13's answer key.

If something goes wrong

Apply it to your vertical (optional)

Pick a business you know where a worker would need two or three read-only tools. For example, a clinic's appointment book, a school's timetable, or a shop's stock list. Ask your coding agent to write the tools as JSON Schema, with every field required and no extras. Write one question that needs two of them in one turn. Then write, in plain words, the three stops you would set and why.

Exam notes

CCDV-F is the Claude Certified Developer foundations exam, and CCAR-F the Claude Certified Architect foundations exam. The note on the second API is from the book's OpenAI competency map.

• CCDV-F, debugging (primary): errors by kind, request ids, retries, and traces (20.9, 20.10).
• CCDV-F, API mechanics, configuration and SDKs (primary): streaming, usage and caching, versions, pinning and keys (20.11 to 20.13).
• CCDV-F, building agents (supporting): the loop itself (20.1 to 20.5).
• CCAR-F, agentic loops and session state (primary): the loop, stop reasons, tool results and limits (20.1, 20.3, 20.4, 20.8). CCAR-F (supporting): tool definitions and results (20.2, 20.4).
• The OpenAI competency map (primary): the second API's loop, state and the OpenAI part of 20.14 (20.6, 20.7).
• For the exam, know that the first API's newest version rejects temperature, top_p and top_k, and that forcing a tool is rejected on its newest models. If the official guide still teaches either as current, expect the exam to follow the guide.

Artifact checklist

• tools/ap-tools.json, prompts/ap-worker.v1.md, src/loop/config.json with prices and checked_on
• src/loop/claude.ts, openai.ts, tools.ts, limits.ts, trace.ts, stream.ts, and their Python twins in py/brightline_ap/loop/
• scripts/ask.ts, break.ts, stateful.ts, stream-demo.ts, and py/brightline_ap/ask.py and experiments.py
• tests/loop.spec.ts, tests/practice-connector.spec.ts, py/tests/test_loop.py, and tests/fixtures/loop/
• docs/loop.md, and ports/ch20.md with every prediction, result and request id
• Tag ch20