Lesson 20.820.8 The stops you add yourself

Where this fits. These checks run before step 1, every time around the loop. So far the loop ends when the model says it is finished. But what if it never does? This concept adds stops that your program controls.

In everyday life. A company card for a sales trip has three limits: a number of nights, a dollar total and a return date. The traveler may come home early when the work is done, and usually does. But if a client keeps saying "come back tomorrow", the card still stops at the limit, and the traveler calls the office. In this concept, the traveler's own "work is done" is the model's stop, and the card's three limits are your turn limit, cost cap and time limit. This concept covers those three stops first, then two other kinds of limit.

What you will learn. Which stops your own program adds to a loop, besides the model's own "I'm finished", and which kinds of limit can be crossed.

Why it matters. A model can keep asking forever, and every round costs more than the last. Your own stops hold even when the model has been fooled.

One request, start to finish. In a test, the connector's search_policy tool is broken: whatever it is asked, it answers "try again". Ana asks: "Is there an early-payment discount?"

1. Turn 1. The model asks for search_policy. The tool says "try again".
2. Turns 2 to 6. The model, reasonably, tries again, with the same or slightly different words. Each time, the same answer. Each turn sends the whole growing conversation and costs a little more than the last.
3. Before turn 7, your loop checks its limits. It has made six model calls, and the turn limit is six. It stops, with the reason "turn limit", and keeps whatever answer it has.
4. Your log shows six requests, six tool calls for the same search and the stop reason, so the broken tool is obvious.

Without that check, the loop would have gone on until the model gave up, if it ever did.

Why is the model's own stop not enough?

The model's own stop ("finished") is the stop you hope for. It is not enough on its own, for three reasons: a tool can fail in a way that invites endless retries, a fooled or confused model can keep asking (fooled, for example, by text hidden in a document that tells it to keep going), and every turn costs more than the last, because the conversation grows. So the loop checks three stops of its own, before every model call:

• A turn limit: at most so many model calls in one run. The lab uses 8 for a question, and 6 in this example.
• A cost cap: the run's cost so far, worked out from each response's usage (20.12) and the model's published prices, must be under a cap. The lab uses 10 cents.
• A time limit: the run's time so far must be under a limit. The lab uses 2 minutes. It is checked between calls, so one slow call is ended by the request deadline below, not by this.

Figure 20.14. Four of the ways a loop ends. Only the first is the model's choice.

Can a run go over its cost cap?

The cap is checked before each call, so the last call can go slightly over it. The loop checks the cost before each model call. A call already under way finishes and is paid for, so a run can end slightly over its cap. Set the cap with that in mind.

What are budgets, deadlines and boundaries?

The three stops are one kind of limit, budgets, for the whole run. Two other kinds follow. A deadline is for one call. A boundary is about which tools may run at all. For example, a budget says "no more than 10 cents for this run". A deadline says "give up on any one call after 60 seconds". A boundary says "this worker may never call pay_vendor". It matters which is which:

A budget limits how much. A deadline limits how long one call may hang. A boundary limits what can happen at all, and it is the only one that keeps a fooled model from causing an effect. Chapter 24 builds more boundaries, and Part IV puts them in a governed layer.

Where does the cost cap get its prices?

A cap is only as good as its prices. Cost is worked out from token counts and the prices you copied from each provider's pricing page (F5). If the prices are wrong, so is the cap. The lab's ask refuses to run until the prices are filled in, because a cap on tokens priced at zero never stops anything.

Is the cost cap only about money?

This is a safety control as well as a cost control. Chapter 17 taught that tool output is untrusted: a vendor's note could say "check every invoice again". The turn limit and the cost cap hold even when the model has been fooled. Chapter 22 hands the cost cap to a hosted platform, which enforces it for you.

At work. In Lab 20, Task 8, your coding agent runs the recorded test where search_policy always says "try again", and shows you the six captured requests and the turn_limit stop. It also runs two edge tests: the cost cap crossed by the last call, and a pay_vendor call refused by allowOnly(). Then it shows you RunMeter.beforeTurn() in src/loop/limits.ts and before_turn() in py/brightline_ap/loop/limits.py, and you find the three checks.

The operating rule: give every loop a turn limit, a cost cap and a time limit, checked before every model call, a deadline on every request, and an allow-list of tools. Never rely on the model alone to stop.