Lesson 20.620.6 The same loop, different shapes

Where this fits. This is 20.5's array that you own, in another shape. Everything so far used one model service, the first API. Other companies offer models too, and their APIs look different. This concept runs the same five steps on a second API, so you see what changes and what stays the same.

In everyday life. You rent a car of a brand you have never driven. The pedals are in the same order, and the steering wheel turns the same way. The rules of the road are the same too: you still stop at a red light. But the headlight switch sits where your own car has the wipers, and the screen's menus are laid out differently. A good driver is a good driver in both cars, once they have learned where the controls are. In this concept, the rules of the road are the loop, and the layout of the controls is each API's JSON. Put simply: the same rules, in a different layout.

What you will learn. How the same five steps look on a second API, and the four differences in how a tool's request and its answer travel there.

Why it matters. Most model services copy one of these two shapes. Learn the loop once, and each new API is a matter of new names.

One request, start to finish. Ana asks the same question on the second API: "Which invoices are due this week?" Here is the same loop, as neutral sketches:

1. Your code sends the first request: the instructions, the input list (just Ana's question), the four tools, and a storage setting turned off, so the provider keeps no copy of the response.
2. The model answers with a list of output items: a reasoning item (its thinking, encrypted, that is, locked, so you can pass it back but not read it) and a function call item with a call id, A1, a name, list_open_invoices, and its arguments as a string of JSON text.
3. Your code adds both items to its input list, unchanged, then parses the arguments string into an object (turns the text back into named values the code can use), and runs the tool.
4. It adds one function call output item, with the same call id and the tool's result as text.
5. It sends the second request: the instructions again, and the whole input list, now four items long. (On the first API it was three entries, because the model's thinking and its tool call sat together in one entry. Here, the reasoning and the call are separate items.)
6. The model answers with a message item, in words, and no function calls. Your code sees no calls and ends the loop.

Same five steps. Same three decisions in your code. Different names for everything.

What changes between the two APIs?

The second API calls its pieces items where the first calls them content blocks, and its conversation is one flat list of items rather than a list of entries, one per speaker. Side by side, in plain words:

The four rows marked ★ are the four differences in how a tool call travels. Learn those four. The lab's checkpoints ask you to name them. The other rows are for looking up.

These two shapes are two products' versions of one idea. They are not a standard, and no standard sits between them. The loop is what they share.

Figure 20.11. One loop, two shapes. Learn the loop once, then learn each API's names for it.

How does this chapter keep the second API stateless?

Stateless on purpose. The second API can keep the conversation on the provider's side, and by default it stores each response for a while. This chapter turns that off, with a setting called storage (store in the API), set to false, and never names a previous response. So the input list is yours, as on the first API: your code builds it, sends it whole every time, and can print, log and test it. 20.7 shows what the other way hides.

Which habits matter on the second API?

Three habits that matter on the second API.

• Parse the arguments, and catch a failure. The arguments come as text. A well-behaved model writes valid JSON, and strict mode guarantees the shape. An open model on your own computer may not (Task 12). The lab's loop parses inside the same try that runs the tool, so a broken string becomes an error result, never a crash.
• Send back every reasoning item. Without storage, the reasoning that led to a function call must go back with the call's output. If you leave it out, the request may be rejected, or the model may reason worse on the next turn. Task 6 asks you to predict which, then try it.
• Send the instructions every time. Nothing is kept for you, so the instructions go with every request. (Even in the stateful mode of 20.7 they are not carried over.)

At work. In Lab 20, Task 6, your coding agent shows you the second API's loop file, in TypeScript and in Python, beside the first API's. You find the five steps in each, and name the line that parses the arguments. Then it runs Ana's question on the second API.

Is every agent built on these two shapes?

Nearly. The first and second APIs are the two main ways programs talk to language models, and most other model vendors offer an API compatible with one of them. Agent libraries and hosted platforms run this same loop on top of them, out of sight. 20.14 names the two APIs and says more.

The operating rule: learn the loop once, and each API's shapes separately. Keep the second API stateless too, parse every arguments string inside the error handling, and send every item back unchanged.