# 20.7 What a stateful mode hides (/building-ai-workers/agent-loop-by-hand/what-a-stateful-mode-hides) --- type: Document title: "20.7 What a stateful mode hides" description: "What a stateful mode keeps on the provider's side, what it saves and does not save, and what you can no longer see." status: stable order: 320.07 ksor: owner: team:panaversity audience: [ public ] approval: by: process:panaversity at: 2026-11-06T10:53:06Z chapter: "20" part: III expert_status: builder-only concepts: [ "20.7" ] last_verified: 2026-11-05 generated: at: 2026-11-06T10:53:06Z by: process:panaversity trust_tier: unverified build_id: sha256:21db26ecbf7fc84b043f434d194e035ffe0481398b00a039c42d852801046bca dirty: true ksor_version: 0.0.60 --- **Where this fits.** This is [20.5](the-conversation-is-an-array-you-own.md)'s array, kept by someone else. Some APIs offer to keep the conversation for you, so your program sends only what is new. This concept shows what that saves, and what you can no longer see. **In everyday life.** Two clinics treat the same patient. The first hands the visiting doctor the whole paper folder. The second keeps the folder in a back room, and the doctor gets a number to quote: "continue from visit 42". The second is lighter to carry. But when the doctor wants to know exactly what the last doctor wrote, the answer is in a room she cannot enter, and the clinic charges for every page read aloud. In this concept, the paper folder is your conversation array, and the number is a previous-response id. **What you will learn.** What changes when the provider keeps the history for you, what that saves, and what you can no longer check. **Why it matters.** A convenience that hides what the model saw also hides the cause of a bad answer. Knowing what it keeps lets you choose it on purpose. **One request, start to finish.** Ana's question runs again on the second API, this time **stateful**: 1. **Your code sends the first request** with storage on. The provider stores the response and gives it an id. 2. **The model asks for a tool,** as before. Your code runs it. 3. **The second request sends only two new things:** the tool's output, in its input list, and the id of the previous response, in a field of its own. No question, no reasoning, no function call. (The instructions and tools still go with it, as on every request.) The provider joins them to the history it kept. It still bills that stored history as input on every call, like the clinic's charge for each page read aloud. 4. **The model answers in words.** 5. **Your code prints what it sent.** Call 1: the question. Call 2: one function call output and an id. The model's function call and its reasoning are nowhere in your program's record. They live under that id, on the provider's side. The run worked. But your own log can no longer show what the model saw. ### What does a stateful API keep for you? A **stateful** API keeps the conversation on the provider's side between calls, and each new request names the earlier response instead of resending it. The second API offers this with a previous-response id, and also as a longer-lived conversation object (a saved conversation the provider keeps for you, with its own id). The first API has a stateful option too, in beta (still being tested) at the time of writing: message threads, which keep a thread for a few minutes. Both save you from resending the history. Here is what that trade costs: | | Stateless (this chapter) | Stateful | | --- | --- | --- | | **What you send each call** | Everything, every time | Only what is new, and an id (on the second API, plus the instructions and tools, which go on every call) | | **What you can see** | The whole conversation, in your own array | Only what you sent. The rest is on the provider's side | | **What your tests can replay** | Every request, exactly | Only your half. The provider's half must be fetched or trusted | | **What it costs** | Every turn pays for the whole history (caching helps, [20.12](tokens-caching-and-what-a-call-costs.md)) | The same. On the second API, each call is billed again for every input token of the earlier calls. Stateful saves sending. It does not save paying | | **Where the data lives** | On your computer, for as long as you keep it | On the provider's servers, for 30 days by default for a stored response, longer for a conversation object | | **What carries over** | Whatever you send | Not everything: on the second API, the earlier request's instructions are not carried over | ### Why does this chapter teach the stateful mode last? A stateful mode is not wrong. It is convenient, and for a chat app it can be the right choice. But it hides the very thing this chapter teaches: what the model sees on each call. Learn the loop with the whole array in your hands. Then, when you choose a stateful mode, or an SDK that uses one (Chapter 21), you know exactly what it is keeping for you, and what you can no longer inspect. ![Two halves. Left, stateless, your array: four gold items, the question, the reasoning, the function call and the function call output, labeled every item in your own log, on your computer. Right, stateful, what you sent: one box holding the function call output and the previous id, with an arrow down to a dashed box on the provider's side holding the question, the reasoning and the function call, labeled stored 30 days by default, still billed. Below both: stateless shows you everything, stateful shows you what you sent.](img/chapter-20-hidden-array.png) *Figure 20.12. Your array against the provider's hidden one.* **At work.** In [Lab 20](lab.md), Task 7, your coding agent runs `npm run stateful` with Ana's question. At the end it prints everything your program sent, call by call, beside the full input list from Task 6. You write down, in `ports/ch20.md`, three things the stateless run showed that the stateful one does not. **The operating rule:** keep the conversation yourself unless you have a reason not to. When you choose a stateful mode, write down what it keeps, where, for how long, and what you can no longer see. **Part B in one minute.** The second API runs the same loop with different shapes: items instead of blocks, a call id, arguments as a string you parse, reasoning items you send back, and "no calls" for done ([20.6](the-same-loop-different-shapes.md)). With storage off, the array is still yours. A stateful mode keeps the history on the provider's side, which saves sending but not paying, and hides what the model saw (20.7). Now do [Sitting 2](lab.md#sitting-2-the-same-loop-on-the-second-api-about-1-hour-10-minutes).