Lesson 14.614.6 JSON and REST APIs

In everyday life. A restaurant's takeout window has a menu taped beside it. You ask for item 12 with no onions, and you get it in a bag with a receipt that lists what is inside. You never see the kitchen. The menu, the way you ask, and the receipt are all you see. So you check the bag before you leave, because the kitchen can change a recipe without telling you.

What you will learn. How to read a request to another program and the answer that comes back. How to act on each kind of answer on purpose, fetch the whole result, keep secrets out of the code, and check what arrives.

Why it matters. The program on the other side belongs to someone else, who can change it without telling you. So your program checks every answer before it uses it.

What is JSON?

JSON is a text format for records. { } holds a dictionary and [ ] a list, text goes in quotes, and numbers, true, false and null are written plainly. JSON has no dates, so a date travels as text, as in {"number": "INV-8306", "dueDate": "2027-03-08"}. It is how most programs send records to each other.

What are the parts of a request?

An API, an application programming interface, is a way for one program to ask another for something. A REST API does it over the web, with one address, or URL, for each kind of thing. Each of those addresses is an endpoint. At the takeout window, the menu is the list of endpoints, your order is the request, and the bag with its receipt is the response.

Pages. An API rarely returns thousands of records at once. It returns a page of them and tells you how many pages there are. A client, the program that sends the requests, has a common bug if it reads only the first page. The bug gives no warning.

At work. Brightline's repository includes a small sample REST API that runs on your own computer and serves Brightline's sample data. It needs the key from .env, serves five invoices per page, and is slow on purpose, so you can see the effect of the async code in 14.7. Ana first asks the agent, "Before we write any code, could you send the sample API a few single requests and show me what comes back, with no key, with the right key, and for page 2 of the open invoices?" No key gives 401, the right key gives 200, and page 2 of the open invoices starts at INV-8306.

In TypeScript. fetch makes a request, and await waits for the answer, which 14.7 explains. localhost in the address means your own computer. process.env.AP_API_KEY reads the key from the environment, as Where does the key go? explains, and ?? "" uses empty text if there is none. The agent starts the API first, with npm run api. The last line uses JSON.stringify, which turns a record into JSON text. Predict each line.

const response = await fetch("http://localhost:4010/invoices?status=open&page=2", {
  headers: { "X-API-Key": process.env.AP_API_KEY ?? "" },
});
console.log(response.status, response.headers.get("content-type"));
const body = await response.json();
console.log(body.page, body.pages, body.invoices.length, body.invoices[0].number);
console.log(JSON.stringify({ number: "INV-8306", amountCents: 158200, dueDate: "2027-03-08" }));

200 application/json
2 3 5 INV-8306
{"number":"INV-8306","amountCents":158200,"dueDate":"2027-03-08"}

In Python. httpx is the library this book uses to send web requests. Its params builds the query string for you.

import json
import os

import httpx

response = httpx.get(
    "http://localhost:4010/invoices",
    params={"status": "open", "page": 2},
    headers={"X-API-Key": os.environ["AP_API_KEY"]},
)
print(response.status_code, response.headers["content-type"])
body = response.json()
print(body["page"], body["pages"], len(body["invoices"]), body["invoices"][0]["number"])
print(json.dumps({"number": "INV-8306", "amountCents": 158200, "dueDate": "2027-03-08"}))

200 application/json
2 3 5 INV-8306
{"number": "INV-8306", "amountCents": 158200, "dueDate": "2027-03-08"}

Where the languages differ. TypeScript's null and Python's None both become JSON null. JSON.stringify writes compact JSON, and Python's json.dumps adds a space after each colon and comma. Both are valid JSON. response.json() in TypeScript must be awaited, because the body may still be arriving. In Python's plain httpx.get, it is ready.

What should the client do for each status code?

Act on each one on purpose: read the body, fix the request, check the key, or wait and try again. Neither fetch nor httpx.get raises an error for a 401 or a 503. Each hands back the answer with its code, so a program that ignores the status code reads an error message as if it were invoices.

Why did the tests pass and the live run fail?

Because the tests and the code shared one picture of the data, and the API had its own. An API is an edge you do not control. Its owner can rename a field, change a format or fail for a day without telling you, and your code finds out on its next run. Only the live answer shows what the API really sends, and the schema at the edge is what reads it.

At work. Ana's Python client passes every test against a fake API, a stand-in that answers like the real one, which the agent built for it. It uses the field names she expected, such as vendor_id. Then she has the agent run it against the sample API for the first time. It is refused on the first invoice: the schema reports vendor_id, invoice_date, due_date and amount_cents as missing. This API's JSON names fields in camelCase, such as vendorId, and Python code uses snake_case, such as vendor_id. Her tests could not show this, because the fake API used her names. The schema caught it at once, before any invoice was used. She compares the names in the error with the JSON the agent fetched by hand earlier, and asks for the model to accept both names. The client prints 8 vendors and 13 open invoices, in about one second.

Where does the key go?

Into an environment variable, never into the code. An API key is a secret that proves who is calling. It is like the PIN for the company card, kept apart from the card, and treated as known once it has been written down anywhere. An environment variable is a named value the operating system gives a running program, usually loaded from a .env file that git ignores. Never put a key in code, in a test or in a commit. If a key is ever committed, treat it as leaked, even if you delete it in the next commit, because git keeps every commit.

The operating rule: read an API call by its method, endpoint, query, headers and status code, and handle each status code on purpose. Read every page, keep keys in the environment, and validate the body with a schema before you use it, even when your tests pass.

Figure 14.7. One request, one response. Every part has a label, and every label is something to check.

Practice it. Practice: JSON and REST APIs, in your own tool, fifteen minutes.