Lesson 14.414.4 Data models and schemas

In everyday life. A loading dock checks each delivery at the door: the labels, the paperwork, any damage. A broken pallet is refused there, with a note on the delivery slip. Nobody wants to find it three days later on a shelf, after it has been counted, priced and sold. The dock does not open every box, so a box with the right label and the wrong goods inside still gets through.

What you will learn. How to write down what a valid record looks like, so the program checks every record that comes in, and what those rules let through.

Why it matters. Your tests cover the rows you can see. Written rules for a good record, checked on every record as it comes in, also cover next month's rows, which no test has seen.

What is a schema?

A schema is a written description of what valid data looks like: which fields, which types, which values. A schema validates data: it checks it, and either returns the data, now trusted and typed, or returns a list of what is wrong. A data model is the type of a valid record, often produced from the schema so the two can never disagree. In TypeScript, Brightline uses a library called Zod. In Python it uses Pydantic. Types are not checked while the program runs, so in 14.2 isTerms checked one field by hand as the text came in. A schema checks every field of a record that way, from one written description.

A schema is a deterministic engine. A schema applies the same rules to every record, every time. It never gets tired, and nobody can persuade it to skip a rule. That is why accounting rules, such as "amounts are whole cents, zero or more", belong in a schema or in tested code, not in an instruction to an AI worker. The agent can write the schema. The schema enforces the rule.

In TypeScript. An excerpt from src/schemas.ts, then one bad invoice. z.enum is Zod's own way to list a fixed set, not the TypeScript enum that 14.3 says Node refuses. { ...inv8304, amountCents: -23450 } is a copy of INV-8304 with its amount changed to -23450. Predict which field the first problem names.

import * as z from "zod";

export const InvoiceSchema = z.object({
  number: z.string().regex(/^INV-\d+$/, "must look like INV-8301"),
  invoiceDate: z.iso.date("must be a real date, YYYY-MM-DD"),
  dueDate: z.iso.date("must be a real date, YYYY-MM-DD").nullable(),
  terms: z.enum(TERMS, "must be one of the known terms"),
  amountCents: z.int("must be whole cents").min(0, "must be zero or more"),
  // ...vendorId and status
});

const result = InvoiceSchema.safeParse({ ...inv8304, amountCents: -23450 });
console.log(result.success, result.error?.issues[0].path, result.error?.issues[0].message);

false [ 'amountCents' ] must be zero or more

safeParse never throws. It returns success, and either the typed data or the list of issues.

In Python. An excerpt from schemas.py, then one bad invoice, copied from INV-8310 the same way with **. A Pydantic model is a class whose fields are the schema. try and except catch the error so it can be printed, as 14.5 explains.

class InvoiceModel(BaseModel):
    number: str = Field(pattern=r"^INV-\d+$")
    invoice_date: date
    due_date: date | None
    terms: Terms
    amount_cents: int = Field(ge=0)
    # ...vendor_id and status

try:
    InvoiceModel.model_validate({**inv_8310, "invoice_date": "2026-13-02", "terms": "Net 10"})
except ValidationError as err:
    for e in err.errors():
        print(e["loc"], e["msg"])

('invoice_date',) Input should be a valid date or datetime, month value is outside expected range of 1-12
('terms',) Input should be 'Due on receipt', 'Net 15', 'Net 30', 'Net 45' or 'Net 60'

model_validate raised one ValidationError holding both problems, and errors() lists them, each with the field it is about.

Why check at the edge?

Because every record passes there, on every run. An edge is wherever data comes in: a file, an API, a form or another program. Validate once, at the edges you marked in 14.1's practice, and the code inside can rely on what it holds. That is true as long as the schema is right and nothing later changes the data without checking it again. Code that checks deep inside, a little in each function, misses cases and repeats itself. Code that never checks is Marcus's script.

On the day nobody reads the code, the schema still reads every record.

At work. Ana decides the schema's test cases first, from rows she can see, and the agent writes them as tests. One good invoice, INV-8301, passes. Each kind of bad value is refused: a negative amount, a fraction of a cent, unknown terms and month 13. Then she asks the agent to add Zod and Pydantic, and to write the schemas that pass. She compares every rule with the table above, and checks that the lock files changed in the same commit.

What can a schema not check?

Two kinds of rule. The first kind needs more than the one record the schema is checking. The invoice schema here looks at one record at a time, and it does not have the vendor list. So it cannot know that vendor V099 does not exist. Schemas can describe lists and records inside other records too, and some can check several fields together, but only with the data they are given. Rules that compare records, or compare a record with other data, are code you write after the schema. Concept 14.5 shows where they go.

The second kind is a wrong value of the right shape. A schema checks shape, not meaning. If INV-8301's amount arrives as 48250 instead of 482500, it is whole cents, zero or more, and it passes. Your count and total from Chapter 12 catch it, and so does a test that names the right amount. The edge does not replace your check. It stops what has the wrong shape, on every run.

Does the library change values for me?

Pydantic does, by default. Zod does not. Pydantic is like a clerk who tidies an amount before she checks it. She trims the spaces, and reads "4825.0" as 4825. Zod sends back any amount that arrives as text.

Where the languages differ.

The second row matters. Pydantic converts some values for you. This is convenient for CSV files, and it is also a risk. By default it accepts " 482500 " with spaces, "4825.0" as 4825, and even True as 1. Pydantic's own strict mode, not the type checkers' setting in 14.2, turns those conversions off. Zod converts nothing.

The safe pattern in both languages is controlled conversion, then validation. The loader, src/load-invoices.ts, which reads the file and checks each row, turns text into a number only when the text is a plain whole number. It passes anything else on as a value the schema will refuse. In JavaScript, Number("") and Number(" ") are both 0. So a loader that calls Number on every cell turns a blank amount into $0.00.

Brightline's own code has that bug. Every invoice row becomes a record in invoiceFromRow, which converts the amount with Number and checks nothing. due-by-friday, the list of Friday's payments, reads its invoices through it. If next week's export leaves INV-8301's amount blank, the list still shows 8 invoices, with INV-8301 at $0.00, a total of $14,404.53, and no warning. It is the same kind of bug as Marcus's. Read which conversions happen before you trust a schema.

How does a schema become JSON Schema?

Both libraries can write their schema out in a standard format called JSON Schema, a description of valid data that other programs read. Chapter 17 uses it to describe the inputs of a tool an agent can call. Chapter 24 uses it to make an AI model's answer follow a fixed shape. The schema you write here is the one those chapters build on.

Here is the terms and amountCents part of the invoice schema, written out by z.toJSONSchema. Read it against the table above: the allowed terms, and an amount that is a whole number, zero or more.

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "type": "object",
  "properties": {
    "terms": {
      "type": "string",
      "enum": ["Due on receipt", "Net 15", "Net 30", "Net 45", "Net 60"]
    },
    "amountCents": {
      "type": "integer",
      "minimum": 0,
      "maximum": 9007199254740991
    }
  },
  "required": ["terms", "amountCents"],
  "additionalProperties": false
}

required lists the fields that must be there. additionalProperties: false tells other programs to send no other fields. Zod itself does not refuse an extra field when it checks a record: it drops it, and the record passes. The large maximum is the largest whole number TypeScript can hold exactly.

The operating rule: validate every record where it enters the program, with a schema whose rules you have read. Keep the rules that compare records in code right after it, and keep your own count and total for the wrong values a schema lets through.

Figure 14.5. Check at the edge. Inside the program, every record has already passed the schema. The amounts start as text, and the loader turns plain whole numbers into numbers before the schema checks them.

Practice it. Practice: data models and schemas, in your own tool, fifteen minutes.