Lesson 16.316.3 Writing a skill with the expert

In everyday life. A family wants Grandma's soup recipe written down before she moves. She knows the soup. Her grandson knows how to write a recipe a stranger can follow. She says "a handful of barley, never the quick kind". He writes "1/2 cup pearl barley (not quick-cooking)" and cooks it twice to check. Neither could write it alone.

What you will learn. How the person who owns the rules and the person who builds the files split the work. How to write the one line that decides when an agent loads the work. How to test it.

Why it matters. The person who knows the rules rarely builds the files, and the person who builds them does not know the rules. Each one's mistakes are the kind the other one catches.

Who writes what?

A good skill needs two kinds of knowledge. This split is what co-build means.

The expert's words become the skill's references. The expert's cases become the tests, written first and failing for the right reason, exactly as in Chapter 13. The tests' expected values come from the rules sheet only, never from what the code happens to print.

How does an agent know when to load it?

From the description. It is the trigger, the line that makes the agent load the skill. The agent sees only names and descriptions before it decides to load a skill (16.1). So the description must answer two questions: what does this do, and when should it be used. Some products cut long descriptions in their skill list, so the main use goes first.

• Too vague: "Helps with invoices." It can load for any invoice question, including ones it cannot answer.
• Too narrow: "Run triage.py on invoices-intake.csv." It can miss someone who says "sort this week's batch".
• Right: "Sort new vendor invoices into the AP team's four piles using a tested script. Use when someone asks to triage, sort or check a batch of new invoices. Never use it to decide whether to pay."

The last sentence matters. It says when not to use the skill, which keeps it from loading for a payment decision it must never make.

What goes in the body?

Short numbered steps. Name the script for every number, and say "copy the numbers it prints, never recompute them". Say what to do when an input is missing: ask, do not guess. Give one example of a request and the expected shape of the answer. Put long material in references/, one level below the skill, and point to it by its relative path.

Here is Brightline's skill, shortened:

---
name: invoice-triage
description: Sort new vendor invoices into the AP team's four piles using a tested script. Use when someone asks to triage, sort or check a batch of new invoices. Never use it to decide whether to pay.
metadata:
  version: "0.1.0"
  policy: AP policy v3
---

1. Find the new batch, the vendor list and the AP export. If one is missing, ask.
2. Find the run date. If the person gives none, ask. Do not assume today.
3. Run the tested tool: `npm run triage -- <batch.csv> <run date>` in the repository, or `scripts/triage.py` anywhere else. Never sort or count by reading the file yourself.
4. Report each pile exactly as the tool printed it. Copy the numbers, and never recompute one.
5. End with who acts: exceptions to the AP lead, large amounts to the controller, urgent invoices to the AP clerk today.

Never say an invoice should or should not be paid. Never edit the batch, the vendor list or the AP export.

The rules are in `references/ap-triage-rules.md`. Read them to explain a pile.

How do I test a skill?

By asking two questions.

1. Does it load when it should, and only then? Write a few requests that should load it and a few that should not, and record what happened. "Triage the new batch" should load it. "What is our controller's name?" should not. If it loads wrongly, fix the description, not the steps.
2. Does it follow the rules on known cases? Run it on a batch whose answer you already know, from the expert's cases. The counts must match the tested script's output. If the agent's report differs from what the script printed, the skill's wording is letting the agent improvise, which means make up its own answer.

Models vary from run to run, so run each check more than once, and keep the requests and results in a file. That file becomes the skill's repeatable test cases (16.6).

At work. Ana writes docs/ap-triage-rules.md. It has six rules, and the first one that matches wins: an unknown vendor, a possible duplicate, terms that do not match the vendor's, an amount over the controller's limit, a due date within three days, and everything else. They put each invoice in one of four piles: exception, needs approval, urgent or ready to pay. She also writes six cases she checks by hand. The sorting rules are hers. The limit in them belongs to Dave Kowalski, the controller, who owns AP policy. Four of the cases test two boundaries from both sides. An invoice of $5,000.01 needs the controller, and one of $5,000.00 does not. An invoice due three days after the run date is urgent, and one due four days after is not. The builder turns each case into a failing test, then asks the agent for src/triage.ts, the engine, the tested code that does the sorting. Then the builder writes the skill. When Ana reads the skill, she asks for one change: the report must end with who acts on each pile. That line was missing because the builder did not know who acts. Only Ana did.

The operating rule: the expert owns the rules and the cases. The builder owns the files and the tests. Nobody releases a skill until it loads when it should and its numbers match the script.

Figure 16.5. The expert's sheet and the builder's files. Every rule has a test, every case has an expected value, and every "never" has a line in the skill, plus a setting where it must hold.

Practice it. Practice: writing a skill with the expert, in your own tool, fifteen minutes.