Lesson 16.816.8 Watching it run: telemetry

In everyday life. A delivery company tracks every package. The scan at each stop says where the package was and when. A late truck shows up as a long gap between two scans. But the tracking never tells you whether the right item is inside the box. Only opening it does.

What you will learn. How to see a plugin at work: which steps ran, how long they took and what they cost. What that data cannot show. What must stay out of it.

Why it matters. People need to watch software they cannot sit beside, so you must know what this evidence proves, and what it does not.

What is telemetry?

Telemetry is data that running software sends about what it did, so people can watch it without sitting beside it. A plugin packages abilities, and telemetry is how you see them run. OpenTelemetry (opentelemetry.io, checked November 2, 2026) is the open standard for it, not tied to any one company. A product that follows it can send its data to any collector, a service that receives telemetry, and any tool can read it.

What does it send?

Three main kinds of signal:

Each signal carries attributes, name and value pairs such as a skill's name. An exporter sends the signals, to a collector or simply to the console, which means your terminal. A host that exports telemetry can report when a plugin loaded and which skill ran. It can also report each model request with its tokens and estimated cost, each connector call, and each tool it allowed or denied. A host may also export the plugin version it loaded, though most do not. Products differ in how much they export (16.10).

Can one trace cross programs?

Yes, if both sides take part. OpenTelemetry's semantic conventions are shared names for the same things, so every product's data reads the same way. Its conventions for MCP, the standard for connectors, are at Development status, so the names may still change. They also say how a connector call joins the agent's trace. The agent passes trace context, a small identifier, inside each connector request. A connector that reads it adds its own spans under the agent's. 16.10 lists the names.

What can the host not see?

What happens inside a program. The host can time a connector call or a script, but it cannot see inside, such as which rule each invoice matched. Only code you own can add spans of its own. Brightline's connector is ready-made, so in this chapter it is seen only from outside. Chapter 17's connector is yours, and you instrument it there, which means you add code that sends its own telemetry.

Does a clean trace mean the answer is right?

No. Telemetry shows that work ran, never that it was right. A trace can show a skill that ran fast, cost little and had no errors, and still printed a wrong total. Correctness comes from tests and evaluations (16.3, 16.6).

What must stay out of it?

Prompts, replies and tool content, which can hold customer names, amounts and bank details. Most products leave them out of telemetry unless you turn each one on. Leave them off, and send telemetry only to a collector your organization controls.

At work. The builder turns on one product's telemetry for one session, sends it to the console, and runs the Monday report. The log events show the plugin loading and a skill being used, and the cost records show the run's tokens and cost. The product hides the skill and plugin names until the builder asks for them. NOTES.md says that the evidence for the counts 4, 2, 2 and 5 is the test.

The operating rule: export telemetry to the open standard so any tool can read it. Keep sensitive content out of it, and instrument the code you own. Never take a clean trace as proof of a right answer.

Figure 16.12. One triage run as a trace. The host times what it can see. Only the test says the answer is right.

Practice it. Practice: watching it run, in your own tool, ten minutes.