Skill · production · observability
Trace the loop
Trace every LLM and tool hop (Langfuse, OpenTelemetry). Use when debugging agents, shipping production, distilling traces into skills/evals, or the user cannot see why the agent spent $4 and did the wrong thing.
GET /api/canon/skills/observability?format=md
Spans around llm and tool. Cost, cache, and latency are design constraints. Traces feed evals and skills.
When
You cannot improve what you cannot see. The loop must be inspectable.
Do
- 01
Span every hop
Prompt, tool name, args (redacted), observation (compact), tokens, USD, latency.
- 02
Show the loop to humans
The operator sees the path, not a spinner. Streaming the visible loop is part of the ACI.
- 03
Export into evals
Failed traces become golden cases. Good traces become skills.
Don't
- Log secrets or raw PII into the trace store.
- Debug production agents by reading stdout on one box.
Hard rules
- If you cannot name the last tool call, you do not have observability.
- Cost and cache prefixes are product metrics.
Refuse
- Eval by demo — Architecture first, golden set never. Success is a recorded GIF.
- Memory as unfiltered logs — Session transcripts stuffed into the next session. The user cannot inspect or delete them.
Load with this
Load next
Trigger tests
Should fire
- “Add Langfuse to the agent”
- “Why did this run cost $4?”
- “We can't see the tool calls”
Should not
- “Add Google Analytics”