Skill · job · document-agent

Document / RAG agent

Question-answering over a private corpus with citations in the schema. Use for RAG, LlamaIndex, 'chat with PDFs', knowledge bases. The hard problem is the corpus; orchestration is secondary. Retrieved text is untrusted.

GET /api/canon/skills/document-agent?format=md

LlamaIndex (and equivalents) own indexes and query engines. A ReAct loop over raw PDFs stuffed into context is context hoarding.

When

Users ask questions over private documents and you must not hallucinate a missing page.

Do

01
Index the corpus properly
Chunk, metadata, access control. Prefer a document framework over a blank loop.
02
Citations are schema
Every answer names pages/ids. Missing page → refuse, do not invent.
03
Label retrieved text
Untrusted data. Injection cases in the eval set.

Don't

Stuff six PDFs into the window 'just in case'.
Skip evals because the demo cited the right paragraph once.

Hard rules

Hallucinated citation is a failing eval, not a nit.
Permissions on the corpus are executor problems, not prompt problems.

Refuse

Context hoarding — Raw HTML, full CI logs, six PDFs, entire JSON envelopes stuffed 'just in case'. The goal is the first thing forgotten.
Eval by demo — Architecture first, golden set never. Success is a recorded GIF.
Security by system prompt — 'Never delete files' as the only guard. The model is a confused deputy and will try to help.

Load with this

Job · Document / RAG agent Playbook · Design an agent Playbook · Write evals first Playbook · Own the window Playbook · Defend injection Retrieval and Document Agents Own Your Context Window Injection, Poisoning, Confused Deputy Evaluation

Load next

Trigger tests

Should fire

“Chat with our policy PDFs”
“RAG agent with citations”

Should not

“Public web search with no corpus”