Job pack · supervisor
Production agent product
Durable reducer, gateway from anywhere, traces, evals, HITL, owned prompts. The last 20%.
When: Someone will pay, wait, or sue. The demo is not the product.
Name the five: termination, sandbox, context builder, eval set, pause API. One blob. One orchestrator. Gateway not a text box.
GET /api/canon/jobs/production-agent?format=md
Playbooks
- Ship durableThe reducer over a blob, with pause, resume, traces, and evals. This is production.
- Add a human gateApproval and missing info are tools. Pause is an API. The blob survives the wait.
- Write evals firstThe golden set is the product specification. Architecture waits on it.
- Defend injectionUntrusted speakers everywhere: pages, PDFs, MCP descriptions, other agents. Executor enforces.
- Pick a frameworkPick for the constraint you have. Steal modular concepts. Do not rewrite the company around a README.
Doctrine to load
- The Twelve FactorsTwelve engineering constraints that make LLM-powered software reliable enough for paying customers. Not a framework — a checklist you can apply to any stack.
- Durable ExecutionAgents run longer than request timeouts. Checkpoint after every turn. Resume as a first-class API. LangGraph won production on this axis.
- The Agent as a Stateless ReducerFactor 12: (state, event) → (newState, effects). The model is a pure function over a context you serialize. Everything else is infrastructure.
- Unify Execution and Business StateFactor 5: there is one serializable blob. The ticket is the agent state. A second 'agent memory' that drifts from the business object is how you lose the thread and the customer.
- ObservabilityA trace is the loop laid out in time: prompts, tool I/O, token counts, costs, user id. Without it you cannot eval, debug, or bill.
- Trigger from AnywhereFactor 11: the loop does not live in one text box. Mail, chat, tickets, cron, webhooks, and IDE events are all legal starts. The agent is a reducer over events, not a chatbot with extra steps.
- Anti-PatternsThe field's repeating failures, collected from the 12-factor journey, Anthropic's warnings, and every framework README's implicit 'please do not'.
Refuse
- HITL that cannot resume. The human answers and the run is gone. Approval as a side channel, not a tool.
- Double side effects. Non-idempotent tools, no journal. A retry refunds twice.
- Unowned prompt. The policy lives in a framework black box you cannot diff, test, or revert.
- Unbounded loop. No max-turns, token, wall-clock, or money budget. The agent rambles until the bill or the context dies.
- Eval by demo. Architecture first, golden set never. Success is a recorded GIF.
Checklists
Ready to build
- Named termination: done schema, max turns, token/money cap.
- Sandbox / allowlist for every side effect. Secrets in the executor.
- An explicit context builder: pinned, working, recalled, compacted.
- A golden set of 20–50 real tasks, run in CI on prompt change.
- Launch / pause / resume APIs. HITL is a tool that checkpoints.
Security review
- Retrieved docs, web, and MCP descriptions are labeled untrusted data.
- MCP servers pinned, hashed, reviewed. No marketplace auto-trust.
- Secrets injected at the executor, never into the prompt or the workspace dump.
- Subagents are reduced principals. No parent deploy keys.
- Allow / ask / deny. Irreversible actions are high-friction asks.
Eval quality
- 20–50 real tasks, not toys. Include the failures you have already seen.
- CI on prompt, tool schema, and skill changes.
- Written rubric. If you cannot score it, you cannot loop it.
- Generator and evaluator do not share prompt or incentive.
Recipes
Default corpus
langgraphinngestlangfusepromptfoomastra