Vol. III · Single-Agent Patterns · III.4
Evaluator–Optimizer
One model generates, another critiques against explicit criteria, loop until the critic accepts or the budget dies. The missing unit test of generation.
6 min read
Doctrine
- Write the rubric first. If you cannot score it, you cannot loop it.
- The evaluator should not have the same prompt as the generator.
- Cap the iterations. Diminishing returns arrive fast.
Where it shines
Translation, code review, retrieval completeness. MetaGPT's QA engineer is this pattern with a job title. DSPy teleprompters automate the outer loop when you have a metric. promptfoo is the batch version.
Anti-patterns
- A critic that only says 'make it better' with no checklist.
- Generator and evaluator sharing weights and prompt — they collude.