№ 01 — Oct 2026
My agent saw the price change and still didn't tell anyone
In Sutra's real-model eval, a small free model correctly spotted a price change on a page, then skipped the alert and never saved it to memory. The code was fine; the model just dropped steps. That's why the evals now separate code bugs from model mistakes, and why the important rules (allowed tools, fixed recipients, step and cost caps) live in code instead of the prompt.
- AI Agents
- Evals
- Guardrails