An agent that passes your tests has been tested on one thing: the run where every tool call came back fine.
More test cases will not close that. It is structural. In a clean environment the dependencies answer, the schemas line up, the provider is not rate-limiting you, so the failure branches do not execute and nobody wrote them.
Production hands them over on day one. A timeout on a call that actually committed. A 429 halfway through a fan-out. A stale read. A provider answering 200 with an empty body.
The mechanic that turns that into silent damage: in most agent setups a tool failure goes back to the model as text. The error is just another token in the context, and the model's job is to keep going plausibly. So it keeps going. It retries a write that was not idempotent, or swaps in an approximation, or calls the task done.
You have handed error handling to the one component that cannot be relied on to fail the same way twice.
What I do instead: raw failures do not reach the model. Deterministic code above the loop classifies every tool result as retryable, fatal, or degraded, and only that classification enters the context. Fatal stops the run with a structured outcome. Degraded gets recorded as degraded, not smoothed into prose.
The rule I hold to: the model may decide what to do next. It may never decide whether something worked.
I write up more about how I build these pipelines here: https://polycratia.com/c/97c3ce35
If you run agents against real systems, I'd like to hear which failure mode slipped past your tests first...