The agent passed every test I wrote. Then it went live against the real catalog feed and started confidently describing products that did not exist.
Not hallucinating from nothing. Hallucinating from garbage.
My fixtures were product descriptions I had picked by hand: clean text, one language, complete attributes. Production input came from retail catalog parsers, and parsers break quietly. A truncated field. Leftover HTML. A size chart flattened onto one line. Two languages in the same description. A unit that changed without anyone noticing.
The model never objected. It has no concept of "this is not a product description." Whatever arrived, it produced something fluent and plausible, and fluent-and-plausible is what a reviewer skims past...
The cost was pulling a batch of listings back and reprocessing them. What stung more was losing the assumption that output could be sampled instead of checked.
What I do differently now: the model sits in the middle of the pipeline, never at the edge. A gate in front decides whether an input is even eligible to be processed. Hard post-conditions sit behind it: numbers, units, SKUs and required attributes have to survive the transformation unchanged, and anything that fails gets routed to review instead of published. My test inputs are sampled from live parser output now, not written by me.
The uncomfortable part is that my test set was not too small. It was too clean.
If you run an LLM step in production, I'd want to know what you assert about its output that does not depend on a human reading it.