I had a test set for an LLM catalog step: a small batch of products I had read end to end, with the output I expected sitting next to each one. It passed. Then it went live behind the parsers, and the thing I had actually been testing disappeared.
In testing I knew the right answer every time. In production that answer does not exist.
The adapted listing comes out fluent and well-formed, and confidently wrong about a dimension. No reviewer sits on the row, and there is no expected value to diff against. Nothing throws. The queue drains. The dashboard is green.
And what broke it was the input, not the model. My test items were clean because I had picked them myself. Real catalog rows come off scrapers with truncated titles, HTML fragments, mixed-language source text, units glued to numbers. The agent did not refuse any of it. It smoothed it over, which I think is the worst behaviour available here.
So I stopped testing outputs and started testing invariants that have to survive the step. Model numbers, size tokens, units, the attribute set: those go in and must come out unchanged, character for character. If a dimension vanishes or a unit shifts, the item goes to quarantine instead of the catalog. The model gets to be creative with prose and with nothing else.
That changed what passing means for me too. The gate before release is now a sample of real production rows, including the ugly ones, and not a curated set I assembled myself.
I wrote up more on how I structure these pipelines on the blog: https://polycratia.com/c/f351a84f
If you run agents in production, I would like to know your detection signal for output that is wrong but well-formed.