The test inputs for my catalog pipeline were a dozen records I had typed by hand: clean fields, nothing merged, no stray markup.
Then production fed it a scraper.
The agent step (translate and adapt a product listing for another market) never failed once. It just got a record where the spec block had collapsed into the description, and it wrote fluent, confident copy around the merged text. Dimensions turned into prose. A packaging note turned into a product feature.
Nothing threw. Some of those outputs read better than the correct ones.
The cost: a batch of published listings describing something that didn't exist, caught by a human reading the storefront, not by anything in the pipeline.
What I had missed is that a model does not error on malformed input. Plausibility is its default behaviour, and broken input comes back as plausible output at exactly the same confidence as good input. In testing I was the input source, so that failure mode had no way to show up.
Now the model boundary has a contract. Fields get validated and normalized before the call (expected units, no markup, every attribute in its own slot), and a record that fails the contract is parked for review instead of passed through. The model gets a narrow job on data I can describe.
I wrote more about the parser-to-model boundary here: https://polycratia.com/c/223cf7b3
If you run a model behind a scraper or an OCR step, the thing worth checking is what happens to a record your pipeline can't parse cleanly: it gets parked, or the model gets to guess...