The adaptation step ran clean on every item I tested. Then I pointed it at the real catalog, and it started inventing product attributes.
My test inputs were items I had picked myself: full descriptions, consistent units, one language in and one language out. The pipeline looked finished.
Production was the long tail. Descriptions scraped into half-broken markup. Specs the seller truncated mid-sentence. Fields someone else had already machine-translated once. Listings where the only real text lived inside an image.
Nothing failed. That was the problem. Handed almost nothing to work with, the model filled the gap confidently, and those guesses landed in a live catalog as product facts.
The lesson was not about the model. I had sampled the easy head of the input distribution and measured the pipeline in the one place it was never going to break.
Now I build it the other way around. The ugliest inputs come first (shortest fields, lowest text-to-markup ratio, wrong-language detections) and those become the acceptance set. Before any model call there is a cheap pre-check: does the input even contain the thing I'm asking to transform? If it doesn't, the item goes to a quarantine queue for a human, not to the model. And an empty or near-empty source field is an explicit outcome in the pipeline, with its own status, instead of an invitation to be creative.
Agents don't degrade the way code degrades. Code throws. A model answers anyway.
If you run an LLM step in production, you probably already know what your pipeline does with garbage input: stop, or guess...