The output read perfectly. A clean translation of a product title with a spec table glued onto the end of it.
My LLM pipeline adapts retail catalogs for other markets. In testing it looked solid: I fed it real product records, read the results, checked terminology and tone. The model did its job every time.
Then a source site changed its layout. The scraper kept returning a value for the title field, it just wasn't a title anymore. Fluent, well-formatted localized garbage went into the catalog, and nothing in the pipeline objected, because every check I had was on the output.
Testing can't show you that part. Output quality is conditional on the input being the kind of thing you built for, and nothing in my stack was checking that condition. In testing I was the condition: I picked the inputs by hand.
So now the contract sits in front of the model instead of behind it. Shape checks on every field before a prompt is ever built: length ranges, expected language, and the markers a real record carries. Anything that fails goes to a reject queue instead of to the model.
With a tool-calling agent this matters more, because it doesn't only produce text: it writes.
The model is allowed to be wrong. It is not allowed to be the first component that notices something upstream is broken.
I wrote up more on where these pipelines actually break here: https://polycratia.com/c/72c27c10
If you run agents in production, I'd want to hear your first check on the input, before the prompt gets built...