polycratia

The parser kept running. Prices kept updating. Nothing errored, and nobody paged me.

A retail source had started redirecting requests to a different regional storefront, based on where it thought the client was sitting. My scraper followed that redirect without complaint. Same page structure, same selectors, numbers in exactly the shape I expected.

Different currency.

The bug was that a price in my pipeline was a number. Just a number. Currency lived somewhere upstream as a property of the source, configured once and then assumed forever. So the parser did its job perfectly and wrote confidently wrong rows for days.

The expense was not in the wrong prices. Everything downstream had already consumed them: delivery quotes, comparisons, what customers saw. Re-running the parser fixes the catalog, and it still does not tell you which decisions were made on the bad version, and it does not restore anyone's trust in the number on the page.

Two things changed in how I build parsers after that.

A price does not cross a boundary as a bare number anymore. It carries its currency and its minor-unit scale, and anything that receives it has to accept both or reject the row. If a value can be misread, I make the misreading impossible to express.

And the parser now asserts the identity of what it actually fetched before writing anything: final URL after redirects, locale, the currency marker on the page. A source is allowed to change its mind about who I am. It is not allowed to do that silently.

The cheap version of this check is a per-source sanity hold: if today's price distribution for a source shifts beyond a band, stop writing and flag it. It catches the whole family of bugs where the scrape succeeds and the meaning changes.

I write up more of these failure patterns from production systems here: https://polycratia.com/c/fa736460

If you run scrapers or third-party integrations at scale, the thing worth locating is probably the point in your pipeline where you would notice that a source quietly started answering a different question than the one you asked...

react

$ new-project --brief

or email hey@polycratia.com