polycratia

On my laptop the migration ran clean in under a minute. In production it stopped partway through, on rows older than the code that was reading them.

I was moving an inherited service off Python 2. My local database was convenient in a way I had never examined: every row in it had been created by the new code. Text was text. Optional columns were populated, because my own write path populated them every time. The data agreed with my assumptions because my assumptions had written it.

Production data had history. Rows written years earlier by the old runtime. Rows imported by a parser that does not exist anymore. Rows from a schema version where that column was genuinely nullable. Some held bytes where the new path expected a decoded string.

The cost was not the crash itself. The job had no resume point, so it left part of the table in the new shape and part in the old, and the code meant to tolerate both shapes was the thing I was in the middle of deploying.

So now, before any migration that touches data, I start with a read-only census: null counts per column, which types and encodings are actually present, how far back the oldest writes go. Then I make the job resumable, idempotent per row, and able to quarantine a bad row instead of aborting the batch.

Fixtures I write myself only test my code against itself.

I keep longer notes on legacy modernization work like this on my blog: https://polycratia.com/c/e6282887

If you check one thing this week, check the oldest row in your main table, and which version of your code wrote it...

react

$ new-project --brief

or email hey@polycratia.com