polycratia

A moderator hit approve, and the queue took about three seconds to come back. Every time, and only on that one screen.

The search query was fast. The database write was fast. The three seconds sat in between.

The write path pushed the record into Elasticsearch and then waited for the document to become visible before returning, so the reloaded queue would show the item as already handled. That wait has nothing to do with your document or your query: it depends on how busy the index is.

Catalog parsers were bulk-indexing against the same cluster. The one-document moderation write queued up behind their refresh work. Latency on a compliance screen had turned into a side effect of a scraper schedule living in an entirely different service.

What it cost: moderators paced themselves to the UI instead of to the work, and every query-time dashboard looked healthy, because the query was never the slow part. Nobody measured the wait for visibility, only the calls around it.

What I do differently now: read-your-writes does not go through the search index. The endpoint returns the authoritative row from Postgres, indexing happens asynchronously, and the screen reads freshness from the store that owns the record. Search stays with what it is good at, lists and filters, and not with proving that the action you just took landed.

The broader habit out of this one: instrument waits, not just calls. A span around "index the document" would have shown me this in an afternoon. A span around "search" never will.

I write up more of these production post-mortems here: https://polycratia.com/c/0ae66f78

If you run search alongside a write path, I'd look at where your UI gets its freshness guarantee from: the index, or the database.

react

$ new-project --brief

or email hey@polycratia.com