polycratia

A cache with a near-zero hit rate runs slower than no cache at all. That is where a lot of those mysterious three-second endpoints actually live.

The bug is not in the cache layer. It is in the key.

Someone wraps an expensive call (a delivery rate quote, a catalog page, a pricing lookup) and builds the key out of the whole incoming request. Session id, cursor, a timestamp, a locale that arrives in two spellings. Alone, each field is harmless. Together they make every request unique, so every request misses, pays the full upstream cost, then pays again to store a value nobody will ever read.

Dashboards hide this well: calls to the cache are happening, no errors show up, and the latency chart just shows a slow endpoint. So attention moves to the query, and the query is fine.

When something is slow and a cache sits in the path, I check key cardinality before I touch the database. Distinct keys per hour against requests per hour tells you the truth in one number. If the two are close, what you built is a write-only store.

The fix is almost never a faster upstream. It is deciding what the response actually varies by, and accepting that it varies by less than the request does.

So which field smuggled itself into your cache key: locale, auth context, or a timestamp?

Longer write-up, including how I separate identity from inputs in a key: https://polycratia.com/c/7ffdc97c

react

$ new-project --brief

or email hey@polycratia.com