polycratia

Every dependency in the request was healthy. The endpoint was still slow.

Both of those were true at the same time.

This is where a lot of latency work goes wrong. You open the database dashboard: queries are fine. You open each downstream service, and each one reports a reasonable average. Nothing looks broken, and users still wait.

A user request does not experience averages. It experiences the slowest thing on its own critical path.

I hit this over and over on shipping and checkout endpoints in cross-border commerce. One call would read the catalog, then fan out to several carriers for live rates, then touch payment and currency logic before it could answer anything. Each integration was fine on its own. But once a single request depends on six things, the chance that at least one of them is having a slow moment stops being an edge case. The tail latency of that endpoint ends up much closer to the maximum of its calls than to any average you happen to be staring at.

So I stopped measuring components and started measuring the request. One timeline per request, every external call on it, ordered, waiting time visible. That view usually answers the question in minutes, and what it turns up is almost always something nobody suspected: a serial chain of calls that could have run in parallel, or a retry policy quietly tripling one leg, or an enrichment step nobody needs before the response goes out.

The fixes are unglamorous. Give every external call an explicit time budget instead of the default timeout it inherited from some library. Run independent calls concurrently instead of chaining them. And decide up front what the response looks like when a dependency does not answer in time, because a degraded answer (a cached rate, a narrower list) beats holding a request hostage to the slowest participant.

Latency is a property of the composition, not of the parts. That is why a fast database and a slow API are not a contradiction.

I write up more of these production trade-offs here: https://polycratia.com/c/7234d751

The last time you chased a slow endpoint, the time was probably hiding somewhere you weren't looking...

react

$ new-project --brief

or email hey@polycratia.com