polycratia

The query logs were clean. Every statement under a few milliseconds. And the checkout page still took seconds to show shipping options.

This was a cross-border shopping platform. The endpoint rendering delivery choices did something that reads as harmless in code: for each carrier, ask for a rate. Each call was a separate HTTP request to a separate provider, sequential, each one waiting on someone else's infrastructure. The database was never the bottleneck. It was barely involved.

What made it hard to see: every instrument I had pointed inward. Query timings, ORM counts, process CPU. All healthy. The time was going somewhere none of the dashboards described: waiting.

Then a carrier had a bad afternoon. Their rate endpoint went from fast to merely slow, not down, just slow. My workers filled up holding connections open to them. Requests that had nothing to do with delivery started queueing behind the ones that did. A third party's degradation became my outage, and I found out from users instead of from monitoring.

What I do differently now.

I treat any outbound call on a request path as a latency budget item with a name, not an implementation detail. Every integration gets a timeout chosen deliberately and a bounded retry, because an unbounded wait is a decision to let someone else own your availability.

Anything that can be precomputed or cached stops being synchronous. Rates that change slowly do not need a live call per page view.

Where the work genuinely has to happen live, it goes to a queue and the client polls or gets pushed the result, so a slow partner costs one job slot instead of one worker.

And I log the resolved duration of each external call per attempt, separately from total request time. The gap between those two numbers is the part of your latency nobody owns, and it is usually where the seconds are.

The pattern generalises past delivery. Payment providers, verification vendors, pricing feeds. Anywhere your process stops computing and starts waiting, your database speed stops being the relevant question.

For those of you running APIs with heavy third-party dependencies: I'd like to know whether you enforce latency budgets per integration, or handle it at the edge with one global timeout.

react

$ new-project --brief

or email hey@polycratia.com