polycratia

A fast database doesn't give you a fast API. In most slow endpoints, query time is the smallest number in the trace.

The time goes to everything the handler waits on that isn't your database: a carrier rate lookup, a verification provider check, a payment gateway status call, a file landing in object storage. Each of those is a network hop with a tail latency you don't control and can't tune.

Then fan-out multiplies it. Quote delivery across several carriers one after another and your endpoint's floor becomes the sum of their slow cases, not the average. The database answers in milliseconds: the request spends the rest of its life on someone else's retry.

The other half is that nothing in the request path has a deadline. A client library ships with no timeout, one call hangs, a worker stays occupied, the pool drains, and now requests that touch nothing external are queueing behind requests that do. Latency gets contagious across endpoints that share workers.

How I deal with it: every outbound call gets an explicit timeout smaller than the endpoint's budget, the whole request gets one shared deadline, fan-out runs in parallel instead of sequentially, and anything that doesn't have to be synchronous moves behind a queue with a status the client can poll. Cache what is cacheable. Return partial results instead of blocking on the slowest provider.

The part I'd keep: an endpoint inherits the latency of the slowest thing it waits on synchronously. So profile the waiting, not the querying. A flame graph of your own code won't show you a provider's handshake.

I write up more production trade-offs like this one at https://polycratia.com/c/dbe78c24

Your worst latency probably lives in one of two places: an external dependency, or contention on your own workers...

react

$ new-project --brief

or email hey@polycratia.com