The endpoint everyone complained about was doing nothing wrong.
Balance checks were taking around 3 seconds in production. My own timing inside the handler said 90 milliseconds, consistently. For most of a day I looked at the wrong layer: query plans, an index I added that changed nothing, serializer overhead.
The 3 seconds was time before the handler ever ran.
A document-upload endpoint in the same service called a verification provider with no timeout. The provider hadn't broken. It had gotten slow. Every sync worker sat parked in that call, and everything else queued up behind them. Nothing in per-route latency showed it, because the clock inside a request only starts once a worker picks the request up.
What it cost me: a day of optimizing a healthy query path, plus an index I later removed.
What I do differently now. Every outbound call to a third party gets an explicit timeout, written when the call is written, and not in some hardening pass later. Endpoints that depend on someone else's availability don't share a worker pool with endpoints that answer from my own database. And I watch busy workers and queue wait as first-class metrics, because a per-route latency chart can look fine while the service is effectively full.
A slow dependency rarely shows up as a slow dependency. It shows up as your simplest endpoint getting slow.
So it matters where you measure request time from: the edge, or the handler.