An API does not take three seconds. Three seconds is the average of two code paths, and usually neither of them takes three seconds.
That is why so many latency investigations stall. You profile an endpoint with a cheap path and an expensive path, and the mean lands somewhere no real request ever lived. Every line looks mildly guilty. Nothing looks responsible.
The mechanics are boring. One branch gets served from something already computed: a cached result, a stored match, a materialized row. The other computes it right there. Payment attribution makes this very visible. An incoming transfer that matches deterministically on requisites is almost free, while an unmatched one kicks off a candidate search across time windows, partial amounts and near-duplicate references. Same endpoint, same payload shape, two different cost classes.
So latency moves when nothing in the code changed. A bank file arrives dirtier than usual, the unmatched share rises, the mean drifts up. You read that as a performance regression, and it is a traffic-mix change. Optimizing the code in that window is work spent on the wrong object.
Before I measure time, I split traffic by branch (hit or miss, matched or unmatched, first page or deep page) and measure each population on its own. A fast path with a stable tail plus a slow path you can name is a fixable system. One blended number is not a question yet.
I wrote up more on how I approach this kind of debugging here: https://polycratia.com/c/b02bc493
In your hottest endpoint there is probably a branch nobody measures separately...