polycratia

An API response went from 3 seconds to 200ms, and not one query got faster.

None of that time was computing. It was waiting for a database connection.

The endpoint wrapped the whole request in one transaction: the framework default, and the kind of thing nobody flags in review. Inside that transaction it called a payment provider over HTTP to confirm a charge, then wrote the result. Readable code. Also one database connection held open for the full length of somebody else's network call.

With a pool of a few dozen connections that behaves fine in staging, where you are the only caller. In production the arithmetic turns over: hold time times concurrent requests goes past the pool, and every new request now starts by queueing for a connection. Most of those 3 seconds were other people's provider calls.

So what changed was the scope of the transaction, not the speed of anything. The provider call moved out. What stayed inside shrank to the writes that actually needed atomicity: read state, call out, then open a short transaction to apply the result, guarded by an idempotency key, because once the call happens outside the transaction a retry can otherwise apply the same charge twice.

The part I'd keep: latency that only shows up under concurrency is a capacity number rather than a performance number. Profilers show you work. They don't show you waiting on a resource another request is holding.

I wrote up how I decide where a transaction should start and end around external calls over on my blog: https://polycratia.com/c/57629433

Probably worth checking where your own transactions end, at the last write or at the end of the request...

react

$ new-project --brief

or email hey@polycratia.com