Why a scheduled JWKS refresh skips the throttle and still spends it
A resource server that verifies JWTs against a remote key set has two reasons to go fetch that key set: a token arrived naming a key id the server does not hold, or the server's own ticker came round. Those two triggers want opposite throttling rules, and they share one piece of state: the time the issuer was last asked. Conflate them and you get one of two bad outcomes, either a key set that never catches a rotation, or an issuer that anyone can drive into the ground by minting tokens with invented key ids.
I hit this writing the JWKS cache in mcpaudience, the resource-server half of MCP authorization. The resolution fits in one function signature. The interesting part sits elsewhere: in what the two triggers are forced to share.
The gate belongs to the arriving token#
A token shows up with kid: rsa-9. The server does not hold rsa-9. There are two explanations and the server cannot tell them apart without asking: either the issuer rotated and the copy in memory is behind, or nobody ever published that key id at all. That asymmetry is the whole problem. Minting a token with a random kid costs a caller nothing, and it does not even have to be signed by anything real, because the signature gets checked after the key is found. The fetch it provokes costs an HTTPS round trip to the authorization server. Without a floor, an unauthenticated caller turns one cheap loop into sustained load at the issuer, and the first thing that breaks is token verification for every legitimate request.
So the request path gets a floor: MinRefresh, 30 seconds by default. Not a cache TTL. A minimum gap between two attempts to reach the issuer.
The ticker's period is already its bound#
The scheduled fetch has none of that exposure. Nobody outside the process can make it happen more often, because the only thing that triggers it is a ticker the server configured. Its period is the rate limit, and the way to ask the issuer less often is to lengthen the period. Put a second gate underneath it and a configured period shorter than MinRefresh silently becomes MinRefresh: two knobs, one of them lying.
Which leaves one decision. The fetch has to know who asked. In jwks.go that is a named boolean, not a flag threaded through four call sites:
// trigger is what asked for a fetch, which is what decides whether MinRefresh
// has anything to say about it.
type trigger bool
const (
arrivingToken trigger = false
ownSchedule trigger = true
)
func (j *JWKS) refresh(ctx context.Context, kid string, asked trigger) (*KeySetRefresh, error) {
now := j.now()
if since := now.Sub(j.attempted); asked == arrivingToken && !j.attempted.IsZero() && since < j.minRefresh() {
return nil, tooSoon{since: since, interval: j.minRefresh()}
}
// Recorded before the fetch rather than after it, so an attempt that fails
// or is abandoned spends the interval too.
j.attempted = now
fetch := &KeySetRefresh{KeyID: kid, At: now}
body, err := j.get(ctx)
// ...
}The two callers differ in exactly two arguments. The request path reaches it through the key lookup, with the kid the token named and arrivingToken. The schedule reaches it with no kid at all:
func (j *JWKS) scheduledRefresh(ctx context.Context) (*KeySetRefresh, error) {
j.mu.Lock()
defer j.mu.Unlock()
return j.refresh(ctx, "", ownSchedule)
}The empty key id is not a placeholder. It is reported as-is through OnRefresh, which is how an operator reading the fetch log tells a rotation the server went looking for from one a token dragged out of it: KeyID set means some request wanted that key, KeyID empty means the ticker.
What both kinds share is the record of the attempt#
This part took me a second pass to get right. The gate is skipped for the scheduled fetch, but the line above it, j.attempted = now, is not. Every fetch records its attempt, whoever asked for it.
That is deliberate, and it follows from what the request path is actually reading. It is not reading who asked. It is reading when the issuer was last asked, because that is the only thing that bounds load at the issuer. A scheduled fetch that went out two seconds ago has already spent that round trip, and a token arriving now with an unknown key id has no claim to another one. So a tick holds the request path off for the whole interval, exactly like a token-triggered fetch would:
// The attempt it recorded is what holds the request path off: a key id the
// refreshed set does not name does not start a fetch of its own.
clock = now.Add(2 * time.Minute)
if _, err := set.Key(context.Background(), "rsa-9"); err == nil {
t.Fatal("a key id nobody published was answered")
}
if got := issuer.count(); got != 2 {
t.Errorf("the issuer was asked %d times after the scheduled fetch recorded its attempt, want twice", got)
}The same reasoning covers the failure cases, and that is where the ordering matters most: the attempt gets written down before the fetch goes out, not after. A fetch that got a 500, and a fetch whose request context was cancelled before it finished, both spend the interval, because what costs the issuer is the attempt and not the result. An issuer that is down is precisely the situation where refetching on every arriving token does the most damage.
Both refusals carry the wait they imply, so the HTTP layer does not have to guess:
func (t tooSoon) RetryAfter() time.Duration { return t.interval - t.since }
func (u unreachable) RetryAfter() time.Duration { return u.interval }A declined fetch names what is left of the interval. A fetch that went out and failed names the whole of it. The guard turns an ErrUnavailable into a 503 with that Retry-After, rounding the division up, because a client told to wait 0 comes straight back and is told 0 again. Note also that the floor is inclusive: a gap of exactly MinRefresh is no longer shorter than it, so a client that waits precisely as long as it was told gets through rather than being told to wait the interval over again.
A ticker is infrastructure, so it gets checked like infrastructure#
The schedule is written out in the package rather than left to the caller, and what that mainly gets you is sane behaviour when things break.
Start(ctx, every) validates the key set URL before it starts the goroutine. A key set URL that is not HTTPS is refused at startup, as a returned error, rather than producing a goroutine that fails quietly once a period into a log nobody is reading. A zero every becomes CacheFor, 15 minutes by default, which makes that field the ticker period and nothing else.
Nothing is fetched before the first tick. The request path still fetches on demand, so startup is not blocked on the issuer being up. The ticker is there to meet rotations without a token having to ask for them.
On each tick the goroutine checks ctx.Err() before refreshing, because a tick and a stop can both be ready at once, and a fetch on a cancelled context fails immediately while still spending MinRefresh for every request queued behind it. Then stopping:
return func() {
cancel()
<-done
}, nilstop cancels the derived context and waits on the done channel, so a fetch in flight has returned before stop does. A server that has shut down is not still asking its issuer for keys, and a test that stops the ticker is not racing a goroutine that still holds the mutex.
What I would weigh differently#
The cost of recording the attempt before the fetch is that recovery waits for the interval to lapse rather than for the next request. When the issuer comes back up, the first arriving token inside the interval is still refused with a 503 even though a fetch would now succeed. I took that trade knowingly, since the alternative is that an outage plus traffic equals a retry storm, but it is a real cost, and on a key set with a long MinRefresh it is probably the thing that will look like a bug to whoever is on call.
The other tension is that one attempted field serves two policies. Separating them, a budget for scheduled fetches and another for token-triggered ones, is not what this package does, and I would want a concrete reason before adding it, because the moment the two are independent the issuer no longer has a single bound on how often it is asked.
One clock, two triggers, one record of the attempt. The gate knows who asked; the interval does not care.