Exactly-once deposits: a broker cannot deduplicate a reorg

2026-09-02 · 7 min read

A service that credits user balances from on-chain deposits has to tell its consumer about each deposit exactly once. The usual reflex is to push the problem onto the transport: turn on deduplication in the queue, set a dedup id, move on. That solves the wrong half. A broker deduplicates the delivery of a message you already decided to send. Nothing in it deduplicates the decision, and on a chain the decision is the part that repeats.

I have been building custodial wallets, on-chain payment systems and fiat-to-crypto onramps since 2018, through three market cycles, and every exactly-once bug I have had to fix in that time lived on the producer side of the queue.

Two problems wearing one name#

A deposit watcher polls. Every poll re-reads a range of blocks that overlaps the previous one, because that overlap is the only way to notice that a block you already read is gone. So the same transfer comes back on poll after poll, by design, not by accident. Add a process restart, a node resync, an operator replaying from height zero after a bad deploy, and the same payment is presented to your code an unbounded number of times over an unbounded stretch of wall-clock time.

Broker deduplication only helps if you hand it the same dedup id each time. But knowing that this transfer is the same one you already reported is the state problem. If you can compute a stable id, you have already solved deduplication and the broker is doing nothing for you; if you cannot, the broker has nothing to work with. Exactly-once at the queue is a property you supply, not one you receive.

The key names the payment, not the message#

This is where most implementations go wrong, and it is a modelling error rather than a distributed-systems one. The identity you deduplicate on has to name the payment itself, independent of when you learned about it and where it was mined.

In chain-watch, a small library I maintain (https://github.com/polycratia/chain-watch), that identity is DepositKey: the (tx_id, output_index) pair naming the transaction output the money landed on. Every deposit carries it as deposit.key.

from chain_watch import DepositWatcher

watcher = DepositWatcher(source, ["addr-1"], policy=3, reorg_depth=20)

result = watcher.poll()
for deposit in result.confirmed:
    credit(deposit.key, deposit.address, deposit.amount, deposit.asset)
for deposit in result.reverted:
    withdraw(deposit.key)

Compare that with the two keys people reach for instead. A broker message id is attempt-scoped: two polls that see the same transfer produce two ids, so it deduplicates retries of one publish and nothing else. A block-scoped identity — block hash plus index, or a poll cursor — is location-scoped: it survives ordinary repetition and then changes precisely when the transfer is mined into a different block, which is the one moment you needed it to hold. Location-based keys fail exactly when the chain gets interesting.

(tx_id, output_index) says what was paid, not where it was mined. It holds across polls, duplicated blocks, restarts and reorgs, and it is also the natural unique constraint downstream.

Dedup windows are measured in time; finality is measured in blocks#

Even a broker that does deduplicate on a key you supply does it inside a window, and that window is expressed in minutes. Chain finality is not expressed in minutes. It is expressed in depth, and depth per asset:

from chain_watch import ConfirmationPolicy

policy = ConfirmationPolicy(default=6, per_asset={"ETH": 12, "USDT": 12})

policy.depth_for("BTC")   # 6, the default
policy.depth_for("USDT")  # 12, the override

Six blocks is an hour on one chain and a couple of minutes on another. A resync replays a year of history in the time it takes to read it. There is no minute count you can configure that means "never report this key again, whatever the source says afterwards" — the units do not match, so the guarantee cannot be stated in the broker's vocabulary at all.

It can be stated in the watcher's. The state carries three things: pending transfers that are not deep enough yet, reported deposits already notified but still inside the reorg window, and settled — the keys of deposits buried below it, final, never reverted and never notified again.

import json
from chain_watch import DepositWatcher, WatcherState

state = WatcherState.from_dict(json.loads(snapshot.read_text()))
watcher = DepositWatcher(source, ["addr-1"], policy=3, state=state)

settled only grows, and the snapshot grows with the number of deposits ever seen. That is the price of the guarantee, and it is worth being honest about it rather than dressing it up. A time window is cheap because it forgets. Forgetting is the bug.

A retraction is not a duplicate#

There is a second thing a broker cannot model: an event that was true and stopped being true. Blocks disappear. A deposit you credited at three confirmations can be reorged out, and the consumer needs to hear about that too. A poll therefore returns two lists, not one.

@dataclass(frozen=True, slots=True)
class PollResult:
    confirmed: tuple[Deposit, ...] = ()
    reverted: tuple[Deposit, ...] = ()

The useful part is the invariant between them: a key that was reverted may confirm again, but only after the revert was reported. Credits and withdrawals always alternate. That single rule is what lets the consumer stay dumb — it can apply each side blindly, without comparing amounts, reconciling against a running balance, or deciding whether this confirmation is the first or the third.

On the consumer side the whole thing collapses into one transaction:

def apply(watcher, result, db):
    with db.transaction():
        for deposit in result.confirmed:
            db.execute(
                "insert into deposit_credit (tx_id, output_index, address, amount, asset)"
                " values (%s, %s, %s, %s, %s) on conflict do nothing",
                (*deposit.key, deposit.address, str(deposit.amount), deposit.asset),
            )
        for deposit in result.reverted:
            db.execute(
                "delete from deposit_credit where tx_id = %s and output_index = %s",
                deposit.key,
            )
        db.execute(
            "update watcher_snapshot set data = %s",
            (json.dumps(watcher.state.to_dict()),),
        )

The unique constraint on (tx_id, output_index) is the downstream mirror of DepositKey, and on conflict do nothing is a seatbelt rather than the guarantee. The guarantee is that the credit and the snapshot commit together.

Which is the whole point about placement, stated once: commit the snapshot in the same transaction as the notification and you get exactly once; commit it after and a crash in between replays the poll, so you get at least once; commit it before and the same crash drops the notification, so you get at most once. The broker sits downstream of that choice and cannot improve on it. If you do need to fan out to other services, publish from that same transaction — the transport's job is transport, and its own deduplication becomes a nicety rather than a load-bearing part of the design.

One demand this puts on the chain source: transfers() must describe the current best chain, so a transfer it stops returning is a transfer the chain no longer has. If your node or provider cannot promise that, no amount of bookkeeping above it will save you, and that is worth checking before anything else.

What I would do differently#

Earlier custodial systems I built keyed deduplication on things that were not the payment. One keyed on the provider's webhook id, which is really "this notification", so a provider that reissued notifications after an outage produced duplicate credits that looked like a balance bug for a week. Another keyed on block-scoped identity, which held beautifully until the first reorg deep enough to matter.

The subtler mistake was structural. I kept a notified boolean on the same row the scanner was updating with confirmation counts — mixing the record of what the chain says with the record of what I told downstream. Those are two different facts with two different lifetimes, and once they share a row every reorg becomes a data-repair task rather than an event.

What I would build first now is the identity: a key that names the output rather than the message or the block, and a persisted, restorable set of what has already been said. Everything else — the polling loop, the depth policy, the choice of transport — is replaceable around that. The queue is a pipe. Exactly-once was decided before the message reached it.