polycratia

Reversals are a poll result, not an incident

· 9 min read

A deposit that already cleared your confirmation threshold can stop existing. The chain reorganises, the block that carried the transfer gets replaced, and a balance you credited minutes ago is backed by nothing. That event does not belong in an alerting runbook, to be cleaned up by hand later. It belongs in the return value of the same poll that reports confirmations, next to a declared depth below which it can no longer happen.

I have run custodial deposit flows for BTC and ETH, and stablecoin rails in daily production use. What made them calm was not a cleverer confirmation counter. It was accepting that a deposit watcher has exactly two jobs: say this arrived once, say this went away once, and publish the number that tells the ledger when the second sentence has become impossible.

Height is not identity#

Depth is arithmetic over two block references:

python
def confirmations_for(block: BlockRef, tip: BlockRef) -> int:
    """Depth of ``block`` under ``tip``, counting the block itself as one."""
    if tip.height < block.height:
        return 0
    return tip.height - block.height + 1

That is a pure function of two heights. It tells you how deep the block you remember would be, if the chain still contains it. It says nothing about whether the block now sitting at that height is the one you saw.

This is the whole reason a block reference has to be a pair:

python
@dataclass(frozen=True, slots=True)
class BlockRef:
    """Where something happened on the chain."""

    height: int
    hash: str

A reorg of the kind that costs money leaves no gap in the heights. It replaces the occupant. A watcher that stores only a height for each reported deposit cannot detect that: it keeps counting confirmations on a block that no longer exists, and reports ever-growing certainty about a payment that has been undone. In chain-watch a reported deposit still inside the window is reverted when its DepositKey is missing from what the source answers now, or when the block it currently sits in differs from the recorded one by height or hash. Both branches encode one sentence: the chain no longer agrees with what I credited.

Two numbers, not one#

The depth a deposit must reach before you credit it is a per-asset decision. A chain with ten-second blocks and a chain that settles in minutes do not deserve the same number, and a token worth more per transfer than the coin it rides on can deserve a stricter one than its own chain:

python
policy = ConfirmationPolicy(default=6, per_asset={"ETH": 12, "USDT": 12})

policy.depth_for("BTC")   # 6, the default
policy.depth_for("USDT")  # 12, the override

The depth at which a credit stops being reversible is a different number. It is the height of the window the watcher re-reads on every poll: the floor is tip.height - reorg_depth + 1, never below start_height, and that floor is exactly the since_height the chain source gets asked for. Everything at or above the floor is re-checked against what the source says now. Everything below it is final: settled, with no reversal and no second report.

So there are two knobs, and the constructor couples them unless you say otherwise:

python
self._policy = ConfirmationPolicy.coerce(policy)
if reorg_depth is None:
    reorg_depth = self._policy.max_depth
elif reorg_depth < 1:
    raise ValueError(f"reorg_depth must be at least 1: {reorg_depth}")

The default is policy.max_depth, the deepest requirement anywhere in the policy. The test suite pins it: a policy with default=2 and {"ETH": 4} gives watcher.reorg_depth == 4. That default is a real position, not a placeholder. With the window equal to the deepest requirement, a transfer reaches its depth exactly at the floor of the window, and the next block the chain produces buries it. You get effectively no reversal room past the poll that reported it. The default says: credit is final the moment it confirms.

If you want room to take a credit back, you buy it explicitly:

python
watcher = DepositWatcher(source, ["addr-1"], policy=policy, reorg_depth=24)

The gap between reorg_depth and policy.depth_for(asset) is how many blocks of regret you have. That is the actual risk decision, and writing it as one number per watcher is the only way I have found to discuss it with someone who is not an engineer. "We credit USDT at twelve confirmations and can reverse it up to twenty-four" is a sentence a finance owner can agree or disagree with. "We handle reorgs" is not.

The window is also observable. Over a ten-block chain with reorg_depth=4, the watcher asks the source for since_height=7, the same floor, every poll, with no hidden catch-up logic.

A reversal is a field, not an exception#

One poll returns both things that can happen:

python
@dataclass(frozen=True, slots=True)
class PollResult:
    """What one poll changed: deposits that matured and deposits that died."""

    confirmed: tuple[Deposit, ...] = ()
    reverted: tuple[Deposit, ...] = ()

    def __bool__(self) -> bool:
        return bool(self.confirmed or self.reverted)

Two tuples and a truth value. The falsiness matters more than it looks: most polls change nothing, and if not watcher.poll() is the whole quiet path. What counts is that reverted is a peer of confirmed in the same type. You cannot write the credit path without seeing the debit path sitting next to it, and the debit path gets the same Deposit values (the same address, amount, asset and block) instead of an error carrying a transaction id and a hope.

Note also that a transfer which disappears before it ever reached its depth is simply dropped. It never appears in reverted, because it never appeared in confirmed. Only things you were told about can be taken back, which is what keeps your handler finite.

The same key can be credited twice#

This is the part that bit me. A reverted key leaves the reported set. If the source still returns the transfer, it is re-pended, and it can confirm again from scratch: a reorg while pending restarts the count. The identity is the DepositKey, the (tx_id, output_index) pair, and that pair names what was paid, not where it was mined. It is deliberately stable across polls, restarts and reorgs.

So the second confirmation arrives carrying the key you already used for the first credit. If your ledger dedupes credits by deposit key alone, and that is probably the natural thing to do, because it is what makes replay safe, then the re-credit gets silently swallowed after you already debited the reversal. The customer's money quietly evaporates, and the only trace is a debit with no matching credit.

The fix is to make the ledger entry identity the key plus the block it was credited in:

python
result = watcher.poll()

for deposit in result.reverted:
    ledger.reverse(entry_id=(deposit.key, deposit.block.hash))

for deposit in result.confirmed:
    ledger.credit(
        entry_id=(deposit.key, deposit.block.hash),
        address=deposit.address,
        amount=deposit.amount,
        asset=deposit.asset,
    )

ledger is yours: the watcher opens no sockets and writes no rows. What it hands you is enough information to form that identity, because Deposit forwards the transfer's fields and the block reference travels with it. Amounts travel as Decimal: a float is refused on construction in favour of Decimal, int or str, so nothing gets rounded on the way into the ledger.

What exactly-once costs#

The promise "reported once, never again" has a price, and the state type states it plainly:

python
@dataclass(frozen=True, slots=True)
class WatcherState:
    start_height: int = 0
    pending: tuple[Transfer, ...] = ()
    reported: tuple[Deposit, ...] = ()
    settled: tuple[DepositKey, ...] = ()

pending and reported are bounded by the window. settled only grows... That set is the memory of every promise already kept, and it is what lets a restarted process ignore a source that replays a range it already served. Pruning it is not an optimisation, it is a second and quieter reorg decision: "keys older than this can be forgotten" is the same judgement as reorg_depth, made without writing it down. If I prune, I want the cutoff derived from the window, not from a retention default.

The state is a value, so persistence stays your problem: to_dict renders JSON-ready data with amounts as strings, from_dict reads it back and refuses an unknown STATE_VERSION with InvalidState. A process that dies between polls resumes with the same two numbers in force.

Determinism is what makes the pair testable#

Both lists in PollResult are sorted by block height, then tx_id, then output_index. Two runs over the same chain produce the same result, which is the only reason a reorg path can be tested at all: you can script a chain that forks, re-mines, stalls and forks again, then assert the exact sequence of confirmed and reverted keys. A stalled chain changes nothing (the state before and the state after compare equal) so "nothing happened" becomes an assertion rather than an absence.

I care about this because reversal paths are the code that runs least often and matters most. If the output is a set with incidental ordering, your reorg tests either assert too little or flake, and in both cases you stop trusting them.

What I would do differently#

In earlier systems, reorg detection lived in a reconciliation job that compared the ledger against the node on a schedule. It worked, in the sense that discrepancies were eventually found. But the discovery happened in a different process from the one that had credited the money, hours later, and it arrived as a list of rows for somebody to interpret. Nobody could answer "can this specific credit still be reversed" without reading code.

Two things I would now do from the start. First, do not accept the default window: set reorg_depth explicitly per chain, as a decision separate from the credit threshold, so the gap is visible in the constructor call. Second, record the window in effect alongside the credit, so support can answer the reversibility question from a row instead of from a repository.

A reversal is not an exception. It is the chain exercising a right it never gave up, and the depth at which it loses that right is a number you choose and should be able to say out loud.

react

$ new-project --brief

or email hey@polycratia.com