polycratia

Decimal is exact only inside its context, so size the context per operation

· 8 min read

Switching a balance from float to Decimal removes binary rounding, but it does not make the arithmetic exact. Decimal is exact only within the precision of the context it runs in, and the default context stops at 28 significant digits. An ETH balance held in wei carries more than that the moment you multiply it by anything, so a sum you believed was exact gets rounded silently, and a quantize you believed was safe raises instead.

That is the problem the arithmetic in cryptomoney is built around. The package refuses floats at the door, which is the part everybody writes about. The quieter half: refusing floats buys you nothing if the decimal arithmetic that follows runs in a context too narrow for the numbers you hold.

28 digits is not a lot of money#

Every Money is padded to its asset's full precision at construction, so an ETH amount always spends eighteen digits on its fraction. Ten digits are left for whole ether before the default context starts cutting:

python
>>> from decimal import Decimal, getcontext, localcontext
>>> getcontext().prec
28
>>> balance = Decimal("1234567890.123456789123456789")   # 28 digits, at the limit
>>> balance * Decimal("1.5") == Decimal("1851851835.1851851836851851835")
False

The exact product needs 29 digits. Under prec=28 the last one is dropped. No exception, no flag anybody reads, just a wei that stopped existing between two lines of code. Widen the context and the same expression comes out exact:

python
>>> with localcontext() as ctx:
...     ctx.prec = 60
...     balance * Decimal("1.5") == Decimal("1851851835.1851851836851851835")
True

Quantizing is the loud half of the same problem. quantize will not produce a result whose coefficient is longer than the context allows:

python
>>> Decimal("1851851835.1851851836851851835").quantize(Decimal("1E-18"))
decimal.InvalidOperation: [<class 'decimal.InvalidOperation'>]

So the two things a money type does constantly (combine amounts, then fit the result back to the asset's precision) are the two things the default context is too narrow for. One fails quietly, the other fails violently. In a ledger neither is acceptable.

Counting the digits an operation actually needs#

The fix is not a bigger global precision. You compute, per operation, how many digits that operation needs, and you run it in a local context of that size. Everything rests on one small function:

python
def _digits_of(value: Decimal) -> int:
    return len(value.as_tuple().digits) + max(int(value.as_tuple().exponent), 0)

Coefficient digits plus any positive exponent. The second term matters more than it looks: Decimal("1E+20") carries a single digit of coefficient but occupies twenty-one digit positions once you write it out, and a context sized from the coefficient alone would round it away.

On top of that sits the bound for the steps that must not be rounded at all:

python
def _exact_precision(*values: Decimal) -> int:
    """Enough digits that a sum, difference or product is never rounded.

    Every amount carries its asset's full precision, so the default context of
    28 digits is not wide enough for a large balance in wei.
    """
    return sum(_digits_of(value) for value in values) + 10

Summing the operands is the worst case of the three operations this serves. A product of an m-digit and an n-digit number needs at most m + n digits; a sum or difference needs at most max(m, n) + 1. One expression covers both, with ten digits of headroom on top. __add__, __sub__, __mul__, multiply and convert all size their context with it, so none of them can be rounded by whatever context they happen to be running in.

That is the real payoff. Whether a result loses a digit stops being a function of ambient process state; it gets decided in exactly one place, by the rounding mode the caller passed to the quantization at the end.

Division and quantization get target places plus a guard band#

Division cannot be exact, so it gets a different bound, one sized from the target rather than from the full product:

python
def _working_precision(places: int, *values: Decimal) -> int:
    return max(_digits_of(value) for value in values) + places + 10

divide uses this for the quotient and sets ctx.rounding to the caller's mode, so the single rounding that happens is the one that was asked for. _quantize uses it again:

python
def _quantize(amount: Decimal, places: int, rounding: str) -> Decimal:
    quantum = Decimal((0, (1,), -places))
    with localcontext() as ctx:
        ctx.prec = _working_precision(places, amount)
        return amount.quantize(quantum, rounding=rounding)

This is the function that would otherwise raise InvalidOperation on a long product. Because the context is sized from the amount actually in hand plus the places being quantized to, the result fits, and the only thing deciding the last base unit is rounding. The same helper runs inside _to_decimal, so even a plain constructor call pads to the asset's precision under a context wide enough to do it.

Which is why nothing in the package has a default rounding mode. If the only rounding in the system is the one the caller asked for, then the caller has to ask:

python
from decimal import ROUND_DOWN, ROUND_HALF_UP

from cryptomoney import BTC, Money

Money("0.5", BTC) / 3                                      # TypeError: / is refused
Money("0.5", BTC).divide(3, rounding=ROUND_DOWN)           # 0.16666666 BTC
Money("1", BTC).multiply("0.015", rounding=ROUND_HALF_UP)  # 0.01500000 BTC

Validate the operand before you size a context from it#

multiply, divide and convert run their checks in a fixed order: _check_rounding first, which refuses any mode outside the eight names from decimal, then _operand, which raises through _refuse_float for a float and ValueError for anything not finite. divide raises ZeroDivisionError for a zero divisor; convert raises ValueError for a rate at or below zero.

The order is not decoration. A context sized from an unvalidated operand is a context sized from whatever the caller happened to pass. A NaN does not even have an integer exponent, so _digits_of on one fails with an error that tells the caller nothing about what they did wrong. Validate first and the precision calculation only ever sees a finite decimal, and every error message names the actual mistake.

The global context is never touched#

The shortcut for all of this is one line at import time: getcontext().prec = 60. It works, and it turns your library's correctness into a property of the whole process. Another library can set it back. Application code can wrap a call site in its own localcontext() and silently narrow you. A notebook can do it by accident. The results of a money type should not move because of code it has never heard of.

So no code path in the package alters the global context. Every widening happens inside a with localcontext() block that unwinds on exit, and results come back through Money.__post_init__, which runs _to_decimal like any other constructed amount. One door in, one normalization, whichever operation produced the number.

What I would do differently#

The + 10 guard band is a bare literal in two functions. It should be a named constant with a sentence explaining what it covers, because "ten" currently reads as superstition when it is in fact headroom for the extra digits quantize carries while it decides the last one.

_exact_precision also sums every operand when the operation is addition, where max + 1 would do. I would keep the loose bound (a few extra digits in a short-lived local context cost nothing measurable, and one bound serving three operators is one fewer thing to get wrong), but the docstring should say it is deliberately generous rather than leaving the next reader to wonder whether the author confused the two cases.

The last one is a dependency I would write down. An asset's precision is capped at 36 decimals; Asset("TOOFINE", 37) raises. That cap bounds the fractional side of every operand these functions ever see, which is what keeps the computed precisions sane. Today it reads as an input validation rule. It is actually load-bearing for the precision strategy, and it belongs in the same paragraph as _exact_precision.

Close#

Decimal does not hand you exactness. It hands you a way to ask for it, one operation at a time, and the default answer is 28 digits because somebody had to pick a number for the general case. Money is not the general case. How many digits a step needs is computable from its operands, and computing it costs a few microseconds, considerably less than finding out the hard way, from a reconciliation report, which wei went missing and when...

react

$ new-project --brief

or email hey@polycratia.com