Catastrophic Cancellation

A risk system computes an option's delta by finite differences: price it with the spot bumped up a tiny amount, price it again with spot bumped down, subtract, divide by the bump size. The two prices are both computed to roughly 15-16 significant digits — but if they're nearly equal, most of those digits cancel out, and what's left is dominated by rounding error rather than the real signal. This is catastrophic cancellation: subtracting two close floating-point numbers can throw away almost all of your working precision in a single operation.

Why it happens

Each floating-point number already carries a tiny rounding error from how it was computed or stored, somewhere around the 16th significant digit. Subtraction itself is exact given two floating-point inputs — the danger is what the inputs already contained. If $a = 1.00000000012345$ and $b = 1.00000000012340$, both numbers might individually be accurate to full double precision, but $a - b = 0.00000000000005$ has thrown away 13 leading digits of agreement, and the relative error in that tiny result can be enormous, because the absolute rounding error in $a$ and $b$ (roughly $10^{-16}$ relative to their size, so around $10^{-16}$ absolute) is now comparable to the entire answer.

Worked example

Compute the numerical derivative of $f(x) = x^2$ at $x = 1{,}000{,}000$ using a central difference with a naive small step $h = 10^{-8}$: $\frac{f(x+h) - f(x-h)}{2h}$. Mathematically the true derivative is $2x = 2{,}000{,}000$. But $f(x+h) = (1{,}000{,}000.00000001)^2 \approx 1{,}000{,}000{,}000{,}000.02$ and $f(x-h)$ is nearly identical — both values are around $10^{12}$, computed to about 16 significant digits, so their absolute precision floor is around $10^{12-16} = 10^{-4}$. The true difference we need is on the order of $2x \cdot 2h = 4 \times 10^{-2}$ — comparable to that noise floor. In practice this computation returns a derivative estimate with only 1-2 correct digits instead of the 6-8 a well-scaled version would give, purely because $h$ was too small relative to $x$'s magnitude, and the subtraction cancelled far more than it should have.

What this means in practice

Catastrophic cancellation shows up in P&L attribution (today's portfolio value minus yesterday's, both huge numbers, difference tiny), numerical Greeks via finite differences, and variance computed as $E[X^2] - (E[X])^2$ on data with a large mean and small spread — that formula is a textbook cancellation trap and is why NumPy's and most statistics libraries' variance implementations use a different, cancellation-resistant algorithm internally rather than that direct formula.