Richardson Extrapolation

Many numerical approximation methods — numerical differentiation, integration, solving differential equations — have an error that shrinks in a predictable, known way as the step size $h$ gets smaller, often proportional to $h^2$ or some other known power. Simply using a smaller step size does reduce error, but it costs more computation and still leaves a residual error term you're just chipping away at slowly. Richardson extrapolation instead computes the same approximation at two different step sizes (say $h$ and $h/2$) and combines the two results algebraically in a way that cancels out the leading error term exactly, leaving only a much smaller, higher-order error behind.

For a method with error proportional to $h^2$, the combination is $\frac{4 A(h/2) - A(h)}{3}$, where $A(h)$ is the approximation using step size $h$. As a concrete feel: if a numerical derivative estimate gives $A(h) = 2.10$ at the coarse step and $A(h/2) = 2.025$ at half the step, the extrapolated estimate is $(4 \times 2.025 - 2.10)/3 = (8.1 - 2.1)/3 = 2.0$ — combining two moderately accurate estimates to land on a much more accurate one, without evaluating the function at an even smaller step size directly.

The intuition is a bit like triangulating a target's true position from two biased guesses whose bias you understand: if you know a rifle scope is off by an amount proportional to distance, you can combine two shots taken at two known distances to back out where the true center actually is, even though neither individual shot was aimed correctly. Here, the "bias" is the numerical method's error, and because it shrinks in a known, predictable way as $h$ shrinks (proportional to $h^2$, or $h^3$, depending on the method), you can solve for what the answer would be if that leading error term were exactly zero, without ever actually taking the limit $h \to 0$ directly. The same idea generalizes: combining three approximations at three step sizes can cancel two leading error terms at once, pushing accuracy even higher, which is the basis of more elaborate schemes like Romberg integration.