Interpolation vs Extrapolation

A cartographer who has surveyed every street between two towns can confidently draw the road connecting them — every foot was actually walked and measured. Ask the same cartographer to extend the map fifty miles past the edge of the survey, and they can still draw something, guided by the pattern of roads they've seen, but nothing on that extension was ever checked, and the terrain out there could do anything — a cliff, a river, a different landscape entirely. Both drawings look identical on paper: confident lines on a map. Only one is backed by evidence. Interpolation and extrapolation are exactly this distinction, applied to any model's predictions.

The definition

Interpolation is a prediction made for an input $x_0$ that falls within the range spanned by the training data — surrounded on multiple sides by examples the model actually learned from. Extrapolation is a prediction made for an input $x_0$ that falls outside that range — beyond the highest, below the lowest, or in a region of a multi-dimensional feature space where no nearby training examples existed at all. Formally, if the training inputs span $[x_{\min}, x_{\max}]$, then $x_0 \in [x_{\min}, x_{\max}]$ is interpolation and $x_0 \notin [x_{\min}, x_{\max}]$ is extrapolation. In higher dimensions the idea generalizes to "inside the convex hull of the training data" versus "outside it," which is why extrapolation risk can appear even for a feature that individually stays within its historical range, if the combination of features never occurred together before.

The critical fact is that a model has no mechanism to know which regime it's operating in — it computes $\hat f(x_0)$ the exact same way regardless of whether $x_0$ sat inside or far outside the training range, and reports a number with the same apparent confidence either way. Nothing about the output format changes; only the trustworthiness of that output changes, silently.

Drag the exponent here and watch how, within the region shown, several different curves can look almost identical — but they diverge dramatically once extended past the plotted range. Any of those curves could be an equally good fit to data sampled only from the visible window; the data simply cannot tell you which one is right beyond its edge, no matter how good the fit looks inside it.

Worked example 1: two predictions from the same model, judged differently

A model trained on companies with market caps between $1bn and $50bn predicts expected volatility as a function of size, fitting $\hat\sigma(x) = 25 - 0.3x$ (with $x$ in billions), calibrated on that $1bn–$50bn range. Asked to predict for a $20bn company — squarely inside the training range — it outputs $\hat\sigma = 25 - 0.3(20) = 19\%$. This is interpolation: dozens of similarly sized companies were in the training set, and the relationship at $x=20$ was directly informed by nearby evidence.

Asked to predict for a $500bn company — a scale it never saw a single example of — the same formula outputs $\hat\sigma = 25 - 0.3(500) = -125\%$, a nonsensical negative volatility. The model didn't refuse or flag anything; it applied the identical straight line and produced a number, because nothing in a linear regression's mechanics distinguishes "confidently extending a validated pattern" from "blindly continuing a line past the edge of the map." The absurdity of $-125\%$ is a lucky, obvious tell here; most extrapolation failures are far less obviously wrong and go unnoticed.

Worked example 2: extrapolation hiding inside "normal-looking" inputs

A credit model is trained on years where interest rates ranged 2%–6% and unemployment 3%–8%, individually. In a new period, rates hit 7% (just outside range) while unemployment sits at 4% (well within range). Individually, 4% unemployment looks unremarkable. But the combination — rates above 6% with unemployment below 5% — never co-occurred in training; historically the two happened in separate macro regimes. The model predicts anyway, using patterns learned from combinations that don't resemble this one, with no automatic warning. Checking each feature's range separately would have missed this entirely; only checking the joint region catches it.

What this means in practice

Markets routinely deliver genuinely new conditions — a rate environment never seen before, a correlation regime that breaks historical patterns, a crisis that combines stresses that were previously independent — and any model trained on history is, by construction, being asked to extrapolate exactly when it matters most: during regime change, precisely when its training data is least representative of the present. This is why risk models built purely from historical fitting tend to fail hardest during genuine tail events, and why stress testing deliberately constructs scenarios outside the historical range rather than trusting the model's ordinary output there.