Gaussian Processes

Most models fit one specific curve to data — a regression line, a fitted volatility surface — and treat that curve as the answer. A Gaussian process (GP) instead treats the unknown curve itself as random, and asks: given the data observed so far, what's the whole distribution over curves that could plausibly have produced it? That distribution naturally comes with a built-in notion of uncertainty — tight where data is dense, wide where it's sparse — without needing to bolt on a separate uncertainty calculation afterward.

The idea

A Gaussian process is defined by saying: for any finite collection of input points you choose, the function's values at those points are jointly normally distributed, with a mean (usually taken as zero, or a simple trend) and a covariance between any two points determined by a kernel function — a rule for "how similar are the function's values expected to be at these two inputs, based on how close together the inputs are." Points close together (in whatever sense the kernel defines) are assumed to have highly correlated function values — the curve can't jump wildly between neighbors — while distant points are assumed nearly independent. Conditioning this joint normal distribution on the observed data points gives, at any new input, both a predicted mean value and a predicted variance: the mean is the GP's best guess, and the variance shrinks near observed points (where the data pins the curve down tightly) and grows in gaps between data (where many different curves remain consistent with what's been seen).

A small illustration of the shape this takes: near an observed data point, the GP's predicted uncertainty is nearly zero — the curve is essentially forced to pass through what's already been measured — while a few steps away from any observed point, in either direction, the uncertainty band widens out, reflecting genuine ambiguity about what the function does in that unmeasured region, before eventually settling back to the GP's prior uncertainty far from any data.

What this means in practice

Gaussian processes are used to fit implied volatility surfaces and yield curves with calibrated uncertainty bands rather than a single deterministic fit, to guide Bayesian optimization when searching over expensive-to-evaluate model configurations, and more generally whenever a smooth, data-driven curve is needed alongside an honest sense of where that curve is well-supported by data versus where it's extrapolating. Their main practical limitation is computational cost — the core computation scales roughly with the cube of the number of data points — which is why GPs are typically applied to moderately sized datasets rather than very large ones.