Kernel Regression and the Nadaraya-Watson Estimator

You have scattered data pairs — say, option moneyness against implied volatility — and you want a curve through them, but you don't trust any particular parametric shape (quadratic, cubic, whatever) to be right. Fitting the wrong shape everywhere is worse than fitting no fixed shape at all. Kernel regression sidesteps the choice entirely: instead of fitting one global curve, it estimates the value at each point separately, as a weighted average of whatever data happens to be nearby.

An analogy: pricing a house from its neighbors

To estimate what a house is worth, one approach is a weighted average of nearby houses' recent sale prices — with weights that fade smoothly as a comparable house gets farther away in location or size, rather than a hard cutoff. A house next door counts a lot; a house across town barely counts at all. Kernel regression does exactly this for any $x$-$y$ relationship: to estimate $y$ at some value of $x$, take a weighted average of the $y$'s from data points whose $x$'s are nearby, letting the weight smoothly decay with distance rather than picking a fixed neighborhood or a fixed functional shape.

The idea, one symbol at a time

The Nadaraya-Watson estimator at a point $x$ is

$$\hat{m}(x) = \frac{\sum_{i=1}^{n} K\left(\frac{x - x_i}{h}\right) y_i}{\sum_{i=1}^{n} K\left(\frac{x - x_i}{h}\right)},$$

where $K(\cdot)$ is a kernel function (commonly a Gaussian bump) that assigns higher weight to $x_i$ closer to $x$, and $h$ is the bandwidth controlling how quickly that weight decays. In plain English: the estimate at $x$ is a weighted average of all the observed $y_i$, where the weight on each observation is determined entirely by how close its $x_i$ is to $x$ — points right at $x$ count heavily, points far away barely count at all, and the denominator simply rescales the weights so they add up to 1 (turning them into a proper weighted average rather than a raw weighted sum).

Worked example 1: implied vol at a specific strike, by hand

Suppose you have three observed (moneyness, implied vol) pairs: $(0.95, 22\%)$, $(1.00, 18\%)$, $(1.05, 21\%)$, and you want to estimate implied vol at moneyness $x = 0.98$, using a simple triangular kernel with bandwidth $h = 0.05$ that gives weight $\max(0, 1 - |x - x_i|/h)$. Distances: $|0.98-0.95|=0.03 \to$ weight $1 - 0.03/0.05 = 0.4$; $|0.98-1.00|=0.02 \to$ weight $1-0.02/0.05=0.6$; $|0.98-1.05|=0.07 \to$ weight $\max(0, 1-0.07/0.05) = 0$ (outside the window entirely). The estimate:

$$\hat{m}(0.98) = \frac{0.4 \times 22 + 0.6 \times 18 + 0 \times 21}{0.4 + 0.6 + 0} = \frac{8.8 + 10.8}{1.0} = 19.6\%.$$

Notice the far point at $1.05$ contributes nothing — outside the triangular window it's given zero weight — while the closer point at $1.00$ carries more influence than the slightly farther point at $0.95$.

Worked example 2: what happens with a wider bandwidth

Redo the same estimate with $h = 0.15$ instead, which now includes the third point too: weights become $1-0.03/0.15=0.8$, $1-0.02/0.15\approx0.867$, $1-0.07/0.15\approx0.533$.

$$\hat{m}(0.98) = \frac{0.8(22) + 0.867(18) + 0.533(21)}{0.8+0.867+0.533} = \frac{17.6+15.6+11.2}{2.2} \approx \frac{44.4}{2.2} \approx 20.2\%.$$

The wider bandwidth pulls in the point at $1.05$ (21%) and shifts the estimate up from $19.6\%$ to $20.2\%$ — a direct, hand-computable illustration of why bandwidth choice (see Bandwidth Selection for Smoothers) changes the answer, sometimes substantially, even with the exact same data.

What this means in practice

Kernel regression is used wherever a relationship is believed to be smooth but not well described by any specific functional form — smoothing an implied volatility smile, estimating how expected returns vary with a continuous factor exposure, or building a nonparametric hedge-ratio curve. It requires no assumption about linearity or curvature, at the cost of needing more data to estimate well (see Parametric vs. Nonparametric Models) and needing a well-chosen bandwidth to avoid either overfitting or oversmoothing.