Bandwidth Selection for Smoothers

Every nonparametric smoother — a kernel density estimate of return distributions, a kernel regression of implied vol against strike, a LOESS fit through noisy scatter — has one number you must choose before anything else: the bandwidth. It controls how wide a "neighborhood" of nearby data points gets averaged together at each point of the fit. Get it wrong in one direction and your fit is a jagged mess that chases noise; get it wrong in the other and it's an overly smooth blur that hides real structure. Every other decision in nonparametric smoothing is secondary to getting this one number right.

An analogy: a camera's focus and blur

Think of bandwidth like the blur radius on a photo filter. A tiny blur radius leaves every pixel nearly untouched — you preserve fine detail, but also every speck of sensor noise, so the image looks grainy and jittery. A huge blur radius smooths away the noise beautifully, but it also smears out real edges and features you wanted to see — a face becomes an unrecognizable smudge. There's no blur radius that's "correct" in some absolute sense; the right amount depends on how much genuine fine detail exists versus how much of what looks like detail is actually noise. Bandwidth selection is choosing that blur radius for your data.

The idea, one symbol at a time

For a kernel smoother with bandwidth $h$, the expected squared error at a point decomposes as

$$\text{MSE}(h) \approx \underbrace{c_1 h^4}_{\text{bias}^2} + \underbrace{\frac{c_2}{nh}}_{\text{variance}},$$

where $n$ is the sample size and $c_1, c_2$ are constants depending on the true underlying function's curvature and the noise level. In plain English: a larger $h$ averages over more points, which smooths out noise (shrinking the variance term $c_2/(nh)$) but also blurs together points where the true function is actually changing, introducing systematic bias (growing as $h^4$). A smaller $h$ does the opposite — low bias, high variance. Minimizing the sum over $h$ gives the theoretically optimal bandwidth $h^* \propto n^{-1/5}$ for typical kernel smoothers: it shrinks as data accumulates, but only slowly, at the fifth-root rate rather than the square-root rate you might expect from simple averaging.

Worked example 1: kernel density estimate of returns

Suppose you're estimating the density of 1,000 daily returns using a Gaussian kernel. Silverman's rule-of-thumb bandwidth is $h \approx 0.9 \times \min(\hat{\sigma}, IQR/1.34) \times n^{-1/5}$. With $\hat{\sigma} = 1.2\%$ (daily return standard deviation) and $n = 1000$: $n^{-1/5} = 1000^{-0.2} \approx 0.251$, so $h \approx 0.9 \times 1.2 \times 0.251 \approx 0.27\%$. Using $h = 0.27\%$ produces a density estimate that's smooth enough to reveal the overall shape (fat tails, slight skew) without artificially breaking it into spurious little bumps around individual return values, which a much smaller $h$ (say $0.02\%$) would do.

Worked example 2: cross-validation picks it automatically

Rather than a formula, you can pick $h$ by leave-one-out cross-validation: for a grid of candidate bandwidths $h \in \{0.05, 0.15, 0.27, 0.5, 1.0\}\%$, refit the smoother leaving out each point in turn and measure how well the fit (built without that point) predicts it. Suppose the resulting average prediction errors are $1.8, 1.1, 0.9, 1.0, 1.4$ (in some error units) for the five candidates respectively — the error dips at $h=0.27\%$ and rises on both sides, tracing out exactly the U-shaped bias-variance tradeoff. Cross-validation picks the bandwidth at the bottom of that U, here confirming the rule-of-thumb value.

The explorer above plots the same tradeoff for model complexity in general; for a smoother, bandwidth plays the role of complexity in reverse — small h is high complexity (low bias, high variance), large h is low complexity (high bias, low variance). Drag it and watch the U-shaped total error curve.

What this means in practice

Bandwidth selection is the practical crux of every nonparametric method used in quant work: estimating a return density for VaR, smoothing an implied volatility surface with kernel regression, or fitting a LOESS curve through noisy factor exposures over time. Whenever a nonparametric fit looks suspiciously jagged or suspiciously flat, the first thing to check is not the method itself but the bandwidth it was given — cross-validation is the general-purpose fix when a rule-of-thumb formula doesn't fit the data well.