Local Polynomial Regression and LOESS

Kernel regression estimates a curve by taking a local weighted average at every point — but a flat average is a bad local approximation whenever the true curve is actually sloping, and it gets systematically worse right at the edges of the data, where the local window only has points on one side to average. Local polynomial regression, and its popular implementation LOESS, fix this by fitting a small weighted straight line (or parabola) in each local window instead of a flat average, and sliding that window across the full range of the data.

An analogy: a tailor's local measurements

Fitting one single formula to describe a person's whole body shape from head to toe would be absurd — instead a tailor takes local measurements, and near any specific point (the waist, the shoulder), a straight taper line fit just to that region works far better than any single global curve, and works especially well right at the very top or bottom of the body where a flat local average would clearly be wrong (an average of "just below the shoulder" measurements doesn't describe the shoulder's actual slope). LOESS does this systematically for a whole dataset: at every point along $x$, it fits a small local line (or curve) using only nearby, weighted data, and the local line adapts to the true slope of the data right up to the boundary.

The idea, one symbol at a time

At a query point $x_0$, local linear regression solves a weighted least squares problem using only data near $x_0$:

$$(\hat{\alpha}, \hat{\beta}) = \arg\min_{\alpha, \beta} \sum_{i=1}^{n} K\left(\frac{x_i - x_0}{h}\right) \left(y_i - \alpha - \beta(x_i - x_0)\right)^2,$$

and the fitted value at $x_0$ is $\hat{m}(x_0) = \hat{\alpha}$. In plain English: fit a straight line through the data, but weight each point's contribution to the fit by how close it is to $x_0$ (using kernel $K$ and bandwidth $h$, exactly as in Nadaraya-Watson), then read off the line's height at $x_0$ itself as your estimate — and repeat this whole fitting procedure fresh at every point along the x-axis you want a fitted value for. Unlike Nadaraya-Watson's flat local average, a locally linear fit can capture and extrapolate a local slope, which is precisely what removes the systematic pull-toward-the-middle bias at the boundary — the local line can tilt to match the true trend even when data only exists on one side.

Worked example 1: comparing flat average vs local line at a boundary

Suppose near the left boundary of the data, you have points $(1, 10), (2, 12), (3, 14)$, evenly weighted, and you want the estimate at $x_0 = 1$ (right at the edge — no data exists below it). Nadaraya-Watson's flat weighted average of the three $y$'s is $(10+12+14)/3 = 12$, even though the true local trend is clearly rising by 2 per unit $x$, meaning the honest value at $x_0=1$ should be closer to 10, not 12. A local linear fit through the same three points gives slope $\hat{\beta}=2$, intercept such that the line passes through $(2,12)$, giving $\hat{y} = 12 - 2(2-1) = 10$ at $x_0=1$ — matching the actual boundary value far better than the flat average's inflated 12.

Worked example 2: fitting a small yield curve segment

Suppose four nearby maturities and yields: $(1, 3.0\%), (2, 3.3\%), (3, 3.5\%), (5, 3.9\%)$, and you want a smoothed yield at maturity 2.5 using a local linear fit with roughly equal weight on all four (wide bandwidth). A least-squares line through these four points has slope $\hat\beta \approx 0.22\%$ per year and passes near $(2.75, 3.5\%)$; extrapolating to $x_0=2.5$: $\hat{y} \approx 3.5 - 0.22(0.25) \approx 3.45\%$. This locally linear smoothing gives a yield curve value at 2.5 years that respects the visible upward slope in the surrounding data, rather than just averaging the four yields flatly (which would give $\approx 3.425\%$, close here but diverging more sharply whenever the local slope is steeper).

What this means in practice

LOESS is the default "just show me the trend" smoother in most plotting software for a reason: it handles curved trends and boundary regions gracefully with minimal tuning beyond the bandwidth. In finance it's used for smoothing noisy scatter — realized vol against implied vol, factor exposure against forward return — without committing to a specific parametric shape, and it's often preferred over Nadaraya-Watson specifically because trading data frequently needs a reliable estimate right at the edge of the observed range (e.g., the shortest or longest maturity available).