Leverage and Influential Observations

Fit a regression line to a cluster of points and it usually settles wherever most of the data agrees it should. But add one extreme point — an observation far out on the x-axis, unlike anything else in the sample — and the line can swing dramatically toward it, sometimes flipping a slope's sign entirely, even though it's only one data point among hundreds. Not every unusual point does this; some sit far out but still land right where the existing trend would predict, and barely move the fit at all. Telling these two cases apart — a point that's merely far out versus a point that's actually dragging the whole regression around — is what leverage and influence measure.

An analogy: the far end of a see-saw

Think of a regression line as a see-saw balanced across the data. A person sitting near the center of the see-saw, even a heavy one, doesn't tip it much — their position is close to the pivot. A person sitting at the very far end, however, can tip the whole plank dramatically with barely any effort, purely because of where they're sitting, not how much they weigh. Leverage measures how far out on the x-axis a point sits — its position on the see-saw. But leverage alone doesn't tell you if the point actually tips anything: a far-out point that happens to sit exactly where the see-saw was already balanced does nothing. Influence measures the actual tip — how much the fitted line changes if that point is removed — and it depends on both the leverage (position) and the residual (whether the point actually disagrees with the existing trend).

The mechanics, one symbol at a time

Leverage for observation $i$, denoted $h_{ii}$, comes from the "hat matrix" that maps actual values to fitted values ($\hat{y} = Hy$); it ranges from 0 to 1 and is large when $x_i$ is far from the average of the predictor. A common rule of thumb flags a point as high-leverage when

$$h_{ii} > \frac{2p}{n},$$

where $p$ is the number of parameters (including the intercept) and $n$ is the sample size — in plain English, a point has "too much" leverage if it's carrying more than twice its fair share of influence over the fit, given how many parameters and how many total points there are. To combine leverage with how much the point disagrees with the trend, Cook's distance measures the actual change in all the fitted values if observation $i$ is dropped:

$$D_i = \frac{e_i^2}{p \cdot s^2} \cdot \frac{h_{ii}}{(1-h_{ii})^2},$$

where $e_i$ is the residual and $s^2$ is the regression's mean squared error. In plain English: Cook's distance is large when a point has both a big residual (it disagrees with what the current trend predicts) and high leverage (it's sitting far enough out to have the power to drag the fit toward itself) — either alone is not enough to cause trouble, but the combination is.

Worked example 1: a lever with no push

A regression of monthly hedge fund returns on a market factor has 24 observations, mostly with the factor between −5% and 5%, plus one month where the factor was 12% (a big outlier in $x$) but the fund's return that month, 0.9%, was almost exactly what the existing trend would have predicted anyway. This point has high leverage — it's far out on the x-axis — but a tiny residual, so its Cook's distance is small: $e_i \approx 0$ makes the whole product in the Cook's distance formula collapse toward zero regardless of the large $h_{ii}$. Removing this point barely moves the fitted line, because it agreed with the trend the other 23 points already established.

Worked example 2: a lever with a big push

Same setup, but suppose instead that in the 12%-factor month, the fund's actual return was −8% — wildly inconsistent with the trend. Now both ingredients of Cook's distance are large: high leverage ($h_{ii}$ far out) combined with a large residual ($e_i$ far from what the trend predicted). With $p = 2$ and, say, $h_{ii} = 0.35$, $e_i = -8.6$ (residual after accounting for the trend), $s^2 = 4$:

$$D_i = \frac{(-8.6)^2}{2 \times 4} \times \frac{0.35}{(1-0.35)^2} = \frac{73.96}{8} \times \frac{0.35}{0.4225} \approx 9.245 \times 0.828 \approx 7.66 .$$

Compared against a common threshold of $4/n = 4/24 \approx 0.167$, this Cook's distance of 7.66 is enormous — this single month is dominating the regression, and removing it would substantially change the fitted slope. That's the kind of point worth investigating individually rather than letting it silently drive the whole model's conclusions.

Drag a single point out to the far right or far left of the cloud and watch how much the fitted line tilts — that visible tilt is what Cook's distance quantifies.

What this means in practice

Leverage and influence diagnostics matter anywhere a regression is fit on a modest number of observations and one bad data point — a data error, a one-off market event, a stale price — could otherwise dictate the conclusion: factor regressions on monthly returns, calibrations of a pricing model to a handful of quoted instruments, or event studies with a small number of events. Flagging high-Cook's-distance points doesn't automatically mean deleting them; it means investigating whether that observation is a genuine data problem or a real, important extreme event that the model should actually be capturing.