Heteroskedasticity-Robust Standard Errors

Financial data is never equally noisy. Volatility clusters, so a regression spanning 2007 and 2013 mixes wild months with placid ones. Small caps scatter more than mega caps. Illiquid names print stale prices. Meanwhile the standard regression output on your screen assumes every observation was drawn from the same calm world, and quietly reports an error bar built on that assumption. The coefficient is fine. The error bar is not, and every t-statistic and p-value you read off it inherits the problem.

Ten people estimating a building

You ask ten people how tall a building is. Five are surveyors with laser rangefinders; five are guessing from a café across the road. Average all ten answers and you will land close to the truth — the guessers are wrong in both directions and largely cancel out. The average is unbiased.

Now report your confidence. If you assume all ten estimates were equally reliable, you compute a spread from the typical disagreement and quote a tight interval. But the honest picture is five tight answers and five wild ones, and the true precision of your average depends on which of them happened to be extreme. Treat the guessers as surveyors and you will claim precision you never had.

That is the entire problem. The estimate is fine. The claimed precision is fabricated, and it is usually fabricated in the dangerous direction: too tight, so borderline results look conclusive.

Two formulas, side by side

Take the simplest regression, one slope $\beta$ fitted to $n$ paired points. Let $\bar{x}$ be the average of the drivers and $\hat{e}_i$ the residual for observation $i$ — the vertical gap between the point and the fitted line. The classical error bar is

$$\widehat{\operatorname{Var}}(\hat\beta) = \frac{s^2}{\sum_i (x_i - \bar{x})^2}, \qquad s^2 = \frac{\sum_i \hat{e}_i^2}{n-2} .$$

In words: compute one average squared residual, $s^2$, and hand that same number to every observation, then divide by how spread out the drivers are. Notice what it assumes — one noise level for all points.

The robust version drops that assumption:

$$\widehat{\operatorname{Var}}_{\text{rob}}(\hat\beta) = \frac{\sum_i (x_i - \bar{x})^2\,\hat{e}_i^2}{\Big[\sum_i (x_i - \bar{x})^2\Big]^2} .$$

In words: let each point bring its own squared residual, and weight it by how far that point sits from the middle of the drivers. That reweighting is the whole innovation, and it is due to Halbert White in 1980.

Why weight by distance from the middle? Think of the fitted line as a see-saw balanced on the average of the $x$ values. A point sitting right at the pivot can move up or down without tilting anything — its noise barely touches the slope. A point far out at one end has enormous leverage: wobble it and the whole line swings. So when you ask "how uncertain is my slope?", noise at the far ends matters far more than noise in the middle, and the robust formula is simply the version that keeps track of where the noise was, rather than averaging it away.

There is one thing worth pausing on, because it looks like cheating. You cannot possibly estimate a separate noise level for each observation — you have one residual per point and no way to tell luck from volatility. But you never needed to. All the formula requires is one weighted sum, and a sum of $n$ noisy terms is estimated perfectly well even though no individual term is. That is why robust standard errors work at all.

Worked example one: same slope, different verdict

Eight observations with drivers spread symmetrically: $x = -4, -3, -2, -1, 1, 2, 3, 4$, so $\bar{x} = 0$ and $\sum (x_i - \bar x)^2 = 16+9+4+1+1+4+9+16 = 60$. Fitting the line gives $\hat\beta = 0.50$ with residuals

$$-3.0,\; 0.3,\; -0.2,\; 0.2,\; -0.2,\; 0.2,\; -0.3,\; 3.0 .$$

The two extreme observations are enormously noisier than the middle six — classic heteroskedasticity.

Step 1 — classical route. Squared residuals sum to $9 + 0.09 + 0.04 + 0.04 + 0.04 + 0.04 + 0.09 + 9 = 18.34$. With $n - 2 = 6$:

$$s^2 = \frac{18.34}{6} = 3.057, \qquad \widehat{\operatorname{Var}} = \frac{3.057}{60} = 0.0509 .$$

Taking the square root, the classical standard error is $0.226$.

Step 2 — robust route. Weight each squared residual by its squared $x$: $16(9) + 9(0.09) + 4(0.04) + 1(0.04) + 1(0.04) + 4(0.04) + 9(0.09) + 16(9)$, which is $144 + 0.81 + 0.16 + 0.04 + 0.04 + 0.16 + 0.81 + 144 = 290.0$. Divide by $60^2 = 3600$:

$$\widehat{\operatorname{Var}}_{\text{rob}} = \frac{290.0}{3600} = 0.0806 .$$

The robust standard error is $0.284$, about 26% larger.

Step 3 — the verdict flips.

$$t_{\text{classical}} = \frac{0.50}{0.226} = 2.22, \qquad t_{\text{robust}} = \frac{0.50}{0.284} = 1.76 .$$

Same coefficient, same data, one formula clears the conventional 1.96 bar and the other does not. Notice why: the two noisiest points also sit at the extremes of $x$, so they dominate the slope and deserve extra weight. The classical formula diluted their noise across all eight observations.

Drag points in the explorer and watch the residual sticks. The classical formula only ever looks at their average length. The robust formula looks at each stick individually and cares most about the ones at the far left and far right of the plot.

Worked example two: when the noise really is even

Same eight drivers, but now every residual has magnitude $1.5$, alternating sign.

Step 1 — classical. Squared residuals sum to $8 \times 2.25 = 18$, so $s^2 = 18/6 = 3.0$ and $\widehat{\operatorname{Var}} = 3.0/60 = 0.05$, giving a standard error of $0.224$.

Step 2 — robust, raw form. Every squared residual is 2.25, so the weighted sum is $2.25 \times 60 = 135$, and $135/3600 = 0.0375$, a standard error of $0.194$ — noticeably smaller.

Step 3 — apply the small-sample correction. The raw form (called HC0) has no degrees-of-freedom adjustment, which makes it too optimistic in small samples. HC1 multiplies the variance by $n/(n-k)$, here $8/6$:

$$0.0375 \times \frac{8}{6} = 0.05 .$$

That is exactly the classical variance, and a standard error of $0.224$ again. This is the reassuring result: when the noise genuinely is even, the robust formula reproduces the classical answer. You give up essentially nothing by using it, which is why the standing advice on most desks is to use robust standard errors by default and stop thinking about it.

What this means in practice

Reach for HC1 or HC3 rather than the raw HC0 — HC3 inflates each residual by its leverage and is the safest choice under a few hundred observations. Above a thousand rows the variants agree to two decimals and the choice stops mattering.

Robust standard errors only fix unequal noise. They assume observations are still independent of one another. In finance that assumption fails constantly: monthly returns are serially correlated, and firms in the same industry or month move together. For those you need Newey-West HAC Standard Errors for time dependence or Clustered Standard Errors for group dependence, both of which generalise the same weighting idea. Getting the wrong one is a common interview trap.

Finally, do not go hunting for heteroskedasticity with a formal test and only then switch formulas. That two-step procedure makes your final p-value depend on the outcome of the first step, which is its own small piece of p-Hacking and the Garden of Forking Paths. Just use the robust version from the start.

The underlying condition and its diagnostic tests live in Heteroskedasticity; the assumption being relaxed is number five of the The Classical Linear Regression Assumptions.