Generalized Least Squares (GLS)

Ordinary least squares comes with fine print. It's the best linear unbiased estimator only when the errors are all the same size and uncorrelated with each other, statisticians call that "spherical" errors. Financial data almost never obliges. Volatility clusters, so some observations are noisier than others (Heteroskedasticity). Overlapping returns and momentum make consecutive errors move together (Autocorrelation and Serial Correlation). When that fine print is violated, OLS still gives an unbiased answer, but it's no longer the most precise one, and worse, its reported standard errors are simply wrong.

Generalized least squares is the fix. The idea is elegant: if you know (or can estimate) the pattern of the errors, you can transform the data to undo that pattern, then run plain OLS on the cleaned-up version. GLS reweights the noisy observations down and untangles the correlated ones, restoring efficiency.

The core idea

Write the error covariance matrix as $\Omega$. For OLS, $\Omega = \sigma^2 I$, a diagonal matrix with equal entries: every error the same size, none correlated. GLS lets $\Omega$ be anything: unequal diagonal entries (some errors bigger), off-diagonal entries (errors correlated). The GLS estimator is

$$\hat\beta_{\text{GLS}} = \left(X^\top \Omega^{-1} X\right)^{-1} X^\top \Omega^{-1} y.$$

Compare it to the OLS formula $(X^\top X)^{-1} X^\top y$: GLS just sandwiches in $\Omega^{-1}$, the inverse of the error covariance. That inverse is what does the reweighting, observations with large variance get pushed down, and correlated observations get partly discounted so you don't double-count the same information. The magic is that this is exactly OLS run on a transformed dataset where the errors have been made spherical again (a step called "whitening").

The payoff is the Aitken theorem, the GLS generalisation of Gauss-Markov: when $\Omega$ is known, GLS is the best linear unbiased estimator, it has the smallest variance among all linear unbiased methods. OLS gives up that crown the moment the errors stop being spherical.

The three cousins

GLS, weighted least squares, and OLS are the same idea at three levels of generality, distinguished entirely by what shape you allow $\Omega$ to take:

Method	Error covariance $\Omega$	Use it when
OLS	$\sigma^2 I$ — equal variances, zero correlation	textbook conditions hold
WLS	diagonal, unequal entries	errors differ in size but stay uncorrelated (Heteroskedasticity)
GLS	full matrix, off-diagonals allowed	errors are unequal and correlated (autocorrelation, panel structure)

So WLS is just GLS with a diagonal $\Omega$, and OLS is WLS with all the weights equal. Learn GLS and you've learned all three.

Worked example: combining two noisy estimates

The simplest GLS is combining measurements of unequal reliability. Suppose two independent desks each estimate the same fair value. Desk A reports $102$ with error variance $\sigma_A^2 = 1$; Desk B reports $108$ with variance $\sigma_B^2 = 4$ (four times noisier). What's the best combined estimate?

The naive average is $(102 + 108)/2 = 105$, but that treats the noisy Desk B as equal to the precise Desk A. GLS weights each by the inverse of its variance:

$$\hat\mu = \frac{\tfrac{1}{\sigma_A^2}\,102 + \tfrac{1}{\sigma_B^2}\,108}{\tfrac{1}{\sigma_A^2} + \tfrac{1}{\sigma_B^2}} = \frac{1 \cdot 102 + 0.25 \cdot 108}{1 + 0.25} = \frac{102 + 27}{1.25} = 103.2.$$

The answer sits much closer to the reliable Desk A, exactly as it should. And the combined estimate is more precise than either alone: its variance is $1/(1 + 0.25) = 0.8$, below Desk A's own $1.0$. That inverse-variance weighting is the whole spirit of GLS, trust the clean signals, discount the noisy ones, and combine correlations correctly.

The catch: you rarely know Omega

In practice $\Omega$ is unknown, so you estimate it from the data first (say, model the variance as a function of some variable, or estimate an autocorrelation coefficient), then plug the estimate in. That's feasible GLS (FGLS). It works well in large samples but carries a warning.

• Model risk in $\Omega$. Getting the error pattern wrong doesn't just cost efficiency, it can bias your inference. Robust standard errors on OLS are the more defensive move when the structure is uncertain.
• Small samples. Estimating a big covariance matrix from limited data is noisy; the "efficiency gain" can evaporate.
• It doesn't fix bias. Like OLS, GLS assumes the regressors are exogenous. If a regressor is correlated with the error (endogeneity), GLS is just as biased as OLS, no amount of reweighting saves you.

The mental model: OLS asks every data point to shout equally loudly. GLS listens more carefully, quieting the noisy voices and recognising when several observations are really echoing the same information. When you know the error structure, that's strictly better; when you only guess it, tread carefully.