Random Effects and the Hausman Test

A panel regression of fund returns on manager characteristics, tracked across many funds over many years, needs to handle the fact that some funds are just persistently better or worse for reasons the model doesn't observe — skill, culture, whatever it is. Fixed effects (see Panel Data and Fixed Effects) handles this by giving every fund its own intercept, which is safe but can't estimate the effect of anything that doesn't vary within a fund over time. Random effects offers a more efficient alternative — treating each fund's baseline as a random draw from a shared distribution rather than a free parameter — but only if a specific assumption holds, and the Hausman test is the standard way to check whether that assumption is safe to rely on.

An analogy: knowing every student's grade history versus assuming a distribution

Imagine grading students across many exams, where each has their own baseline ability. Fixed effects estimates each student's baseline individually from their own history — safe, but it can't say anything about a trait that never changes for a given student (like their hometown). Random effects instead assumes baselines are drawn from a shared population, uncorrelated with anything else, borrowing strength across students for more precise estimates with fewer parameters. That only works honestly if the assumption is true; if stronger students disproportionately choose harder classes, the shortcut gives a biased answer. The Hausman test checks whether that assumption is safe.

The mechanics, one symbol at a time

Both approaches start from the same panel model, $y_{it} = \beta x_{it} + \alpha_i + \varepsilon_{it}$, where $\alpha_i$ is entity $i$'s individual baseline and $x_{it}$ is a predictor that varies across entities and time. Fixed effects estimates each $\alpha_i$ directly (or equivalently, removes it by demeaning within each entity — a special case of the Frisch-Waugh-Lovell theorem, see The Frisch-Waugh-Lovell Theorem) and makes no assumption about how $\alpha_i$ relates to $x_{it}$. Random effects instead assumes $\alpha_i$ is drawn independently of $x_{it}$ from a distribution with mean zero, $\alpha_i \sim (0, \sigma_\alpha^2)$, and uses a more efficient estimator that partially pools information across entities rather than estimating each $\alpha_i$ separately. The Hausman test compares the two estimators' coefficients on $x$:

$$H = (\hat{\beta}_{FE} - \hat{\beta}_{RE})' \left[ \text{Var}(\hat{\beta}_{FE}) - \text{Var}(\hat{\beta}_{RE}) \right]^{-1} (\hat{\beta}_{FE} - \hat{\beta}_{RE}),$$

which follows a chi-squared distribution under the null hypothesis that random effects' assumption holds. In plain English: if the entity baseline is genuinely unrelated to the predictors, both fixed effects and random effects should give roughly the same coefficient on $x$ (fixed effects is just noisier), so a big gap between the two estimates — more than sampling noise would explain — is evidence that random effects' convenient assumption is false and only fixed effects can be trusted.

Worked example 1: a Hausman test that passes

A panel of 200 mutual funds over 15 years regresses fund flows on expense ratio and past performance. Fixed effects gives a coefficient on past performance of $\hat{\beta}_{FE} = 0.42$; random effects gives $\hat{\beta}_{RE} = 0.39$ — close together. The Hausman statistic comes out to $H = 1.8$ against a chi-squared critical value of 5.99 (2 degrees of freedom, 5% level). Since $1.8 < 5.99$, the test fails to reject the null — the small gap between 0.42 and 0.39 is consistent with sampling noise, and random effects' assumption looks safe here. The researcher can use the more efficient random effects estimates, which also allow estimating the effect of time-invariant fund characteristics (like a fund's founding decade) that fixed effects would have absorbed and made inestimable.

Worked example 2: a Hausman test that fails

A different panel regresses fund returns on manager tenure, where longer-tenured managers plausibly ended up at their funds partly because those funds were already performing well (an unobserved fund quality correlated with both tenure and returns). Fixed effects gives $\hat{\beta}_{FE} = 0.15$; random effects gives a noticeably different $\hat{\beta}_{RE} = 0.31$ — random effects makes the tenure effect look twice as large. The Hausman statistic comes out to $H = 14.2$, well above the 5.99 critical value, rejecting the null. This large gap signals that fund baseline quality really is correlated with manager tenure, violating random effects' core assumption — its 0.31 estimate is biased, and only the fixed-effects estimate of 0.15, which makes no such assumption, should be trusted here.

What this means in practice

Random effects is attractive whenever it's valid: it's more efficient than fixed effects and can estimate variables that don't change within an entity over time, which fixed effects mechanically cannot do. But its core assumption — the entity baseline is unrelated to the predictors — is often implausible in finance, where unobserved entity-level quality (a fund's true skill, a firm's risk culture) is frequently correlated with the variables under study. Many researchers default straight to fixed effects for exactly this reason.