Model Selection with AIC and BIC

You've fit two models to the same year of factor returns: a 3-factor model and a 12-factor model. The 12-factor model has a lower in-sample error, almost guaranteed, because more free parameters give it more room to bend toward every wiggle in the training data — including the noise. A lower in-sample error alone tells you nothing about which model will generalize better. AIC and BIC solve this by scoring a model on how well it fits and how many parameters it spent to get there, penalizing the second thing directly instead of relying on a separate hold-out sample.

An analogy: paying for a tailored suit

Think of model fit like a suit's fit and model complexity like its price. A suit tailored with 50 individual measurements will fit you better than one made from 3 standard measurements — of course it will, it had far more freedom to match your exact body. But if you're choosing between two tailors and one needs 50 measurements to achieve only a slightly better fit than the other's 3, you'd reasonably prefer the simpler one: the marginal improvement in fit isn't worth the extra complexity, time, and risk that comes with over-customizing to today's measurements, which might not hold next month. AIC and BIC formalize exactly this trade-off between fit and complexity for statistical models.

The formulas, in words

Both scores combine a goodness-of-fit term with a complexity penalty:

$$AIC = 2k - 2\ln(\hat{L}), \qquad BIC = k\ln(n) - 2\ln(\hat{L}) .$$

Here $k$ is the number of parameters the model estimated, $n$ is the number of observations, and $\hat{L}$ is the model's maximized likelihood — a measure of how well it fits the data, where a higher $\hat{L}$ means a better fit. In plain English: $-2\ln(\hat{L})$ is a "badness of fit" term that gets smaller as the model fits better, and $2k$ (for AIC) or $k\ln(n)$ (for BIC) is a penalty that grows with the number of parameters used. Lower AIC or BIC is better. The key difference is that BIC's penalty, $k\ln(n)$, grows with the sample size and, for any reasonable $n$, penalizes each extra parameter more harshly than AIC's flat $2k$ — so BIC tends to favor simpler models than AIC, especially with lots of data.

Worked example: comparing 3-factor and 12-factor models

Suppose both models are fit on $n=250$ trading days. The 3-factor model has $k=3$ parameters and achieves $-2\ln(\hat{L}) = 300$. The 12-factor model has $k=12$ and a better fit, $-2\ln(\hat{L}) = 280$. Computing AIC:

$$AIC_3 = 2(3) + 300 = 306, \qquad AIC_{12} = 2(12) + 280 = 304 .$$

AIC narrowly prefers the 12-factor model (304 < 306) — its 20-point fit improvement outweighs the extra 18 points of complexity penalty. Now check BIC, where $\ln(250) \approx 5.52$:

$$BIC_3 = 3(5.52) + 300 = 316.6, \qquad BIC_{12} = 12(5.52) + 280 = 346.2 .$$

BIC strongly prefers the simpler 3-factor model (316.6 < 346.2) — its steeper per-parameter penalty at this sample size outweighs the 12-factor model's fit advantage. The two criteria disagree precisely because they encode different philosophies: AIC is tuned to pick the model that predicts best, even if that means it isn't the exact "true" model; BIC is tuned to find the simplest model consistent with the data, useful when you believe a small set of factors truly drives returns and want to identify it.

What this means in practice

AIC and BIC let you compare non-nested models fit on the same data without carving out a separate validation set, which matters when data is scarce, as it often is with low-frequency factor or macro models. They're not interchangeable with cross-validation, though: both assume the likelihood is computed correctly and comparably across models, and both are asymptotic approximations that can behave oddly with small $n$ or heavily regularized models. When you have enough data to spare, cross-validated out-of-sample error is generally the more trustworthy check, with AIC/BIC as a fast complementary screen.