Marginal Likelihood and Model Evidence

Two candidate models could explain a strategy's returns: a simple one-factor model and a richer three-factor model. The richer model can always fit the observed data at least as well — more parameters mean more flexibility. So "which model fits better" can't be the tiebreaker; a model with enough free parameters will always seem to fit best, right up until it fails on new data. The marginal likelihood is the Bayesian answer to "which model should I prefer," with a built-in penalty for needless flexibility baked directly into its definition — no separate step required.

An analogy: judging a fortune teller by every possible prediction, not just the one that came true

Imagine two fortune tellers. One makes a single, precise, testable prediction. The other hedges by offering a huge menu of vague possible outcomes, one of which is bound to roughly match whatever happens. After the event, both "got it right" in some loose sense. But if you'd asked, before the event, how likely each fortune teller's full menu was to include the actual outcome with reasonable specificity, the precise one looks far better — they staked real credibility on a narrow claim and it paid off, while the vague one spread their bets so thin that being "right" barely means anything. The marginal likelihood is exactly this pre-registered accounting: how much probability a model assigned, in advance, averaged honestly over its own parameter uncertainty, to the data that actually showed up.

The maths, one symbol at a time

Given a model $M$ with parameters $\theta$, a prior $p(\theta \mid M)$, and observed data $D$, the marginal likelihood (also called the model evidence) is:

$$p(D \mid M) = \int p(D \mid \theta, M)\, p(\theta \mid M)\, d\theta$$

In words: instead of picking the single best-fitting $\theta$ and reporting how well that $\theta$ explains the data, average the likelihood over every possible $\theta$ the model allows, weighted by the prior probability of each $\theta$. A model with a wide prior spread across many implausible parameter regions gets penalized automatically — most of that prior mass explains the data poorly, dragging the average down — even if some value of $\theta$ fits perfectly. This is precisely the mechanism behind Occam's razor in Bayesian statistics: unnecessary flexibility is punished by the averaging itself, with no separate penalty term bolted on.

Worked example 1: comparing two priors on the same simple model

Suppose a coin is flipped 10 times, landing heads 7 times, and we compare two priors under the same Bernoulli model: Prior A, Beta$(8,8)$ (fairly confident the coin is close to fair), and Prior B, Beta$(1,1)$ (uniform, "any bias is equally plausible"). Prior A's marginal likelihood for this data comes out noticeably higher than Prior B's, because Prior A concentrates its belief where $\theta$ actually explains 7-heads-in-10 well (near 0.5–0.7), whereas Prior B spreads belief thinly across values (like $\theta$ near 0.05 or 0.95) that explain the data terribly, dragging its average down. Even though both priors could fit the data if $\theta$ happened to land right, Prior A "bet" its probability more wisely and scores higher.

Worked example 2: a one-factor versus three-factor return model

A one-factor model has a tight, well-justified prior on its single beta coefficient. A three-factor model has the same prior on the first coefficient plus two more coefficients with wide, only weakly-justified priors. Even if the three-factor model's best-fit parameters explain historical returns slightly better in raw sum-of-squares terms, its marginal likelihood can come out lower, because most of the wide prior mass on the two extra coefficients corresponds to parameter combinations that explain the data poorly — that wasted mass drags the average down more than the modest fit improvement makes up for. The marginal likelihood is effectively saying: "you paid for two extra dimensions of flexibility, and most of it didn't pay off."

Widening a prior's spread in the explorer above is exactly the mechanism that lowers marginal likelihood — more prior mass spread over implausible parameter values pulls the weighted-average fit down, even without touching the best-fit point.

What this means in practice

Marginal likelihood is the quantity behind Bayes factors (see Bayes factors) and Bayesian model averaging (see Bayesian model averaging), and it's the principled Bayesian alternative to picking a model by raw in-sample fit. It's most useful exactly where overfitting risk is highest — comparing factor models with different numbers of factors, or a simple mean-reversion model against an elaborate regime-switching one — because it penalizes unjustified flexibility automatically, through the prior, rather than a hand-tuned complexity penalty.