Bayes Factors

A p-value tells you how surprising your data would be if the null hypothesis were exactly true — but it never answers the question you actually want, which is "how much more should I now believe the alternative over the null, given what I saw?" It's also silent about how good the alternative is; a tiny p-value can come from a null that's mildly wrong or spectacularly wrong, and it can't distinguish evidence against the null from evidence for a specific alternative. The Bayes factor answers that comparative question directly: given the data, how many times more plausible is model 1 than model 2?

An analogy: two suspects and a piece of evidence

A detective has two suspects for a crime, and a fingerprint is found at the scene. The fingerprint alone doesn't prove guilt — the real question is how much more likely it is if suspect A did it versus suspect B. If the fingerprint matches a pattern common under A's story but rare under B's, that should shift the detective's belief substantially toward A, in proportion to exactly how much more likely the evidence is under A's story than B's. A Bayes factor formalizes this: the ratio of how likely the observed data was under one model versus another — exactly the factor by which the data should update your relative belief between the two.

The maths, one symbol at a time

Given two models $M_1$ and $M_2$ with marginal likelihoods $p(D \mid M_1)$ and $p(D \mid M_2)$ (see marginal likelihood), the Bayes factor in favor of $M_1$ is:

$$BF_{12} = \frac{p(D \mid M_1)}{p(D \mid M_2)}$$

In words: it's the ratio of how well each model, averaged honestly over its own prior uncertainty, explains the observed data. A $BF_{12}$ of 10 means the data is 10 times more likely under $M_1$ than under $M_2$. This connects directly to updating actual beliefs: prior odds times the Bayes factor equal posterior odds,

$$\frac{p(M_1 \mid D)}{p(M_2 \mid D)} = BF_{12} \times \frac{p(M_1)}{p(M_2)}$$

so the Bayes factor is literally the multiplier that turns your before-data odds into your after-data odds — nothing more, nothing less. A commonly used rough scale (Kass & Raftery): $BF_{12}$ between 1–3 is barely worth mentioning, 3–20 is positive evidence, 20–150 is strong, above 150 is decisive.

Worked example 1: testing whether a signal has any edge

Model $M_1$: a signal's daily hit rate is exactly $0.50$ (no edge). Model $M_2$: the hit rate is unknown with a Beta$(2,2)$ prior (mild edge plausible either way). Over 100 trades, 62 hit. Computing the two marginal likelihoods gives roughly $p(D\mid M_1) \approx 0.0086$ and $p(D \mid M_2) \approx 0.088$, so $BF_{21} \approx 10.2$. The data is about 10 times more likely under "the hit rate is genuinely uncertain, possibly above 0.5" than under "exactly 0.5" — positive-to-strong evidence of a real edge, by the Kass-Raftery scale, even though 62/100 alone might not have looked dramatic.

Worked example 2: a Bayes factor that a p-value would mislead on

Suppose a much larger dataset of 10,000 trades produces a hit rate of $0.507$ — technically significant at the 5% level (a tiny but real deviation, given the huge sample). The Bayes factor comparing $M_1$ (exactly 0.5) against $M_2$ (uncertain, wide prior) here can actually favor $M_1$, with $BF_{12} > 1$: the deviation is so small it's more consistent with "no real edge, just noise" than with a diffuse alternative whose wide prior wastes probability on much larger edges that didn't materialize. This is the well-known divergence where a p-value flags "significant" on a giant sample while the Bayes factor says the effect is too tiny to prefer "real edge" over "no edge" — a caution against equating significance with practical evidence.

Feeding in different amounts of data in the explorer above and comparing how far the posterior moves is the same underlying comparison a Bayes factor formalizes numerically between two named models.

What this means in practice

Bayes factors are used in quant research to compare a "no signal" baseline against a "signal exists" model directly, folding in how plausible the proposed effect size was in the first place — which is exactly what raw significance testing can't do. They're also the building block for Bayesian model averaging (see Bayesian model averaging) across several candidate models at once, and for reporting evidence strength in a way that doesn't get more "significant" just because a sample got bigger, unlike a p-value.