Spike-and-Slab Priors

You've assembled 40 candidate factors that might predict a stock's return, but you strongly suspect only a handful actually matter — most are noise. An ordinary Bayesian regression with a normal prior on every coefficient shrinks the useless ones toward zero, but never quite to zero; every coefficient ends up with some small nonzero posterior estimate, leaving you to eyeball which ones "really" matter. A spike-and-slab prior answers a sharper question directly: for each coefficient, is it plausibly exactly zero, or genuinely nonzero and if so how big?

An analogy: a suspect lineup with a genuine option for "not present"

Ordinary variable selection via shrinkage is a bit like a police lineup where every suspect is assumed at least a little guilty, and the only question is how guilty each looks — nobody is ever fully cleared. A spike-and-slab prior instead gives each suspect a genuine possibility of being cleared entirely (the "spike": a point of pure innocence, coefficient exactly zero) versus being a real suspect whose degree of guilt is estimated on its own wide scale (the "slab"). The model doesn't just shrink everyone toward "probably not guilty" — it makes an actual binary call, in probability terms, about whether each predictor belongs in the model at all.

The maths, one symbol at a time

For each coefficient $\beta_j$, the spike-and-slab prior is a mixture:

$$\beta_j \mid \gamma_j \;\sim\; (1 - \gamma_j)\,\delta_0 + \gamma_j\, N(0, \tau^2)$$

with $\gamma_j \in \{0, 1\}$ a latent inclusion indicator, itself given a prior probability $P(\gamma_j = 1) = \pi$ (the prior belief that predictor $j$ belongs in the model at all). In words: with probability $1-\pi$, coefficient $j$ is drawn from $\delta_0$, a "spike" that places all its mass exactly at zero — the predictor is excluded. With probability $\pi$, it's drawn from a wide normal "slab" $N(0,\tau^2)$ — the predictor is included, and its size is estimated with real uncertainty. Fitting the model (typically via MCMC) produces, for every predictor, a posterior inclusion probability $P(\gamma_j = 1 \mid \text{data})$ — a direct, interpretable answer to "how likely is it that this predictor actually belongs in the model," alongside a posterior distribution over its size conditional on being included.

Worked example 1: sorting real signal from noise among candidate factors

Fit a spike-and-slab regression with 10 candidate factors on 300 months of returns, using prior inclusion probability $\pi = 0.3$ per factor. Suppose the posterior inclusion probabilities come out as: factor A, $0.94$; factor B, $0.87$; factor C, $0.05$; and the remaining seven below $0.10$. This reads cleanly: A and B are almost certainly real, the rest almost certainly noise, with no ambiguous middle ground — a much sharper statement than a ridge regression's shrunk-but-nonzero coefficients on all ten would give.

Worked example 2: a borderline factor and what it means to sit near 50%

Suppose an eleventh factor, "overnight gap size," gets a posterior inclusion probability of $0.52$ — right near the coin-flip line. This is itself informative: the data genuinely can't decide whether this factor belongs, a meaningfully different conclusion from either "definitely include" (0.94) or "definitely exclude" (0.05). A quant seeing 0.52 knows to treat it as speculative — worth revisiting once more data accumulates — rather than committing to it or discarding it outright, a distinction a plain shrinkage coefficient wouldn't communicate nearly as clearly.

Each factor's inclusion probability is itself a mini Bayesian update, prior belief in $\pi$ combined with how much the data favors "real" over "noise" — the same posterior-updating mechanic shown in the explorer above, just run once per candidate predictor.

What this means in practice

Spike-and-slab priors are used in quant research wherever the question is genuinely "which of these candidate predictors belong in the model," not just "how should I weight them" — factor screening across a large candidate universe, or deciding which of many technical indicators to retain in a trading signal. Compared to lasso-style sparsity (see ridge and lasso regularization), which also zeroes out coefficients but gives no probability attached to that decision, spike-and-slab gives a genuine posterior probability of inclusion — useful when you want to communicate confidence in the selection itself, not just report a final sparse model.