Horseshoe and Global-Local Shrinkage Priors

Ordinary Lasso shrinks every coefficient toward zero by roughly the same amount, which is a blunt tool when you believe most predictors truly are irrelevant but a handful are strongly predictive — Lasso tends to shrink the genuine signals too, biasing them downward. A global-local shrinkage prior fixes this by giving each coefficient its own "how much to shrink" parameter (the local scale) on top of one shared parameter that sets the overall shrinkage level (the global scale). Coefficients whose local scale happens to be large escape most of the shrinkage; the rest get pulled hard toward zero.

The horseshoe prior is the best-known version: it chooses the local scales from a distribution with an extremely heavy tail near zero and an extremely heavy tail out at large values, so a coefficient is pushed to be either almost exactly zero or left almost fully alone, with very little probability mass in between. Its density, plotted against the shrinkage factor, looks like the cross-section of a horseshoe — heavy at both ends, light in the middle — which is where the name comes from.

Worked example

Fitting a return-predicting regression on 200 candidate signals, a horseshoe prior will typically end up shrinking 180+ of the coefficients to values indistinguishable from zero while leaving a handful — say, momentum and a earnings-surprise term — with coefficients close to their unshrunk regression values. A standard ridge or Lasso fit on the same data would apply a more uniform pull, likely understating the size of those few genuine effects.