Stability Selection

Run the lasso once on a dataset and it hands back a specific list of "selected" predictors — but run it again on a slightly different sample of the same underlying population (say, a different random half of your history) and the list can change substantially, especially when candidate predictors are numerous or correlated with each other. A single lasso run's selected set is really one noisy draw, not a settled truth. Stability selection addresses this directly: repeat the selection procedure many times on random subsamples, and only trust the predictors that show up in the selected set almost every time.

An analogy: asking many juries instead of one

If you wanted to know which of 50 pieces of evidence in a case are genuinely decisive, you wouldn't trust the verdict of a single jury's deliberation — different juries, given slightly different subsets of testimony, might latch onto slightly different details. Instead, empanel 100 independent juries, each shown a different random three-quarters of the evidence, and ask each jury which pieces of evidence it considered essential. A piece of evidence that 95 out of 100 juries flagged as essential is trustworthy; one that only 12 out of 100 flagged was probably just a fluke that one particular subset of evidence happened to make look important. Stability selection is running the equivalent of a hundred juries on your data.

The method, one symbol at a time

Repeat the following many times, say $B$ times (often $B=100$ or more): draw a random subsample of roughly half the observations, run a sparse selection method (typically the lasso) on that subsample across a range of regularization strengths, and record which predictors get selected. Define the selection frequency of predictor $j$ as

$$\hat\Pi_j = \frac{1}{B}\sum_{b=1}^{B} \mathbb{1}\{j \text{ selected in subsample } b\}.$$

In plain English: out of $B$ independent reruns on random half-samples, what fraction of the time did predictor $j$ get selected? Predictors are kept in the final model only if $\hat\Pi_j$ exceeds a chosen threshold $\pi_{\text{thr}}$ (commonly around 0.6–0.9) — meaning they were selected in a large majority of the reruns, not just once on the full dataset. In plain English: don't trust a predictor because one lasso run happened to pick it; trust it because it kept getting picked across many different random slices of the data.

Worked example 1: separating stable signal from noise

Run the lasso 200 times on random half-samples of a 500-predictor, 250-observation dataset with 8 truly predictive variables among the 500. The 8 true predictors come back with selection frequencies around 0.85–0.95 — selected in the vast majority of the 200 reruns. Meanwhile, of the 492 truly irrelevant predictors, most have selection frequencies under 0.10, but a handful of spuriously correlated ones might reach 0.3–0.4 by chance in any single dataset. Setting the threshold at $\pi_{\text{thr}} = 0.6$ correctly retains all 8 real predictors and correctly excludes every noise variable, including the handful that had gotten temporarily lucky in isolated reruns.

Worked example 2: correlated predictors splitting selection frequency

Two candidate predictors, C and D, are highly correlated with each other (correlation 0.9) and both are genuinely related to the target, but the lasso's usual behavior with correlated predictors is to arbitrarily pick one and zero out the other in any given fit. Across 200 subsample reruns, C gets selected in about 55% of runs and D in about 50% (their combined "the pair matters" signal is consistently strong, but which specific one gets chosen varies run to run). With a naive single-run lasso, you might see only C selected and conclude D is irrelevant; stability selection instead reveals that both C and D have meaningfully elevated selection frequencies, correctly signaling "this pair jointly matters" even though neither one alone dominates every single run.

What this means in practice

Stability selection is used whenever a sparse variable-selection result needs to be trusted for a real decision — deploying a factor model, choosing risk drivers for a hedging program, or reporting "these are the predictors that matter" to a portfolio manager — rather than treated as a single, possibly fragile lasso run's output. It converts an unstable point selection into a graded confidence measure per predictor, and it comes with theoretical guarantees bounding the expected number of falsely selected predictors, given a threshold choice.