Debiased Lasso and Post-Selection Inference

A researcher runs lasso on 200 candidate factors, it selects 12 as nonzero, and the natural next step is to ask: which of these 12 are statistically significant, and what are their confidence intervals? It's tempting to just read off lasso's coefficients and standard errors as if the model had been designed from the start with only those 12 variables. That instinct is wrong in two compounding ways: lasso's coefficients are deliberately biased toward zero (that's the whole point of the shrinkage), and the very act of using the data to choose which 12 variables survived makes ordinary standard-error formulas invalid, because they don't account for the search that happened before the "final" model was fixed. Debiased lasso and post-selection inference exist to repair both problems at once.

An analogy: judging a team chosen by tryouts

Imagine picking a basketball team by running tryouts and keeping only the players who scored above a cutoff on a single practice game. Two things go wrong if you then treat that single practice game's scores as the team's "true" ability level. First, players who scored well partly through luck on that one game will look overrated by that same score — regression to the mean means their future performance will typically be a bit lower than the score that got them selected, an inherent bias. Second, if you then run a statistical test asking "is this team's average score significantly above league average," using only the tryout scores of the players you kept, you're cheating: you already used those scores once to decide who made the team, so testing them again on the same data double-counts the evidence and overstates your confidence. Debiased lasso corrects the first problem (removing the shrinkage-driven bias) and post-selection inference corrects the second (accounting for the fact that variable selection already used the data).

The mechanics, one symbol at a time

Ordinary lasso solves $\min_\beta \sum_i (y_i - x_i'\beta)^2 + \lambda \sum_j |\beta_j|$, and the resulting nonzero coefficients $\hat{\beta}_j$ are systematically shrunk toward zero by the penalty — a form of bias introduced deliberately to control variance. The debiased (or "desparsified") lasso estimator adds a correction term back:

$$\hat{\beta}_j^{\text{debiased}} = \hat{\beta}_j + \frac{1}{n} \hat{z}_j' \left( y - X\hat{\beta} \right),$$

where $\hat{z}_j$ is a specially constructed vector (from an auxiliary regression of predictor $j$ on all the others) chosen so that the added term approximately cancels out the shrinkage bias in $\hat{\beta}_j$. In plain English: the correction estimates how much bias the penalty introduced for that specific coefficient, given how it's entangled with the other predictors, and adds back roughly that amount, producing an estimate whose sampling distribution is approximately normal — which means ordinary-looking confidence intervals and p-values can be validly constructed around it, unlike around the raw lasso coefficient.

Worked example 1: a single coefficient's bias correction

A lasso fit on a factor model with $\lambda$ chosen by cross-validation gives a raw coefficient of $\hat{\beta}_{\text{value}} = 0.18$ on a value factor. The debiasing procedure computes an auxiliary regression of the value factor on the other 30 candidate predictors, uses its residuals to build the correction term, and adds back a correction of $+0.09$, giving $\hat{\beta}_{\text{value}}^{\text{debiased}} = 0.27$. The debiased estimate then comes with a standard error of, say, 0.06, giving a 95% confidence interval of roughly $0.27 \pm 1.96 \times 0.06 = 0.27 \pm 0.118$, i.e. about 0.15 to 0.39 — an interval that does not include zero, so the debiased procedure supports concluding the value factor's effect is statistically distinguishable from zero, a claim the raw shrunk lasso coefficient alone could not support with a valid standard error.

Worked example 2: why testing the selected-only model is misleading

A different researcher instead runs lasso, keeps the 8 selected variables, and re-fits an ordinary (non-penalized) regression on just those 8, then reports the resulting p-values as if the 8 variables had been the plan all along. Suppose one of the 8 shows a p-value of 0.04 in that follow-up regression, just under the usual 0.05 cutoff. Because those 8 variables were themselves chosen by searching through 200 candidates for ones that looked promising, the true chance of at least one variable showing a "significant" p-value by luck alone is far higher than 5% — closer to what you'd expect from many implicit comparisons, similar in spirit to the multiple-testing inflation described in p-values and Multiple Testing. The 0.04 p-value from the naive re-fit understates how easily this result could have arisen from noise; a properly constructed post-selection or debiased-lasso interval accounts for the search step and gives an honestly wider, more conservative range.

What this means in practice

Debiased lasso and post-selection inference matter whenever a model is built with automatic variable selection — lasso or elastic net over a large factor library — and someone wants to make a statistical claim (a confidence interval, a p-value, a significance statement) about the surviving coefficients rather than just use the model for prediction. If the model is only ever used to generate predictions, this correction matters less; it becomes essential the moment the coefficients themselves are being interpreted or reported as evidence that a particular factor "matters."