Elastic Net Regularization

Lasso regression shrinks some coefficients all the way to zero, which is great for automatically discarding useless features — but when several features are highly correlated, Lasso tends to pick one almost arbitrarily and zero out the rest, even if they were all genuinely useful. Ridge regression handles correlated features gracefully, shrinking them all together, but it never zeroes anything out, so it can't do feature selection on its own.

Elastic net combines both penalties in one objective:

$$\min_\beta \; \text{Loss}(\beta) + \lambda \left[ \alpha \|\beta\|_1 + (1-\alpha) \|\beta\|_2^2 \right]$$

In words: minimize the usual prediction error, plus a penalty that's a weighted mix of Lasso's absolute-value penalty and Ridge's squared penalty, where $\alpha$ dials between the two extremes — $\alpha=1$ is pure Lasso, $\alpha=0$ is pure Ridge.

Worked example. A model has two nearly identical, highly correlated predictors (e.g. two ways of measuring firm size). Pure Lasso, fit once, might set one coefficient to 0.9 and the other to exactly 0 — a coin-flip outcome that changes with small data perturbations. Elastic net with $\alpha = 0.5$ on the same data tends to split the effect between them, giving both a moderate positive coefficient (e.g. 0.45 each), which is both more stable across resamples and more faithful to the fact that both variables genuinely carry the same signal.