Group Lasso

Ordinary Lasso decides, coefficient by coefficient, which ones to zero out. That works fine when each feature stands alone, but breaks down when features naturally come in bundles — a one-hot-encoded categorical variable with five levels produces five coefficients that are really one decision ("does this categorical variable matter at all?"), yet Lasso is free to zero out three of the five and keep two, a nonsensical partial answer.

Group Lasso fixes this by penalizing predefined groups of coefficients together rather than individually:

$$\min_\beta \; \text{Loss}(\beta) + \lambda \sum_{g=1}^{G} \sqrt{p_g} \, \|\beta_g\|_2$$

In words: instead of penalizing the absolute size of each coefficient separately, penalize the overall magnitude (the $\ell_2$ norm) of each whole group $g$ of coefficients, scaled by that group's size — so an entire group is driven to zero together, or kept together, never split arbitrarily.

Worked example. A model includes a "region" categorical variable one-hot encoded into 4 dummy columns and a "sector" variable encoded into 6 dummy columns. Fit with group Lasso treating each categorical variable as one group, the penalty might zero out all 4 region coefficients together (region isn't predictive) while keeping all 6 sector coefficients nonzero (sector is) — a clean, interpretable "keep this whole variable or drop it" outcome that ordinary per-coefficient Lasso cannot guarantee.