Why R-squared can only go up, and what to use instead

Question

A colleague keeps adding regressors because the model's $R^2$ never falls, and concludes each addition is an improvement.

Explain why $R^2$ can only increase, why that makes it useless for model selection, and what to use instead.

Accepted Answer

Why R^2 never decreases. By definition R^2 = 1 - 	ext{RSS}/	ext{TSS}, where RSS is the residual sum of squares and TSS is fixed by the data. Adding any regressor gives least squares more freedom, so the minimized RSS can only stay the same or shrink, never grow. Even a column of random noise gets a small coefficient that trims RSS slightly. Therefore R^2 mechanically rises (or ties) with every added variable, whether or not the variable carries real signal. Why that breaks model selection. If a metric always favors bigger models, it cannot tell you when to stop. Chasing R^2 leads straight t…

Why R-squared can only go up, and what to use instead

Your answer

More Statistics questions