R-squared and Goodness of Fit

R-squared answers one question: of all the up-and-down variation in the thing you're trying to predict, how much does your model actually account for? If your outcome bounces around a lot and your fitted line follows most of that bouncing, R-squared is high. If your line is nearly flat while the data scatter wildly around it, R-squared is low. It's the most quoted number in regression, and also the most misread.

Start with two piles of "leftover wiggle." First, how far each point sits from the flat line at the average $\bar y$, this is the total variation you could explain. Second, how far each point sits from your fitted line, the part you failed to explain. R-squared is one minus the ratio of the second to the first:

$$R^2 = 1 - \frac{\text{SS}_{\text{res}}}{\text{SS}_{\text{tot}}}, \qquad \text{SS}_{\text{res}} = \sum_i (y_i - \hat y_i)^2, \quad \text{SS}_{\text{tot}} = \sum_i (y_i - \bar y)^2 .$$

Here $y_i$ is the actual value, $\hat y_i$ is the model's prediction, and $\bar y$ is the plain average of the $y$'s. If the model's misses ($\text{SS}_{\text{res}}$) are tiny compared to the raw spread ($\text{SS}_{\text{tot}}$), the ratio is near 0 and $R^2$ is near 1. If the model does no better than guessing the average, $\text{SS}_{\text{res}} = \text{SS}_{\text{tot}}$ and $R^2 = 0$. Those leftover misses are the residuals, the vertical gaps below.

A useful shortcut: it's just correlation, squared

In simple regression with one predictor, R-squared is literally the correlation coefficient squared, hence the name. A Correlation of $0.9$ means $R^2 = 0.81$; a correlation of $0.3$ (respectable in finance) means $R^2 = 0.09$, the model explains only 9% of the variation. That single fact deflates a lot of hype: a "strong-looking" correlation of $0.5$ still leaves three quarters of the movement unexplained.

Drag $\rho$ below and read the two numbers together. Notice how much faster $r^2$ falls than $\rho$: at $\rho = 0.5$ the cloud still looks tilted, yet $r^2$ is only $0.25$ — the line explains just a quarter of the scatter.

Worked example: computing it by hand

Fit a line to five points: $x = (1,2,3,4,5)$, $y = (1,3,2,3,5)$. Ordinary least squares (see Ordinary Least Squares (OLS)) gives the line $\hat y = 0.4 + 0.8x$. The predictions are $(1.2,\ 2.0,\ 2.8,\ 3.6,\ 4.4)$, so the residuals are $(-0.2,\ 1.0,\ -0.8,\ -0.6,\ 0.6)$ and

$$\text{SS}_{\text{res}} = 0.04 + 1.00 + 0.64 + 0.36 + 0.36 = 2.40 .$$

The mean of $y$ is $\bar y = 2.8$, so the total variation is

$$\text{SS}_{\text{tot}} = (1-2.8)^2 + (3-2.8)^2 + (2-2.8)^2 + (3-2.8)^2 + (5-2.8)^2 = 8.80 .$$

Therefore

$$R^2 = 1 - \frac{2.40}{8.80} = 0.727 .$$

The line explains about 73% of the spread in $y$. (And indeed $\sqrt{0.727} \approx 0.85$, the correlation between $x$ and $y$.)

Common pitfalls

• It never goes down when you add predictors. Throw in a useless variable and $R^2$ stays flat or nudges up, purely from fitting noise. That's why people report adjusted $R^2$, which docks you for every extra parameter you spend. See Degrees of Freedom for the bookkeeping, and Overfitting for the danger.
• High $R^2$ does not mean a good, or causal, model. Two trending series (both drifting up over time) can post an $R^2$ near 1 while being completely unrelated, a spurious regression. A high fit can also come from look-ahead leakage.
• Low $R^2$ is normal in finance and not a failure. Predicting daily returns, an $R^2$ of $0.01$ can be a genuine, tradable edge. Judge return models by out-of-sample profit and risk, not by $R^2$.
• It says nothing about bias or the size of errors. A model can have a decent $R^2$ yet be systematically off in the tails. Always eyeball the residual plot, not just the single number.

The habit to build: read $R^2$ as "share of variation explained," sanity-check it against the correlation, and never let a big number lull you into skipping the residual plot or the out-of-sample test.