Regressing Realised on Predicted

The single most useful diagnostic for a forecasting model is embarrassingly simple: take every prediction the model made, take what actually happened next, and regress one on the other. This is the Mincer-Zarnowitz regression, and despite its plainness it catches problems that fancier metrics like correlation or Sharpe ratio routinely miss — because it checks not just whether the model is directionally right, but whether its predictions are the right size.

Setting up the regression

$$r_{t+1} = \alpha + \beta \, \hat{r}_{t+1} + \varepsilon_t$$

In words: regress the return that actually happened, $r_{t+1}$, on the model's prediction of it, $\hat{r}_{t+1}$, made in advance. A model with no systematic bias and correctly scaled predictions should return $\alpha \approx 0$ and $\beta \approx 1$; anything else is a specific, fixable diagnosis rather than a vague "the model isn't great."

What each parameter reveals

An intercept $\alpha$ meaningfully different from zero means the model has a constant bias — it's predicting returns that are, on average, too high or too low, independent of the actual signal content, often because the training sample's average return doesn't match the live period. A slope $\beta$ below one means the model's predictions swing too far: when it predicts a big move, the actual move tends to be smaller, which is the classic signature of overfitting — the model has learned to be confident on noise. A slope above one is rarer and means the opposite: the model is under-stating genuinely large moves.

Worked example

A researcher regresses realized 5-day returns on a model's predicted 5-day returns across 400 out-of-sample predictions and gets $\hat{r}_{t+1} = 0.0004 + 0.62\,\hat{r}_{t+1}^{model}$, with $R^2 = 0.04$. The slope of 0.62 says that for every 1% the model predicts, the market only delivers about 0.62% on average — the model's predictions are about 60% too extreme. Rather than discard the model, the practical fix is to keep using its direction and relative ranking of predictions but multiply every future prediction by 0.62 before sizing trades off it, which corrects the scale without needing to refit the whole model.

The explorer above shows an OLS line fit to a scatter with visible residuals — imagine the x-axis as predicted return and the y-axis as realized return; a flatter-than-45° fitted line is exactly the overconfident-model pattern described here.

What this means in practice

This regression is cheap to run and should be a standard part of evaluating any return forecast before it goes live, alongside (not instead of) hit rate and correlation. A model can have a respectable correlation with future returns and still be badly miscalibrated in scale — correlation only measures co-movement, not whether the magnitude of the prediction matches the magnitude of the outcome.