The Bias Hiding in Every Predictive Regression: Stambaugh

Here is one of the most-run regressions in the history of finance.

Regress next month's stock market return on this month's dividend yield.

The idea is old and appealing. When stocks are cheap relative to their dividends, future returns should be higher. Run this regression on decades of US data and you will find a positive coefficient with a respectable t-statistic. An enormous literature on "return predictability" was built on regressions of exactly this shape: returns on dividend yield, on the earnings-price ratio, on the book-to-market ratio, on the short rate, on the term spread.

In 1999, Robert Stambaugh showed that this regression has a built-in bias, arising purely from its structure, that pushes the coefficient in exactly the direction researchers were hoping to find. Not because of data mining. Not because of overfitting. Because of arithmetic.

The problem: two features that seem harmless and together are poison

The regression has two properties. Each looks innocent on its own. Together they are lethal.

Property one: the predictor is highly persistent. Dividend yield does not jump around. This month's dividend yield is very close to last month's. Its autoregressive coefficient is up near 0.99. It is a slow-moving, sticky variable. So is the book-to-market ratio. So is the earnings yield. So is essentially every valuation ratio ever used as a predictor.

Property two: shocks to the predictor are strongly negatively correlated with returns. This one requires a moment's thought, and once you see it you cannot unsee it.

Dividend yield is dividends divided by price. Price is in the denominator. So when the market has a great month and prices shoot up, the dividend yield mechanically falls. When the market crashes, the yield mechanically jumps.

That means: a positive return shock is almost automatically a negative dividend-yield shock. The two are joined at the hip by construction, not by economics. The correlation between the return innovation and the predictor innovation is large and negative, often around minus 0.9.

Both properties are true, both are unavoidable, and Stambaugh's contribution was to work out precisely what they do together.

The key idea via analogy: the scale that reads your weight from your last reading

Start with a fact that is old but under-appreciated. When you estimate a persistent autoregressive process from a finite sample, the estimated persistence is biased downward. This has been known since the 1940s. If the true autoregressive coefficient is 0.99, your estimate from 500 observations will, on average, come out below 0.99. The estimator systematically under-states persistence. The nearer to a unit root, the worse it gets.

Now here is Stambaugh's chain of reasoning, and it is a chain worth following link by link.

Your estimate of the predictor's persistence is biased downward. (Old, known fact.)
That bias in the persistence estimate spills over into the predictive regression's slope coefficient, because the two regressions share the same underlying shocks.
The direction of the spillover depends on the correlation between the two shocks.
For dividend yield, that correlation is strongly negative (because price is in the denominator).
A downward bias in persistence, transmitted through a strongly negative shock correlation, produces an upward bias in the predictive slope.

Your regression will report predictability even when there is none.

The analogy: imagine a bathroom scale that, due to a mechanical quirk, nudges its reading slightly in whichever direction the previous reading was. Any pattern you find in your weight over time is partly a pattern in the scale, not a pattern in you. The instrument and the measurement are entangled, and the entanglement runs in a known direction.

The crucial point is that this bias is not the result of anyone doing anything wrong. It is not data mining, it is not p-hacking, it is not a bad choice of sample. It is a structural property of the regression itself, and it exists even if the researcher is perfectly honest and the null hypothesis of no predictability is exactly true.

The magnitude, and why nobody escaped it

The bias is not small. Because the predictor is so persistent (near a unit root) and the shock correlation is so extreme (near minus one), the bias in these classic regressions is large enough to account for a substantial part of the reported coefficient. A regression that reports a coefficient of, say, 0.10 might have a bias contributing a meaningful fraction of that, and the honest bias-corrected estimate, with an honest confidence interval, is often not distinguishable from zero.

Stambaugh worked out the exact form of the bias, provided the correction, and then went further: he derived Bayesian posterior distributions for the regression parameters under different assumptions about the predictor's persistence and about how the initial observation should be treated. That framing is important, because it stops treating the bias as a point correction to be subtracted and starts treating the whole exercise as a question of honest uncertainty. And the honest uncertainty is much larger than the textbook standard errors admit.

Why it mattered

It took a large chunk out of the return-predictability literature. Many celebrated findings that valuation ratios predict returns were substantially weakened, and in some cases eliminated, once the bias was corrected.
"Stambaugh bias" became a standard term. Any serious paper that regresses returns on a persistent predictor now has to address it. If it does not, referees ask.
It identified a failure mode that honesty cannot prevent. This is the deepest lesson and the one most worth carrying. Most warnings in quant research are warnings about researcher behaviour: do not overfit, do not data mine, do not peek at the test set. Stambaugh's bias survives all of that. A completely honest researcher running a completely pre-registered regression on clean data will still get a biased answer. The problem is in the estimator, not the person.
It generalises far beyond dividend yield. Any predictive regression where the predictor is persistent and its shocks correlate with the outcome's shocks has this problem. That describes an enormous number of signals in quant finance: valuation ratios, sentiment measures constructed from prices, volatility-based signals, anything with price in the denominator. If your signal is a ratio with price on the bottom, you are exposed.

The honest limitations

The correction depends on knowing the true persistence, which you also cannot estimate without bias. This is genuinely circular. You correct the slope bias using an estimate of the predictor's persistence, but that estimate is itself biased. Various fixes exist, and none of them fully escapes the loop.
The bias correction is not the whole story. Even after correcting the point estimate, the inference remains difficult. When the predictor is near a unit root, standard t-statistics do not have their usual distribution, and getting honest confidence intervals requires more machinery than a simple bias adjustment. A substantial literature followed on exactly this.
The Bayesian framing requires priors, and the answer depends on them. Stambaugh is explicit about this: he presents results under several specifications precisely because the choice matters. Some readers find that clarifying. Others find it unsatisfying.
It does not prove there is no predictability. Correcting the bias weakens the evidence considerably. It does not eliminate it. Whether valuation ratios genuinely predict long-horizon returns remains one of the live and contested questions in finance, and reasonable people still disagree.
The bias goes the other way if the shock correlation flips. The upward bias is specific to predictors whose shocks are negatively correlated with returns. A different predictor could have a bias in the opposite direction. The lesson is to check, not to assume.

The one-line takeaway

Stambaugh showed that regressing returns on a persistent, price-based predictor produces a coefficient that is biased in exactly the direction that makes predictability look real, purely as an artifact of the regression's structure, which means an honest researcher with clean data and no data mining will still find predictability that is not there unless they correct for it.