The Curve Predicts, but Backwards: Campbell and Shiller's Puzzle

Some empirical results are damaging because they fail to find what theory predicted. This one is worse. Campbell and Shiller found the opposite of what theory predicted, consistently, across nearly every pair of maturities in the US bond market, and the discrepancy got bigger the further out the curve you looked.

The paper's modest title, "A Bird's Eye View," undersells it. It is one of the most damaging pieces of evidence ever assembled against the expectations hypothesis, and the pattern it documents is still called the Campbell-Shiller puzzle.

The problem: the expectations hypothesis makes two predictions, not one

The expectations hypothesis says long yields are just averages of expected future short yields. If the ten-year yield is above the one-year yield, the market must be expecting short rates to rise over the coming decade, because otherwise you would just roll one-year bonds and be better off.

Campbell and Shiller notice this single idea actually implies two separate testable predictions, and they are worth separating because they behave completely differently.

Prediction one, about short rates. If the spread between a long yield and a short yield is positive, then short rates should rise over the life of the long bond. That is the direct statement of the theory: the spread exists because rates are going up.

Prediction two, about the long yield itself. If the spread is positive, then the long yield should rise in the near term. Why? Because a long bond gradually becomes a shorter bond as time passes, and if the whole curve is upward sloping because rates are expected to climb, the long yield must be climbing too. The maths of the expectations hypothesis makes this precise: a steep curve today implies the long yield goes up over the next period.

Two predictions, one theory. Test both.

The key idea via analogy: the thermometer that reads backwards

Campbell and Shiller ran these regressions across essentially every combination of maturities from one month out to ten years, which is why the paper is a "bird's eye view." Not one clever test. A systematic sweep of the whole surface.

The results split cleanly, and both halves are interesting.

Prediction one, about short rates, roughly holds. A high spread does tend to be followed by rising short rates over the long run. Not perfectly, and not with the exact coefficient the theory demands, but the sign is right and the direction is right. The curve does carry genuine information about where policy rates are headed. That is a real vindication of something.

Prediction two, about the long yield, fails catastrophically. And it does not merely fail to work. It works in reverse. When the yield curve is steep, meaning the theory says long yields should rise, the long yield tends to fall. The regression coefficient, which the theory says should be positive and equal to one, comes out negative.

And the failure gets worse in an orderly way: the longer the maturity of the bond, the more negative the coefficient becomes. This is not noise. It is a systematic, structured, sign-flipped rejection of the theory.

The analogy is a thermometer that reads backwards. If it just gave noisy readings you would call it broken and move on. But it reliably says "cold" when it is hot, and the discrepancy is bigger the hotter it gets. That is not a broken instrument. That is an instrument measuring something else entirely, and the interesting question is what.

What it is actually measuring

The answer, and this is what the paper points to, is the term premium.

The yield spread is not a pure forecast of future rates. It is a mixture of two things: what the market expects rates to do, and how much extra return investors are demanding to hold long bonds. When the spread is wide, both explanations are on the table, and the data says the second one dominates.

Specifically: a steep curve mostly means investors are demanding a large risk premium on long bonds right now. A large risk premium means long bonds are cheap. Cheap bonds subsequently deliver high returns, and a high return on a bond means its yield falls. Hence the negative sign. The curve was never predicting rising long yields; it was signalling a fat risk premium, and the fat premium was subsequently earned.

This is exactly the same conclusion Fama and Bliss reached four years earlier from the forward rate side. Two independent routes, the same destination: the shape of the curve is dominated by a time-varying risk premium, not by expectations. And the curve therefore predicts bond returns, which is a strictly more useful thing than predicting yields.

Why it mattered

• It is the definitive rejection of the expectations hypothesis. Not a failure to confirm, but a systematic, sign-reversed contradiction across the whole maturity spectrum. There is no reading of these results that rescues the theory.
• The wrong sign is what makes it famous. A theory that fails is one thing. A theory that gets the direction backwards, more and more so as you extend the horizon, is telling you that your model is not merely imprecise but pointed the wrong way. That is a far more informative failure, and it is why the result has a name.
• It confirmed that bond risk premia move, and move a lot. Together with Fama and Bliss, this paper established the central empirical fact of modern fixed income: the compensation for holding duration is not a constant. It swings around, predictably, and it dominates the shape of the curve.
• It reframed what a yield curve is for. After this, the honest question is not "what is the curve forecasting?" but "how much of this spread is expectation and how much is risk premium?" Decomposing yields into those two pieces became the central task of term structure modelling, and it is exactly what central banks now do (with essentially affine models, with all their fragility) every time they publish a term premium estimate.
• The regressions became a permanent test. "Campbell-Shiller regressions" are run on every new bond market and every new sample. The wrong sign shows up almost everywhere, in almost every country.

The honest limitations

• The econometrics is genuinely fragile, and Campbell and Shiller knew it. Yield spreads are extremely persistent, and the regressions use overlapping observations from a limited sample. That is a recipe for biased coefficients and understated standard errors. A serious literature argues that the rejection is weaker than the headline numbers suggest, and some of it argues the results are partly a statistical artefact of persistence and small samples. The puzzle is robust enough that few people think it is only an artefact, but the confidence intervals are wider than the tables imply.
• A rejection is not an explanation. The paper shows the term premium moves. It cannot say why. Is it rational compensation that rises when investors are genuinely more frightened? Is it a behavioural underreaction, investors failing to update fast enough? Is it institutional, pension and insurance demand for long duration overwhelming everything else? Thirty years later this is still argued about, and Vayanos and Vila's preferred-habitat model is one serious attempt at an answer.
• The predictability is not a money machine. A negative coefficient means steep curves precede high bond returns. Acting on that means being long duration when the curve is steep, which is usually when the economy looks terrible and everyone is frightened. The premium is a payment for enduring exactly that discomfort, and it will occasionally, and violently, not pay.
• The sample is a particular history. Postwar US data, dominated by the great inflation and then the great forty-year disinflation. A period in which long bonds delivered spectacular returns as yields fell from double digits to near zero. It is fair to ask how much of the "risk premium was earned" result is a feature of that specific, unrepeatable rate path.
• It says nothing about how to model this. Diagnosis without prescription. Building term structure models flexible enough to accommodate a wildly time-varying risk premium took another decade, and Duffee's essentially affine class is the direct response.

The one-line takeaway

Campbell and Shiller found that a steep yield curve does not predict rising long yields as the expectations hypothesis demands. It predicts falling ones, more strongly the longer the bond, because the steepness is not a forecast at all: it is the market paying a fat risk premium that long-bond holders subsequently collect.