Roll's Critique: The Paper That Said You Can Never Test the CAPM

By the mid-1970s the CAPM was the reigning theory of finance and a small industry had grown up around testing it. Researchers would take a stock index, measure each stock's sensitivity to it, and check whether higher sensitivity really did come with higher average returns. Results were mixed but people kept refining the tests.

Then Richard Roll published a paper arguing that all of these tests, past and future, were essentially meaningless. Not wrong. Meaningless: they were not testing what their authors thought they were testing. It landed like a bomb, and nearly fifty years later nobody has properly defused it.

The problem: the theory is about something you cannot see

The CAPM says a stock's expected return depends on its sensitivity to the market portfolio. And "the market portfolio," in the theory, does not mean the S&P 500. It means every risky asset that exists, weighted by value. Every stock in every country. Every bond. Every privately held business, every apartment building, every gold bar, every painting. And, most awkwardly, human capital: the discounted value of everybody's future wages, which is plausibly the single largest asset class on the planet and is completely untraded.

Nobody has ever measured this portfolio. Nobody ever will. So every empirical test has quietly substituted a stock index as a stand-in and hoped it was close enough.

Roll's contribution was to prove that "close enough" is not a thing here.

The key idea, via analogy

Suppose a theory says: "Everything falls toward the center of the Earth." To test it, you need to know where the center of the Earth is. Now suppose you cannot locate it, so you pick a nearby landmark and test whether everything falls toward that. Two things go wrong. If objects do not fall toward your landmark, you have learned that your landmark is not the center of the Earth, which you already knew. And if objects do seem to fall toward your landmark, you have learned that your landmark happens to line up with the real center, not that the theory is true.

Roll's mathematical result is a sharper version of this. He showed that the CAPM's central equation, the straight-line relationship between beta and expected return, is not an economic claim that can pass or fail a test. It is a mathematical identity that holds, exactly, for any portfolio that sits on the efficient frontier, and fails for any portfolio that does not. It is baked into the algebra of mean and variance. No economics required.

The consequences are brutal:

• If you pick an index that happens to be efficient, the beta-return line will look perfect. Not because the CAPM is true, but because it must look perfect. It is arithmetic.
• If you pick an index that is not efficient, the line will look broken. Not because the CAPM is false, but because your proxy is off the frontier.
• Therefore every "test of the CAPM" is really just a test of whether the index you chose is mean-variance efficient. It tells you nothing whatsoever about the actual theory, which is a claim about a portfolio you did not use and cannot observe.

Roll added a second knife. Common proxies are highly correlated with each other, so a proxy can be almost perfectly correlated with the true market portfolio and still give completely wrong answers about individual stock betas and about whether the model holds. Being close is not enough. The error does not shrink gracefully.

Why it mattered

• It reframed a generation of empirical finance. After 1977, honest researchers had to describe their work as testing a joint hypothesis: the CAPM is true and my index is a good proxy for the market. When the test fails, you cannot tell which half broke. That framing is now standard, and Fama would generalize it into the famous joint-hypothesis problem for market efficiency.
• It explains why the anomaly literature is so contested. Every claimed "anomaly," meaning a return the CAPM cannot explain, might just be evidence that the S&P 500 is not the true market portfolio. Roll's critique is the permanent escape hatch for the model's defenders, and the permanent frustration of its attackers.
• It pushed the field toward multi-factor models. If a single unobservable portfolio is a dead end, maybe use several observable factors instead. The critique is part of why APT and, later, Fama-French style models became attractive: they do not depend on identifying one true portfolio.
• It is a masterclass in scientific skepticism. The paper's real lesson is broader than finance: before you run the test, ask whether the test can possibly distinguish the hypothesis from its negation. A shocking amount of empirical work fails that check.

The honest limitations

• It is nihilistic, and not everyone accepts that. Critics point out that if you take Roll seriously and literally, the CAPM becomes unfalsifiable, and an unfalsifiable theory is arguably not science at all. Some see this as an indictment of the CAPM rather than a defense of it.
• Practitioners have to use something. "The true market portfolio is unobservable" is correct and completely unhelpful to someone who needs a discount rate by Friday. In practice people use a broad index, accept the bias, and move on. Roll does not offer a replacement.
• Later work softened the blow. Stambaugh, among others, examined how much conclusions change when you broaden the proxy to include bonds and other assets, and found that some inferences are more robust than Roll's starkest reading implies. The critique is real, but its practical bite is debated.
• It is Part I. The paper is explicitly the first installment of a longer argument, and the sequels are less famous than the bombshell.

The one-line takeaway

Roll showed that the CAPM's beta-return line is an algebraic consequence of using an efficient portfolio, not an economic prediction, so every test of the CAPM is really a test of the index you happened to pick, and since the true market portfolio can never be observed, the theory can never be honestly tested at all.