The Riskiest Stocks Paid the Least: the Volatility Puzzle

Every finance textbook contains a reassuring lesson: you do not get paid for taking risk you could have diversified away. A stock's own private wobble, the part unrelated to the market, is called idiosyncratic volatility, and since you can wash it out simply by owning many stocks, nobody should compensate you for it. The market only pays for risk you cannot escape.

The prediction that follows is clear. Idiosyncratic volatility should have no relationship at all with expected returns. It should be an irrelevance.

In 2006, Andrew Ang, Robert Hodrick, Yuhang Xing and Xiaoyan Zhang looked and found a relationship. And it went the wrong way, hard.

The problem: the stocks with the most private wobble did worst

Sort every stock by how much it bounces around for reasons unrelated to the market. Put the calmest fifth in one bucket, the wildest fifth in another. Then look at what they earned going forward.

The finding: the high idiosyncratic volatility bucket earned abysmally low average returns. Not "the same as everything else," which is what theory predicted. Substantially, painfully worse. And the effect was not explained away by size, by value, by momentum, or by liquidity. It was its own thing, and it was pointing in the direction no model wanted.

This is worse than an anomaly that says you get paid for something you should not. This is an anomaly that says you get punished for taking on extra risk. Nobody has a comfortable theory for that.

The paper also documented a second, related result about aggregate volatility: stocks that do well when overall market volatility spikes are valuable as insurance, so investors accept lower returns to hold them, which is exactly what the data showed. That result actually makes sense. It is the idiosyncratic finding that broke things.

The key idea via analogy: everybody wants the lottery ticket

Why would investors willingly hold stocks that are wild and pay badly? The dominant explanation is that some investors are not really investing at all. They are buying lottery tickets.

Consider what a high idiosyncratic volatility stock actually looks like from the outside. It is the speculative biotech with one drug in trials. It is the tiny company that might get bought out tomorrow. It is the meme stock. These are names with a small chance of a spectacular payoff.

Now recall how people behave with actual lotteries. They cheerfully pay one dollar for a ticket whose true expected value is fifty cents, because the tiny chance of the enormous prize is worth more to them than the arithmetic says. People systematically overpay for a small chance of a big win.

Stocks with huge idiosyncratic volatility are the stock market's lottery tickets. If a chunk of investors overpays for that lottery-ticket quality, then those stocks are, by definition, expensive. And expensive stocks earn low subsequent returns. The negative relationship is not mysterious once you accept that some buyers are not maximising expected return, they are buying a thrill.

A second, complementary explanation is about the people who should be correcting this. Arbitrageurs ought to short these overpriced lottery stocks and drive them back to fair value. But shorting a stock that could triple overnight is terrifying, expensive, and career-threatening. High idiosyncratic volatility is precisely what makes an overpriced stock hard to arbitrage. So the mispricing survives because the very feature that creates it also protects it.

Why it mattered

• It broke a textbook prediction cleanly. Not "the effect is smaller than we thought," but "the effect has the wrong sign." That commands attention.
• It became the reference point for the whole low-volatility literature. The idea that boring stocks beat exciting stocks now has several strands: low beta (Frazzini and Pedersen), low total volatility, and low idiosyncratic volatility. This paper is the anchor for the last of these, and it turned "the low-volatility anomaly" into a research programme.
• It handed practitioners a simple, powerful filter. Avoiding the highest-volatility stocks turned out to improve portfolios rather than reduce their returns, which is close to a free lunch and became a widely used screen.
• It fed the lottery-preference literature. It helped establish the idea that a meaningful slice of investors have a taste for skewness, for the small chance of a big score, and that this taste has measurable consequences for prices.

The honest limitations

• The measurement choices matter more than they should. How you estimate idiosyncratic volatility, over what window, with which model, and how long you hold, all change the results. Later papers showed that using a different measurement window can weaken or even flip the effect, which is a serious concern for something this simple.
• The effect is concentrated in the nastiest stocks. Much of the return comes from the short leg, which is full of tiny, illiquid, expensive-to-borrow names. That is exactly where a paper profit does not survive contact with real trading costs.
• The explanation is disputed. Lottery preferences, arbitrage limits, and various measurement artefacts have all been proposed. Some researchers argue the effect is mostly a story about a handful of extreme stocks around earnings announcements rather than a broad phenomenon.
• There is no risk-based story at all. Frazzini and Pedersen at least explain low beta with a mechanism (leverage constraints). The idiosyncratic volatility result has no comparably clean economic account, which leaves it looking more like a documented fact than an understood one.

The one-line takeaway

Theory says a stock's private, diversifiable wobble should earn you nothing extra. Ang, Hodrick, Xing and Zhang found it earns you less than nothing, most likely because the wildest stocks are the market's lottery tickets and investors reliably overpay for the chance of a jackpot.