Give Your Sharpe Ratio a Haircut: Harvey and Liu on Honest Backtesting

There is an old rule of thumb in the quant world: whatever Sharpe ratio your backtest shows, cut it in half before you believe it. Everyone has heard it. Nobody knows where it came from. It has the comforting feel of hard-won wisdom.

Campbell Harvey and Yan Liu looked at that rule and asked the obvious question, which is: why half? Why not a third? Why not ninety percent? And why would the same discount apply to a strategy you found on your first try and a strategy you found on your five-thousandth?

Their paper does the work the rule of thumb never did. It builds an actual method for deciding how much to discount a backtested Sharpe ratio, and the answer turns out to be considerably more interesting than "halve it."

The problem: the number in your backtest is not the number you will get

Start with why a discount is needed at all.

Your backtest reports a Sharpe ratio. That number is not an estimate of your strategy's true performance. It is an estimate of your strategy's true performance plus the amount of luck required for this particular strategy to be the one you noticed.

If you tested one strategy, that luck term is small. If you tested a thousand and reported the best, the luck term is enormous, because the winner of a thousand-strategy contest is, by construction, the one that got the best draw of noise. This is the same multiple-testing logic that Harvey, Liu and Zhu applied to published factors, turned inward on your own research.

So the observed Sharpe is inflated, and the inflation grows with the size of your search. The task is to work out how much to deflate it.

The key idea via analogy: the winner of a big tournament is not that good

Picture a golf tournament. If a golfer shoots a spectacular round, how good are they really?

If it is a two-person match, a spectacular round is strong evidence of genuine skill. If it is a field of ten thousand amateurs, then somebody was always going to shoot a spectacular round, and the winner is probably a decent golfer who had a very good day. Your best guess of their true ability is well below what you just watched. This is the statistician's idea of shrinkage: pull the extreme observed result back toward the average, and pull harder when the field was bigger.

Harvey and Liu apply exactly this to Sharpe ratios. They take the multiple-testing corrections that statisticians use to control false discoveries, the classic Bonferroni correction, the Holm correction, and the Benjamini-Hochberg false discovery rate procedure, and turn each of them around. Instead of asking "does this pass?", they ask "what Sharpe ratio would have passed, and by how much did we exceed it?" From that they back out an adjusted Sharpe: the performance level that is actually defensible once you account for the search.

The percentage you have to give up is what they call the haircut.

The result that kills the fifty percent rule

Here is the finding that makes the paper worth reading, and it is genuinely counterintuitive.

The haircut is not a fixed percentage. It is non-linear, and it depends heavily on how good the raw Sharpe was in the first place.

• A strategy with a marginal backtested Sharpe, say something that only just looks interesting, gets absolutely destroyed. Its haircut is severe, often approaching total. Why? Because a mediocre result is exactly what a large search produces from nothing. It is the most likely thing to be pure noise, so almost none of it survives.
• A strategy with a genuinely outstanding backtested Sharpe gets a comparatively gentler haircut. Why? Because random search struggles to produce a truly extreme result. The bigger the observed number, the more of it is likely to be real.

So the fifty percent rule is wrong in both directions at once. It is far too kind to weak strategies (which deserve to be thrown out entirely) and unnecessarily harsh on strong ones (which deserve more credit than half). Harvey and Liu are explicit that using a flat fifty percent discount is a serious mistake, and their framework tells you where the real cut falls.

And of course the haircut also grows with the number of tests. The same raw Sharpe means one thing after ten trials and something very different after ten thousand.

Why it mattered

• It made the discount defensible. Before, an allocator would eyeball a backtest and apply a gut discount. Now there is a documented, reproducible procedure with a statistical justification. The authors even published code, which is why this paper actually got used rather than just cited.
• It reframed the interview question. "What Sharpe did your backtest show?" is now an incomplete question, and any serious quant will ask the follow-up: over how many trials, and on how much data?
• It rescued the strong strategies. This is a nice, under-appreciated twist. The paper is usually filed under "sceptical warnings about backtests," but it actually says the sceptics have been over-penalising the genuinely good results. That is a constructive message, not just a destructive one.
• It sits at the heart of a small family of tools. Together with the Deflated Sharpe Ratio and the Probability of Backtest Overfitting, it forms the practical toolkit for asking "is my edge real?" rather than merely worrying about it.

The honest limitations

• You still have to count your trials, and you cannot. The whole apparatus needs a number for how many strategies were tested. Your honest answer includes every idea you tried and dropped, every parameter you nudged, and every insight you absorbed from other people's published backtests. Nobody knows that number. Any figure you plug in is a guess, and it is almost certainly too low.
• It assumes your tests are of a certain shape. The corrections carry assumptions about how the test statistics behave and how they relate to one another. Strategies tested on the same data are correlated in complicated ways, and the adjustments handle this only approximately.
• A haircut Sharpe is still just a Sharpe. It says nothing about drawdowns, about whether the returns are horribly skewed, about capacity, or about whether the strategy quietly stops working the moment you add transaction costs. Passing the haircut is necessary, not sufficient.
• It corrects for luck, not for lies. If your backtest suffers from look-ahead bias, survivorship bias, or a bug, no statistical adjustment will save you. The haircut assumes your backtest is honest and merely lucky. That is a generous assumption.
• It can be gamed by under-reporting. The framework's output depends on an input the researcher supplies and nobody can audit. A researcher who wants a flattering number simply reports a smaller trial count.

The one-line takeaway

Harvey and Liu replaced the folk rule "halve your backtested Sharpe" with an actual method, and showed the correct discount is non-linear: marginal backtest results deserve to be discounted almost to nothing, while genuinely outstanding ones deserve a much lighter cut than tradition assumed.