The best of 100 random strategies looks brilliant

Question

A researcher tunes a strategy across 100 parameter combinations and reports the best one, which has an in-sample annualized Sharpe of 2.0 that looks highly "significant." Assume, for argument, that none of the 100 variations has any real edge.

Explain why the maximum Sharpe over many trials is inflated, roughly how much, and what "deflating" the Sharpe means.

Accepted Answer

Sharpe is estimated with noise. Even a genuinely zero-edge strategy produces a nonzero sample Sharpe by chance. The standard error of an estimated annualized Sharpe over T years of returns is roughly \sqrt{(1 + 	frac12 SR^2)/T}; under the null (SR = 0) that's about \sqrt{1/T}. So each backtest's Sharpe is a draw from a distribution centered near zero with that spread. Taking the max over many trials. When you run 100 independent zero-edge trials and keep the best, you're sampling the maximum of 100 noisy estimates. The expected maximum of N standard normals grows like \sqrt{2 \ln N}: \mathbb…

The best of 100 random strategies looks brilliant

Your answer

More Statistics questions