Best of 100 backtests over 9 years: where is the luck bar?

Question

A team screens 100 signal variants over 9 years of data and reports the best, which has an annualized Sharpe of 1.5. Suppose none of the 100 has any genuine edge.

What Sharpe would the best of 100 zero-edge variants show by luck alone, and what does that imply about the reported 1.5?

Accepted Answer

Each junk backtest is noise. A zero-edge strategy over T years produces a sample Sharpe with standard error about \operatorname{SE}0 \approx \sqrt{1/T}: each variant lands near zero with that typical spread. Best of many. The expected maximum of N standard-normal draws is about \sqrt{2\ln N}, so the luck bar for the best-of-N Sharpe is 	ext{luck bar} \approx \sqrt{\frac{2\ln N}{T}}. Plug in the numbers. With N = 100 and T = 9: \sqrt{2\ln 100} = \sqrt{2 	imes 4.605} = \sqrt{9.21} \approx 3.03,\qquad 	ext{luck bar} \approx \frac{3.03}{\sqrt{9}} = \frac{3.03}{3} \approx 1.01. Interpretation. …

Best of 100 backtests over 9 years: where is the luck bar?

Your answer

More Statistics questions