Best of 200 backtests over 6 years: deflate the winner

Question

A researcher grid-searches 200 configurations over 6 years and reports the best, which has an annualized Sharpe of 1.4. Suppose none of the 200 has any genuine edge.

What Sharpe would the best of 200 zero-edge configurations show by luck alone, and what does that imply about the reported 1.4?

Accepted Answer

Each junk backtest is noise. A zero-edge strategy over T years yields a sample Sharpe with standard error about \operatorname{SE}0 \approx \sqrt{1/T}. Best of many. The expected maximum of N standard-normal draws is about \sqrt{2\ln N}, so the luck bar for the best-of-N Sharpe is 	ext{luck bar} \approx \sqrt{\frac{2\ln N}{T}}. Plug in the numbers. With N = 200 and T = 6: \sqrt{2\ln 200} = \sqrt{2 	imes 5.298} = \sqrt{10.60} \approx 3.26,\qquad 	ext{luck bar} \approx \frac{3.26}{\sqrt{6}} = \frac{3.26}{2.449} \approx 1.33. Interpretation. Testing 200 configurations pushes the luck bar up to…

Best of 200 backtests over 6 years: deflate the winner

Your answer

Discussion

Learn the concepts

Related questions