The best of fifty backtests shows a Sharpe of 1

Question

You backtested 50 candidate strategies and kept the single best one, which shows an annualized Sharpe of 1.0 over 4 years, a t-statistic of t = SR\sqrt{T} = 1\cdot 2 = 2. Is that t-stat of 2 real evidence? What threshold should you actually require after searching 50 strategies?

Accepted Answer

A naive t of 2 looks "significant at 5%", but that threshold assumes you ran one test. You ran 50 and reported the maximum, so the bar must rise. Bonferroni view. To hold the family-wise error rate at 5\% across 50 tests, each test must clear \alpha = 0.05/50 = 0.001. The two-sided critical value for 0.001 is about z = 3.29, not 2. So the winner needs SR\sqrt{T} \ge 3.29 \implies T \ge \left(\frac{3.29}{1}ight)^2 \approx 10.8 	ext{ years}, nearly triple the 4 years you have. At 4 years a Sharpe-1 strategy only reaches t = 2, comfortably short of the 3.29 that 50-way searching demands. Expe…

The best of fifty backtests shows a Sharpe of 1

Your answer

More Statistics questions