The False Strategy Theorem

A researcher tests 200 variants of a mean-reversion signal — different lookbacks, different entry thresholds, different universes — on five years of daily data. One of them comes back with an annualised Sharpe of 1.5. It looks like a discovery. It is almost exactly what you'd expect to see if every single variant were pure noise with zero true edge.

Think of 200 people flipping a fair coin 20 times each. Someone is going to get 15 or more heads. That person did not discover a biased coin; they were the extreme of a crowd. Run enough backtests on the same data and the best-looking one is, by construction, an extreme value.

The formula, in words first

If a strategy has true Sharpe ratio zero, its estimated Sharpe over a finite sample still bounces around, because you only observed one finite history. The standard error of that estimate shrinks with the square root of the number of observations, $T$: roughly $\sigma_{SR} \approx \sqrt{1/T}$ for the per-period ratio.

Now run $N$ independent backtests, all with zero true edge, and keep the best one. The expected value of the maximum of $N$ noisy draws is not zero — it grows with $N$:

$$E[\max SR_N] \;\approx\; \sigma_{SR}\sqrt{2\ln N}.$$

In words: the Sharpe you should expect from the luckiest strategy in a pile of worthless ones equals the per-trial noise level times a factor that grows only slowly, with the square root of the log of the trial count. Slowly is the trap — the illusion never stops growing, and it outpaces most researchers' intuition for "I only tried a few things."

Worked example 1: 200 trials, 5 years of daily data

Take $T = 1260$ daily observations (five years). The per-period Sharpe noise annualises as $\sigma_{SR} = \sqrt{252/1260} = \sqrt{0.2} \approx 0.447$. With $N = 200$ independent trials, $\sqrt{2\ln 200} = \sqrt{2 \times 5.30} = \sqrt{10.6} \approx 3.26$.

$$E[\max SR_{200}] \approx 0.447 \times 3.26 \approx 1.46.$$

That 1.5 Sharpe the researcher found is almost exactly what 200 worthless strategies produce by chance. Nothing needs to be true for that number to appear.

Worked example 2: how many trials explain a Sharpe of 2.0?

Flip the question. A researcher reports Sharpe 2.0 on three years of daily data ($T = 756$, so $\sigma_{SR} = \sqrt{252/756} = \sqrt{0.333} \approx 0.577$). Solve for $N$:

$$\sqrt{2\ln N} = \frac{2.0}{0.577} \approx 3.47 \;\Rightarrow\; \ln N \approx 6.01 \;\Rightarrow\; N \approx 407.$$

A grid search over roughly 400 parameter combinations — five lookbacks times four thresholds times four universes times five entry rules is already 400 — is expected to hand you a 2.0 Sharpe out of pure noise. Most quant shops run grid searches bigger than that without a second thought.

Drag the standard deviation on the plot above and picture 200 draws from that curve: the maximum lands far out in the right tail even though the curve is centred on zero — exactly the number the theorem predicts.

What this means in practice

The theorem isn't an argument against searching — it's an argument for tracking $N$ and discounting for it, as the The Deflated Sharpe Ratio does. Log every configuration tried, not just the winner; hold out a final validation window no configuration ever touched; and treat an in-sample Sharpe near $\sigma_{SR}\sqrt{2\ln N}$ for your actual trial count as indistinguishable from luck until it survives that untouched data.