Specifying the Null Hypothesis for a Strategy

A systematic long-only equity strategy reports 11% annualized return over ten years, a t-statistic of 3.9 against a null of zero return, and gets funded on that basis. Over the same ten years, a simple buy-and-hold of the same universe's cap-weighted index returned 10.4%. The strategy's actual edge over the thing an investor could get for free was 0.6% a year, with a t-statistic against that null of about 0.4 — indistinguishable from zero. The 3.9 was real. It also answered the wrong question, because "beats zero" was never the null that mattered to anyone deciding whether to fund this over doing nothing.

Why the null hypothesis is a choice, not a default

Every significance test compares an observed statistic against some baseline of "nothing interesting is happening." For a trading strategy, "nothing interesting" has several honest candidates, and they are not remotely equivalent:

• Zero return. Tests only whether the strategy makes money at all — the weakest possible bar, cleared by holding any asset with positive long-run drift.
• Risk-free rate. Tests whether the strategy compensates for taking risk at all, a slightly higher bar.
• A relevant benchmark (cap-weighted index for equity long-only, a passive carry basket for a carry strategy). Tests whether the strategy beats a naive, investable alternative — usually the bar that actually matters to an allocator with that alternative available.
• A matched-risk naive rule (e.g., a random long-short portfolio with the same turnover and sector exposures). Tests whether the specific construction — not just directional exposure — adds value.

Each choice changes both the effect size and its standard error, because the benchmark's own variance and its correlation with the strategy both enter the calculation.

Worked example: the real t-stat against the right null

Strategy return: mean 11.0% annualized, SD 14%, over $T=10$ years (annualized SE $\approx 14\%/\sqrt{10} = 4.43\%$). Against zero: $t = 11.0/4.43 \approx 2.48$ — still significant.

Now test against the benchmark using the difference series (strategy minus benchmark each day), the statistically correct way to net out a common factor. That difference series has mean 0.6% and SD 5.2% annualized (much lower than either return alone, since strategy and benchmark share most of their variance). SE of the mean difference: $5.2\%/\sqrt{10}\approx1.64\%$.

$$t = \frac{0.6}{1.64} \approx 0.37.$$

Against the benchmark, outperformance is nowhere near significant. Both t-statistics are computed correctly; only one — beating what an investor could hold for free — determines whether this strategy deserves capital over a cheap index fund.

Worked example: choosing the null changes the ranking

Manager P returns 9% with volatility 10%. Manager Q also returns 9% at 10% vol, but Q's book is explicitly long high-beta, high-momentum names, so the matched-risk null (a naive momentum basket) itself returns 8.2% at 9.5% vol. Against zero, P and Q look identical. Against their correctly-matched nulls, P clears its bar comfortably while Q barely beats its own naive comparison — exposing that Q's "9%" is mostly a bet the market pays for free.

Drag the correlation on the scatter above toward 1: that's what a strategy and its benchmark look like when they share most of their variance, which is exactly why the difference series — not each series' own mean — is the statistically correct thing to test.

What this means in practice

Write down the specific, investable alternative your strategy needs to beat before running any significance test, and compute the t-statistic on the difference series against it — not on each series' mean separately. For long-only equity that's almost always a cap-weighted benchmark; for a factor tilt, a matched-risk version of the same tilt; only for genuinely uncorrelated strategies is zero the honest choice.