Economic Versus Statistical Significance

A cross-sectional signal on 6,000 stocks over 15 years of daily data reports a t-statistic of 4.8 on its average return. Any statistics textbook calls that overwhelming — the probability of seeing a t-stat that large if the true mean return were zero is under one in a million. The average daily return behind that t-stat, though, is 1.8 basis points. Round-trip cost to trade the signal, including spread and market impact, is 6 basis points. The signal is real. It also loses money on every single trade, guaranteed, before you've placed a single order.

Two different questions

Statistical significance answers: is the average return distinguishable from zero, given how much data I have? With enough observations, almost any nonzero effect eventually clears a significance bar — the t-statistic for a mean is $t = \bar{r}/(s/\sqrt{n})$, and $\sqrt{n}$ in the denominator means the bar gets easier to clear the longer your sample, even while $\bar{r}$ itself stays the same tiny number.

Economic significance answers a completely different question: is the average return large enough, after realistic frictions, to be worth trading at all? That comparison has nothing to do with sample size — it's simply the return per trade against the cost per trade.

Worked example: the vanishing edge

Daily long-short signal, 6,000-stock universe, 15 years (≈3,780 trading days). Average daily long-short return: $\bar{r} = 0.018\%$ (1.8 bps). Daily standard deviation of that return: $s = 0.65\%$.

$$t = \frac{\bar{r}}{s/\sqrt{n}} = \frac{0.00018}{0.0065/\sqrt{3780}} = \frac{0.00018}{0.0065/61.5} = \frac{0.00018}{0.0001057} \approx 1.70.$$

Not yet overwhelming at $t \approx 1.7$ — but real signals like this carry autocorrelation from overlapping holding periods, and correcting for it can raise the effective significance. Say the corrected estimate is $t=4.8$, as reported. The signal clears every statistical bar you'd set.

Now the economic side: round-trip cost is estimated at 6 bps (half-spread plus impact, in and out). Net average return per trade: $1.8 - 6 = -4.2$ bps. The strategy is statistically proven to make 1.8 bps gross and mechanically guaranteed to lose 4.2 bps net, every single time it trades.

Worked example: the reverse trap

A separate momentum signal shows an average monthly return of 45 bps with a t-stat of only 1.4 — below the conventional 1.96 bar, so a strict significance filter would discard it. But the strategy trades monthly, in large-cap names, at 3 bps round-trip cost. Net return per trade: $45 - 3 = 42$ bps, compounding to roughly 6% a year net of costs, on modest position turnover. The signal is not "statistically significant" by the usual convention, because 15 years of monthly rebalancing is only 180 observations and monthly returns are noisy — but it is economically enormous relative to its costs. A researcher who filters ideas by p-value alone throws this one away and keeps the 1.8 bps one from before.

The t-statistic's denominator shrinks as $1/\sqrt{n}$ — drag the exponent on the curve above toward a square-root shape and you're looking at exactly how the statistical bar gets easier over time while the economics, which don't depend on $n$ at all, stay fixed.

What this means in practice

Compute net return per trade against your best cost estimate before computing a t-stat at all — if it's negative, the test is answering a question nobody needs answered. If positive, still report the t-stat: it tells you how much of that edge might be noise. Both numbers together, not either alone, are what a portfolio manager should see.