One-Sample and Two-Sample t-Tests

A strategy's daily returns average +0.05% over the last year. Is that a real edge, or exactly the kind of average you'd get from pure noise centered at zero? If you knew the true underlying volatility of the noise with certainty, this would be a simple calculation. In practice you never know it — you only have the same handful of data points to estimate both the average and how noisy that average's estimate is. The t-test is what you get when you honestly account for that extra layer of uncertainty, and it is the single most common hypothesis test in applied statistics.

An analogy: judging a shaky hand with a shaky ruler

Imagine measuring how far off-center a dart lands, but your ruler itself is slightly unreliable and you don't know exactly how unreliable — you have to estimate the ruler's own wobble from the same handful of measurements you're using to judge the dart. That double layer of uncertainty — not knowing the true average and not knowing the true noise level, estimating both from the same small sample — makes your confidence in "the dart missed center" weaker than if you had a perfectly calibrated ruler. The t-distribution is exactly this: a version of the normal (bell curve) with fatter tails, built to honestly reflect that extra layer of not-knowing when you must estimate your own noise level.

The one-sample formula, one symbol at a time

Testing whether a sample's true mean $\mu$ equals some hypothesized value $\mu_0$ (often zero), with sample mean $\bar{x}$, sample standard deviation $s$, and sample size $n$:

$$t = \frac{\bar{x} - \mu_0}{s/\sqrt{n}}$$

In plain English: take the gap between what you observed and what you're testing against, and divide by the estimated standard error of your mean — how much your sample mean would naturally bounce around from sample to sample, estimated from the same data. Compare $t$ against the t-distribution with $n-1$ Degrees of Freedom; a $t$ far from zero, beyond the relevant critical value, means the observed mean is unlikely under the null.

The two-sample formula, one symbol at a time

Comparing two independent groups' means, $\bar{x}_1$ and $\bar{x}_2$, with sample sizes $n_1, n_2$ and a pooled standard deviation $s_p$ combining both groups' spread:

$$t = \frac{\bar{x}_1 - \bar{x}_2}{s_p\sqrt{1/n_1 + 1/n_2}}$$

In plain English: the gap between the two group averages, divided by the estimated standard error of that gap. The logic is identical to the one-sample case; only the quantity being tested (a difference of two means, instead of one mean against a fixed value) and the degrees of freedom (roughly $n_1+n_2-2$) change.

The t-distribution looks like the bell curve pictured above but with visibly heavier tails, especially at small degrees of freedom — a concrete penalty for having estimated the noise level from the same limited sample instead of knowing it in advance.

Worked example 1: one-sample t-test on a strategy's daily returns

Twenty-five days of returns average $\bar{x} = 0.08\%$ with sample standard deviation $s = 0.30\%$. Test against $\mu_0 = 0$:

$$t = \frac{0.08 - 0}{0.30/\sqrt{25}} = \frac{0.08}{0.30/5} = \frac{0.08}{0.06} = 1.33$$

With $n-1=24$ degrees of freedom, the two-sided critical value at $\alpha=0.05$ is about $2.064$. Since $1.33 < 2.064$, this result is not statistically significant — 25 days of data is not enough to distinguish this average from pure noise around zero.

Worked example 2: two-sample t-test comparing strategies

Strategy A: $n_1=40$, mean $\bar{x}_1=0.10\%$. Strategy B: $n_2=40$, mean $\bar{x}_2=0.03\%$. Pooled standard deviation $s_p = 0.20\%$.

$$t = \frac{0.10-0.03}{0.20\sqrt{1/40+1/40}} = \frac{0.07}{0.20\sqrt{0.05}} = \frac{0.07}{0.20(0.2236)} = \frac{0.07}{0.0447} = 1.566$$

With roughly $n_1+n_2-2=78$ degrees of freedom, the critical value at $\alpha=0.05$ is close to $1.99$. Since $1.566 < 1.99$, the difference between the two strategies' average returns is not statistically significant at this sample size, even though Strategy A's raw average is more than three times Strategy B's.

What this means in practice

The t-test is the default first check for "did this strategy actually make money on average, or is the headline number indistinguishable from noise?" and the two-sample version is the default check for "is Strategy A actually better than Strategy B?" Both examples above show a common and important pattern in quant research: a strategy or comparison can look impressive in raw numbers (a 0.08% daily average, more than triple the comparison's average) and still fail to clear statistical significance, because daily return noise is large relative to typical edges.