Welch's Test for Unequal Variances

The standard two-sample t-test compares the means of two groups, but its usual formula for the standard error assumes both groups share the same underlying variance and just pools them together. Welch's test drops that assumption: it computes the standard error from each group's own variance separately, then uses a special formula — the Welch-Satterthwaite equation — to work out an adjusted, often non-integer, degrees-of-freedom for the resulting t-distribution.

Why it matters: if one group is genuinely noisier than the other — say, comparing a strategy's returns in a high-volatility regime against a low-volatility one — the pooled-variance t-test can understate or overstate the true uncertainty in the difference of means, giving a p-value that's too optimistic. Welch's version stays accurate in that mismatched-variance case, at the cost of only a small loss of power when the variances actually are equal, which is why most statistical software now defaults to it.

Say strategy A has 40 daily returns with variance 4 and strategy B has 60 daily returns with variance 1. A pooled t-test would blend these into one shared variance estimate, which understates B's genuine precision and overstates A's. Welch's formula instead computes the standard error as $\sqrt{4/40 + 1/60} \approx 0.324$ using each sample's own variance, and derives degrees of freedom around 46 rather than the pooled test's 98 — a smaller, more conservative number that reflects how much less reliable the noisier, smaller sample A actually is.