The Mann-Whitney U Test

A standard t-test asks whether two groups have different means, but it assumes the underlying data is roughly normal — an assumption that's often shaky for skewed data like trading returns, latency measurements, or anything with occasional large outliers. The Mann-Whitney U test answers a related but more robust question — is one group's values systematically larger than the other's? — using only the ranks of the data, with no distributional assumption at all.

The procedure pools both groups' observations together, ranks every value from smallest to largest across the combined pool, and then sums the ranks belonging to one of the two groups. If the two groups really come from the same distribution, that group's observations should occupy a roughly even spread of high and low ranks; if one group's values tend to be systematically larger, its rank sum will be noticeably higher than chance would produce. The test statistic $U$ converts this rank sum into a measure of how many pairwise comparisons (one observation from each group) favor one group over the other, and its distribution under the null hypothesis is known, letting you compute a p-value.

For example: comparing 5 days of returns for strategy A against 5 days for strategy B, pool all 10 values and rank them 1 (smallest) through 10 (largest). If strategy A's five observations happen to occupy ranks 6 through 10 entirely, that's an extreme configuration under the null of "no difference," and it would produce a small p-value — evidence A's returns are systematically higher, without ever assuming either return series is normally distributed.

Because it only uses ranks, the Mann-Whitney U test is far less sensitive to outliers than a t-test comparing means directly, which is why it's a common robustness check for comparing two strategies' or two regimes' return samples when the data looks non-normal.