When a permutation test's exchangeability assumption breaks

Question

You compare two trading strategies, A and B, each with one daily return for the same $250$ trading days. To test whether A beats B, an analyst pools all $500$ daily returns, then runs a two-sample permutation test: repeatedly relabel which returns are "A" and which are "B," and compare the group means. The test reports A significantly ahead.

Explain why this permutation test is invalid here, and what resampling scheme respects the structure of the data.

Accepted Answer

The core mistake: broken exchangeability. A permutation test is valid only when, under the null hypothesis, the labels are exchangeable, meaning every relabeling is equally likely. Pooling all 500 returns and shuffling the A/B labels freely assumes any A-return could just as well have been a B-return on any day. That is false here for two reasons: 1. The two series are paired by day. A's Monday return and B's Monday return both react to the same market on Monday; they are correlated, not interchangeable with returns from other days. 2. Cross-sectional dependence. On a big up-day for the whole…

When a permutation test's exchangeability assumption breaks

Your answer

More Statistics questions