Counting arrangements for a ten-item permutation test

Question

You run an exact two-sample permutation test that splits 10 items into two groups of 5, using a one-sided statistic. What is the smallest one-sided p-value this test can return?

Accepted Answer

The permutation null has one entry per way to choose which 5 of the 10 items go in the first group: \binom{10}{5} = 252. If the observed split is the single most extreme arrangement, the one-sided p-value is p{\min} = \frac{1}{252} \approx 0.0040. So this design can resolve p-values down to about 0.004, comfortably below 0.05 and even 0.01. Note how sharply resolution depends on the split: five-versus-five gives 252 arrangements, whereas two-versus-eight would give only \binom{10}{2} = 45, a much coarser floor of 1/45 \approx 0.022. Takeaway. For a fixed n, balanced groups maximize the number…

Counting arrangements for a ten-item permutation test

Your answer

More Statistics questions