Permutation test resolution for three-versus-three

Question

You run an exact two-sample permutation test comparing a group of 3 against another group of 3 (n = 6 total), using a one-sided statistic. What is the smallest one-sided p-value this test can return?

Accepted Answer

The permutation null has one entry per way of choosing which 3 of the 6 units form the first group: \binom{6}{3} = 20. If the observed split is the single most extreme of these, the one-sided p-value is p{\min} = \frac{1}{20} = 0.05. So the best this design can do, one-sided, is exactly 0.05: it sits right on the conventional threshold and cannot go below it. A two-sided version would bottom out at 2/20 = 0.10, which never clears 0.05. Small balanced designs are borderline by construction. Takeaway. With \binom{6}{3} = 20 arrangements, a three-versus-three permutation test can reach exactly p…

Permutation test resolution for three-versus-three

Your answer

More Statistics questions