Minimum two-sided p-value for six-versus-six

Question

You run an exact two-sample permutation test comparing two groups of $6$ ( $n = 12$ total), using a two-sided statistic whose null distribution is symmetric.

What is the smallest two-sided p-value this test can produce?

Accepted Answer

The number of distinct group assignments is \binom{12}{6} = 924. For a symmetric two-sided test, the most extreme observed split is matched by its mirror image on the opposite tail, so the two most extreme arrangements share the smallest two-sided p-value: p{\min} = \frac{2}{924} \approx 0.0022. With 924 arrangements the test has fine resolution, easily able to report p-values well below 0.01. Doubling the group sizes from three to six per side takes the two-sided floor from 0.10 down to about 0.002. Takeaway. Resolution improves fast with sample size: a six-versus-six permutation test can re…

Minimum two-sided p-value for six-versus-six

Your answer

More Statistics questions