A permutation test p-value by hand

Question

You test whether a "treatment" tweak beats "control" on four measurements. Two units got the treatment and two got control, with values

$1, \quad 2, \quad 3, \quad 8.$

The two treatment units are the ones that scored $3$ and $8$ . Your test statistic is the sum of the treatment group's values (larger favors treatment). To get an exact one-sided p-value, you consider every way to split the four numbers into two groups of two.

What is the one-sided permutation p-value?

Accepted Answer

Under the null hypothesis, the group labels are arbitrary: any of the four values could have been "treatment." So list all ways to choose which 2 of the 4 values are treatment, and compute the statistic (treatment sum) for each. There are \binom{4}{2} = 6 splits: Treatment pair Sum ------ \{1,2\} 3 \{1,3\} 4 \{2,3\} 5 \{1,8\} 9 \{2,8\} 10 \{3,8\} 11 The observed split is \{3,8\} with sum 11, the largest of all six. The one-sided p-value is the fraction of splits whose statistic is at least as extreme (as large) as observed. p-value. \frac{\{	ext{sums} \ge 11\}}{6} = \frac{1}{6} \approx 0.16…

A permutation test p-value by hand

Your answer

More Statistics questions