The two-sided exact p-value for 10 of 12

Question

A signal is right on 10 of 12 independent calls. This time you had no pre-committed direction: you want to test whether the signal differs from a coin flip in either direction (unusually right or unusually wrong), under the null that each call is correct with probability $0.5$ .

Compute the two-sided exact p-value and explain how it relates to the one-sided value.

Accepted Answer

Why two-sided. With no pre-registered direction, a result is "at least as extreme" if it is far from the expected 6 correct in either tail. By the symmetry of 	ext{Binomial}(12, 0.5) about 6, the outcomes as extreme as "10 or more correct" are matched by "2 or fewer correct." So the two-sided p-value doubles the one-sided tail: p{	ext{two}} = \mathbb{P}(X \geq 10) + \mathbb{P}(X \leq 2) = 2 	imes \frac{79}{4096} = \frac{158}{4096} \approx 0.0386 \approx 0.039. Relation to one-sided. The one-sided p-value was 79/4096 \approx 0.019. Because the null distribution is symmetric here, the two-si…

The two-sided exact p-value for 10 of 12

Your answer

More Statistics questions