The exact p-value for 10 correct calls out of 12

Question

A forecaster makes 12 independent up-or-down calls on a market and gets 10 correct. Under the null hypothesis that they are just guessing, each call is correct with probability $0.5$ .

Compute the exact one-sided p-value (the probability of getting 10 or more correct by pure guessing), and state what it means.

Accepted Answer

Set up the exact test. Let X be the number correct out of 12 under guessing, so X \sim 	ext{Binomial}(12, 0.5). The one-sided p-value is the probability of a result at least as impressive as observed: p = \mathbb{P}(X \geq 10) = \frac{\binom{12}{10} + \binom{12}{11} + \binom{12}{12}}{2^{12}}. Count and compute. The binomial coefficients are \binom{12}{10} = 66, \binom{12}{11} = 12, \binom{12}{12} = 1, summing to 79. The total number of equally likely outcomes is 2^{12} = 4096. So p = \frac{79}{4096} \approx 0.0193 \approx 0.019. What it means. If the forecaster were truly guessing, they woul…

The exact p-value for 10 correct calls out of 12

Your answer

More Statistics questions