How many tests until a coin-flip of a false positive

Question

Testing at the 5% level, each independent test of a truly-null hypothesis has a 5% chance of a false positive.

How many such tests does it take before the probability of getting at least one false positive first exceeds 50%?

Accepted Answer

Set up the condition. With independent tests, the chance that none of k tests fires is 0.95^{k}, so the chance of at least one false positive is 1 - 0.95^{k}. We want the smallest k with 1 - 0.95^{k} 0.5 \quad\Longleftrightarrow\quad 0.95^{k} < 0.5. Solve for k. Take logs: k \frac{\ln 0.5}{\ln 0.95} = \frac{-0.6931}{-0.0513} \approx 13.5. So k = 14. Check: 0.95^{13} \approx 0.5133 (chance of at least one \approx 48.7\%, not yet past half), while 0.95^{14} \approx 0.4877 (chance of at least one \approx 51.2\%, over half). The takeaway lesson. It takes only 14 independent tests at the standard …

How many tests until a coin-flip of a false positive

Your answer

More Statistics questions