Only the true nulls can raise a false alarm

Question

A researcher tests 100 hypotheses, each at the 5% level. This time not all are worthless: suppose 20 of them describe a real effect, and the other 80 are truly null.

On average, how many false positives should the researcher expect, and which hypotheses can produce them?

Accepted Answer

Where false positives come from. A false positive means declaring a hypothesis "significant" when it is actually null. Only the truly null hypotheses can do this, a real effect that tests significant is a correct call, not a false alarm. So the 20 real hypotheses contribute zero false positives. Count from the nulls only. That leaves 80 truly null hypotheses, each with a 5% chance of a false alarm. The expected number of false positives uses the number of nulls, not the total: 	ext{expected false positives} = m0 	imes \alpha = 80 	imes 0.05 = 4, where m0 is the count of true nulls. So expe…

Only the true nulls can raise a false alarm

Your answer

More Statistics questions