Bonferroni cutoff for a 200-signal screen

Question

A quant screens 200 candidate signals and wants the family-wise error rate held at 5% (a 5% chance of any false positive across all 200).

Using the Bonferroni correction, what per-test p-value cutoff must each signal beat?

Accepted Answer

Divide by the count. Bonferroni splits the 5% budget across all 200 tests: \frac{\alpha}{m} = \frac{0.05}{200} = 0.00025. So a signal is significant only if its p-value is at most 0.00025, a very demanding bar. Why so strict. With 200 chances for a false alarm, the only way to keep the total risk at 5% is to make each individual chance minuscule. Summing the worst case gives 200 	imes 0.00025 = 0.05, exactly the family target. The cost. Such a tiny cutoff means many genuinely useful signals with, say, p = 0.01 get rejected too. This is Bonferroni's well-known downside: with hundreds of tests…

Bonferroni cutoff for a 200-signal screen

Your answer

More Statistics questions