The genome-wide bar: correcting for 1000 tests

Question

A study runs 1000 two-sided z-tests and controls the family-wise error rate at 5% with Bonferroni. A single test would clear at z = 1.96. What critical z must each of the 1000 tests now clear?

Accepted Answer

Shrink the per-test level. Bonferroni splits the 5% budget over 1000 tests: \frac{\alpha}{m} = \frac{0.05}{1000} = 0.00005 \quad (	ext{two-sided}). Convert to a z-hurdle. A two-sided level of 0.00005 puts 0.00005/2 = 0.000025 in each tail, so the critical value is the z with cumulative probability 1 - 0.000025 = 0.999975: z \approx 4.06. So each test must clear about 4.06, more than double the usual 1.96. Why this is brutal. A bar at z = 4.06 means a true effect must sit more than four standard errors from zero to be reliably caught. Most genuine but modest effects have almost no chance of c…

The genome-wide bar: correcting for 1000 tests

Your answer

More Statistics questions