The critical z after correcting for 20 tests

Question

A team runs 20 two-sided z-tests and holds the family-wise error rate at 5% with Bonferroni. A single test would use the familiar critical value $|z| = 1.96$ .

What critical $|z|$ must each of the 20 tests now clear?

Accepted Answer

Shrink the per-test level. Bonferroni divides the 5% budget across 20 tests: \frac{\alpha}{m} = \frac{0.05}{20} = 0.0025 \quad (	ext{two-sided}). Convert to a z-hurdle. A two-sided level of 0.0025 places 0.0025/2 = 0.00125 in each tail, so the critical value is the z with cumulative probability 1 - 0.00125 = 0.99875: z \approx 3.02. So each test must clear about 3.02 instead of 1.96. What it costs. Moving the bar from 1.96 to 3.02 is a jump of about one standard error. A real effect that a single test catches most of the time now slips under the taller bar more often, so power falls. Recover…

The critical z after correcting for 20 tests

Your answer

Discussion

Learn the concepts

Related questions