Šidák vs Bonferroni: the slightly sharper correction

Question

You're running 10 independent tests and want the family-wise error rate, the chance of any false positive, held at 5%. The reflex is Bonferroni: use $\alpha/10 = 0.005$ per test.

Derive the Šidák correction for independent tests, compare it to Bonferroni numerically, and say when each is appropriate.

Accepted Answer

The exact target. With m independent tests each at per-test level \alpha', the probability that all correctly stay quiet is (1 - \alpha')^m, so the family-wise error rate is 1 - (1-\alpha')^m. To hold that at \alpha = 0.05, solve 1 - (1 - \alpha')^m = \alpha \quad\Longrightarrow\quad \alpha' = 1 - (1 - \alpha)^{1/m}. That's the Šidák correction. For \alpha = 0.05, m = 10: \alpha' = 1 - (0.95)^{1/10} = 1 - 0.99488 \approx 0.005116. Bonferroni for comparison. Bonferroni just uses \alpha/m = 0.05/10 = 0.005. Notice Šidák's threshold (0.005116) is a hair larger, slightly more permissive, hence sl…

Šidák vs Bonferroni: the slightly sharper correction

Your answer

More Statistics questions