The exact correction for 5 independent tests

Question

You run 5 independent tests and want the family-wise error rate (the chance of any false positive) held at 5%. Bonferroni would use $0.05/5 = 0.01$ per test.

Compute the exact Šidák per-test cutoff for independent tests, and compare it to Bonferroni.

Accepted Answer

Set up the exact target. With m independent tests, each using per-test level \alpha', the chance that a single null test stays quiet is 1 - \alpha', and the chance that all m stay quiet is (1-\alpha')^m. So the family-wise error rate (at least one alarm) is 1 - (1-\alpha')^m. We want that to equal 5%: 1 - (1-\alpha')^m = \alpha. Solve for the per-test level. Rearranging gives the Šidák cutoff: \alpha' = 1 - (1-\alpha)^{1/m} = 1 - (0.95)^{1/5}. Numerically, (0.95)^{1/5} \approx 0.98979, so \alpha' \approx 1 - 0.98979 = 0.0102. Compare to Bonferroni. Bonferroni gives 0.05/5 = 0.01. Šidák's cuto…

The exact correction for 5 independent tests

Your answer

More Statistics questions