Šidák cutoff for 20 independent tests

Question

You run 20 independent tests and want the family-wise error rate held at 5%. Bonferroni would use $0.05/20 = 0.0025$ per test.

Compute the exact Šidák per-test cutoff, and note how the gap to Bonferroni changes as the number of tests grows.

Accepted Answer

Exact formula. For m independent tests with family-wise target \alpha, the Šidák cutoff comes from setting 1 - (1-\alpha')^m = \alpha and solving: \alpha' = 1 - (1-\alpha)^{1/m} = 1 - (0.95)^{1/20}. Compute. (0.95)^{1/20} \approx 0.997439, so \alpha' \approx 1 - 0.997439 = 0.00256. Compare. Bonferroni gives 0.05/20 = 0.0025. Šidák's 0.00256 is barely larger. Notice the gap has shrunk compared with small m: at m = 4 the two differed in the third decimal, at m = 20 they agree to nearly four decimals. Why the gap vanishes. As each per-test level gets tiny, the chance of two false alarms overlapp…

Šidák cutoff for 20 independent tests

Your answer

More Statistics questions