Bonferroni correction and the sample it demands

Question

A single two-sided test at \alpha = 0.05 with 80\% power needs 900 users per arm. But you actually plan to test 5 variants against control, and to keep the overall false-positive rate at 5\% you apply a Bonferroni correction, testing each comparison at \alpha = 0.05/5 = 0.01. Power and effect stay the same. Roughly how many users per arm does each comparison now need?

Accepted Answer

Why the sample grows. Bonferroni splits the 5\% budget across the five comparisons, so each is tested at \alpha = 0.01. A stricter per-test \alpha raises the critical value and therefore the sample. The two constants. - \alpha = 0.05: (1.96 + 0.84)^2 = 7.84. - \alpha = 0.01: (2.576 + 0.84)^2 = 11.67. The ratio and result. \frac{n{0.01}}{n{0.05}} = \frac{11.67}{7.84} \approx 1.49, \qquad n \approx 1.49 	imes 900 \approx 1{,}340 	ext{ per arm}. So guarding against the multiple-comparisons inflation costs about 49% more data per arm. This is the sample-size price of testing many things at once…

Bonferroni correction and the sample it demands

Your answer

Discussion

Learn the concepts

Related questions