Does p = 0.006 survive Bonferroni across 10 tests?

Question

You ran 10 hypothesis tests at a 5% family-wise target. Your most promising result has a p-value of 0.006, which comfortably beats the usual 0.05 line.

After applying the Bonferroni correction, is this result still significant?

Accepted Answer

Find the corrected bar. Bonferroni divides the 5% family-wise budget across all 10 tests: \frac{\alpha}{m} = \frac{0.05}{10} = 0.005. So a result counts only if its p-value is at most 0.005. Compare. Your p-value is 0.006, which is larger than 0.005. It just misses the corrected cutoff, so it is not significant once you account for having run ten tests. The uncomfortable point. A raw p-value of 0.006 looks impressive on its own, it would sail past a single 5% test. But it was the best of ten attempts, and the honest bar for "best of ten" is 0.005. Being close does not count: 0.006 exceeds 0.0…

Does p = 0.006 survive Bonferroni across 10 tests?

Your answer

Discussion

Learn the concepts

Related questions