When 50 tests are really only 5 independent ones

Question

You plan to Bonferroni-correct across 50 signals, dividing your 5% budget by 50. But the 50 signals are really 5 groups of 10, and within each group the 10 signals are near-identical (correlation close to 1), essentially the same bet copied ten times.

Roughly how many effectively independent tests are you running, and why does this make plain Bonferroni too harsh here?

Accepted Answer

Bonferroni counts tests, not information. The correction divides your budget by the number of tests, assuming each one is a fresh, independent chance for a false alarm. But ten near-identical copies of a signal are not ten independent chances, they rise and fall together, so they behave almost like a single test. Estimate the effective count. If each group of 10 is essentially one bet (correlation near 1 collapses the group to a single degree of freedom), then across 5 groups you have roughly m{	ext{eff}} \approx 5 effectively independent tests, not 50. Why plain Bonferroni over-corrects. Di…

When 50 tests are really only 5 independent ones

Your answer

Discussion

Learn the concepts

Related questions