Detecting a real effect on at least one of two tries

Question

A test has only 60\% power. You run it twice, independently, on the same real effect. What is the probability it detects the effect on at least one of the two runs?

Accepted Answer

Work through the complement. Each run misses a real effect with probability \beta = 1 - 0.60 = 0.40. Because the runs are independent, both miss with probability 0.40 	imes 0.40 = 0.16. So at least one run catches it with probability 1 - 0.16 = 0.84. Running a weak (60%-powered) test twice lifts the effective detection chance from 60% to about 84%. Replication buys power: repeated independent looks make it much harder for a real effect to hide. Takeaway. With independent runs, the chance of at least one detection is 1 - \beta^k. Two runs of a 60%-powered test reach about 0.84, well above the…

Detecting a real effect on at least one of two tries

Your answer

More Statistics questions