Why the replication shrinks toward zero

Question

A signal is "discovered" as significant ( $p < 0.05$ ) in a small, low-powered study. The team then runs a clean, independent, well-powered replication and finds a much smaller effect, well short of the original estimate.

Explain why this shrinkage is predictable rather than surprising, and what it says about acting on freshly discovered significant effects.

Accepted Answer

Why shrinkage is expected. The original discovery survived a significance filter in a low-power setting. As the winner's curse shows, such surviving estimates are selected from the upward-biased tail: they are significant partly because noise pushed them high. The true effect is smaller than the discovery estimate. A replication, run without conditioning on the outcome, samples the effect afresh, so its estimate centers on the true (smaller) value. The drop from discovery to replication is the built-in bias unwinding, not a new phenomenon. Regression toward the truth. This is a form of regres…

Why the replication shrinks toward zero

Your answer

More Statistics questions