Forty factors and near-certain false positives

Question

A factor-mining pipeline screens 40 independent candidate factors, all in truth useless, at the 5% significance level.

Find the expected number of false positives and the probability that at least one factor looks significant.

Accepted Answer

Expected false positives. Each useless factor is flagged with probability 0.05, so across 40: 40 	imes 0.05 = 2. You should expect two spurious "significant" factors even though none has any power. Probability of at least one. With independence, the chance none of the 40 fire is 0.95^{40}, so \mathbb{P}(	ext{at least one}) = 1 - 0.95^{40} = 1 - 0.1285 \approx 0.8715 \approx 0.87. There is about an 87% chance of finding at least one "significant" factor by luck alone. At this scale, discovering something significant is almost guaranteed and therefore almost meaningless without correction. Wh…

Forty factors and near-certain false positives

Your answer

More Statistics questions