Storey q-Values and the Null Proportion

When a quant researcher backtests thousands of candidate signals, most standard multiple-testing corrections (like Bonferroni or Benjamini-Hochberg) implicitly assume that essentially all the tested hypotheses are truly null — that is, that almost none of the signals actually work — which makes the correction unnecessarily conservative if, in reality, a meaningful fraction genuinely do have skill. Storey's q-value approach fixes this by first estimating $\pi_0$, the actual proportion of tests that are true nulls, directly from the data, and using that estimate to calibrate a less pessimistic threshold.

The estimate of $\pi_0$ typically comes from looking at the distribution of p-values across all tested signals: truly null signals produce p-values spread uniformly between 0 and 1, so the density of p-values in the "boring" region near 1 (say, above 0.5) estimates how much of the whole population is null. If 4,000 signals are tested and 1,800 of them have p-values above 0.5, a rough estimate is $\hat{\pi}_0 \approx 1800 / (4000 \times 0.5) = 0.9$ — suggesting about 90% of the tested signals are genuinely null and 10% may carry real signal.

A q-value for a given test is then this estimated null proportion multiplied by the usual false-discovery-rate-adjusted p-value, so it's always smaller than (or equal to) the more conservative Benjamini-Hochberg-adjusted value whenever $\hat{\pi}_0 < 1$. In practice this means fewer real signals get thrown out simply because thousands of hopeless candidates were tested alongside them.