Benjamini-Yekutieli FDR Under Dependence

The standard Benjamini-Hochberg procedure for controlling the false discovery rate — the expected fraction of "significant" findings among thousands of tested factors that are actually false positives — assumes the tests are independent or only positively correlated in a specific way. In quant research, that assumption is routinely violated: backtested factors built from overlapping stock universes, overlapping time windows, or correlated underlying data are dependent in ways that can be arbitrarily complicated, not just mildly positively correlated.

Benjamini-Yekutieli (BY) fixes this by using a stricter threshold that provably controls the false discovery rate under any dependence structure among the tests, positive or negative. It does this by dividing the usual Benjamini-Hochberg critical value by the harmonic-sum correction factor $\sum_{i=1}^{m} \frac{1}{i}$, where $m$ is the number of tests — a term that grows (slowly, like $\log m$) with the number of tests, making BY meaningfully more conservative than Benjamini-Hochberg as the number of candidate factors grows into the thousands.

The practical tradeoff is straightforward: BY guards against dependence you can't rule out — exactly the situation when screening thousands of correlated backtests — at the cost of a real loss in power, meaning genuinely good factors are more likely to be missed. Researchers who know their tests are only mildly dependent, or independent, often stick with Benjamini-Hochberg and accept BY only when correlation among the tested hypotheses is severe or unknown.