Two analysts, one filters on significance

Question

Two analysts each run the same large batch of independent, unbiased backtests of genuine (but weak) effects. Analyst One records the estimated edge from every backtest. Analyst Two records the edge only from backtests that reached $p < 0.05$ .

Explain which analyst's average estimate is unbiased for the true edge and which is biased, and why.

Accepted Answer

Analyst One is unbiased. Each individual estimate is unbiased: its expected value equals the true edge, with noise scattered symmetrically above and below. Averaging all of them lets the upward and downward noise cancel, so Analyst One's average converges to the true edge. No selection has occurred. Analyst Two is biased upward. Analyst Two keeps an estimate only when it is far enough from zero to be significant, roughly \hat	heta 1.96 standard errors. For a weak effect, that condition is met mainly on draws where the noise happened to push the estimate away from zero, predominantly upward. …

Two analysts, one filters on significance

Your answer

Discussion

Learn the concepts

Related questions