Tag: multiple-testing
Concepts
Interview Questions
- The signal that correlated 0.9 with returns
- Why the best backtest is probably a fluke
- The best of 100 random strategies looks brilliant
- You found 30 significant signals. How many are real?
- Test 20 signals and one will "work" by accident
- How likely is even one false positive across 20 tests?
- Why Benjamini-Hochberg is a step-up procedure
- How many of your discoveries are fake?
- When your signals are correlated, BH needs a haircut
- Testing 1000 signals, false discovery rate vs Bonferroni
- Should you control FWER or FDR for 1,000 signals?
- The hidden multiplicity in a single reported test
- Ten metrics on the dashboard, one lit up green
- A hundred signals at the 1% level
- How many backtests beat t = 2 by luck?
- Šidák versus Bonferroni
- Benjamini-Hochberg on six p-values
- Twenty signals, one spurious discovery
- The best of 100 backtests has t = 3, is it real?
- Twenty backtests, one hit at p < 0.05
- How many false positives across many null tests
- The best of fifty backtests shows a Sharpe of 1
- Šidák vs Bonferroni: the slightly sharper correction
- The overall test failed, but it "works for left-handed users"