Automated Alpha Search

A human researcher testing signals one at a time might try a dozen ideas in a week. A search program can combine a library of basic data fields — price, volume, earnings, sentiment — with a library of mathematical operators — rank, delay, correlation, moving average — and mechanically generate and backtest thousands or millions of candidate formulas overnight. This is automated alpha search, and formulas produced this way (sometimes called "formulaic alphas") can look like rank(delay(close, 5) / volume) — expressions no researcher necessarily sat down and hypothesized, but that a search process discovered by combinatorial trial.

The appeal is obvious: far more of the space of possible signals gets explored than any team could cover by hand. The danger is exactly proportional to that appeal — testing a million formulas against the same historical data guarantees that some of them will look profitable purely by chance, with no real edge behind them at all.

Why more search needs a higher bar

The standard safeguards are the same ones used against any multiple-testing problem: hold out a chunk of history the search never sees and validate survivors only on it; require an economic rationale for why a mechanically discovered formula should work, not just that it backtested well; and raise the statistical bar for significance as the number of formulas tested grows, since testing 10,000 formulas at a 5% significance level should produce roughly 500 "significant" results out of pure noise.

Worked example

A search evaluates 5,000 candidate formulas against five years of daily data and finds 30 with an in-sample t-stat above 2 (roughly a 5% chance of occurring by luck for any single test). Naively, that looks like 30 promising signals. But at a 5% false-positive rate, pure chance alone predicts about $5{,}000 \times 0.05 = 250$ formulas would clear that bar with no real signal behind them — so finding only 30 total already suggests most tested formulas had no edge, and even among those 30, a large share are likely false positives rather than genuine signals. Running the 30 survivors against a held-out year of data the search never touched, only 4 remain statistically significant — those 4 are far more trustworthy, precisely because they were never allowed to influence the search that selected them.