Automated Feature Generation

Building good features by hand is slow: a researcher has to think up a transformation, code it, test it, and repeat, often for hundreds of candidate signals. Automated feature generation tools instead take a set of raw fields — say price, volume, and sector — and mechanically apply a library of transformations (ratios, lags, rolling statistics, interactions between two fields) to generate thousands of candidate features at once, then hand the result to a feature-selection step that keeps only the useful ones.

The appeal is coverage: a human researcher might try twenty reasonable transformations of a variable, while an automated system can try every combination in a defined search space without getting tired or having blind spots. The cost is that most of what gets generated is noise, and without a disciplined selection and validation step, an automated system will happily "discover" features that fit historical data by chance alone.

Worked example

Given daily price, volume, and sector for 500 stocks, a feature-synthesis tool might automatically generate a 20-day rolling volume-to-average-volume ratio, a price momentum over 60 days divided by sector momentum, and a lagged interaction between the two. Out of a few thousand such combinations, a feature-selection filter based on out-of-sample correlation with future returns might retain only a dozen — the rest are discarded as indistinguishable from noise.