Nonlinear Signal Transformations
Not every signal's relationship to future returns is a straight line — value only matters below some threshold of cheapness, and a squashing function that respects that shape often extracts more edge than feeding the raw number straight into a model.
Prerequisites: Cross-Sectional Signal Normalization, Polynomial and Spline Regression
Feeding a raw signal value straight into a linear model assumes its relationship with future returns is a straight line: twice the signal, twice the expected edge, all the way out to whatever extreme value the signal happens to take. That assumption is frequently wrong. A stock's dividend yield being 2% versus 4% might matter a lot; being 20% versus 22% usually means the market is pricing in a dividend cut, not extra reward — the relationship flattens, or even reverses, in the tail. A nonlinear transformation reshapes the raw signal to better match its true relationship with returns before it's used.
Common transforms include a sigmoid or rank-based squashing (compresses extreme values so a handful of outliers can't dominate), thresholding (only "on" past some level — a signal for financial distress might only matter once leverage crosses a critical point), and quantile bucketing (throws away exact magnitude, keeps only which decile a stock falls in, useful when the relationship is monotonic but not linear).
A linear model implicitly assumes a signal's edge scales proportionally with its raw magnitude all the way to the extremes — when that's false, transforming the signal first (squashing, thresholding, bucketing) captures the true shape of the relationship, while a plain linear fit either ignores the nonlinearity or lets a few extreme values distort the whole model.
Shape before magnitude
Drag the logistic curve's parameters to see how a sigmoid squashes large raw inputs toward a bounded range while still discriminating clearly between small and moderate values — this is the same shape used to compress an extreme raw signal (like a leverage ratio that occasionally spikes to 50x) before it reaches a model that would otherwise be dominated by those spikes.
Worked example
A leverage signal (debt / equity) ranges from 0.2 for a conservative company to 40 for one company in severe financial distress with equity nearly wiped out. Fed raw into a linear model, that one outlier at 40 has as much influence on the fitted slope as forty ordinary companies at 1.0 combined, badly distorting the estimated relationship for the typical firm.
Applying a rank transform instead — replacing each company's raw leverage with its percentile position among that day's universe — turns 0.2, 1.0, 3.0, and 40 into ranks like 0.05, 0.50, 0.85, and 0.99. The distress case still ranks near the top (correctly flagging it as high-leverage), but its extreme raw magnitude of 40 no longer swamps the fit; a company at rank 0.85 contributes proportionally, not 40 times more, to estimating the leverage-return relationship.
A transform should be chosen because the true relationship is nonlinear, not to force a stubborn signal to "look significant." Squashing outliers can also hide a genuinely important extreme event — a leverage ratio of 40 might be exactly the case an investor most needs to identify, and a transform that treats it the same as a rank-0.99 company with leverage of 5 can throw away real information along with the noise.
Related concepts
Practice in interviews
Further reading
- Grinold and Kahn, Active Portfolio Management (ch. 9)