Projection Pursuit Regression

Ordinary linear regression assumes the outcome moves in a straight line as each input changes. Real relationships are often curved, and with many inputs at once it's hard to know which combinations of them matter. Projection pursuit regression (PPR) handles this by picking a handful of directions through the input space — each direction is a weighted combination of the original variables, like "twice momentum minus half volatility" — projecting the data onto each direction, and then fitting a separate flexible curve (not a straight line) to how the outcome behaves along that one direction. The final prediction adds up the outputs of all these curves.

What makes this powerful is that the directions themselves are learned from the data, not chosen by hand, and the algorithm alternates between improving the curves given the current directions and improving the directions given the current curves until neither can be improved further. With enough directions, PPR can approximate almost any smooth relationship, which is why it was an important precursor to modern neural networks — a single-hidden-layer neural network is, structurally, projection pursuit with the flexible curves replaced by a fixed activation function.

The tradeoff is interpretability: a linear regression coefficient has an obvious meaning, but a projection pursuit direction is a blend of variables whose curve must be inspected visually to understand what pattern it has captured.