Generalized Additive Models and Splines

A linear regression forces every feature into a straight-line relationship with the outcome — double the feature, double the effect, forever. Real relationships are rarely that obliging: a stock's return relationship to trading volume might rise then flatten, or a customer's spending relationship to age might peak in middle age and taper at both ends. A generalized additive model (GAM) replaces each feature's straight line with its own flexible curve, built from splines (smooth, piecewise curves), while still adding the pieces together — keeping the "look at one feature's effect at a time" readability of linear regression without forcing every effect to be a straight line.

The analogy: a recipe where each ingredient has its own dial, not just a volume knob

A linear model is like a recipe where every ingredient has a single volume knob — twice the salt always means twice the saltiness, in a straight line, no matter how much salt is already in the dish. A GAM gives every ingredient its own custom dial shaped like its real effect: salt might taste linearly stronger for a while, then plateau once the dish is already salty enough. Each ingredient still contributes independently to the final taste (additive), but the shape of each contribution is allowed to be whatever the data says it actually is.

Worked example: an additive model with two curved features

A model predicting a bond's yield spread from two features, credit rating score and years to maturity, is fit as:

$$\hat y = \beta_0 + f_1(\text{rating}) + f_2(\text{maturity})$$

where $f_1$ and $f_2$ are smooth curves, not straight lines, each fit from the data. Suppose the fitted $f_1$ shows spread barely changing across the best credit ratings (roughly flat near 0), then rising sharply as rating quality drops below a threshold, while $f_2$ shows spread rising steeply for the first few years of maturity and then flattening out for longer maturities. For a bond with a rating just below that threshold and 3 years to maturity, the model reads $f_1$'s value at that rating (say, $+40$ basis points) and $f_2$'s value at 3 years (say, $+15$ basis points), and adds them to the baseline: $\hat y = \beta_0 + 40 + 15$. Each curve can be plotted and read on its own — "here's exactly how rating alone moves the spread" — something a black-box model can't offer directly.

The curve above — rising steeply then flattening — is the same qualitative shape a GAM's fitted $f_2$ (spread versus maturity) often takes: a straight line would either underfit the steep early rise or overfit by extending that steepness into the flat region, while a spline bends to fit both parts.

What this means in practice

GAMs are a natural middle ground when a linear model is too rigid but a tree ensemble or neural network is too opaque to explain to a risk committee or a regulator — each feature's fitted curve can be plotted and sanity-checked on its own. The trade-off is that GAMs are additive by default: they don't automatically capture interactions between features (how rating and maturity might jointly move spread beyond their separate effects) unless those interaction terms are explicitly added, unlike a tree ensemble which finds interactions on its own.