MARS: Multivariate Adaptive Regression Splines

A relationship between a feature and a target sometimes bends — it's flat, then it kinks upward past some threshold, then it kinks again. Fitting a single polynomial to that shape is awkward: the whole curve has to bend everywhere to accommodate a kink that only exists in one region. Multivariate Adaptive Regression Splines (MARS) takes a different, more surgical approach: it builds the model out of pairs of hinge functions, each one flat on one side of a threshold and linear on the other, and it searches for where to put the thresholds itself rather than requiring the analyst to guess them.

The analogy: a tax bracket schedule

A progressive tax schedule is flat-then-sloped-then-more-sloped: 0% up to $10,000, 15% on the next bracket, 25% above that. Each bracket boundary is a kink — the marginal rate is genuinely different on either side, and no single smooth curve captures that as naturally as a sequence of straight segments joined at specific breakpoints. MARS builds regression models the same way: instead of one smooth curve, it stitches together straight (hinge) segments, choosing both how many kinks to use and exactly where to place them by searching the data, the same way a tax authority chooses bracket cutoffs to match a policy goal rather than any predetermined formula.

The building block: hinge functions

MARS's basis functions are pairs of hinge functions at a chosen knot $t$:

$$h^+(x, t) = \max(0, x - t), \qquad h^-(x, t) = \max(0, t - x)$$

Plain English: $h^+$ is zero everywhere below the threshold $t$ and rises linearly above it; $h^-$ is zero above $t$ and rises linearly below it — together they let the fitted function have a completely different slope on each side of $t$, with a sharp kink exactly at $t$. The full MARS model is a sum of many such hinge terms (and products of hinges, for interactions), with coefficients fit by least squares:

$$\hat y = \beta_0 + \sum_{m=1}^{M} \beta_m \, h_m(x)$$

MARS builds this greedily: it adds hinge-pairs one at a time (forward pass), picking whichever variable and knot location reduces training error the most, then prunes back less useful terms afterward using generalized cross-validation to avoid overfitting.

Worked example 1: a single kink fit by hand

Data suggests a return-vs-momentum relationship that's flat for momentum below 0 and rises with slope 0.4 above 0. MARS selects knot $t=0$ and fits $\hat y = 1.0 + 0.4 \cdot h^+(x, 0)$, dropping the $h^-$ term (coefficient shrunk to 0 by the fitting process, since there's no relationship below the knot). At $x=-2$: $h^+(-2,0)=\max(0,-2)=0$, so $\hat y = 1.0$. At $x=3$: $h^+(3,0)=\max(0,3)=3$, so $\hat y = 1.0+0.4(3)=2.2$. The model correctly stays flat below zero and rises linearly above it, using two simple hinge terms rather than a curved polynomial.

Worked example 2: an interaction term

Suppose the true relationship is that momentum only predicts returns when volatility is low — an interaction. MARS can capture this by multiplying two hinges together: $\beta \cdot h^+(\text{mom}, 0) \cdot h^-(\text{vol}, 20)$. With $\beta = 0.05$: at mom=10, vol=15 (low vol, positive momentum), $h^+(10,0)=10$, $h^-(15,20)=\max(0,5)=5$, contributing $0.05 \times 10 \times 5 = 2.5$ to the prediction. At mom=10, vol=30 (high vol), $h^-(30,20)=\max(0,-10)=0$, so the term contributes exactly $0$ — the interaction term automatically switches off when volatility is high, without the analyst having to specify that condition manually; MARS's forward search found the vol=20 threshold and the interaction structure from the data.

A hinge function is piecewise-linear rather than smoothly curved, but adjusting where a curve bends and how steeply is the same visual idea — MARS just uses straight segments joined at data-chosen knots instead of one continuously curving function.

What this means in practice

MARS sits between plain linear regression (too rigid for kinked relationships) and fully nonparametric methods like random forests (flexible but harder to interpret) — its fitted model is still a sum of simple, interpretable pieces, each one readable as "this feature matters, and only above/below this threshold," which is valuable when a model's output needs to be explained to a risk committee, not just validated on a backtest. It's commonly used for threshold effects in factor models — a value spread that only predicts returns once it crosses some level, or a volatility regime switch.