Surrogate Splits and Missing Values in Trees

A tree's split — "if leverage ratio > 3.0, go right" — has an obvious problem if leverage is missing: which way does the row go? The usual fix elsewhere in machine learning is imputing the value first. Classical CART trees instead build a surrogate split: at every node, alongside the primary rule, the tree finds a backup rule on a different feature that best mimics the primary split's assignment, so a row with the primary feature missing can still be routed sensibly.

The analogy: a substitute who plays the same role

A sports team's star player sometimes doesn't show up, so the coach has already identified a backup whose on-field decisions tend to match the star's closely enough — not the same player, but reading the game the same way. When the star is out, the backup makes a very similar call. A surrogate split is exactly this backup plan, decided in advance at every node.

The mechanics: finding a substitute split at every node

At a node with primary split "$X_1 > t_1$," a surrogate search checks every other feature $X_j$ and threshold $t_j$ for how well "$X_j > t_j$" reproduces the same assignment, using rows where both are observed:

$$\text{agreement}(X_j, t_j) = \frac{\#\{\, i : \mathbb{1}[X_{1,i} > t_1] = \mathbb{1}[X_{j,i} > t_j] \,\}}{n} .$$

In plain English: a surrogate is judged purely by how often it lands on the same side as the primary feature would have — nothing about accuracy on the actual target, just mimicry. At prediction time, if $X_1$ is missing, the tree falls back to the top surrogate, then the next, only defaulting to the majority branch if every surrogate is also missing.

Worked example: routing a row with missing leverage

If the primary split is "leverage > 3.0," "interest coverage < 2.5" might agree with it on 91% of rows where both are observed — high leverage firms tend to have low coverage — making it the top surrogate. For a company where leverage is missing but coverage of 1.8 is available, the tree applies the surrogate: since $1.8 < 2.5$, the row is sent the same direction a high-leverage company would go, a reasoned guess rather than an arbitrary default.

Picture the surrogate as a second, cheaper boundary drawn using a different feature axis — imperfect, but close enough to the primary boundary above to route most cases correctly when the primary axis is unavailable.

What this means in practice

Surrogate splits let a tree be trained and queried without a separate imputation pipeline, useful in finance where missingness is often informative itself (a private company's disclosure gaps aren't random) rather than something to paper over. The tradeoff is cost: computing surrogates at every node multiplies training cost, and most modern gradient-boosting libraries use simpler strategies instead.