Explainable Boosting Machines

A full gradient-boosted model with deep-ish trees can be extremely accurate, but its prediction for any single input is the sum of hundreds of trees each testing multiple features in combination — there's no clean way to say "feature X contributed this much" without resorting to approximations like SHAP after the fact. An Explainable Boosting Machine (EBM) takes a different approach: it restricts the model's structure during training itself so that every prediction genuinely decomposes into a sum of per-feature (and optionally per-pair) contributions, with nothing hidden.

The core idea: one feature at a time, additively

An EBM is a generalized additive model — the final prediction is a sum of separate functions, one per feature:

$$\hat{y} = \beta_0 + f_1(x_1) + f_2(x_2) + \dots + f_p(x_p) .$$

In plain English: instead of one big model that tangles all features together, the prediction is built by adding up independent "how much does this one feature push the prediction up or down" curves, one per input variable. What makes an EBM different from an ordinary GAM is how each $f_j$ is learned: rather than a simple parametric curve, each $f_j$ is itself a small gradient-boosted ensemble of shallow trees, fit by cycling through every feature many times with a very small learning rate. This lets each $f_j$ capture flexible, non-linear shapes (kinks, plateaus, thresholds) a straight line would miss, while keeping the overall model additive and fully interpretable.

Worked example: reading off a contribution

Suppose an EBM predicts default risk from two features: borrower income and days-past-due. For a borrower with income $45,000 and 12 days past due, the per-feature curves report $f_{\text{income}}(45{,}000) = -0.8$ and $f_{\text{days-past-due}}(12) = +1.3$, with intercept $\beta_0 = 0.1$. The predicted (log-odds) score is exactly $0.1 - 0.8 + 1.3 = 0.6$ — no approximation, no post-hoc explanation needed, since that sum genuinely is how the model computed its answer. Plotting $f_{\text{days-past-due}}$ over its range might show risk barely rising from 0 to 5 days late but jumping sharply after day 10 — a shape a linear coefficient could never show.

The accuracy-interpretability tradeoff, narrowed

Restricting the model to be additive (no feature interactions, unless explicitly added as pairwise terms) does cost some accuracy relative to an unrestricted gradient-boosted model that can freely combine any number of features per tree. In practice, on many tabular datasets this gap is surprisingly small — EBMs are frequently within a percentage point or two of a full gradient-boosted model's accuracy, because much of the signal in typical tabular data really is additive, feature-by-feature — while offering exact, plot-able explanations rather than approximate ones.

The curve above is the kind of per-feature shape function ($f_j$ in the additive sum) an EBM learns for a single input: not a straight line, but a flexible, plottable curve whose contribution to the final prediction you can read off directly at any value of that feature.

What this means in practice

EBMs are a strong choice whenever a model's predictions need to be explained to a regulator, a risk committee, or a non-technical stakeholder — credit scoring, fraud flagging, or any setting where "why did the model say this" needs a precise, auditable answer rather than an approximation. Where feature interactions genuinely matter and interpretability is secondary, a full gradient-boosted model or random forest will usually out-perform an EBM; the right choice depends on whether the accuracy gap is worth trading for exact transparency.