Monotonic Constraints in Gradient Boosting
A monotonic constraint forces a gradient-boosted model's prediction to only ever increase (or only decrease) as one input rises, trading a small amount of raw fit for a guarantee that matches domain knowledge.
A gradient-boosted tree model, left unconstrained, is free to fit whatever pattern minimizes training error — including noisy wiggles that contradict domain knowledge. A credit model might predict that default risk drops slightly as debt-to-income rises from 40% to 42%, purely because of sampling noise in that narrow band, even though everyone agrees more debt relative to income should never make default less likely.
A monotonic constraint rules that out directly: for a chosen feature, it forces every split the model considers to respect a fixed direction — the model's prediction must weakly increase (or weakly decrease) as that one feature increases, holding all else fixed, everywhere in the feature's range, not just on average.
A monotonic constraint doesn't just clean up the output afterward — it restricts which splits the tree-building algorithm is allowed to make in the first place, so the guarantee holds everywhere, including regions with little training data where an unconstrained model would otherwise be free to overfit noise into a nonsensical direction.
Worked example. Training a default-probability model with debt-to-income as a feature but no constraint, the model's predicted default probability might trace 8%, 9%, 8.5%, 11% at debt-to-income levels of 30%, 35%, 40%, 45% — that dip at 40% is noise, not signal, and would embarrass a lender asked to explain it. Adding a monotonic-increasing constraint on that feature forces the fitted curve to be non-decreasing throughout, giving something like 8%, 9%, 9.5%, 11% instead — a curve that both matches domain knowledge and is far easier to defend to a regulator, usually at the cost of a small amount of training accuracy.
Further reading
- XGBoost/LightGBM documentation, 'Monotonic Constraints'