Oblique Trees and Linear Model Trees

A standard decision tree splits on one feature at a time — "is age over 40?" then "is income over $50,000?" — which always produces boundaries that are stacked rectangles, since every cut is parallel to a feature axis. If the true boundary between two classes runs diagonally, a plain tree needs a staircase of many small rectangular cuts to approximate it.

Two different fixes

An oblique tree generalizes the split test from "feature X > threshold" to "a·X + b·Y > threshold," letting one split capture a diagonal boundary that an axis-aligned tree would need a dozen stair-step cuts to match. The cost is that a single split is now harder to explain to a non-technical reader, since it depends on a combination of features rather than one clean rule.

A model tree keeps ordinary axis-aligned splits for deciding which region a point falls into, but instead of predicting a single constant value per leaf (as a regression tree does), it fits a small linear regression using only the data in that leaf. This helps when the target varies smoothly within a region rather than jumping in steps.

Worked example

Predicting house price from square footage within a leaf that only contains 3,000-4,000 sq ft homes: a plain regression tree would just predict the leaf's average price for every home in that range, while a model tree fits price ≈ $150 + $180 × square footage inside that leaf, capturing the fact that price still rises steadily with size even within a narrow bucket.