Linear model versus decision tree, on test error

Question

You compare two models on the same problem, where expected test error is $\text{squared bias} + \text{variance} + \text{irreducible noise}$ and the noise is 5:

Linear model: squared bias 10, variance 2
Decision tree: squared bias 1, variance 9

Which model has the lower expected test error, and what is it?

Accepted Answer

The idea. The linear model is rigid: it is steady (low variance) but cannot bend to the true shape (high bias). The tree is flexible: it captures the shape (low bias) but wobbles (high variance). Add each model's three parts and compare. Total the pieces (noise 5 for both): - Linear model: 10 + 2 + 5 = 17 - Decision tree: 1 + 9 + 5 = 15 The winner. The tree, with test error 15. Here the truth is curved enough that the linear model's bias (10) hurts more than the tree's extra variance. If the underlying relationship were closer to a straight line, the bias would shrink and the steadier linear …

Linear model versus decision tree, on test error

Your answer

More Statistics questions