The error floor no model can beat

Question

Suppose you had a perfect model, with zero bias and zero variance. Expected test error is $\text{squared bias} + \text{variance} + \text{irreducible noise}$ , and the irreducible noise is 6.

What is the lowest expected test error any model can possibly achieve?

Accepted Answer

The idea. Bias and variance are properties of your model, and in principle both can be driven to zero. But the irreducible noise is a property of the world: the outcome has randomness that no model can explain, because it does not depend on your inputs at all. Compute the floor. With bias and variance both zero, 	ext{Test error}{\min} = 0 + 0 + 	ext{noise} = 6. No model, however clever, can push expected test error below 6. That number sets the ceiling on achievable accuracy. If your best model already has test error near 6, stop tuning: the remaining error is noise, not a modeling failure …

The error floor no model can beat

Your answer

More Statistics questions