Overfitting versus underfitting, and how to tell which you have

Question

Explain overfitting and underfitting in plain terms (the bias-variance tradeoff), and describe how you would diagnose which problem a model has.

Accepted Answer

Every model makes two kinds of error beyond the irreducible noise. Underfitting (high bias). The model is too simple to capture the real pattern, like fitting a straight line to a clearly curved relationship. It does poorly on the training data and on new data, and the two errors are close together and both high. Adding more data does not help much, the model simply lacks the flexibility to do better. Overfitting (high variance). The model is too flexible and memorizes the noise in the training data, like a wiggly high-degree curve threading through every point. It does great on the training …

Overfitting versus underfitting, and how to tell which you have

Your answer

More Statistics questions