Quant Memo
Statistics/●●●●●

Bias, variance, and irreducible noise in prediction error

Suppose y=f(x)+εy = f(x) + \varepsilon where ε\varepsilon has mean 00 and variance σ2\sigma^2, independent of everything else. You fit a model f^\hat{f} on random training data and predict at a fixed point xx.

Show that the expected squared prediction error at xx splits into irreducible noise plus bias squared plus variance, and use it to explain overfitting.

Your answer

This one is open-ended. Work it through, then check your reasoning against the full solution.

More Statistics questions