Is it ever smart to shrink your estimate toward zero?

Question

You observe a single noisy measurement x \sim N(\mu, \sigma^2) and want to estimate \mu. Instead of using x directly, consider the shrunken estimator \hat\mu = c\,x for some constant c \in [0,1]. Find the c that minimizes mean squared error and interpret the result.

Accepted Answer

Decompose the error. For \hat\mu = cx, the mean squared error is variance plus squared bias: 	ext{MSE}(c) = \operatorname{Var}(cx) + \big(\mathbb{E}[cx] - \mu\big)^2 = c^2\sigma^2 + (c-1)^2\mu^2. Minimize. Differentiate with respect to c and set to zero: \frac{d}{dc}\,	ext{MSE} = 2c\sigma^2 + 2(c-1)\mu^2 = 0 \implies c^\star = \frac{\mu^2}{\mu^2 + \sigma^2}. Interpretation. The optimal c^\star is strictly less than 1 whenever \sigma^2 0, so some shrinkage always lowers MSE below that of the raw estimator x (which corresponds to c = 1). The amount of shrinkage is governed by the signal-to-no…

Is it ever smart to shrink your estimate toward zero?

Your answer

More Statistics questions