The variance divisor that beats n minus 1 on error

Question

For i.i.d. Normal data X1, \dots, Xn with unknown mean, consider estimators of the variance of the form \hat\sigma^2 = c\sumi (Xi - \bar X)^2. The choice c = 1/(n-1) is unbiased and c = 1/n is the maximum likelihood estimate. Which constant c minimizes the mean squared error, and how does it compare to those two familiar choices?

Accepted Answer

Set up the total error. Let Q = \sumi (Xi - \bar X)^2. For normal data, Q/\sigma^2 is a chi-square with n-1 degrees of freedom, so \mathbb{E}[Q] = (n-1)\sigma^2, \qquad \operatorname{Var}(Q) = 2(n-1)\sigma^4. Write a = n-1. The mean squared error of cQ is variance plus squared bias: 	ext{MSE}(cQ) = c^2\operatorname{Var}(Q) + \big(c\,\mathbb{E}[Q] - \sigma^2\big)^2 = c^2\,2a\sigma^4 + \sigma^4\,(ca - 1)^2. Minimize over c. Divide out \sigma^4 and differentiate: \frac{d}{dc}\big[2a c^2 + (ca-1)^2\big] = 4ac + 2a(ca - 1) = 0. Cancel 2a: 2c + (ca - 1) = 0, so c(a + 2) = 1 and c^\star = \frac{1}{…

The variance divisor that beats n minus 1 on error

Your answer

Discussion

Learn the concepts

Related questions