Pick the lower-error estimator of a click rate

Question

You are estimating a true click rate 	heta for an ad. Estimator P is unbiased but has variance 25. Estimator Q is biased 2 units too low on average but has variance only 20. Mean squared error splits as 	ext{MSE} = 	ext{Variance} + 	ext{Bias}^2. Which estimator has the smaller MSE, and what is that value?

Accepted Answer

The idea. Total squared error is scatter plus the square of the systematic offset. Add the two. Estimator P. Unbiased, so bias is 0: 	ext{MSE}P = 25 + 0^2 = 25. Estimator Q. Bias 2, variance 20: 	ext{MSE}Q = 20 + 2^2 = 20 + 4 = 24. The winner. Estimator Q edges it out, 24 versus 25. Its small bias costs only 4 units of squared error, while the 5 units of variance it saves are worth more. The margin is thin here, which is the honest lesson: trading bias for variance helps most when the variance saving is large and the bias you take on is small. A tiny bias is cheap because it enters squared.…

Pick the lower-error estimator of a click rate

Your answer

More Statistics questions