Does adding a pseudo-count beat the raw click rate?

Question

From n trials with k successes you can estimate the success probability p either with the raw rate \hat p = k/n or with the smoothed estimate that adds one pseudo-success and one pseudo-failure, 	ilde p = \frac{k+1}{n+2}. Does the smoothed estimator have a lower mean squared error, and where?

Accepted Answer

The raw estimator. The rate \hat p = k/n is unbiased, with variance and hence mean squared error 	ext{MSE}(\hat p) = \frac{p(1-p)}{n}. The smoothed estimator. Write 	ilde p = \frac{k+1}{n+2}. Using \mathbb{E}[k] = np and \operatorname{Var}(k) = np(1-p), \mathbb{E}[	ilde p] = \frac{np + 1}{n+2}, \qquad \operatorname{Var}(	ilde p) = \frac{np(1-p)}{(n+2)^2}. The smoothing pulls the estimate toward 1/2, adding bias but cutting variance because of the larger denominator. Check at a fair coin. Take p = 1/2 and n = 10. The raw rate gives 	ext{MSE}(\hat p) = \frac{0.25}{10} = 0.025. The smoothed…

Does adding a pseudo-count beat the raw click rate?

Your answer

Discussion

Learn the concepts

Related questions