Why the score function averages to zero

Question

The score is the derivative of the log-density with respect to the parameter, U(	heta) = \dfrac{\partial}{\partial	heta}\ln f(X;	heta). Maximum likelihood works by setting the sample average of the score to zero. Show that at the true parameter value the score has expectation zero, and explain why this justifies the MLE estimating equation.

Accepted Answer

The identity. Every density integrates to one for every 	heta: \int f(x;	heta)\,dx = 1. Differentiate both sides with respect to 	heta and pass the derivative inside the integral (regularity permitting): \int \frac{\partial f(x;	heta)}{\partial	heta}\,dx = 0. Now use the log-derivative trick \dfrac{\partial f}{\partial	heta} = f \cdot \dfrac{\partial \ln f}{\partial	heta}: \int \frac{\partial \ln f(x;	heta)}{\partial	heta}\, f(x;	heta)\, dx = \mathbb{E}\big[U(	heta)\big] = 0. So at the true 	heta, the score is a mean-zero random variable. (Differentiating once more gives \operator…

Why the score function averages to zero

Your answer

More Statistics questions