The Score Function and Likelihood Equations

Maximum likelihood estimation says: pick the parameter value that makes your observed data most probable. In practice, "search over every possible parameter value and check which one gives the highest likelihood" is not how it's actually done — that would be painfully slow. Instead, you use calculus: find where the likelihood stops rising and starts falling, which is exactly where its slope is zero. The score function is that slope, and the equation that sets it to zero is how virtually every maximum-likelihood estimate is actually computed.

An analogy: finding the top of a hill blindfolded

You're standing somewhere on a hill, blindfolded, trying to find the summit. You can't see the whole hill, but you can feel the ground's slope under your feet at your current position. If the ground tilts upward ahead of you, you're not at the top yet — move forward. If it tilts downward, you've overshot — step back. The summit is exactly the one spot where the ground feels flat under your feet: zero slope. The score function is that "felt slope," measured not on physical ground but on the log-likelihood surface, as you vary a candidate parameter value.

The definitions, one symbol at a time

Let $\theta$ ("theta") be the parameter and $L(\theta)$ the likelihood — how probable the observed data is, as a function of $\theta$. Work with the log-likelihood, $\ell(\theta) = \log L(\theta)$, because logs turn the products in a likelihood into sums, which are far easier to differentiate. The score function is its derivative:

$$S(\theta) = \frac{\partial}{\partial \theta}\, \ell(\theta)$$

Read this as: "the score is the slope of the log-likelihood curve at a given value of $\theta$." In plain English, the score tells you which direction, and how strongly, increasing $\theta$ would improve how well your data is explained. A positive score means "push $\theta$ up, the fit is still improving"; a negative score means "you've gone too far, pull back."

The maximum likelihood estimate $\hat\theta$ is found by solving the likelihood equation — setting the score to zero:

$$S(\hat\theta) = 0$$

In plain English: the MLE is the parameter value where the log-likelihood is flat — not still climbing, not falling — which, for a well-behaved single-peaked likelihood, is exactly its summit.

The explorer above shows a curve and lets you see where its slope is zero — that flat point is the maximum of the curve. The log-likelihood plays the exact same role here: $S(\theta)$ is its slope, and $\hat\theta$ is the flat point.

The binomial explorer above lets you drag $p$ and watch which value makes the observed count of heads look most probable — that dragging, done by hand, is exactly what solving $S(p)=0$ does automatically, landing on the $p$ that peaks the likelihood of what you actually saw.

Worked example 1: the MLE of a coin's bias, from its score function

You flip a coin $n=10$ times and observe $k=7$ heads. Model each flip as Bernoulli with unknown heads-probability $p$. The log-likelihood of the whole sample is $\ell(p) = k\log p + (n-k)\log(1-p) = 7\log p + 3\log(1-p)$.

The score function is its derivative with respect to $p$:

$$S(p) = \frac{7}{p} - \frac{3}{1-p} .$$

Set it to zero and solve: $\dfrac{7}{p} = \dfrac{3}{1-p} \implies 7(1-p) = 3p \implies 7 - 7p = 3p \implies 7 = 10p \implies p = 0.7$.

The MLE is $\hat p = 0.7 = k/n$ — exactly the sample proportion, derived here from first principles by finding where the score is zero, not assumed. Check the score at a nearby value, $p=0.5$: $S(0.5) = \frac{7}{0.5} - \frac{3}{0.5} = 14 - 6 = 8$, positive — confirming the log-likelihood is still climbing at $p=0.5$ and $\hat p$ must be higher, consistent with landing at $0.7$.

Worked example 2: the score function of a normal mean

You observe $n=4$ daily returns, in percent: $1.2, 0.8, 1.5, 0.9$, assumed normal with known standard deviation $\sigma = 1$ and unknown mean $\mu$. The log-likelihood (dropping constants that don't involve $\mu$) is $\ell(\mu) = -\frac{1}{2}\sum_{i=1}^{4} (x_i - \mu)^2$.

The score function is $S(\mu) = \sum_{i=1}^{4} (x_i - \mu) = (1.2-\mu)+(0.8-\mu)+(1.5-\mu)+(0.9-\mu) = 4.4 - 4\mu$.

Set it to zero: $4.4 - 4\mu = 0 \implies \mu = 1.1$.

The MLE is $\hat\mu = 1.1$, which is exactly the sample mean $\frac{1.2+0.8+1.5+0.9}{4} = \frac{4.4}{4} = 1.1$ — again derived by finding the flat point of the score, not asserted. Try $\mu = 1.0$: $S(1.0) = 4.4 - 4(1.0) = 0.4 > 0$, meaning the log-likelihood is still rising as $\mu$ increases past $1.0$, correctly pointing toward the true maximum at $1.1$.

What this means in practice

• Every "fit this distribution's parameters" routine in a statistics library — fitting a GARCH model, a Poisson jump intensity, a logistic regression — is, under the hood, solving a score equation, usually numerically because a closed-form solution like the two above is rare in realistic models.
• When numerical optimization fails to converge or gives a suspicious answer, the score function is the diagnostic: plot it (or the log-likelihood directly) against the parameter and check whether it actually crosses zero cleanly, or whether the surface is flat, multi-peaked, or unbounded.
• The score function also has a mean of exactly zero when evaluated at the true parameter, a property called unbiasedness of the score — this fact is the seed that eventually leads to the Cramér-Rao bound on how precise any unbiased estimator can be.