The Score (Lagrange Multiplier) Test

The Wald test needs you to fit the full, complicated model and look at how far the estimate sits from the null value. The likelihood-ratio test needs you to fit both the full model and the restricted one and compare them. Sometimes fitting the full model is expensive, unstable, or simply the thing you're trying to decide whether to bother doing at all — you want to know before fitting the complicated model whether it's even worth the effort. The score test answers exactly that: fit only the simple, restricted model, and check whether the data still seems to be pulling on the extra parameters you left out.

An analogy: is the extra weight still pulling on the scale

Imagine a two-pan balance where one pan is fixed with a small locking pin, so it can't move even if something is pulling on it. Fit the "restricted" model by only using the free pan. Now ask: if you removed the locking pin, would that pan want to move? You can answer this without ever removing the pin — you just measure the force still straining against it. If the force is small, the pin wasn't holding back much and the restriction was fine. If the force is large, the data is straining hard against a constraint that shouldn't be there. That straining force is the "score," and the test is built entirely around measuring it.

The rule, one symbol at a time

Let $\ell(\theta)$ be the log-likelihood — a curve whose peak marks the best-fitting parameter value. The score function $U(\theta) = \dfrac{d\ell(\theta)}{d\theta}$ is the slope of that curve at any point $\theta$; it is exactly zero at the maximum-likelihood estimate $\hat{\theta}$, by definition of "peak." Under the null hypothesis $H_0: \theta = \theta_0$, evaluate the score at $\theta_0$ instead of at the peak — this is the "how hard is it still pulling" measurement. The score statistic is

$$S = \frac{U(\theta_0)^2}{I(\theta_0)} ,$$

where $I(\theta_0)$ is the Fisher information at $\theta_0$, a measure of how sharply curved the likelihood is there (more curvature means more precision, so the same slope counts as stronger evidence). In plain English: square the slope of the log-likelihood at the null value, and divide by how sharply peaked the likelihood is there — a big score relative to the local sharpness means the null is a bad fit. Under $H_0$, $S$ follows a chi-squared distribution with degrees of freedom equal to the number of restricted parameters, exactly like the Wald and likelihood-ratio statistics.

Think of the curve above as the log-likelihood itself, not the data. If $\theta_0$ sits at the curve's peak, the slope there is zero and the score test finds nothing wrong. The further $\theta_0$ sits from the true peak, the steeper the slope at that point, and the larger the score statistic grows.

Worked example 1: testing a Poisson rate without fitting further

You observe $n=20$ counts from a Poisson process and want to test $H_0: \lambda = 5$. The Poisson log-likelihood score function is $U(\lambda) = \sum x_i/\lambda - n$, and its Fisher information is $I(\lambda) = n/\lambda$. Suppose the observed counts sum to $\sum x_i = 130$.

$$U(5) = \frac{130}{5} - 20 = 26 - 20 = 6 .$$ $$I(5) = \frac{20}{5} = 4 .$$ $$S = \frac{U(5)^2}{I(5)} = \frac{6^2}{4} = \frac{36}{4} = 9.0 .$$

Compare $9.0$ to a chi-squared distribution with 1 degree of freedom; the 5% critical value is $3.84$. Since $9.0 > 3.84$, reject $H_0: \lambda=5$ — note this test never needed to compute the actual MLE, $\hat{\lambda} = 130/20 = 6.5$, at all, though we can check it's consistent: the data really does look like it comes from a higher rate.

Worked example 2: comparing to the Wald test on the same data

Using the same data, the MLE is $\hat{\lambda} = 6.5$ with standard error $\text{SE}(\hat{\lambda}) = \sqrt{\hat{\lambda}/n} = \sqrt{6.5/20} = \sqrt{0.325} = 0.570$. The Wald statistic squared is

$$W^2 = \left(\frac{6.5-5}{0.570}\right)^2 = (2.632)^2 = 6.93 .$$

Both tests reject $H_0$ (both exceed $3.84$), but they don't agree exactly — $9.0$ versus $6.93$ — because the Wald test evaluates curvature at $\hat{\lambda}$ while the score test evaluates it at $\theta_0$. The two coincide only in the large-sample limit or in special cases like the normal mean; in finite samples, and especially far from the null, they can diverge noticeably.

What this means in practice

• Cheap pre-screening. Before adding an expensive extra parameter to a large risk model, a score test tells you whether it's likely to matter, using only the model you already have fitted.
• Diagnostic tests are score tests. Many classic diagnostics — tests for heteroskedasticity (Breusch–Pagan), for serial correlation (Lagrange multiplier tests in time series) — are score tests in disguise, chosen specifically because they avoid refitting.
• Same asymptotic footing as its rivals. Wald, score, and likelihood-ratio tests are asymptotically equivalent — they agree in large samples — but choosing among them in finite samples is a real, practical decision about what's cheap to compute.