Generalized Method of Moments

Maximum likelihood asks a lot of you. To write one down you must commit to the entire distribution of your data — every tail, every dependency, every shape. In finance you rarely know that much, and often know what you wrote is wrong. What you usually do know are a few relationships that must hold on average: a regression says the errors average to zero and are uncorrelated with the regressors; a queue model says the average waiting time is one over the arrival rate. The Generalized Method of Moments turns exactly that kind of partial knowledge into an estimate, and asks for nothing more.

An analogy: several witnesses, one suspect

Three witnesses describe a suspect's height. One says "about six feet", another "a little under six", a third "closer to five nine". No single height satisfies all three, and insisting on a perfect fit is hopeless. What you do instead is obvious: pick the height that leaves the smallest total disagreement, listening hardest to the witness who stood closest and least to the one who glimpsed the suspect from across the street.

That is GMM in full. Each moment condition is a witness saying something your parameter should satisfy on average. With more witnesses than unknowns you cannot satisfy them all, so you minimise the weighted disagreement. The disagreement left over is itself informative, and there is a formal test for whether it is too large to believe.

The pieces, one symbol at a time

Write $\theta$ (theta) for the unknown parameter and $x_i$ for the $i$-th observation. A moment condition is a function $g(x, \theta)$ chosen so that, at the true parameter, its average is exactly zero:

$$E\big[g(x, \theta_0)\big] = 0 .$$

In plain English: the model claims a particular quantity averages out to nothing when you plug in the correct parameter. For example, if the model says the mean of $x$ is $\theta$, then $g(x, \theta) = x - \theta$ works: the deviations from the true mean average to zero.

You cannot compute a population average, so you use the sample one, written $\bar{g}$ ("g-bar"):

$$\bar{g}(\theta) = \frac{1}{n}\sum_{i=1}^{n} g(x_i, \theta) .$$

In plain English: the same quantity, averaged over the data you actually have. By the The Law of Large Numbers this converges to the population version, which is the entire reason the method works.

Now count. If you have as many conditions as parameters, set $\bar{g}(\theta) = 0$ and solve — that is the classical Method of Moments and GMM. If you have more conditions than parameters, no $\theta$ makes them all zero, and GMM minimises a weighted total instead:

$$\hat{\theta} = \arg\min_{\theta} \ \bar{g}(\theta)' \, W \, \bar{g}(\theta) .$$

In plain English: choose the parameter that makes all the sample conditions as close to zero as possible at once, where $W$ decides how much each one counts. $W$ is the weighting matrix — a table of how much you trust each condition. The efficient choice is the inverse of the conditions' covariance, which is the formal version of "listen to the closest witness."

The whole approach leans on sample averages settling onto population averages. Drag the sample size in the explorer below and watch that happen; the shrinking envelope is what makes a moment condition usable as evidence.

Worked example 1: exactly identified

Trades arrive at a desk with waiting times that you model as exponential with rate $\lambda$ (lambda), the average number of trades per second. The model gives one moment condition:

$$E[X] = \frac{1}{\lambda} \quad \Longrightarrow \quad g(x, \lambda) = x - \frac{1}{\lambda} .$$

Your 100 observed gaps average $\bar{x} = 4.0$ seconds. One condition, one unknown, so set the sample average to zero and solve:

$$4.0 - \frac{1}{\lambda} = 0 \quad \Longrightarrow \quad \hat{\lambda} = \frac{1}{4.0} = 0.25 \text{ trades per second} .$$

That is the classical method of moments, and it is the special case of GMM where the weighting matrix is irrelevant because you can drive every condition exactly to zero.

Worked example 2: over-identified, with a real compromise

Now use a second fact the exponential model implies: $E[X^2] = 2/\lambda^2$. Suppose the sample average of $x^2$ came out at $40$. Two conditions, still one unknown:

$$\bar{g}_1(\lambda) = 4.0 - \frac{1}{\lambda}, \qquad \bar{g}_2(\lambda) = 40 - \frac{2}{\lambda^2} .$$

Step 1 — see the disagreement. Condition one alone gives $\hat{\lambda} = 1/4.0 = 0.250$. Condition two alone gives $\hat{\lambda} = \sqrt{2/40} = \sqrt{0.05} = 0.224$. The two witnesses disagree. Neither is wrong; the data simply does not fit the model perfectly.

Step 2 — set the weights. Weight each condition by one over the variance of its sample average. For the first, the variance of a mean of exponentials is $(1/\lambda^2)/n = 16/100 = 0.16$, so $w_1 = 1/0.16 = 6.25$. For the second, the variance of $x^2$ under this model works out to $20/\lambda^4$, so its sample average has variance $5120/100 = 51.2$ and $w_2 = 1/51.2 = 0.0195$. The first condition is trusted far more, exactly as the analogy demands. (A full GMM would also account for the overlap between the two conditions; here we keep them separate to stay hand-computable.)

Step 3 — minimise the criterion. The quantity to minimise is

$$J(\lambda) = 6.25 \left(4.0 - \frac{1}{\lambda}\right)^2 + 0.0195\left(40 - \frac{2}{\lambda^2}\right)^2 .$$

Evaluate it on a grid, by hand:

$\lambda$	$\bar{g}_1$	$\bar{g}_2$	$J(\lambda)$
0.2236	$-0.472$	$0.00$	1.394
0.2300	$-0.348$	$2.19$	0.850
0.2370	$-0.219$	$4.39$	0.677
0.2400	$-0.167$	$5.28$	0.718
0.2500	$0.000$	$8.00$	1.250

The minimum sits at $\hat{\lambda} \approx 0.237$: neither witness gets exactly what they asked for, and both get close.

Step 4 — test the model. The leftover disagreement $J = 0.677$ is Hansen's J-statistic. Under the hypothesis that all conditions are correct it behaves like a chi-square with degrees of freedom equal to the number of extra conditions, here $2 - 1 = 1$. The 5% cutoff for one degree of freedom is 3.84, so 0.677 is easily inside. The exponential model survives this test — it is not contradicted by the data.

What this means in practice

• You already use it. OLS is GMM with the condition "errors are uncorrelated with the regressors"; instrumental variables is GMM with "errors are uncorrelated with the instruments." Seeing them as one method is most of the payoff.
• It survives where likelihood cannot. Asset pricing models specify an Euler equation, not a distribution. GMM estimates them directly — that is what Hansen's Nobel was for.
• Extra conditions are free information, until they are not. In finite samples the estimated weighting matrix is itself noisy, and piling on conditions can push the estimate badly off. Two-step GMM on short samples is a known trap.
• The J-test is a genuine specification test. A large J says at least one condition is false. It does not say which, which is the frustrating part.