Effective Sample Size in MCMC

When you draw samples from a Markov chain to approximate a posterior distribution, each new draw depends on the one before it — the chain moves in small steps, so neighboring samples tend to look alike. That correlation means a chain of 10,000 draws does not tell you as much as 10,000 truly independent draws would. Effective sample size (ESS) answers the practical question: how many independent samples would give you the same precision as this correlated chain?

ESS is computed from the chain's autocorrelation at increasing lags. Roughly, $\text{ESS} = n / (1 + 2\sum \rho_k)$, where $n$ is the number of draws and $\rho_k$ is the correlation between samples $k$ steps apart. A chain that mixes fast — jumping around the posterior with little memory of where it just was — has autocorrelations that die off quickly, so the sum stays small and ESS is close to $n$. A slow, "sticky" chain has autocorrelations that decay slowly, and ESS can be a small fraction of $n$, sometimes under 1%.

Worked example. A sampler runs 5,000 iterations for a parameter of interest, and the lag-sum in the formula above works out to 4 (i.e. $\sum \rho_k = 2$). That gives $\text{ESS} = 5000 / (1 + 4) = 1000$ — the chain behaves, for estimation purposes, like 1,000 independent draws rather than 5,000.