Effective Sample Size in MCMC

When you draw samples from a Markov chain to approximate a posterior distribution, each new draw depends on the one before it, the chain moves in small steps, so neighboring samples tend to look alike. That correlation means a chain of 10,000 draws does not tell you as much as 10,000 truly independent draws would. Effective sample size (ESS) answers the practical question: how many independent samples would give you the same precision as this correlated chain?

ESS is computed from the chain's autocorrelation at increasing lags. Roughly, $\text{ESS} = n / (1 + 2\sum \rho_k)$, where $n$ is the number of draws and $\rho_k$ is the correlation between samples $k$ steps apart. A chain that mixes fast, jumping around the posterior with little memory of where it just was, has autocorrelations that die off quickly, so the sum stays small and ESS is close to $n$. A slow, "sticky" chain has autocorrelations that decay slowly, and ESS can be a small fraction of $n$, sometimes under 1%.

Worked example. A sampler runs 5,000 iterations for a parameter of interest, and the lag-sum in the formula above works out to 4 (i.e. $\sum \rho_k = 2$). That gives $\text{ESS} = 5000 / (1 + 4) = 1000$, the chain behaves, for estimation purposes, like 1,000 independent draws rather than 5,000.