Exponentially Weighted Covariance and Half-Life Choice

An equal-weighted covariance matrix over the last two years treats a return from 500 days ago exactly like yesterday's. That's strange: markets change regime, volatility clusters, and correlations that held two years ago may have broken down since. Exponentially weighted covariance fixes this by giving recent observations more weight and older ones progressively less, so the estimate adapts as markets change. But this comes at a cost that's easy to underestimate: the faster you make the matrix adapt, the fewer effective observations it's really using at any moment, and the noisier the estimate becomes. Choosing the decay rate — usually expressed as a half-life — is a real trade-off, not a free upgrade.

An analogy: a rolling weather forecast versus a seasonal average

Imagine forecasting tomorrow's temperature. You could average every day from the last five years (stable, but slow to notice a heatwave starting this week) or just the last three days (catches the heatwave instantly, but jumpy — one unusually cool or warm day swings the forecast a lot). Exponentially weighted averaging is a dial between these extremes: a long half-life behaves like the five-year average, a short one like the three-day average. There's no universally correct setting — it depends on how fast the thing you're forecasting actually changes and how much noise you can tolerate.

The mechanics, one symbol at a time

Let $r_t$ be the return vector at time $t$ and $\lambda \in (0,1)$ be the decay factor. The exponentially weighted covariance estimate at time $t$ is

$$\widehat{\Sigma}_t = (1-\lambda) \sum_{k=0}^{\infty} \lambda^k \, r_{t-k}\, r_{t-k}^\top,$$

where each past observation $r_{t-k}$ is weighted by $\lambda^k$ — geometrically shrinking as $k$ (the number of periods back) grows. In plain English: today's covariance estimate is a weighted sum of all past outer products of returns, with weights that fall off exponentially, so recent data dominates and old data fades but is never fully discarded. The decay factor $\lambda$ is usually reparametrized as a half-life $h$, the number of periods after which a weight has fallen to half its original value:

$$h = \frac{\ln(0.5)}{\ln(\lambda)}, \qquad \text{equivalently } \lambda = 0.5^{1/h}.$$

In plain English: the half-life is a more interpretable dial than $\lambda$ directly — "an observation from 60 days ago counts for half as much as yesterday's" is easier to reason about than "$\lambda = 0.9885$." A short half-life (say, 20 days) makes the matrix track recent volatility and correlation shifts quickly; a long half-life (say, 250 days) makes it behave closer to an equal-weighted estimate over roughly a year, smoother but slower to react.

The effective sample size of an exponentially weighted estimate — roughly $2/(1-\lambda) - 1$, or approximately $2h/\ln 2 \approx 2.9h$ for a half-life of $h$ — tells you how much independent information the estimate is really drawing on. A short half-life doesn't just react faster; it is genuinely estimated from fewer effective data points, which is why it's noisier.

Worked example 1: a regime shift

Suppose two assets have a stable correlation of 0.3 for a year, then a market shock pushes their true correlation to 0.8 starting 10 days ago. With a 20-day half-life ($\lambda = 0.5^{1/20} \approx 0.966$), after 10 more days the weight on data from before the shift has fallen to roughly $\lambda^{10} \approx 0.966^{10} \approx 0.71$ relative weight remaining on old-regime data versus new — the estimate will already have moved meaningfully toward 0.8. With a 120-day half-life ($\lambda \approx 0.994$), the same 10 days of new data barely dents the weighted average — $\lambda^{10} \approx 0.994^{10} \approx 0.94$ — so the estimate is still dominated by the stale 0.3 correlation, badly lagging the true regime shift.

Worked example 2: the noise cost of reacting fast

Now suppose correlation is genuinely stable at 0.3 with no regime shift, just ordinary sampling noise in daily returns. With a 20-day half-life, the effective sample size is roughly $2.9 \times 20 \approx 58$ observations — small enough that random day-to-day noise causes the estimated correlation to bounce visibly, say between 0.15 and 0.45 over a few weeks, even though nothing structural changed. With a 120-day half-life, the effective sample size is roughly $2.9\times120\approx 348$ observations, and the estimate stays much closer to 0.3 throughout, because it's averaging over far more effective data. The fast estimator's extra responsiveness in worked example 1 is bought at the price of exactly this extra noise here — there's no setting that gives you both for free.

What this means in practice

Risk models used for daily VaR or portfolio hedging typically favor shorter half-lives (weeks to a couple months) because reacting to a real volatility spike quickly matters more than smoothness. Strategic asset allocation or long-horizon covariance estimates typically favor longer half-lives (many months to a year-plus) because stability and lower noise matter more than catching short-lived fluctuations. Many production risk systems blend two exponentially weighted estimates at different half-lives — one fast, one slow — precisely because no single half-life is right for every use case.