Evaluating Volatility Forecasts

You built a model that forecasts tomorrow's volatility will be 1.5%. Tomorrow arrives and the return is $+0.3\%$. Was the forecast good or bad? You genuinely cannot tell from that one number, a small return is perfectly consistent with either a correct low-volatility forecast or an incorrect one that simply got lucky. Unlike a point forecast of price, where you can compare prediction to outcome directly, volatility itself is never directly observed on any given day, you only ever see one realized return, not the underlying distribution it was drawn from. Evaluating a volatility forecast is therefore a genuinely harder problem than it first looks, and it requires two separate ingredients: a stand-in for the unobservable truth, and a scoring rule that actually rewards getting close to it.

The analogy

Imagine grading a weather forecaster who predicts "30% chance of rain" every day. On a day it does not rain, was the forecast wrong? Not necessarily, 30% still means it usually doesn't rain. You cannot grade a single probabilistic forecast against a single outcome; you need many days, and a rule that rewards forecasts well-calibrated across all of them. Grading a volatility forecast has the same structure: one squared return is a noisy stand-in for "the truth" on any single day.

Building the two ingredients

The proxy problem. The obvious proxy for "true" daily variance is the squared return, $r_t^2$. It is unbiased, on average across many days it equals the true variance, but it is a dreadfully noisy single-day proxy, because it uses only one price observation. A far better proxy, when available, is realized variance built from high-frequency intraday returns: $RV_t = \sum_{i} r_{t,i}^2$, summing squared 5-minute (or similar) returns across the day. In words: instead of one big number for the whole day, sum up many small squared moves sampled throughout the day, this converges much closer to the day's true variance because it uses far more information (see Realized Volatility Estimation).

The scoring rule. Once you have a proxy $\hat{\sigma}_t^2$ (realized variance, ideally) to compare against your forecast $f_t = \sigma_t^2$, a common loss function is

$\text{QLIKE} = \frac{\hat{\sigma}_t^2}{f_t} - \ln\!\left(\frac{\hat{\sigma}_t^2}{f_t}\right) - 1.$

In words: take the ratio of realized-to-forecast variance; if the forecast is exactly right, that ratio is 1 and the whole expression is 0 (perfect score). Any deviation, forecast too high or too low, makes the expression positive, and it is specifically shaped to penalize an under-forecast (dangerously missing a big variance day) more sharply than an over-forecast of the same relative size, which matches how a risk manager actually feels about the two kinds of mistakes.

Widen or narrow this bell curve's spread and picture drawing a single random return from it each day. That single draw is your only glimpse of "the truth" a naive squared-return proxy has to work with, and it lands near the middle regardless of the true spread most of the time, which is why one squared return is such a weak signal about forecast accuracy.

Worked example 1: QLIKE on three days

A model forecasts daily variance $f_t$ of $0.0002, 0.0002, 0.0002$ (constant $\approx 1.4\%$ daily vol) for three days. Realized variance proxies come in at $\hat{\sigma}^2 = 0.00015, 0.00025, 0.0009$ (the third day had an outsized move).

Day 1: ratio $= 0.75$, QLIKE $= 0.75 - \ln(0.75) - 1 = 0.75 + 0.288 - 1 = 0.038$. Day 2: ratio $= 1.25$, QLIKE $= 1.25 - \ln(1.25) - 1 = 1.25 - 0.223 - 1 = 0.027$. Day 3: ratio $= 4.5$, QLIKE $= 4.5 - \ln(4.5) - 1 = 4.5 - 1.504 - 1 = 1.996$.

Average QLIKE $\approx (0.038+0.027+1.996)/3 \approx 0.687$. Notice how Day 3 dominates the average, a single large under-forecast contributes almost 50 times the loss of the small, roughly symmetric misses on Days 1 and 2, which is precisely the intended behavior: badly missing a volatility spike should hurt a model's score far more than being slightly off on ordinary days.

Worked example 2: comparing two models

Model A forecasts a flat 1.2% volatility every day, regardless of conditions. Model B (EWMA-based) adapts, predicting 1.0% in calm stretches and 2.0% after a shock. Suppose over a month the realized volatility genuinely does spike to around 2.5% for a week after a shock, then settles back near 1.0%.

During the calm weeks, Model A's 1.2% forecast is reasonably close to the 1.0% realized figure (ratio near 1.44, small QLIKE contribution). During the spike week, Model A still forecasts 1.2% against a 2.5% realized figure (ratio near 4.3, a large QLIKE hit for every day of that week), while Model B's 2.0% forecast against 2.5% realized (ratio 1.56, a much smaller hit) tracks the spike far better. Averaged over the month, Model B's total QLIKE will be markedly lower: an adaptive model earns its complexity during the periods that matter most.

What this means in practice

Model selection for VaR engines and margin systems hinges on this kind of evaluation; a model that looks fine on average but consistently under-forecasts spikes will pass casual inspection and then fail a VaR backtest (see VaR Backtesting and the Kupiec Test) exactly when it is needed most. QLIKE is preferred over simple squared-error comparisons because it remains a valid ranking even when the realized-variance proxy itself is noisy.