The Bias Statistic for Risk Model Testing

A risk model says a stock's monthly volatility is $6\%$. Is that number any good? Checking one stock over one month tells you almost nothing, a stock can move more or less than predicted purely by chance. But run the same check across a thousand stocks over a hundred months, and a systematic pattern, the model consistently underestimating or overestimating actual risk, becomes impossible to hide. The bias statistic is exactly this large-scale check, applied every stock and every period, at once.

The analogy

A single weather forecast being off by five degrees on one day proves nothing, forecasts have irreducible error. But if a particular forecaster's "high of 70°" predictions run five degrees too cold on average, across a thousand forecasts, that is a real, correctable bias in the forecasting method itself, not noise. The bias statistic applies exactly this large-sample logic to a risk model: not "was this one volatility forecast right," but "across everything the model has ever forecast, does it run systematically too high or too low."

Building the statistic

For each stock $i$ and period $t$, standardise the realized return by the model's predicted risk at the start of that period:

$b_{i,t} = \frac{R_{i,t}}{\hat{\sigma}_{i,t}}.$

If the risk model is well-calibrated, these standardised returns $b_{i,t}$ should behave like draws from a distribution with standard deviation close to 1, since dividing a return by its own correctly-forecast volatility should, by construction, remove the scale differences between calm and volatile stocks. The bias statistic for a group (say, all stocks in a given month, or one stock across many months) is the sample standard deviation of these standardised returns:

$B = \sqrt{\frac{1}{N}\sum_{i=1}^{N} b_{i,t}^2}.$

In words: if the model's risk forecasts are correctly sized, $B$ should sit close to 1.0. A bias statistic well above 1, say 1.4, means realized returns are routinely bigger than the model predicted, the model is understating risk. A bias statistic well below 1, say 0.7, means the model is overstating risk, forecasting more volatility than actually shows up.

If a risk model is well-calibrated, standardised returns should look like draws from a curve with exactly this width. A bias statistic above 1 means the real distribution of standardised returns is wider than this curve; below 1 means it is narrower.

Worked example 1: computing the bias statistic for four stocks

Four stocks had predicted monthly volatilities of $4\%, 6\%, 8\%, 10\%$ and realized returns of $5\%, 9\%, 6\%, 15\%$ that month. Standardised: $5/4=1.25$, $9/6=1.50$, $6/8=0.75$, $15/10=1.50$. Squaring and averaging: $(1.25^2+1.50^2+0.75^2+1.50^2)/4 = (1.5625+2.25+0.5625+2.25)/4 = 6.625/4 = 1.656$. Taking the square root, $B = \sqrt{1.656} \approx 1.29$. A bias statistic of 1.29 suggests this model understated risk by roughly 29% across these four stocks that month, a single month is far too small a sample to draw a firm conclusion, but the direction is informative.

Worked example 2: tracking bias across many months

Running the same calculation across 200 months for one stock gives bias statistics that average to $1.05$, with individual months ranging from 0.6 to 1.8, but no sustained trend. A 1.05 long-run average is close enough to 1.0 to call the model reasonably well-calibrated for that stock, month-to-month noise around that average is expected and not itself a red flag. Contrast that with a second stock whose bias statistic averages 1.35 over the same 200 months with a tight range (1.2 to 1.5, rarely dipping near 1.0), that consistent, narrow-range elevation is the signature of a genuine, systematic risk underestimate for that name, not noise.

What this means in practice

Risk model vendors publish bias statistics as a standard quality metric, and a systematic drift above 1 across an entire model, not just one stock, is the clearest signal that the model's volatility forecasting method (its decay rate, its lookback window) needs recalibrating before the next rebalancing cycle.