Getting Expected Shortfall Right: Acerbi and Tasche on the Definition That Actually Works

By the early 2000s the risk management profession had reached a consensus. Artzner and coauthors had proved that Value-at-Risk fails a basic sanity axiom: it can report that a diversified portfolio is riskier than a concentrated one. Everyone agreed the fix was to look past the threshold and average the losses in the tail rather than just pointing at where the tail begins.

Then a slightly embarrassing thing happened. It turned out that "the average loss in the tail" can be written down in several different ways, they are not all the same, and some of them are not coherent either.

Acerbi and Tasche's paper is the careful cleanup. It sounds like a technicality, and in one sense it is. But it is the technicality that determines whether the risk measure your bank uses actually satisfies the property you adopted it for.

The problem: several definitions wearing the same name

Here are three plausible ways to formalize "the average loss in the worst cases."

Tail conditional expectation. The expected loss, conditional on the loss exceeding the VaR threshold. Mathematically natural, reads exactly like the English sentence.

Worst conditional expectation. The worst average loss over any set of outcomes with probability at least the tail probability.

Expected shortfall, properly defined. The average of the worst outcomes, defined as an average of quantiles rather than as a conditional expectation, constructed so that you always average exactly the right amount of probability mass.

For a nice, smooth, continuous distribution, all of these coincide. So for years people used them interchangeably and nothing went wrong.

But real loss distributions are not smooth and continuous. They are often discrete: you have a finite set of historical scenarios, or a Monte Carlo sample, or a credit portfolio where a name either defaults or does not. And when the distribution has atoms, meaning lumps of probability sitting at particular values, the definitions come apart.

The key idea via analogy: cutting the cake at exactly the right place

Imagine you want to serve exactly the worst 5 percent of outcomes. If your outcomes are spread perfectly smoothly, you can slice exactly 5 percent. Easy.

But suppose your distribution is lumpy. There is a single scenario, a big default, that by itself carries 3 percent of the probability, and it sits right at the boundary. Now, "the worst 5 percent" is ambiguous. Do you include the whole lump? That gives you more than 5 percent. Do you exclude it? Then you have less than 5 percent, and you have omitted the worst thing that can happen. Neither is right.

The naive tail conditional expectation, the "average of losses given that you exceed the VaR," makes exactly this mistake. Because it conditions on a threshold, it takes whatever probability mass happens to sit beyond that threshold, which may be more or less than the tail probability you intended. And because of that mismatch, the tail conditional expectation can fail subadditivity for discontinuous distributions. It can tell you that diversifying made you riskier, which is the exact failure you switched away from VaR to avoid.

Acerbi and Tasche's expected shortfall fixes this by not conditioning on a threshold at all. Instead of asking "what is the average loss beyond the VaR," it is constructed as an average of the quantiles in the tail. Think of it as slicing the cake by taking exactly the right amount of probability mass, and if a lump straddles the boundary, taking exactly the right fraction of that lump.

The result is a measure that:

• Always uses exactly the tail probability you asked for, whatever shape the distribution has.
• Is coherent for any distribution, discrete, continuous, or a mixture, which is the theorem the paper delivers.
• Reduces to the intuitive "average loss in the worst cases" whenever the distribution is well behaved, so nothing is lost in the ordinary case.

They also give it a beautifully practical estimator. If you have a set of scenarios, sort the losses from worst to best, take the worst k of them, and average. That is it. It is simple, it converges to the true expected shortfall, and it is exactly what a risk system would naturally do anyway. The paper's contribution is showing that this natural estimator is estimating the right thing, and that the slightly different natural-seeming alternative is not.

Why it mattered

• It made expected shortfall safe to standardize on. If you are going to build a global regulatory framework on a risk measure, you had better be sure the measure is coherent for the lumpy, discrete, scenario-based distributions that real risk systems actually produce. This paper is what establishes that. The Basel Committee's eventual move to expected shortfall for market risk rests on a measure being well defined in exactly this sense.
• It settled a genuinely confusing terminology mess. Conditional Value-at-Risk, expected shortfall, tail conditional expectation, average value at risk, and expected tail loss were floating around as near-synonyms. The paper draws the lines: they agree in the smooth case, they diverge in the discrete case, and only one of them is coherent in general.
• It legitimized the obvious estimator. The "sort the losses and average the worst ones" recipe is what every practitioner would do instinctively. This paper is the proof that the instinct is right, which is more valuable than it sounds, because the alternative recipe (average everything past the VaR) is equally instinctive and is subtly wrong.
• It is a lesson in mathematical care. A definition that is obviously equivalent to another one under convenient assumptions can be dangerously different once those assumptions go. In risk, the assumptions always go.

The honest limitations

• It is a paper about definitions, not about the world. It tells you which formalization of tail loss is coherent. It does not make tail losses easier to estimate, or make your model of the tail any less wrong.
• Estimating the tail is still the hard part, and it is hard. Expected shortfall at 97.5 percent is computed from the worst 2.5 percent of your data. That is very few observations, and the estimate is correspondingly noisy. Expected shortfall is more data-hungry than VaR precisely because it uses information from deep in the tail, where there is by definition almost no data. Jorion's point about estimation risk in VaR applies with even more force here.
• Backtesting expected shortfall is genuinely difficult. VaR has a clean test: count how many days you breached it and compare to expectation. Expected shortfall does not have such a simple test, and this remains a live practical complaint from risk managers and regulators. The mathematical property of elicitability, and expected shortfall's lack of it in the simple sense, has generated a whole literature about this.
• Coherence does not imply safety. A coherent measure applied to a distribution that omits the disaster is a coherent way of being wrong. The axioms discipline the formula; nothing disciplines your scenario set except judgment.

The one-line takeaway

Acerbi and Tasche showed that the intuitive "average loss given that you breached your VaR" is not always coherent once your loss distribution is lumpy, and defined expected shortfall properly, as an average of quantiles that always captures exactly the tail probability you asked for, which is the version that is coherent for any distribution and the version the world's risk systems now use.