Aleatoric vs Epistemic Uncertainty

A model outputs "60% chance of an up day" twice, on two different days. On the first day, that 60% is basically as good as it gets — the market genuinely could go either way, and no amount of extra data would sharpen the number. On the second day, the model is guessing, because it has barely seen conditions like today's and a model trained on more data would answer with real confidence. Same number, two entirely different situations, and a single probability cannot tell you which one you're in.

The analogy: a loaded die versus an unfamiliar one

Roll a fair, six-sided die. You cannot predict the outcome better than 1-in-6 no matter how many times you've watched it roll before — the uncertainty is baked into the die itself. That is aleatoric uncertainty, from the Latin for dice: irreducible randomness in the process you're modelling.

Now hand someone a die they've never seen, possibly weighted, and ask them to bet before they've watched it roll even once. Their uncertainty is enormous, but it is a different kind — it comes from not knowing the die yet. Watch it roll a hundred times and that uncertainty shrinks toward the die's true (possibly still random) behaviour. That is epistemic uncertainty, from the Greek for knowledge: uncertainty about the model, not about the world, and it is reducible by more data.

Splitting the two

For a predictive distribution over outcome $y$ given input $x$ and model parameters $\theta$, the total predictive variance decomposes as:

$$\mathrm{Var}(y \mid x) = \underbrace{\mathbb{E}_{\theta}[\mathrm{Var}(y \mid x, \theta)]}_{\text{aleatoric}} + \underbrace{\mathrm{Var}_{\theta}(\mathbb{E}[y \mid x, \theta])}_{\text{epistemic}}$$

In words: imagine many plausible versions of your model, each consistent with the training data (that's the randomness over $\theta$). The aleatoric term averages each version's own built-in noise — how spread out $y$ is even if you knew that version's parameters exactly. The epistemic term measures how much the different versions disagree with each other about the expected outcome. Noise inside one model's head is aleatoric; disagreement between models is epistemic.

Worked example 1: a coin with a known and an unknown bias

Case A. A fair coin, bias known to be exactly $p = 0.5$. Ask "what's the chance of heads next flip?" The answer is $0.5$ and stays $0.5$ no matter how many more flips you watch. Predictive variance is $p(1-p) = 0.25$, entirely aleatoric — there is nothing left to learn.

Case B. A coin with unknown bias, and you've seen 3 heads out of 4 flips. A Bayesian estimate (using a uniform prior, so the posterior is Beta(4, 2)) puts the expected bias at $4/6 = 0.667$, but the posterior itself is spread out — plausible bias values from roughly $0.3$ to $0.95$ given only four flips. If you plug in a bias of $0.4$, predictive variance is $0.4 \times 0.6 = 0.24$; if you plug in $0.9$, it's $0.9 \times 0.1 = 0.09$. Each of those numbers is aleatoric variance conditional on a guessed bias, but the fact that they disagree with each other (0.24 vs 0.09) is epistemic uncertainty: the model doesn't yet know which bias is right. Flip the coin 400 more times and the posterior narrows sharply — the epistemic piece shrinks — while the aleatoric piece, whatever the true bias turns out to be, does not.

Worked example 2: two trading signals with the same confidence score

A classifier assigns "58% probability of an up move" to two different setups.

Setup 1 occurs during normal, liquid trading hours with thousands of similar historical examples in the training set. An ensemble of five independently trained versions of the model all output values between 57% and 59% — tight agreement. Epistemic uncertainty is low; the 58% reflects real, near-irreducible market randomness. Trading this signal at face value is reasonable.

Setup 2 occurs during a rare pre-holiday half session, with only a handful of similar examples ever seen. The same five-model ensemble outputs 30%, 45%, 58%, 70%, and 85% — wildly disagreeing. The average happens to also be roughly 58%, but the number is fragile: it reflects the model's ignorance, not the market's randomness. Sizing a position the same way in both setups is a mistake — Setup 2 needs either a smaller bet or a decision to sit out and gather more data first.

What this means in practice

A single softmax probability or point forecast conflates both sources into one number, which is exactly the trap. To separate them, you need some way of sampling "plausible models" — an ensemble of independently trained networks (see Deep Ensembles for Uncertainty Estimation), Monte Carlo dropout, or a full Bayesian posterior over weights. Their average prediction estimates the outcome; their disagreement estimates epistemic uncertainty; each one's own residual spread estimates aleatoric uncertainty. In production risk systems this split drives concrete decisions: high epistemic uncertainty says "gather more data, or reduce position size, or route to a human," while high aleatoric uncertainty says "the model is doing its job — the world is just noisy here, size accordingly and move on." Conformal prediction (see Conformal Prediction) sidesteps needing this decomposition explicitly by producing valid intervals regardless of the source, but understanding the split is what tells you why an interval is wide, which matters when deciding whether to collect more data or simply accept the risk.