Reconstruction Error as an Anomaly Score

Labeled anomalies are rare almost by definition — a "flash crash," a "spoofing episode," a "broken feed" happens a handful of times a year, if that, and each one is a little different from the last. Training a supervised classifier to spot the next one from a dozen examples of the last few is a losing proposition. What you do have, in abundance, is normal data — millions of ordinary ticks, ordinary days, ordinary spreads. An autoencoder turns that asymmetry into an advantage: train a network to compress and then rebuild only normal data, and its failure to rebuild something well becomes the anomaly detector, with zero labeled anomalies required to train it.

The analogy: a forger who only ever copied Rembrandts

Imagine a forger who spent twenty years perfecting the ability to reproduce Rembrandt paintings, brushstroke by brushstroke, from memory alone — hand them a Rembrandt, take it away, and they can recreate it almost perfectly. Now hand that same forger a Picasso and ask for a faithful copy from memory. It will come out wrong — not randomly wrong, but wrong in a specific, informative way: the forger will unconsciously drag Picasso's shapes back toward Rembrandt-like textures and proportions, because that's the only vocabulary they've built. The size of the mistake — how far the copy strays from the original — tells you the copy's source wasn't a Rembrandt, without the forger ever being shown a Picasso before or being told what "not a Rembrandt" looks like. An autoencoder trained only on normal market data is that forger, and reconstruction error is the size of the mistake.

The maths

An autoencoder is two networks glued together: an encoder that compresses input $x \in \mathbb{R}^n$ down to a lower-dimensional code $z \in \mathbb{R}^m$ (with $m \ll n$), and a decoder that expands $z$ back to a reconstruction $\hat x \in \mathbb{R}^n$:

$$z = f_{\text{enc}}(x), \qquad \hat x = f_{\text{dec}}(z)$$

In words: squeeze $x$ through a narrow bottleneck of size $m$, then try to rebuild it. Because $m$ is small, the network cannot just memorize an identity map — it is forced to learn which patterns in the training data are common enough to be worth encoding efficiently, and it discards the rest as noise it can't afford to keep.

Training minimizes reconstruction error over the normal training set, typically mean squared error:

$$\mathcal{L} = \frac{1}{n}\sum_{i=1}^{n} (x_i - \hat x_i)^2$$

In words: average, over every input dimension, the squared gap between the original value and the rebuilt value. Small loss means the network's compressed vocabulary was rich enough to capture this input well.

At inference time, for a new point $x^*$, compute the same reconstruction error and use it directly as an anomaly score:

$$s(x^*) = \| x^* - \hat x^* \|^2$$

In words: run $x^*$ through the trained encoder-decoder, measure how far the output strayed from the input, and call that number the anomaly score. Points that look like the training distribution get reconstructed well (low $s$); points that don't — because the network never learned a compact way to represent whatever makes them unusual — get reconstructed poorly (high $s$). Flag anything above a chosen threshold $\tau$.

Worked example 1: computing reconstruction error on two inputs

An autoencoder was trained on ordinary 5-minute return vectors. Feed it a normal input $x = (0.1, -0.2, 0.15, -0.05, 0.1)$ and it reconstructs $\hat x = (0.11, -0.18, 0.13, -0.04, 0.09)$:

$$s(x) = (0.1-0.11)^2+(-0.2+0.18)^2+(0.15-0.13)^2+(-0.05+0.04)^2+(0.1-0.09)^2$$ $$= 0.0001+0.0004+0.0004+0.0001+0.0001 = 0.0011$$

Now feed it an anomalous input, a flash-crash-style vector $x^* = (0.1, -0.2, -3.5, 2.8, -0.1)$, and because the network never learned to represent violent one-bar reversals compactly, it reconstructs something close to its normal vocabulary, $\hat x^* = (0.12, -0.15, -0.3, 0.4, -0.05)$:

$$s(x^*) = (0.1-0.12)^2+(-0.2+0.15)^2+(-3.5+0.3)^2+(2.8-0.4)^2+(-0.1+0.05)^2$$ $$= 0.0004+0.0025+10.24+5.76+0.0025 \approx 16.01$$

The anomalous score, 16.01, is roughly 14,500 times larger than the normal score, 0.0011. That ratio — not the absolute numbers, which depend on scaling — is what makes reconstruction error such a strong discriminator here: the network's bottleneck simply has no efficient way to encode a 3.5-standard-deviation reversal followed by an equally violent snapback, because it never had to during training.

Worked example 2: setting a threshold from a normal-only sample

Suppose reconstruction errors on 10 held-out normal days come out as: $0.8, 1.1, 0.9, 1.3, 0.7, 1.0, 1.2, 0.85, 0.95, 1.05$ (arbitrary units). Mean and standard deviation:

$$\bar s = \frac{0.8+1.1+0.9+1.3+0.7+1.0+1.2+0.85+0.95+1.05}{10} = \frac{9.85}{10} = 0.985$$

Sample variance (using $n-1 = 9$):

$$\sum (s_i - \bar s)^2 \approx 0.375, \qquad \sigma \approx \sqrt{0.375/9} \approx 0.204$$

A common rule of thumb sets $\tau = \bar s + 3\sigma \approx 0.985 + 3(0.204) \approx 1.60$. Any new day scoring above 1.60 gets flagged. This threshold was calibrated using only normal data — no anomaly examples were needed, which is exactly the point: you're setting a bar for "unusually bad reconstruction relative to how normal data usually reconstructs," not a bar tuned against known anomaly shapes.

Autoencoder training loss on normal data should settle to a stable low floor as more normal examples are seen — much like a running average converging on the truth. A model whose reconstruction loss hasn't stabilized yet will produce an unreliable, drifting anomaly score, so check convergence before trusting the threshold.

What this means in practice

This pattern shows up as a first-line detector for broken data feeds, spoofing-like order-book shapes, sensor faults on alternative-data pipelines, and pre-screening candidate regime changes before a human or a more expensive model looks at them. Its big advantage is that it needs no examples of the failure mode you're trying to catch — useful precisely because novel anomalies, by definition, don't look like the last one. Its big limitation is the flip side: if an anomaly happens to be reconstructible by the learned bottleneck (say, it's a smooth, low-dimensional shift the network's compact code space can represent), it will slip through with a low score, undetected.