The IID Assumption and When It Breaks

Every number a model hands you with an air of confidence — a standard error, a t-statistic, a cross-validation score, a p-value — rests on one quiet assumption about your rows. It assumes each row is a fresh, unrelated draw from a single fixed process. Break that assumption and the model still trains, still reports, still prints a t-statistic of 3.2. Nothing errors. The number is simply wrong, and usually wrong in the flattering direction.

This matters more in finance than almost anywhere else, because financial data breaks the assumption twice over. Yesterday's market state carries into today, so rows are not independent. And the process generating those rows genuinely changes — regulation, participants, volatility regimes, the fact that everyone else is now running your strategy — so rows are not identically distributed either. A quant who understands nothing else about statistics but understands this will make fewer expensive mistakes than one who knows everything else.

The analogy: two ways to poll a country

You want to know how the country will vote, so you survey a thousand people.

Survey one: a thousand names drawn at random from the electoral roll. Different towns, different ages, different jobs. A thousand genuinely separate opinions.

Survey two: a thousand people queueing outside one football stadium. Same city, same club, mostly the same demographic. You collected a thousand answers, but you learned roughly what you would have learned from asking fifty people and going home. The answers repeat because the people are connected. This is dependence — the second half of your sample largely echoes the first.

Survey three: the random national sample, but you ran it eleven months ago and there has been an economic crisis since. The sampling was flawless; the country simply is not the same country. This is non-stationarity — the distribution moved out from under you.

Overlapping returns are survey two. Backtesting a 2013 model on 2024 markets is survey three. Most real financial datasets are both at once.

Naming the pieces

"IID" stands for independent and identically distributed. Write your observations as $X_1, X_2, \dots, X_n$, where the subscript is just the row number and $n$ is how many rows you have. The assumption says

$$X_1, X_2, \dots, X_n \;\;\overset{\text{iid}}{\sim}\;\; P$$

In plain English: every row is generated by the same underlying mechanism $P$ (that is the identically distributed part), and learning the value of one row tells you nothing whatsoever about any other (that is the independent part).

Why anyone cares comes down to one formula. If the rows really are independent, the variance of a sample average shrinks in a very specific way:

$$\operatorname{Var}(\bar{X}) \;=\; \frac{\sigma^2}{n}$$

where $\bar{X}$ is your sample mean, $\sigma^2$ is the variance of a single observation, and $n$ is the sample size. In words: the wobble in your estimate falls in proportion to how many rows you have, so the standard error falls with the square root of $n$. Every error bar you have ever drawn is this formula. When rows are dependent it is simply false, and the correction is not small.

The explorer below shows the honest version: a running average settling onto the truth inside a $1/\sqrt{n}$ envelope. That envelope is what dependence widens.

Worked example 1: how much of your sample is real

A very common setup. You have four years of daily data, about $1{,}000$ trading days, and your signal predicts the 5-day forward return. You compute that 5-day return on a rolling basis, so you end up with $996$ observations and you feed all $996$ to your regression.

Look at two neighbouring rows. Row 1 covers days 1 to 5. Row 2 covers days 2 to 6. They share four of their five days. They are not two observations; they are one and a bit.

Make that exact. For overlapping windows of length $m$ built from independent daily returns, the correlation between two rows that are $k$ days apart is

$$\rho_k = \frac{m - k}{m} \quad \text{for } k < m, \qquad \rho_k = 0 \text{ otherwise}$$

which says in words: two windows are correlated in proportion to how many days they have in common. With $m = 5$: $\rho_1 = 0.8$, $\rho_2 = 0.6$, $\rho_3 = 0.4$, $\rho_4 = 0.2$, and zero beyond that.

Now the variance of the mean of correlated rows, for large $n$, is

$$\operatorname{Var}(\bar{X}) \;\approx\; \frac{\sigma^2}{n}\left(1 + 2\sum_{k\ge 1}\rho_k\right)$$

In words: take the textbook answer and multiply it by an inflation factor built from all the correlations you were pretending were zero. Compute it:

$$1 + 2\,(0.8 + 0.6 + 0.4 + 0.2) \;=\; 1 + 2(2.0) \;=\; 5.0$$

So the true variance of your estimate is five times what the software reported. Your effective sample size is $996 / 5 \approx 199$, not $996$. Standard errors are understated by $\sqrt{5} = 2.24$, so every t-statistic on the page is inflated by the same factor. A reported $t = 2.2$ — comfortably "significant" — is really $2.2 / 2.24 = 0.98$. Not significant, not close. And notice the number $199$ is exactly what you would have got by taking non-overlapping 5-day blocks: $996/5$. The overlap added no information at all, only false confidence.

Worked example 2: when the distribution itself moves

Now the other half. Independence is fine, but the process changes.

You fit a one-day 99% value-at-risk model on two calm years where annualised volatility ran at 10%. A trading year has about 252 days, so the daily standard deviation is $0.10 / \sqrt{252} = 0.0063$, that is $0.63$% a day. For a normal distribution the 99th percentile sits $2.326$ standard deviations out, so the model says:

$$\text{VaR}_{99} = 2.326 \times 0.0063 = 0.01465$$

In words: on a normal day you should lose more than $1.47$% roughly one time in a hundred. Fitted honestly, validated honestly, backtested cleanly on the calm sample.

Volatility then regime-shifts to 40% annualised. The daily standard deviation becomes $0.40/\sqrt{252} = 0.0252$, that is $2.52$%. Your threshold has not moved, but relative to the new distribution it now sits only

$$\frac{0.01465}{0.02520} = 0.582$$

standard deviations from zero. In words: the loss you called a once-in-a-hundred-days event is now barely half a standard deviation away. The chance of a normal draw falling below $-0.582$ standard deviations is $28$%.

So the breach rate goes from the designed 1% to 28% — roughly one day in four. Over a 250-day year that is about $70$ limit breaches instead of the $2.5$ you budgeted for. The model did not degrade gracefully; the error is a factor of 28. Nobody mis-specified anything. The world changed, and "identically distributed" was the only thing holding the number up.

What this means in practice

Count independent events, not rows. Before trusting any t-statistic, ask how many genuinely separate bets it represents. A monthly rebalanced strategy over ten years has about $120$ decisions, not the $2{,}500$ daily P&L observations you regressed on. Sample-weighting by overlap, as in Sample Weights & Label Uniqueness, is the systematic version of this.

Random cross-validation folds leak. Shuffling time-series rows puts days from the same week in both training and test folds, so the test fold is not independent of the training data and the score is optimistic. This is why finance uses Purged & Embargoed Cross-Validation rather than plain k-fold.

Deflate before you celebrate. If your rows overlap by a factor of $m$, divide your reported t-statistic by roughly $\sqrt{m}$ before deciding anything. Most "significant" backtest results do not survive that single division.

Assume the distribution will move, and measure it. Test on a period after the one you trained on, refit on a rolling window, and monitor input distributions in production so you find out that the world changed from your own dashboard rather than from your P&L. That monitoring is Concept Drift Detection.