The Prosecutor's Fallacy

A DNA sample from a crime scene matches a suspect. The lab says the chance of a random person matching is 1 in a million. The prosecutor tells the jury: "There's a one-in-a-million chance this defendant is innocent." Is that right?

No — and the gap between what was proven and what was claimed is exactly the mistake. The lab measured $P(\text{match} \mid \text{innocent}) = 1/1{,}000{,}000$: how rare a match is if you're not the source. The jury needs $P(\text{innocent} \mid \text{match})$: how likely innocence is given that a match occurred. Swapping the two, treating $P(A \mid B)$ as if it equalled $P(B \mid A)$, is the prosecutor's fallacy, and the two quantities can differ by orders of magnitude.

Worked example

Suppose the crime happened in a city of 10 million adults, any of whom could in principle be the culprit, and exactly one of them actually did it. The match probability for an innocent person is 1 in a million, so among the roughly 10 million innocent adults, about $10{,}000{,}000 / 1{,}000{,}000 = 10$ of them would show a matching profile purely by coincidence. Add the one true culprit, who obviously matches too, and 11 people in the city match the DNA — only one of them guilty.

$$P(\text{innocent} \mid \text{match}) \approx \frac{10}{11} \approx 91\%.$$

A "one in a million" statistic, presented alone, has quietly become a suspect who is more likely innocent than guilty once you account for how many people were eligible to coincidentally match. The lab's number was correct; the sentence built on top of it was not.

The generalizable technique

Whenever a small conditional probability is used to imply guilt, membership, or causation, ask two questions: what is the base rate — how many people or cases were even eligible to produce this evidence by chance? — and what is the other conditional, the one nobody stated? Multiplying the rare-event probability by the size of the eligible pool gives the expected number of false positives, and comparing that to the (usually just one) true positive is what Bayes' rule is doing under the hood. The fallacy survives in finance too: a backtest with a "1-in-10,000 by chance" Sharpe ratio sounds airtight until you remember how many strategies were tried to find it.