Wilson and Clopper-Pearson Intervals

Imagine backtesting a signal that fired 20 times and was right 18 of them — a 90% hit rate. Do you actually believe the true win probability is close to 90%, or could it plausibly be 70%? The textbook "normal approximation" interval, $\hat p \pm 1.96\sqrt{\hat p(1-\hat p)/n}$, is known to behave badly exactly here: with a small sample and a proportion near 0 or 1, it can produce an interval that overshoots 100% or undershoots 0%, which is nonsense for a probability.

The Wilson interval fixes this by solving the interval directly from the underlying test statistic rather than plugging $\hat p$ into a simple formula, which keeps it inside $[0,1]$ and gives much better coverage at small $n$. The Clopper-Pearson interval goes further, inverting the exact binomial distribution rather than any normal approximation at all — it's "exact" in the sense of guaranteeing at least the stated confidence level, though this makes it somewhat wider (more conservative) than Wilson in practice.

For the 18/20 example: the naive normal interval gives roughly $[76\%, 104\%]$ — already broken, since it exceeds 100%. The Wilson interval instead gives approximately $[68\%, 97\%]$, a sensible, bounded range. Clopper-Pearson gives a similar but slightly wider $[66\%, 98\%]$, reflecting its more conservative guarantee.