Wilson and Clopper-Pearson Intervals

Imagine backtesting a signal that fired 20 times and was right 18 of them, a 90% hit rate. Do you actually believe the true win probability is close to 90%, or could it plausibly be 70%? The textbook "normal approximation" interval, $\hat p \pm 1.96\sqrt{\hat p(1-\hat p)/n}$, is known to behave badly exactly here: with a small sample and a proportion near 0 or 1, it can produce an interval that overshoots 100% or undershoots 0%, which is nonsense for a probability.

The Wilson interval fixes this by solving the interval directly from the underlying test statistic rather than plugging $\hat p$ into a simple formula, which keeps it inside $[0,1]$ and gives much better coverage at small $n$. The Clopper-Pearson interval goes further, inverting the exact binomial distribution rather than any normal approximation at all, it's "exact" in the sense of guaranteeing at least the stated confidence level, though this makes it somewhat wider (more conservative) than Wilson in practice.

For the 18/20 example: the naive normal interval gives roughly $[76\%, 104\%]$, already broken, since it exceeds 100%. The Wilson interval instead gives approximately $[68\%, 97\%]$, a sensible, bounded range. Clopper-Pearson gives a similar but slightly wider $[66\%, 98\%]$, reflecting its more conservative guarantee.