Prediction Intervals Versus Confidence Intervals

A model predicts tomorrow's expected return from a factor at 0.5%. Ask "how confident are we in that 0.5% average," and you get one kind of range. Ask "how far might tomorrow's actual return land from that 0.5%," and you get a much wider one — the actual outcome on any single day is far noisier than the average tendency the model has estimated. These are genuinely different questions with genuinely different answers, and conflating them is one of the most common errors in reading a regression's output: a narrow confidence interval does not mean tomorrow's return will land close to the forecast.

An analogy: the average commute versus tomorrow's commute

A transit analyst can estimate, with fairly tight precision, that the average commute time on a given route is 32 minutes — with enough historical data, that average is pinned down quite precisely, maybe to within a minute either way. But that tells you almost nothing about how long your commute will take tomorrow: traffic, weather, and one-off delays mean any single day's commute could easily run anywhere from 20 to 50 minutes, even though the long-run average really is close to 32. The tight estimate of the average and the wide range for any one day's actual experience are both correct, simultaneously — they're answering different questions, and mixing them up leads to false confidence about what tomorrow actually holds.

The mechanics, one symbol at a time

For a regression prediction at a new point $x_0$, both intervals are centered on the same point estimate $\hat{y}_0$, but their widths differ. The confidence interval for the average response reflects only the uncertainty in estimating the regression line itself:

$$\hat{y}_0 \pm t^* \cdot \text{SE} \cdot \sqrt{ \frac{1}{n} + \frac{(x_0 - \bar{x})^2}{\sum (x_i - \bar{x})^2} } ,$$

which shrinks toward zero as the sample size $n$ grows — with enough data, you can pin down the average relationship almost exactly. The prediction interval for one new individual observation adds the leftover, irreducible noise around that average, $\sigma^2$, which never shrinks no matter how much data you collect:

$$\hat{y}_0 \pm t^* \cdot \text{SE} \cdot \sqrt{ 1 + \frac{1}{n} + \frac{(x_0 - \bar{x})^2}{\sum (x_i - \bar{x})^2} } .$$

In plain English: the only difference between the two formulas is that extra "+1" inside the square root for the prediction interval, representing the individual data point's own random scatter around the line — a source of uncertainty that more data can never eliminate, because it isn't about estimating the average better, it's about the fact that any one observation genuinely varies around that average by a random amount.

Worked example 1: a factor return forecast with a huge sample

A regression of daily returns on a factor, fit on 2,000 days, predicts $\hat{y}_0 = 0.5\%$ at a particular factor level, with $\bar{x}$, $x_0$ close together and standard error terms giving a confidence interval of $0.5\% \pm 0.05\%$, i.e., roughly $0.45 to $0.55 per $100 invested — tightly pinned down, because $n=2{,}000$ makes the $1/n$ term almost vanish. But the residual standard deviation of daily returns around the fitted line, $\sigma$, is large — say 1.2 percentage points. The prediction interval is approximately $0.5\% \pm 1.96 \times 1.2\% \approx 0.5\% \pm 2.35\%$, or roughly $-1.85\%$ to $2.85\%$ — more than 20 times wider than the confidence interval, because tomorrow's individual return is dominated by day-to-day noise the model was never going to explain away, no matter how much historical data it was trained on.

Worked example 2: extending both intervals out to an unusual value

Suppose a second forecast is made at a factor value $x_0$ far from the historical average $\bar{x}$ — an unusually extreme reading. The $(x_0 - \bar{x})^2$ term in both formulas grows, widening both intervals somewhat, but the confidence interval still stays comparatively narrow (maybe $0.4\%$ to $0.9\%$, versus $0.45\%$ to $0.55\%$ near the average) because it's still only tracking uncertainty about the average line's position. The prediction interval, by contrast, remains dominated by the same large $\sigma$ term regardless — widening only modestly to something like $-1.7\%$ to $3.0\%$. The lesson holds at extreme $x$ just as it does near the average: the confidence interval reflects how well you know the line; the prediction interval reflects how much any individual point can deviate from that line, and that deviation stays large everywhere.

Picture two shaded bands around the fitted line in a plot like this one: a thin band hugging the line (the confidence interval) and a much wider band enclosing almost all of the actual scattered points (the prediction interval) — the gap between the two bands is the day-to-day noise no amount of data shrinks away.

What this means in practice

Confidence intervals matter when the question is about the model itself — is a factor's average effect reliably positive, is a fair-value estimate well-pinned-down. Prediction intervals matter when the question is about an actual future outcome — how much could tomorrow's P&L on a single position realistically deviate from its expected value, which is the relevant range for setting stop-losses or sizing risk. Reporting a confidence interval when a decision actually depends on individual-outcome risk is a common way to understate real uncertainty and get blindsided by normal, expected day-to-day variation.