Overlapping Forecast Windows and What They Break

Suppose a model forecasts 20-day-ahead returns, and to build a large sample of forecasts to test it on, you compute one forecast every single trading day. Each forecast covers a 20-day window, and consecutive windows overlap by 19 of their 20 days — day 2's window and day 3's window share almost all the same underlying return data. The forecasts, and their errors, are consequently not independent observations even though there appear to be hundreds of them.

This matters because most statistical tests — a t-test on average forecast error, a standard error on a Sharpe ratio, a p-value on a regression coefficient — assume independent observations, and they use that assumption to shrink standard errors as sample size grows. Overlapping windows violate it directly: the effective number of independent data points is closer to (total days) / (window length) than to the number of daily forecasts computed, which can be twenty times smaller than it looks.

The consequence is that naively computed p-values and confidence intervals come out far too tight, making a forecast look statistically significant when it is really just the same handful of independent 20-day periods being counted repeatedly. A ten-year backtest with daily 20-day-ahead forecasts has roughly 2,500 daily observations but only about 125 independent 20-day blocks — treating it as 2,500 independent samples overstates confidence by a wide margin.

The standard fix is a Newey-West (or similar heteroskedasticity-and-autocorrelation-consistent) standard error, which explicitly corrects for the serial correlation overlapping windows induce, or simply subsampling to one non-overlapping observation per window before running ordinary tests.