Prewhitening and Bandwidth Choice for HAC Estimators

Backtest returns are usually autocorrelated — today's residual leaks into tomorrow's. Ordinary standard errors assume no such leakage, so they understate the true uncertainty and make a mediocre strategy look statistically solid. A Newey-West (HAC) standard error corrects this by weighting nearby-in-time covariances, but the correction needs a bandwidth: how many lags of autocorrelation to include. Pick too few and you're back to understating error; pick too many and the estimate gets noisy and can even lose the positive-definiteness that makes it usable.

Prewhitening runs the return series through a low-order autoregression, keeps the residuals (now much closer to independent), applies the HAC estimator to those, then "recolors" the variance estimate back to account for the AR filter. The intuition: instead of asking a wide, sloppy averaging window to soak up correlation, remove the correlation directly and let the window do a smaller job.

Suppose a signal's raw returns have first-lag autocorrelation of 0.4. Fitting an AR(1) and prewhitening drops the residual autocorrelation toward zero, so a HAC estimator with bandwidth 2 on the residuals can do what a bandwidth of 6–8 would have needed on the raw series. Fewer lags means less estimation noise in the covariance matrix, which in turn means a tighter, more trustworthy confidence interval around the strategy's Sharpe ratio.

The tradeoff is that the AR fit itself is estimated with error, and a badly misspecified prewhitening model can introduce its own bias. In practice, quants treat bandwidth choice (often via the Newey-West or Andrews rule-of-thumb formulas) and prewhitening as complements, not substitutes — prewhiten first, then let an automatic bandwidth rule pick a small number of remaining lags.