Cross-validation, and why it is dangerous on time series

Question

You fit a predictive model and want to estimate how well it will do on unseen data using only the data you have.

Explain k-fold cross-validation: the algorithm, why it estimates generalization, and where it fails, especially for financial time series.

Accepted Answer

The algorithm. Split the data into k equal folds. For each fold, train the model on the other k-1 folds and measure its error on the held-out fold. Average the k held-out errors. Every observation is predicted exactly once, by a model trained without it. Why it works. The held-out error is an (approximately) unbiased estimate of generalization error, because the model never touched the points it is scored on. Averaging over folds uses all the data for both training and testing while keeping them separated at each step, which is why it beats a single train/test split for small samples. Where i…

Cross-validation, and why it is dangerous on time series

Your answer

More Statistics questions