Reframing Forecasting as Supervised Learning

Classical forecasting models — ARIMA, exponential smoothing — treat a time series as a single mathematical object with structure (trend, seasonality, autocorrelation) baked into the model's equations. Machine learning models like gradient-boosted trees or random forests don't work that way; they take a table of rows and columns and learn a mapping from inputs to an output. To use one of these models on a time series at all, the series first has to be reshaped into that table. That reshaping — turning one long sequence into many rows of "past window → next value" — is what it means to reframe forecasting as supervised learning, and the choices made during that reshaping matter as much as the choice of model.

The sliding window

The standard technique is a sliding window: pick a lookback length $k$, and for every point $t$ in the series, build one training row whose features are the $k$ most recent values before $t$ — $x_{t-1}, x_{t-2}, \dots, x_{t-k}$ — and whose label is the value at $t$ itself, $y = x_t$. Slide this window forward one step at a time across the whole series, and a single sequence of $n$ observations becomes roughly $n - k$ rows of an ordinary supervised-learning table. Any standard regression model — a random forest, a boosted tree ensemble, even plain linear regression — can now be trained on it exactly as if the rows were independent, unrelated data points, even though they obviously are not.

Worked example: windowing a ten-point series

Take the small daily-close series 100, 102, 101, 104, 106, 105, 108, 110, 109, 112, and use a lookback of $k = 3$. The sliding window produces these rows:

$x_{t-3}$	$x_{t-2}$	$x_{t-1}$	$y = x_t$
100	102	101	104
102	101	104	106
101	104	106	105
104	106	105	108
106	105	108	110
105	108	110	109
108	110	109	112

Ten raw observations became seven training rows. A model trained on this table learns to map "the last three prices" to "the next price," using exactly the same machinery — same loss function, same tree-splitting logic — it would use on any other regression problem, with no special-cased time-series logic anywhere inside the model itself.

What this means in practice

This reframing is what makes gradient-boosted trees viable for forecasting at all, and it's why they can outperform classical models when there are many exogenous features (macro indicators, cross-sectional signals from other assets) that don't fit naturally into an ARIMA-style equation — a tree model can just take them as extra columns in the same table. The tradeoff is that the model has no built-in notion of "time" beyond whatever lag features are explicitly constructed; if a useful pattern lives outside the chosen lookback window, or depends on the order of values within the window in a way tree splits can't easily represent, the model simply cannot see it, whereas a classical model with an explicit seasonal term captures that structure by construction. A second consequence is more subtle: because the rows generated by a sliding window from the same series are highly overlapping and correlated, a naive random train/test split (rather than a strict time-based split) leaks information from adjacent, nearly-identical windows across the split boundary — this is one of the most common sources of inflated backtest performance in ML-based forecasting.

Practice

1. Using the ten-point series above with a lookback of $k=2$ instead of 3, write out the resulting training table.
2. Explain why randomly shuffling the rows of a sliding-window table before splitting into train and test sets produces an overly optimistic backtest, even though each row is technically a valid input-output pair.