Sequence-to-Sequence Forecasting Architectures

Say you want to forecast the next 20 trading days of a volatility index from the last 60 days of history. One option is to predict day 21, feed that prediction back in as if it were real data, predict day 22 from that, and so on — a recursive approach where a bad early guess drags every later guess down with it. A sequence-to-sequence (seq2seq) model instead reads the whole 60-day history once and outputs all 20 future days directly, without ever pretending its own guesses are real data partway through.

The encoder-decoder idea

Think of it like translating a sentence. A human translator doesn't produce the target sentence word by word purely from the previous target word — they first read and absorb the entire source sentence, forming a compressed understanding of it, and only then start producing the translation, drawing on that whole understanding at every step. A seq2seq forecaster works the same way: an encoder network reads the input history and compresses it into a summary (a vector, or a set of vectors), and a decoder network unpacks that summary into the sequence of future values.

Concretely, the encoder is usually a recurrent network (an LSTM or GRU) or a transformer that processes the input window step by step and produces a final hidden state — a fixed-size numerical summary of "everything that happened in this window." The decoder is a second network that starts from that summary and generates the forecast horizon, conditioned on the full input summary rather than only on its own most recent guess. Many modern versions add an attention mechanism, letting the decoder look back at every individual input point (not just the final summary) when producing each output step.

Why not just predict one step and repeat it

A simple recursive model chains one-step predictions: predict $t+1$, feed it in as if observed, predict $t+2$ from that, and so on out to the horizon. This is easy to build but has a structural flaw: any error in the $t+1$ prediction becomes an input to the $t+2$ prediction, so errors compound. Seq2seq architectures sidestep this by producing the whole horizon from the original, real input — the decoder never has to trust its own earlier guesses as ground truth for later ones (though attention-based decoders that generate step by step still face a milder version of this if they condition on their own prior outputs).

Worked example: encoding a volatility window

Suppose the encoder is a small LSTM processing 60 daily VIX-like readings. At each of the 60 steps it updates an internal state vector (say, 32 numbers) that tries to capture "where the level currently sits, how fast it's rising or falling, how choppy it's been recently." After all 60 steps, that final 32-number vector is passed to the decoder. The decoder is a second LSTM (or a small feed-forward stack) that takes this vector and produces 20 numbers in one pass — Monday through the fourth Friday's forecast — each conditioned on the same shared summary rather than on a chain of the model's own prior-day guesses. If the true horizon shows a spike on day 8, an attention-equipped decoder can weight the input days that looked most similar to past pre-spike patterns more heavily when producing that specific output.

Drag the parameters above to see how a single flexible function can be reshaped — the same intuition applies to a decoder reshaping one learned summary vector into many different output shapes depending on what the encoder fed it.

What this means in practice

Seq2seq and its descendants (temporal fusion transformers, N-BEATS-style architectures) are the workhorses behind most modern multi-horizon forecasting products, especially where you have many related series to forecast jointly with shared weights. The practical cost is data and tuning: encoder-decoder networks need enough training series and enough history to learn a genuinely useful summary rather than memorizing noise, and they are considerably harder to debug than a linear model when something goes wrong.