The Temporal Fusion Transformer
A forecasting architecture that combines attention mechanisms with built-in variable-selection layers, so it can both predict multiple steps ahead and show which inputs it actually relied on.
Most deep forecasting models are effectively black boxes — accurate, but giving no clear sense of which inputs drove a given prediction. The Temporal Fusion Transformer (TFT) was designed specifically to close that gap: it mixes recurrent layers for local processing, an attention mechanism for capturing longer-range dependencies, and a dedicated variable-selection network that learns, at each time step, how much weight to give each input feature.
The Temporal Fusion Transformer forecasts multiple steps ahead using attention over the whole input window, while a built-in variable-selection layer reports which specific input features actually mattered for each prediction — accuracy and interpretability together, not traded off against each other.
Handling different kinds of inputs
Real-world forecasting problems mix input types the TFT explicitly separates: static features that never change (a store's location), known future inputs (a scheduled holiday), and observed past values that are only known historically (past sales). The TFT's variable-selection networks handle each category separately before combining them, and its attention layer then decides which past time steps are most relevant to the current forecast.
Because it reports feature-importance weights alongside its forecasts, the TFT is popular in settings like demand and revenue forecasting where stakeholders need to understand why the model expects a change, not just the number itself.
Related concepts
Practice in interviews
Further reading
- Lim, B. et al., 'Temporal Fusion Transformers for Interpretable Multi-Horizon Time Series Forecasting' (2021)