Foundation Models for Time-Series Forecasting

Training a forecasting model for a newly listed stock is a small-data problem — a few months of history at most, nowhere near enough for a neural network to learn much on its own. But that stock's volume, volatility, and price patterns aren't unrelated to the millions of other time series that already exist in the world: retail sales, weather, web traffic, thousands of other stocks. A foundation model for time series is trained once, on a huge and diverse collection of series from many domains, learning general patterns of trend, seasonality, and mean reversion that transfer to a brand-new series it has never seen, with little or no series-specific retraining.

What makes it "foundation"

The term borrows directly from large language models: a single large network is pre-trained on a massive, broad corpus (there, text; here, a huge pool of time series from many domains) and then applied to a new task with minimal adaptation — sometimes literally zero extra training ("zero-shot"), sometimes a small amount of fine-tuning on the new series ("few-shot"). This is the opposite of the traditional approach of fitting one bespoke model (even a simple ARIMA) per series from scratch. Architecturally, most time-series foundation models use a transformer-based sequence model, often treating a time series much like a sequence of tokens — chunking values into patches, or discretizing values into a vocabulary of bins that a decoder-only transformer can predict over, directly reusing techniques from language modeling.

In plain English: the model has already seen an enormous number of "shapes" a time series can take — a slow uptrend followed by a plateau, a weekly seasonal cycle, a sharp spike that reverts — from thousands of unrelated domains, so when it sees the first 30 points of your brand-new, low-history series, it can often recognize which of those general shapes it's most likely following and extrapolate accordingly, without needing months of that specific series' own history to learn the shape from scratch.

Worked example: forecasting a newly listed stock

Suppose a stock listed eight weeks ago, giving you 40 trading days of volume history — far too little to train a dedicated neural forecaster from scratch without badly overfitting. Feeding those 40 points into a pre-trained time-series foundation model, the model can produce a forecast for the next five days using patterns it learned from millions of other series that showed similar early-life volume decay curves (high volume right after listing, tapering off over the following weeks), even though it has never seen this specific stock before. A locally trained model with only 40 data points, by contrast, has nowhere near enough examples to learn a decay curve on its own and would likely default to something close to a flat extrapolation of the last few days.

Where this still falls short

Foundation models trained mostly on non-financial or generic-financial data can miss institution-specific structure — market microstructure effects, a specific corporate action's typical volume pattern, regime shifts unique to a given asset class — that a model built and tuned specifically for that domain would capture. In practice they're often used as a strong, cheap baseline or as one input blended with domain-specific models rather than as a full replacement.

Step through the descent above and notice how a model converges to a good fit given enough steps and data — foundation models sidestep needing many steps on the new series specifically by having already done that convergence on a huge, unrelated pool of data beforehand.

What this means in practice

Time-series foundation models are most valuable exactly where classical forecasting struggles most: series with very little history, cold-start problems (new products, new listings, new markets), and situations where training thousands of bespoke per-series models isn't practical. They're generally not yet a substitute for a carefully tuned, domain-specific model on a well-established series with abundant history.