State-Space Models

Most time-series models you meet, moving averages, autoregressions, trend-plus-noise, look like separate recipes. State-space models reveal they're all the same shape underneath. The trick is to split the world into two layers: a hidden state that you can't see but that follows its own rules, and observations that are noisy, partial glimpses of that state. Almost any time-series problem can be poured into this mold, and once it is, a single set of tools (the The Kalman Filter and its cousins) does the estimating.

Think of tracking the "true" fair value of an asset. You never observe it cleanly, every printed price is the fair value plus a dusting of microstructure noise, the bid-ask bounce, a stale quote, a fat-finger. The fair value is the hidden state; the prices are the observations. Separating the two is exactly what a state-space model lets you do.

The two equations

A state-space model is just a pair of equations. The state (transition) equation says how the hidden state evolves from one step to the next; the observation (measurement) equation says how what you see relates to that hidden state:

$$x_t = f(x_{t-1}) + w_t \qquad \text{(state evolves)}$$ $$y_t = g(x_t) + v_t \qquad \text{(you observe it, noisily)}$$

Here $x_t$ is the hidden state at time $t$, $y_t$ is the observation you actually record, $w_t$ is the process noise that jostles the state as it moves, and $v_t$ is the measurement noise that corrupts your view of it. The functions $f$ and $g$ encode the dynamics and the measurement. Note the Markov structure: tomorrow's state depends only on today's (see Markov Chains), which is what keeps the whole thing tractable.

Why it unifies everything

The framework's power is that swapping in different $f$, $g$, and noise types recovers models you already know:

• Linear $f$ and $g$ with Gaussian noise gives the The Kalman Filter setting, the state and its uncertainty are tracked exactly.
• A discrete hidden state (say "bull regime" vs "bear regime") gives a hidden Markov model.
• ARMA models (ARMA Models) can be rewritten in state-space form, which is how software actually fits them.
• Structural models decompose a series into hidden trend, seasonal, and cycle components, each a piece of the state.

Learn to think in states and observations and you stop memorising separate recipes.

Worked example: the local level model

The simplest useful state-space model is the local level: a hidden level that random-walks, observed with noise.

$$x_t = x_{t-1} + w_t, \quad \operatorname{Var}(w) = Q; \qquad y_t = x_t + v_t, \quad \operatorname{Var}(v) = R.$$

Everything hinges on the signal-to-noise ratio $q = Q/R$: how fast the true level actually drifts versus how noisy your view of it is. Suppose the process noise is $Q = 1$ and the measurement noise is $R = 4$, so $q = 0.25$, the level drifts slowly relative to the observation jitter. Working out the filter's steady state (solving $P^2 = QP + QR$ for the prediction variance) gives $P = \tfrac{1 + \sqrt{17}}{2} \approx 2.56$, and a steady-state gain

$$K = \frac{P}{P + R} = \frac{2.56}{6.56} \approx 0.39.$$

That $0.39$ is beautifully interpretable: in steady state, each new observation gets $39\%$ weight and your running estimate keeps $61\%$. In other words, the optimal filter for a random-walk-plus-noise is exactly an exponential moving average with smoothing $\alpha \approx 0.39$, and the noisier the observations (smaller $q$), the smaller $\alpha$ and the more you smooth. This is why exponential smoothing, which people often use as an ad-hoc trick, falls out of a proper state-space model as the right answer.

Where it misleads

• Identifiability. With two noise sources ($Q$ and $R$), the data sometimes can't tell them apart, only their ratio is well-pinned. The likelihood can be flat, so estimated variances are shaky in short samples.
• Model assumptions bite. The state-space form assumes a specific structure for how the state evolves. Get $f$ or the noise wrong (a regime break, a fat tail) and the filtered state lags or misleads, confidently.
• Non-Gaussian, nonlinear reality. The clean, exact filtering only holds in the linear-Gaussian case. Beyond it you need approximations (extended/unscented filters, particle filters) that trade exactness for flexibility.

The mental model: the world runs a hidden process you'd love to see, and hands you smudged photographs of it. A state-space model formalises that setup, and the The Kalman Filter is the machinery that reconstructs the hidden picture from the smudges, one frame at a time.