ARMA Models

ARMA models are the linear grammar of time-series forecasting: they describe a stationary series as a filtered version of its own past and past shocks. They are the workhorse for the conditional mean of a series, and, just as importantly for a quant, the framework in which you discover that the conditional mean of returns is nearly flat, pushing the interesting structure into the conditional variance (GARCH). Mastering the AR/MA vocabulary, the lag-polynomial conditions, and the ACF/PACF diagnostic is core time-series literacy.

The three processes

Let $\{\varepsilon_t\}$ be white noise (mean zero, variance $\sigma^2$, uncorrelated). Define the lag operator $L$ by $L X_t = X_{t-1}$.

Autoregressive, AR($p$), the series regresses on its own past:

$X_t = \phi_1 X_{t-1} + \dots + \phi_p X_{t-p} + \varepsilon_t, \qquad \phi(L)X_t = \varepsilon_t,$

with $\phi(L) = 1 - \phi_1 L - \dots - \phi_p L^p$. Shocks have infinitely-persistent (geometrically decaying) effects.

Moving average, MA($q$), the series is a finite window of past shocks:

$X_t = \varepsilon_t + \theta_1\varepsilon_{t-1} + \dots + \theta_q\varepsilon_{t-q} = \theta(L)\varepsilon_t.$

A shock affects only the next $q$ periods, then is gone.

ARMA($p,q$) combines both, $\phi(L)X_t = \theta(L)\varepsilon_t$, which parsimoniously captures dynamics that a pure AR or MA would need many terms to match.

Stationarity and invertibility

These are the two conditions that make an ARMA model well-behaved, and they are dual statements about the roots of the two polynomials.

Stationarity concerns the AR part. AR($p$) is (weakly) stationary iff all roots of $\phi(z) = 0$ lie outside the unit circle (equivalently, the roots of the reciprocal characteristic polynomial lie inside). For AR(1), $\phi(z) = 1 - \phi z = 0$ gives $z = 1/\phi$, outside the unit circle iff $|\phi| < 1$, the familiar condition. When a root sits on the unit circle ($\phi = 1$), you have a random walk / unit root and the process is non-stationary. See Unit Roots and the ADF Test. A stationary AR has the Wold/MA($\infty$) representation $X_t = \phi(L)^{-1}\varepsilon_t = \sum_{j\ge 0}\psi_j\varepsilon_{t-j}$ with summable $\psi_j$.

Invertibility concerns the MA part. MA($q$) is invertible iff all roots of $\theta(z) = 0$ lie outside the unit circle, which lets you write the unobserved shocks as a convergent function of observed data, $\varepsilon_t = \theta(L)^{-1}X_t = \sum_{j\ge0}\pi_j X_{t-j}$. Without invertibility the model is not identified from data (two different MA parameters can produce the same ACF), so estimation software imposes it. Stationarity is about the AR roots; invertibility is about the MA roots, a clean symmetry worth remembering.

The ACF/PACF fingerprint

Box–Jenkins identification reads the order off two functions. The autocorrelation function (ACF) $\rho_k$ measures raw correlation at lag $k$; the partial autocorrelation function (PACF) measures the correlation at lag $k$ after removing the effect of the intermediate lags (the coefficient on $X_{t-k}$ in a regression on all lags up to $k$).

Process	ACF	PACF
AR($p$)	decays (geometric / damped sine)	cuts off after lag $p$
MA($q$)	cuts off after lag $q$	decays
ARMA($p,q$)	decays after lag $q$	decays after lag $p$

The mnemonic: AR shows in the PACF, MA shows in the ACF. An AR($p$) has a PACF that is exactly zero beyond lag $p$ (once the true predecessors are conditioned out, nothing is left); an MA($q$) has an ACF that is exactly zero beyond $q$ (shocks older than $q$ don't enter). In practice you supplement these with information criteria (AIC/BIC) to choose orders, since sample ACF/PACF are noisy.

Forecasting and the worked example

Take a stationary AR(1), $X_t = \phi X_{t-1} + \varepsilon_t$ with $|\phi| < 1$. Its unconditional mean is 0, variance $\gamma_0 = \sigma^2/(1-\phi^2)$, and ACF $\rho_k = \phi^k$, geometric decay. The optimal $h$-step forecast is $\hat X_{t+h} = \phi^h X_t$, which decays toward the unconditional mean at rate $\phi^h$: the series is mean-reverting, and the speed is governed by $\phi$. This is the linear skeleton of statistical mean-reversion strategies, an Ornstein–Uhlenbeck process is the continuous-time AR(1), and the half-life of reversion is $\log(0.5)/\log\phi$. See Mean Reversion. With $\phi = 0.9$ the half-life is about $6.6$ periods; with $\phi = 0.5$, one period.

Failure modes in financial data

• Returns have almost no linear autocorrelation. Efficient-market forces drive the ACF of returns to near zero at all lags, so ARMA($p,q$) on returns typically finds $\phi,\theta \approx 0$, the conditional mean is close to unforecastable. The structure lives in the conditional variance: the ACF of squared or absolute returns is strongly positive and persistent, which is precisely what motivates GARCH Volatility Models.
• Non-stationarity in disguise. Fitting ARMA to a series with a unit root (a near-1 AR root) yields unstable, non-convergent estimates; test for and difference out unit roots first. See Stationarity.
• Overfitting the ACF. Sample ACF/PACF are noisy and, under multiple testing, some lag will look "significant" by chance; parsimonious orders chosen by BIC generalize far better than kitchen-sink ARMA.
• Microstructure autocorrelation. Bid–ask bounce induces spurious negative lag-1 autocorrelation in high-frequency returns; illiquidity/stale prices induce spurious positive autocorrelation. Both are artifacts, not forecastable alpha.

In interviews

Be fluent in the AR/MA/ARMA definitions using the lag polynomial and able to state the two conditions with their root locations: stationarity ⇔ AR roots outside the unit circle, invertibility ⇔ MA roots outside the unit circle. The ACF/PACF table is a reliable question, know that AR($p$) cuts off in the PACF at $p$ and MA($q$) cuts off in the ACF at $q$, and be able to explain why. Derive the AR(1) facts ($\rho_k = \phi^k$, forecast $\phi^h X_t$, mean-reversion half-life $\log 0.5/\log\phi$). The punchline interviewers want for finance: ARMA on returns finds nothing because the mean is unforecastable, the predictable structure is in the variance, which is why GARCH exists.