Markov-Modulated Poisson Processes

Order arrivals on an exchange don't come at a single, steady rate all day — there are bursty periods around news releases and auctions, and quiet stretches between, and you can't directly observe which regime you're in, only the arrivals themselves. A plain Poisson process assumes one fixed rate $\lambda$ throughout, which badly misfits both regimes at once — calibrated to the average, it underpredicts bursts and overpredicts lulls. A Markov-modulated Poisson process (MMPP) fixes this by letting the arrival rate itself be controlled by a hidden Markov chain: the chain silently switches between a small number of states ("calm," "bursty"), and in each state, arrivals happen as an ordinary Poisson process at that state's own rate.

An analogy: a hidden traffic light controlling arrivals

Imagine cars arriving at a checkpoint, and behind the scenes a hidden controller flips between "green wave" (cars rush through) and "red wave" (arrivals trickle) — but watching only the checkpoint, you never see the light, just the resulting stream. During green-wave periods, arrivals look like a fast Poisson process; during red-wave, a slow one. Describing the whole day with one "average" rate misrepresents both periods — it can't quantify how bursty a green-wave burst really is, or how long a lull lasts, because both are generated by a genuinely different rate that needs to be modeled explicitly.

The structure, one piece at a time

An MMPP has two coupled parts. First, a modulating Markov chain, with states $1, \ldots, k$ (e.g., "calm," "bursty") and its own transition rates governing how long it stays in each state and which state it switches to next. Second, in state $i$, arrivals occur as a Poisson process with rate $\lambda_i$ — so the observed arrival rate at any moment equals whichever $\lambda_i$ corresponds to the current hidden state. Formally, if $X(t)$ is the hidden Markov state at time $t$, arrivals occur with instantaneous rate

$$\lambda\big(X(t)\big) \in \{\lambda_1, \ldots, \lambda_k\}.$$

In plain English: the process behaves like an ordinary Poisson process at every instant, but which Poisson rate is "currently active" is itself randomly changing according to an unobserved Markov chain — so unlike a plain Poisson process, MMPP arrivals show burstiness: clusters of closely-spaced arrivals during high-rate states, separated by longer gaps during low-rate states, a pattern a single-rate Poisson model cannot reproduce no matter how its one rate parameter is tuned.

Worked example 1: two-state order flow

Suppose the hidden regime is "calm" ($\lambda_1 = 2$ orders/second) 80% of the time on average and "bursty" ($\lambda_2 = 20$ orders/second) 20% of the time, with regime switches happening on average every 30 seconds. The overall long-run average arrival rate is the weighted mix: $\bar\lambda = 0.8 \times 2 + 0.2 \times 20 = 1.6 + 4.0 = 5.6$ orders/second. A plain Poisson model calibrated to this same 5.6/second average would predict a fairly steady 5–6 orders roughly every second; the true MMPP instead predicts long calm stretches averaging close to 2/second punctuated by short, intense bursts near 20/second — the same long-run average, wildly different short-term behavior, which matters enormously for anything sizing liquidity or predicting short-window order-count risk.

Worked example 2: the variance gives the model away

Over a 1-second window, a plain Poisson process with rate 5.6 has variance equal to its mean, $\text{Var} = 5.6$ (a defining Poisson property). The MMPP's 1-second count variance is higher, because it combines both the Poisson variance within each regime and the extra variance from switching between regimes — roughly $\text{Var}(\lambda(X(t))) \times (\text{regime persistence factor})$ added on top. With the calm/bursty split above, the between-regime variance component alone is $0.8 \times 0.2 \times (20-2)^2 = 0.16 \times 324 \approx 51.8$ (times a persistence-dependent scaling factor typically between 0 and 1), which can push the total observed variance to several times the mean — a hallmark of overdispersion that immediately signals a plain Poisson model is wrong, and is one of the standard diagnostic checks (comparing sample variance to sample mean of order counts) used to detect regime-switching arrival behavior in real order flow data.

Compare the plain Poisson shape above (fixed rate 5.6) against what real order-count histograms typically look like: MMPP-generated counts show a fatter right tail (from bursty-regime windows) and more mass near zero (from calm-regime windows) than a single Poisson distribution can produce — visible overdispersion relative to the smooth Poisson curve.

What this means in practice

MMPPs are the standard model wherever arrival-type events (orders, trades, quote updates) show regime-dependent burstiness: sizing order-book capacity for worst-case bursts rather than average load, detecting regime changes in real time from arrival-count statistics, and simulating realistic order flow for backtesting, where a plain Poisson simulator understates the clustering that actually stresses an execution strategy. Fitting one typically uses the Baum-Welch (EM) algorithm, treating the regime as hidden and inferring it from observed arrival counts.