Continuous-Time Markov Chains

A discrete-time Markov chain (see Markov Chains) moves from state to state on a fixed clock — once per tick. But an order book doesn't update on a metronome: an order arrives, or a trade prints, or a cancellation happens, at random, unpredictable moments, and the "state" (best bid, best ask, queue depth) only changes when one of those events fires. A continuous-time Markov chain (CTMC) is the natural model for systems like this: the state stays put for a random amount of time, then jumps.

An analogy: a light that flickers at random moments

Think of a light bulb that's either ON or OFF. It doesn't switch at fixed intervals — it switches at random moments, and how long it stays in each state before switching is itself random (though on average, ON periods might last longer than OFF periods, or vice versa). Crucially, the bulb has no memory: knowing it's been ON for the last five minutes tells you nothing extra about when it will next switch off, beyond knowing it's currently ON. That "no memory of how long you've already waited" property is the defining feature of the exponential holding times that drive a CTMC.

The mathematics, one symbol at a time

A CTMC is defined by two things: a discrete set of states, and a rate matrix $Q$ (also called the generator), where $q_{ij}$ (for $i \neq j$) is the instantaneous rate of jumping from state $i$ to state $j$, and the diagonal entries satisfy $q_{ii} = -\sum_{j \neq i} q_{ij}$, so each row sums to zero. In words: $q_{ij}$ tells you how "eager" the process is to move from $i$ to $j$ per unit time, and the diagonal is just minus the total eagerness to leave $i$ at all. The chain works in two steps: while in state $i$, it waits an exponentially distributed time with rate $|q_{ii}|$ (mean holding time $1/|q_{ii}|$), then jumps to state $j \neq i$ with probability $q_{ij}/|q_{ii}|$. The probability of being in each state at time $t$, given the vector $\pi_0$ at time 0, evolves as

$$\pi_t = \pi_0 \, e^{Qt},$$

read as: the state distribution at time $t$ is obtained by applying the matrix exponential of $Q t$ — the continuous-time analogue of raising a transition matrix to a power in the discrete case.

Worked example 1: a two-state order book indicator

Model the best-quote side as either BUY-pressure or SELL-pressure. Suppose the rate of switching from BUY to SELL is $q_{BS} = 2$ per second, and from SELL to BUY is $q_{SB} = 4$ per second. The mean time spent in BUY before switching is $1/2 = 0.5$ seconds; the mean time in SELL is $1/4 = 0.25$ seconds. Since the chain leaves SELL faster, it spends proportionally less long-run time there — the long-run fraction of time in BUY works out to $q_{SB}/(q_{BS}+q_{SB}) = 4/6 \approx 66.7\%$, versus $33.3\%$ in SELL, simply because BUY periods last twice as long on average.

Worked example 2: expected number of jumps in a window

If a system is in a state with total exit rate $|q_{ii}| = 3$ per minute, the number of jumps out of that state in any given minute (while it stays there) behaves like a Poisson process with rate 3 (see The Poisson Process). The probability of at least one jump in the next 20 seconds ($1/3$ of a minute) is $1 - e^{-3 \times (1/3)} = 1 - e^{-1} \approx 63.2\%$ — over a third of the time, the state survives that 20-second window unchanged.

What this means in practice

CTMCs are the backbone of queueing models (order arrival and service as jumps), credit-rating migration models (a bond hops between rating states at random times), and simplified limit-order-book models where the state is queue depth or spread. The generator matrix $Q$ is usually estimated directly from timestamped event data — count transitions and divide by total time spent in each state — which is exactly what example 1 does in miniature.