Semi-Markov Processes

You model a market as switching between "quiet" and "volatile" regimes using a continuous-time Markov chain, which assumes time spent in each regime is exponentially distributed — memoryless, meaning a regime that's lasted three hours is no more or less likely to end in the next minute than one that just started. But real volatility regimes don't behave that way: a volatile regime that's run an hour often has real momentum to keep going, while quiet regimes rarely end abruptly in their first ten minutes. The exponential assumption is mathematically convenient but empirically wrong here. A semi-Markov process keeps the useful part of the Markov structure — the next regime depends only on the current one, not the whole history — while letting the time spent in each regime follow any distribution, not just exponential.

An analogy: a relay race with variable-length legs

A relay race has a fixed sequence structure — who runs next depends only on who's currently running, following a known handoff order (that part is "Markov"). But unlike an idealized model where every leg takes an identical, memoryless amount of time, real relay legs vary — a hilly leg takes systematically longer than a flat one, and once a runner is 80% through a long hilly leg, their leg is nearly over, not "equally likely to end at any moment" as memorylessness implies. A semi-Markov process is exactly this: fixed transition logic between states, but each state's holding time is drawn from whatever distribution actually fits, carrying real information about "how much longer" once you know how long you've already been there.

The structure, one piece at a time

A semi-Markov process is defined by two pieces. First, a transition matrix $P = (p_{ij})$, giving the probability that, upon leaving state $i$, the process moves next to state $j$ — exactly like an ordinary discrete-time Markov chain. Second, a set of holding-time distributions $F_{ij}(t)$, one for each possible transition, specifying how long the process stays in state $i$ before making that particular jump to $j$ (in the simplest version, holding time depends only on the current state $i$, not the destination). In plain English: at each visit to a state, the process first decides "how long do I stay" (drawn from that state's own, possibly non-exponential distribution) and only afterward decides "where do I go next" (using the same simple transition-probability logic a Markov chain would use). This decouples how long from how memoryless, which an ordinary continuous-time Markov chain cannot do, because it's forced to use the exponential distribution — the only continuous distribution that is both memoryless and consistent with the Markov property holding at every instant, not just at transition times.

Worked example 1: two-regime market with realistic holding times

Suppose "quiet" regimes last, on average, 5 days, but with low variance — most run close to 5 days, tightly peaked (unlike an exponential, which would allow many very short quiet periods). "Volatile" regimes last on average 2 days but with high variance — some end in half a day, others drag on a week. The transition matrix is simple: from quiet, always go to volatile ($p=1$); from volatile, always go to quiet ($p=1$) — the holding times, not the transition probabilities, do the interesting modeling work. The long-run fraction of time in each regime is $\pi_{\text{quiet}} = 5/(5+2) \approx 0.714$ and $\pi_{\text{volatile}} \approx 0.286$ — the same formula an exponential chain would give, since that calculation only needs the means, not the full holding-time shape.

Worked example 2: where the distribution shape actually matters

Where semi-Markov modeling earns its keep is not the long-run fractions (worked example 1) but short-horizon forecasts: given the market has been in a volatile regime for exactly 4 days already, what's the probability it ends in the next day? Under an (incorrect) exponential assumption with the same 2-day mean ($\lambda = 0.5$/day), memorylessness says the answer is the same regardless of the 4 days already elapsed: $1 - e^{-0.5 \times 1} \approx 0.393$. But if the true holding-time distribution is, say, tightly peaked around 2 days (low variance, most volatile regimes end between 1.5 and 2.5 days), then having already survived 4 days is a strong signal the regime is unusually persistent this time, and the correct semi-Markov-based forecast — using the conditional hazard rate of the actual holding-time distribution at $t=4$ — could put the probability of ending in the next day much higher, since 4 days already exceeds the typical range, making an imminent end far more likely than the memoryless model suggests.

Drag the rate parameter above — this is the incorrect memoryless assumption an ordinary continuous-time Markov chain would force onto every state's holding time, regardless of whether the real process actually looks like this.

What this means in practice

Semi-Markov processes fit wherever regime or state durations show real "typical duration" structure that plain exponential models flatten away: volatility regime models, credit-rating transitions where time-in-rating matters, equipment reliability models where holding times reflect genuine wear. The tradeoff: semi-Markov models lose the elegant matrix-exponential machinery of continuous-time Markov chains and generally require simulation rather than closed-form transition probabilities.