Probabilistic Graphical Models

Ten binary variables — say, whether ten different sectors are "risk-on" or "risk-off" today — have a joint probability distribution with $2^{10} = 1{,}024$ possible combinations, each needing its own probability. Fifty variables need $2^{50}$ — more numbers than could ever be estimated from any realistic amount of data, let alone stored. Almost none of that complexity is necessary in practice, because most variables don't directly depend on most other variables — energy sector sentiment doesn't need forty intermediate connections to explain oil prices, it depends on oil prices directly and on little else. A probabilistic graphical model writes down exactly which variables directly depend on which, as a graph, and uses that graph to factor the impossible full joint distribution into a handful of small, estimable pieces.

The analogy: an org chart instead of a full meeting transcript

Imagine trying to predict everyone's mood in a fifty-person company by writing down, for every possible combination of fifty individual moods, how likely that exact combination is — an absurd task. Now imagine instead drawing the org chart: each person's mood depends mainly on their direct manager's mood and their own immediate team's situation, not on the mood of someone in an unrelated department three floors away. The org chart doesn't capture every possible influence — it's a simplification — but it captures the influences that actually matter, and it turns an impossible fifty-variable joint prediction problem into fifty small, local ones: "given my manager's mood, what's my mood likely to be?" A graphical model is exactly this kind of org chart, built for probability distributions instead of moods.

The mechanism: factoring a joint distribution along a graph

A Bayesian network is a graphical model built on a directed graph with no cycles, where each variable's node points from its direct causes to itself. The joint probability of all variables factors as a product over each variable, conditioned only on its direct parents in the graph:

$$P(x_1, x_2, \ldots, x_n) = \prod_{i=1}^{n} P\big(x_i \mid \text{parents}(x_i)\big)$$

Here $x_i$ is the $i$-th variable, $\text{parents}(x_i)$ is the set of nodes with an arrow pointing directly into $x_i$ in the graph (its direct causes, and nothing farther upstream), and $\prod$ means multiply across all $n$ variables. In words: the full joint probability breaks apart into small local factors, one per variable, each conditioned only on its direct parents rather than on every other variable in the system. If a variable has no parents at all, its factor is just its own unconditional probability. This factorization is not an approximation forced onto the data — it is exactly the true joint distribution whenever the graph correctly captures which variables directly influence which, because the graph structure literally encodes a set of conditional-independence assumptions: any two variables not directly connected are assumed conditionally independent given their parents.

The saving in numbers needed is dramatic. Ten binary variables with a naive joint table need $2^{10}-1 = 1{,}023$ independent numbers. The same ten variables, if each has at most 2 parents in a graphical model, need at most $10 \times 2^2 = 40$ numbers (one small conditional table per variable) — a difference of more than 25×, and the gap widens explosively as the variable count grows.

Worked example 1: factoring a 3-variable joint

Take the graph oil price ($O$) → energy sentiment ($E$) → market risk appetite ($M$), a simple chain. The full factorization is

$$P(O, E, M) = P(O)\, P(E \mid O)\, P(M \mid E)$$

Suppose $P(O{=}\text{high}) = 0.3$, $P(E{=}\text{positive}\mid O{=}\text{high}) = 0.8$, and $P(M{=}\text{risk-on}\mid E{=}\text{positive}) = 0.7$. The joint probability of all three specific outcomes together is

$$P(O{=}\text{high},\, E{=}\text{positive},\, M{=}\text{risk-on}) = 0.3 \times 0.8 \times 0.7 = 0.168$$

Only three small numbers were needed — $P(O)$, $P(E\mid O)$, $P(M\mid E)$ — never a giant table over all joint combinations of $O$, $E$, and $M$ simultaneously, and this factored answer, $0.168$, is exactly the same number a full joint table would have given for this specific combination, had one been built directly.

Worked example 2: reading off conditional independence

In the same chain graph, $M$ (market risk appetite) has only $E$ as a parent, not $O$ directly. This graph structure asserts $M$ is conditionally independent of $O$ given $E$ — once you know $E$, learning $O$'s value adds nothing further about $M$. Check this with numbers: suppose $P(E{=}\text{positive}\mid O{=}\text{low}) = 0.75$ (almost as high as the $0.8$ from $O{=}\text{high}$) and $M$'s probability still only depends on $E$: $P(M{=}\text{risk-on}\mid E{=}\text{positive}) = 0.7$ regardless of which value of $O$ produced that $E$. Whether $O$ was high or low, once $E{=}\text{positive}$ is known, $M{=}\text{risk-on}$ has the same $0.7$ probability either way — exactly what the missing direct edge between $O$ and $M$ claims. If instead $M$'s probability changed depending on $O$'s value even after conditioning on $E$, that would mean the graph is wrong — it's missing a real direct dependency that should have been drawn in as an edge.

What this means in practice

Graphical models underlie a huge range of applied probabilistic reasoning in finance: hidden Markov models for regime detection are a graphical model with a chain structure, Bayesian network credit risk models factor default probability through explicit causal chains (macro conditions → sector health → firm-level default), and Markov random fields extend the same factoring idea to undirected graphs where influence flows both ways rather than causally. The core payoff everywhere is the same: turning an intractable full joint distribution into a small set of local, estimable conditional distributions, tied together by a graph that can be inspected, questioned, and revised.

Practice

1. Extend the chain graph with a fourth variable, "credit spreads," as a second parent of "market risk appetite" alongside energy sentiment. Write out the new factorization of the full joint distribution.
2. In the worked example, if $P(M\mid E{=}\text{positive})$ actually differed depending on whether $O$ was high or low, what would that imply about the graph's correctness?
3. A ten-variable system where every variable has at most 3 parents needs roughly how many small conditional-probability numbers, compared to a naive joint table over $2^{10}$ combinations? Use the same counting approach as the worked comparison above.