Markov Random Fields and Factor Graphs

A Markov random field (MRF) describes a set of variables — say, prices at neighboring exchanges, or pixels in an image — where each variable is directly linked to a handful of neighbors rather than to everything at once, and the links have no direction: influence flows both ways, unlike a Bayesian network's arrows. The joint probability over all variables is built by multiplying together small "compatibility" functions, one per cluster of connected variables, then rescaling so everything sums to one. This is exactly what a factor graph makes explicit: it draws the variables as one kind of node and each compatibility function as a separate node, with edges connecting a function to whichever variables it involves — turning "who depends on whom" into a picture you can literally read algorithms off of.

The payoff is computational. Instead of writing out one impossibly large table over all variables jointly, you only ever need the small local functions, and standard algorithms (like belief propagation) exploit the graph's structure to compute exact or approximate answers by passing messages only along the edges that actually exist — dramatically cheaper than brute-force enumeration whenever the graph is sparse.

A simple example: model three neighboring stocks' up/down moves as an MRF with edges between adjacent pairs (stock A–B, B–C) but no direct A–C edge. The factor graph has two function nodes, one for each pair, each connected to its two variable nodes — capturing that A and C only influence each other through B, not directly.