Joint and Marginal Distributions

Most interesting questions involve more than one random quantity: a stock's return and the market's, a customer's age and their spend, two dice at once. The joint distribution is the full bookkeeping of how likely every combination of outcomes is. From it you can answer any question about the variables together, and, by "summing out" the ones you don't care about, recover the behaviour of each variable on its own, its marginal distribution.

For two discrete variables, the joint distribution is just a table of probabilities $P(X = x,\, Y = y)$, one entry per pair, all adding to 1. To get the marginal of $X$, you collapse the table along the other axis, adding up each row:

$$P(X = x) = \sum_{y} P(X = x,\, Y = y) .$$

In words: the chance $X$ takes a value is the total probability of all the joint outcomes where it does, regardless of what $Y$ did. The name "marginal" is literal, it's the total you'd scribble in the margin of the table.

A joint table with its margins

Here's a concrete joint distribution for two 0/1 variables. The interior cells are the joint probabilities; the shaded edges are the marginals, obtained by adding across each row and down each column.

$P(X, Y)$	$Y = 0$	$Y = 1$	Marginal of $X$
$X = 0$	0.30	0.20	0.50
$X = 1$	0.15	0.35	0.50
Marginal of $Y$	0.45	0.55	1.00

Read straight off it: $P(X = 1) = 0.15 + 0.35 = 0.50$ (the row sum), and $P(Y = 1) = 0.20 + 0.35 = 0.55$ (the column sum). The lone corner, 1.00, is a sanity check that the whole table's probability accounts for everything.

Independence: when the joint is just the product

Two variables are independent when knowing one tells you nothing about the other, and that has a crisp signature: the joint probability factorises into the product of the marginals,

$$P(X = x,\, Y = y) = P(X = x)\, P(Y = y) \quad \text{for every pair.}$$

Test it on the table. If $X$ and $Y$ were independent, the top-left cell would be $P(X=0)\,P(Y=0) = 0.50 \times 0.45 = 0.225$. But the actual joint value is $0.30$, not $0.225$, so these two variables are dependent: they show up together more often than chance alone would produce. This single check, does every interior cell equal the product of its margins?, is the whole definition of independence.

Worked example: from joint to conditional

Using the same table, suppose you learn that $X = 1$. What's now the chance that $Y = 1$? Conditioning zooms in on the $X = 1$ row and renormalises it so its entries sum to 1:

$$P(Y = 1 \mid X = 1) = \frac{P(X = 1,\, Y = 1)}{P(X = 1)} = \frac{0.35}{0.50} = 0.70 .$$

Before you knew anything, $Y = 1$ had probability $0.55$; after learning $X = 1$, it jumps to $0.70$. That shift, from marginal to conditional, is dependence made visible: the information in $X$ genuinely changed your view of $Y$. If they'd been independent, the conditional would have equalled the marginal and nothing would have moved.

The continuous version

For continuous variables the ideas carry over with integrals in place of sums. The joint density $f(x, y)$ describes probability per unit area, and you marginalise by integrating the unwanted variable out:

$$f_X(x) = \int f(x, y)\, dy .$$

The bivariate normal at the heart of the The Multivariate Normal Distribution is the most important continuous example: its joint density is a tilted bell over the plane, and every marginal and conditional slice of it is again a normal.

Joint and marginal thinking is the gateway to the rest of multivariable probability. Conditioning on part of a joint distribution leads to Conditional Probability and Conditional Expectation; summing a joint over one variable while weighting by the other is the The Law of Total Probability; and the co-movement the margins hide is exactly what Correlation and the The Multivariate Normal Distribution are built to capture.