Conjugate Priors

Bayesian updating, in general, means multiplying a prior by a likelihood and then wrestling with an ugly normalizing integral to get the posterior. Conjugate priors are the happy exception where that integral has a clean answer, because the prior and the likelihood are chosen to fit together so that the posterior comes out in the very same family as the prior. When that happens, updating your beliefs stops being calculus and becomes bookkeeping: you just add the new data to a running count.

The idea

A prior is conjugate to a likelihood if the resulting posterior belongs to the same family, only with updated parameters. The classic pair is the Beta prior with the Binomial likelihood. If your belief about a success probability $p$ is a Beta distribution with parameters $\alpha$ and $\beta$, and you then observe $s$ successes and $f$ failures, the posterior is simply

$$\text{Beta}(\alpha, \beta) \ \xrightarrow{\ s \text{ successes},\ f \text{ failures}\ } \ \text{Beta}(\alpha + s, \ \beta + f) .$$

That is the whole update: add the successes to $\alpha$, add the failures to $\beta$. No integral. The parameters $\alpha$ and $\beta$ act like pseudo-counts, imaginary successes and failures you are pretending to have seen before the real data arrived, which is a wonderfully intuitive way to encode a prior.

The common conjugate pairs

A handful cover most textbook cases:

Data / likelihood	Conjugate prior	Posterior update
Bernoulli / Binomial (rate $p$)	Beta$(\alpha,\beta)$	Beta$(\alpha + s,\ \beta + f)$
Poisson (rate $\lambda$)	Gamma$(a,b)$	Gamma$(a + \textstyle\sum x_i,\ b + n)$
Exponential (rate $\lambda$)	Gamma$(a,b)$	Gamma$(a + n,\ b + \textstyle\sum x_i)$
Normal, known variance (mean $\mu$)	Normal$(\mu_0, \sigma_0^2)$	precision-weighted blend of prior and data means
Multinomial (probs $\mathbf p$)	Dirichlet$(\boldsymbol\alpha)$	Dirichlet$(\boldsymbol\alpha + \text{counts})$

In every row, the posterior is the prior's family with data folded into the parameters.

Worked example: a biased coin

You suspect a coin is roughly fair but are not sure, so you set a mild prior of $\text{Beta}(2, 2)$, which peaks gently at $p = 0.5$ and encodes the weak belief "as if I had already seen 2 heads and 2 tails." Now you flip it 10 times and get 8 heads, 2 tails. The posterior is

$$\text{Beta}(2 + 8, \ 2 + 2) = \text{Beta}(10, 4) .$$

Its mean is $\tfrac{10}{10 + 4} = \tfrac{10}{14} \approx 0.714$. Compare the three numbers: the prior mean was $0.5$, the raw data (maximum-likelihood) estimate is $8/10 = 0.8$, and the posterior mean $0.714$ sits between them, pulled toward the data but held back by the prior. That shrinkage is the entire value of the Bayesian view, and here you can read exactly how much the prior weighed in: its 4 pseudo-observations against the data's 10 real ones.

Where conjugacy misleads

• Convenience is not correctness. A conjugate prior is chosen because the math is clean, not because it is the right description of your beliefs. Do not let tractability quietly dictate your prior.
• Small data, strong prior. With few observations the prior can dominate the answer. Beta$(50,50)$ would barely have budged from $0.5$ after those same 10 flips, always ask how much your prior is doing.
• Most real models are not conjugate. The moment you add a hierarchy, an unknown variance alongside an unknown mean, or a nonstandard likelihood, closed-form conjugacy breaks. That is exactly when you reach for MCMC, which samples the posterior without needing it in closed form.

Conjugate priors are the exactly-solvable base case of Bayesian inference, a direct application of Bayes' theorem, and the natural contrast to MCMC, which takes over precisely where the clean formulas run out.