Bayesian Inference

Bayesian inference inverts the frequentist picture. Where a frequentist treats the parameter as a fixed unknown and the data as random, a Bayesian treats the parameter as random, an object of belief, and updates that belief as data arrive. This reframing does two things a quant should care about: it gives the interval statement everyone wants (a probability that the parameter lies in a range), and it unifies shrinkage, regularization, and the whole practice of "pulling estimates toward a sensible prior" under one principled operation. In a low-data, strong-prior world like finance, that is not a philosophical luxury, it is the natural machinery.

Bayes' theorem as an updating rule

Encode beliefs about a parameter $\theta$ before seeing data as the prior $p(\theta)$. The likelihood $p(\text{data}\mid\theta)$ says how probable the data are for each $\theta$ (the same object maximized in Maximum Likelihood Estimation (MLE)). Bayes' theorem combines them into the posterior:

$p(\theta\mid \text{data}) = \frac{p(\text{data}\mid\theta)\,p(\theta)}{p(\text{data})} \ \propto\ \underbrace{p(\text{data}\mid\theta)}_{\text{likelihood}}\ \underbrace{p(\theta)}_{\text{prior}}.$

The denominator $p(\text{data}) = \int p(\text{data}\mid\theta)p(\theta)\,d\theta$ is just the normalizing constant, so the working slogan is posterior ∝ likelihood × prior. All Bayesian inference is reading off summaries of the posterior: its mean or mode is the point estimate, its spread is the uncertainty.

Conjugacy: when the update is closed-form

For special prior–likelihood pairs the posterior is in the same family as the prior, conjugacy, and updating is algebra rather than integration. The canonical example is the normal-mean model. With data $x_i \sim \mathcal{N}(\theta, \sigma^2)$ ($\sigma^2$ known) and a normal prior $\theta \sim \mathcal{N}(\mu_0, \tau_0^2)$, the posterior is normal with

$\mathbb{E}[\theta\mid x] = \frac{\tfrac{1}{\tau_0^2}\,\mu_0 + \tfrac{n}{\sigma^2}\,\bar x}{\tfrac{1}{\tau_0^2} + \tfrac{n}{\sigma^2}}, \qquad \operatorname{Var}[\theta\mid x] = \Big(\tfrac{1}{\tau_0^2} + \tfrac{n}{\sigma^2}\Big)^{-1}.$

Read this carefully, it is the heart of Bayesian shrinkage. The posterior mean is a precision-weighted average of the prior mean $\mu_0$ and the data mean $\bar x$, where precision is inverse variance. When data are scarce or noisy (small $n$, large $\sigma^2$), the estimate stays near the prior; as data accumulate ($n\to\infty$) the data term dominates and the posterior mean $\to \bar x$, converging to the MLE. The prior's influence is exactly an extra "$\tau_0^{-2}$ worth" of pseudo-observations. Other conjugate pairs, Beta–Binomial for probabilities, Gamma–Poisson for counts, Inverse-Gamma for variances, follow the same "add pseudo-counts" logic.

Credible intervals, what people actually want

A credible interval is a range that contains the parameter with stated posterior probability:

$\mathbb{P}(\theta \in [L, U]\mid \text{data}) = 0.95.$

This is the statement almost everyone wrongly attributes to a frequentist confidence interval. Here it is literally correct, because $\theta$ is random, "95% probability $\theta$ is in this interval" is a legitimate sentence. The price is that it depends on the prior; the reward is directness. (With a flat prior and symmetric likelihood the two intervals often numerically coincide, which is why the misinterpretation persists.)

The deep connection: regularization is Bayes

The result that should make a quant sit up: penalized estimation is Bayesian MAP estimation. The maximum a posteriori estimate maximizes $\log p(\theta\mid \text{data}) = \log p(\text{data}\mid\theta) + \log p(\theta) + \text{const}$. Take a Gaussian likelihood (so the log-likelihood is $-\frac{1}{2\sigma^2}\lVert y - X\beta\rVert^2$) and:

• a Gaussian prior $\beta_j \sim \mathcal{N}(0, \tau^2)$ contributes $\log p(\beta) = -\frac{1}{2\tau^2}\lVert\beta\rVert_2^2$, the MAP objective is exactly ridge with $\lambda = \sigma^2/\tau^2$;
• a Laplace prior $p(\beta_j)\propto e^{-|\beta_j|/b}$ contributes $-\frac{1}{b}\lVert\beta\rVert_1$, the MAP objective is exactly LASSO.

So ridge and LASSO are not ad-hoc penalties; they are posterior modes under a belief that coefficients are small. The tuning parameter $\lambda$ is the prior's tightness. This is the unifying view behind Ridge and LASSO Regularization and covariance Shrinkage, shrinking a noisy sample estimate toward a structured target is precisely combining a weak likelihood with an informative prior.

Worked example: shrinking a strategy's Sharpe

You observe a new strategy with an in-sample Sharpe of $1.5$ over one year, but you have a strong prior, from having seen hundreds of strategies, that true Sharpes cluster near $0.3$ with modest spread. The one-year estimate is noisy (large $\sigma^2$, small $n$), so the precision-weighted posterior mean pulls $1.5$ heavily toward $0.3$, you might land near $0.6$. This is not pessimism for its own sake; it is the mathematically correct combination of a weak signal with a well-founded prior, and it is why sophisticated allocators discount dazzling short-track-record Sharpes rather than taking them at face value. The Black–Litterman model applies the identical logic to expected returns, blending a market-equilibrium prior with investor views.

Failure modes in financial data

• Prior misspecification. A confidently wrong prior biases the posterior, and with little data the prior dominates, garbage prior, garbage posterior. The discipline is to make priors explicit and test sensitivity to them.
• Overconfident priors on non-stationary parameters. A tight prior on a drifting quantity (a beta, a vol) fights the very adaptation you need; priors should be wide enough to let regime changes register. See Stationarity.
• Computational cost. Outside conjugate families the posterior is an intractable integral requiring MCMC or variational approximation, which can be slow to converge and hard to diagnose for high-dimensional models.
• Pseudo-objectivity. "Non-informative" priors are not truly neutral (they are not invariant to reparameterization) and can be surprisingly influential in small samples; there is no escaping the prior choice, only making it honestly.

In interviews

State Bayes' theorem as posterior ∝ likelihood × prior and be able to explain conjugacy with the normal–normal example, emphasizing that the posterior mean is a precision-weighted average of prior and data that converges to the MLE as $n\to\infty$. Nail the frequentist-versus-Bayesian interval distinction: a credible interval genuinely carries "95% probability the parameter is inside," which a confidence interval does not. The result interviewers love for quant roles is that ridge = Gaussian prior MAP and LASSO = Laplace prior MAP, regularization is Bayesian estimation, and shrinkage toward a prior is the optimal response to parameter uncertainty. Ground it in finance with the Sharpe-shrinkage or Black–Litterman example: in a low-data world, a good prior is worth more than more parameters.