Bayesian Linear Regression

Ordinary least squares regression hands you one number per coefficient — a single best-fit slope — and, separately, a standard error bolted on after the fact. That works fine with plenty of data, but with a short history (say, 40 monthly observations to estimate a 5-factor model) the single-number answer can hide how genuinely uncertain the fit is, and any downstream calculation that treats the coefficients as known exactly will understate its own risk. Bayesian linear regression builds uncertainty into the coefficients from the start, treating them as random variables with a full distribution rather than fixed unknowns estimated once.

An analogy: fitting a ruler through a handful of dots with a shaky hand

Imagine drawing the "best" straight line through five scattered dots, but your hand isn't perfectly steady — small nudges to the line's angle and height still look roughly consistent with the dots. With only five dots, many slightly different lines look plausible; with five hundred dots, the range of plausible lines narrows dramatically because any line straying even a little now visibly misses many points. Bayesian linear regression formalizes "how much can I wiggle this line and still have it look plausible" — instead of drawing one line, it describes the whole bundle of plausible lines, weighted by fit, naturally wide with 5 dots and tight with 500.

The maths, one symbol at a time

The regression model is the usual $y = X\beta + \varepsilon$, where $y$ is the outcome, $X$ the predictors, $\beta$ the coefficients, and $\varepsilon$ noise with variance $\sigma^2$. The Bayesian version puts a prior on $\beta$ — commonly $\beta \sim N(0, \tau^2 I)$, a belief that coefficients are probably modest in size and centered near zero before seeing data. Combining this prior with the data via Bayes' theorem gives a posterior distribution over $\beta$ that, for this conjugate normal setup, is itself normal:

$$\beta \mid y, X \;\sim\; N\!\left(\hat\beta_{\text{post}},\ \Sigma_{\text{post}}\right)$$

In words: after seeing the data, your belief about the coefficients is still bell-shaped, just recentered at a posterior mean $\hat\beta_{\text{post}}$ with a posterior covariance $\Sigma_{\text{post}}$ describing remaining uncertainty. With a normal prior centered at zero, $\hat\beta_{\text{post}}$ turns out to be mathematically identical to a ridge regression estimate — the prior's pull toward zero acts exactly like ridge's shrinkage penalty (see ridge and lasso regularization).

Worked example 1: shrinkage with a short history

Fit a one-predictor regression ($y = \beta x + \varepsilon$) on just 8 monthly observations. The plain OLS slope estimate is $\hat\beta_{\text{OLS}} = 1.8$, but with only 8 points it's noisy. Using a prior $\beta \sim N(0, 1)$ and combining with the likelihood, the posterior mean comes out to roughly $\hat\beta_{\text{post}} = 1.1$ — pulled substantially toward zero because the small sample gives the prior real influence — with posterior standard deviation around $0.5$, so a 90% credible interval of roughly $[0.3, 1.9]$. This interval correctly reflects both estimation noise and the deliberate skepticism of large slopes baked into the prior.

Worked example 2: the posterior narrows with more data

Refitting with 200 monthly observations instead of 8, keeping the same prior, the OLS estimate might again land near $\hat\beta_{\text{OLS}} = 1.7$, but now the posterior mean is about $\hat\beta_{\text{post}} = 1.65$ — barely shrunk, since 200 points overwhelm the modest prior — with a tighter posterior standard deviation around $0.09$, giving a 90% credible interval of roughly $[1.5, 1.8]$. The same prior that meaningfully shifted the estimate with 8 points has almost no effect with 200; more data always wins the tug-of-war with a fixed prior, eventually.

Toggle the noise or sample size in the explorer above — the spread of plausible lines through the points is exactly what a Bayesian posterior over $\beta$ is quantifying, tight with lots of clean data and wide with little.

What this means in practice

Bayesian linear regression is most valuable exactly where classical regression is riskiest to trust blindly: short factor-model histories, small universes of assets, or any setting where coefficients feed into a downstream risk or sizing calculation that needs a real uncertainty band rather than a single number. It's also the cleanest bridge between Bayesian thinking and tools quants already use daily, since a normal-prior Bayesian regression's posterior mean literally is a ridge estimate — running one is, in that sense, running the other with an uncertainty band attached for free.