Link Functions and the Exponential Family

A linear model predicts $\beta_0 + \beta_1 x_1 + \dots$, a sum that can be any real number. That's fine for tomorrow's return, but not for a probability, which must sit between 0 and 1, or a count, which must be non-negative. A link function is the fix: a translator between an unrestricted linear score and an outcome that has to obey its own rules.

The analogy: a dial and the actual temperature

The linear part of the model is like a dial that spins freely from $-\infty$ to $+\infty$. The room temperature it controls can't do that — it's bounded by what the heater and season allow. The thermostat translates "dial position" into "actual temperature" through a fixed curve, so however far you spin the dial, the temperature stays physically sensible. A link function plays that role between the raw linear score and the constrained quantity it stands in for.

The mechanics: linear score in, correct-shaped answer out

Write the linear part as $\eta = \beta_0 + \beta_1 x_1 + \dots$, the linear predictor, living on $(-\infty,\infty)$. The link function $g$ connects it to the outcome's mean $\mu$: $g(\mu) = \eta$. You don't model the mean directly with a straight line — you model a transform of it, chosen so the inverse transform always lands back inside the valid range. For probabilities, the standard choice is the logit link:

$$\log\!\left(\frac{p}{1-p}\right) = \eta \quad\Longrightarrow\quad p = \frac{1}{1+e^{-\eta}} .$$

In plain English: whatever real number $\eta$ comes out, the logistic function on the right squashes it back into $(0,1)$ — that's logistic regression. For counts, the log link, $\log(\mu)=\eta$, gives $\mu = e^\eta$, guaranteed non-negative — that's Poisson regression. These outcome distributions belong to the exponential family, a shared algebraic form from which a natural "canonical" link falls out for each one.

Worked example: score to probability

A default model gives linear predictor $\eta = -1.2 + 0.8 \times \text{leverage}$. At leverage 2, $\eta = 0.4$, so $p = 1/(1+e^{-0.4}) \approx 0.60$. At leverage 4, $\eta = 2.0$, so $p \approx 0.88$. The linear predictor keeps growing linearly with leverage, but the link function ensures $p$ never leaves $(0,1)$, however extreme leverage gets.

Drag the parameters above and watch how, however wide the input range, the output stays pinned between 0 and 1 — the squashing behavior a link function buys a linear model.

What this means in practice

Choosing the link function means choosing what shape of outcome you're honestly modeling: identity for something unbounded, logit for a probability, log for a count or strictly positive quantity like volume. Fitting plain linear regression to a probability or count produces predictions outside the valid range.