Generalized Linear Models

Ordinary linear regression fits a straight line to your outcome and assumes the leftover noise is a symmetric bell curve, evenly spread everywhere. That works fine when the outcome is something like a return, which can be any real number. It falls apart the moment the outcome is a probability that must stay between 0 and 1, or a count that must stay non-negative, or a binary "did the trade fill or not." Fit a straight line to a 0/1 outcome and it will happily predict a probability of 1.4 or $-0.2$, which is meaningless. A generalized linear model is the fix: keep the linear part — you still add up weighted predictors — but pass the result through a bending function first, one chosen to match the shape the outcome is actually allowed to take.

An analogy: a valve, not a straight pipe

Think of ordinary regression as water flowing straight through a pipe: pour in more pressure (predictors), get out proportionally more flow (outcome), no limit either direction. Now imagine the outcome has to obey a physical limit — say it's the fill level of a tank that can't go below empty or above full. You wouldn't connect the pressure straight to the fill level; you'd put a valve in between that translates "more pressure" into "closer to full" in a way that naturally flattens out near the two limits and never overshoots them. The valve is the link function. It lets you keep using a simple, additive combination of your predictors on the inside, while guaranteeing the outcome that comes out the other side always lives in the right range.

The model, one symbol at a time

Let $y$ be the outcome and $x_1, \dots, x_k$ the predictors. First build the linear predictor, exactly as in ordinary regression:

$$\eta = \beta_0 + \beta_1 x_1 + \cdots + \beta_k x_k .$$

In words: $\eta$ ("eta") is just the familiar weighted sum of predictors — still linear, nothing new yet. Then choose a link function $g$ that connects $\eta$ to the mean of the outcome, $\mu$ ("mu"):

$$g(\mu) = \eta, \qquad \text{equivalently} \qquad \mu = g^{-1}(\eta) .$$

In words: the link function is the rule that translates between the unrestricted linear predictor and the restricted, meaningful mean of the outcome; $g^{-1}$ is the valve that un-does the link and produces the actual prediction. For binary outcomes the usual choice is the logit link,

$$g(\mu) = \ln\!\left(\frac{\mu}{1-\mu}\right), \qquad \mu = \frac{1}{1+e^{-\eta}} .$$

In words: the logit turns a probability into "log-odds," which can range over all real numbers, so the linear predictor is free to be linear; inverting it (the logistic function) squeezes any real number back into a valid probability between 0 and 1. For count outcomes the standard choice is the log link, $g(\mu) = \ln(\mu)$, so $\mu = e^{\eta}$, which guarantees the predicted count is always positive since $e^{\eta}$ can never go negative.

Finally, a GLM also specifies a distribution family for the noise around $\mu$ — Bernoulli for binary outcomes, Poisson for counts, Gaussian (which recovers ordinary regression) for continuous unbounded ones. The parameters $\beta$ are fit by maximum likelihood, not by minimizing squared error, because squared-error fitting silently assumes the Gaussian shape that no longer applies.

Worked example 1: logistic regression by hand, one step of fitting

You're modeling whether a trade fills ($y=1$) or not ($y=0$) using order size $x$ (in $10k units). Suppose the fitted model is $\eta = -1.0 + 0.5x$. At $x = 4$ (a $40k order):

$$\eta = -1.0 + 0.5(4) = 1.0, \qquad \mu = \frac{1}{1+e^{-1.0}} = \frac{1}{1+0.368} = 0.731 .$$

So a $40k order is predicted to fill 73.1% of the time. At $x=0$: $\eta = -1.0$, $\mu = 1/(1+e^{1.0}) = 1/3.718 = 0.269$, a 26.9% fill chance for a tiny order. Notice the change in probability per unit of $x$ is not constant — from $x=0$ to $x=1$ ($\eta$ from $-1.0$ to $-0.5$), $\mu$ moves from 0.269 to 0.378, a jump of 0.109; from $x=3$ to $x=4$ ($\eta$ from $0.5$ to $1.0$), $\mu$ moves from 0.622 to 0.731, a jump of 0.109 as well here because we're near the steepest part of the S-curve — but near either extreme, the same 0.5 change in $\eta$ moves $\mu$ by only a few points. That non-constant sensitivity is exactly the point of the link: probabilities compress near 0 and 1, and the logit is what makes that compression automatic.

Worked example 2: Poisson regression for trade counts

You model the number of trades per hour, $y$, as a function of volatility $x$ (in vol points), with fitted log link $\eta = 1.0 + 0.2x$. At $x = 5$: $\eta = 1.0 + 0.2(5) = 2.0$, so $\mu = e^{2.0} = 7.39$ trades expected per hour. At $x = 10$: $\eta = 1.0+0.2(10)=3.0$, $\mu = e^{3.0}=20.1$ trades per hour. Doubling volatility from 5 to 10 didn't double the linear predictor's effect additively — it multiplied the predicted count by $e^{0.2 \times 5} = e^{1.0} = 2.72$. This is the signature of the log link: a one-unit increase in $x$ multiplies the predicted count by $e^{\beta}$, a constant factor, rather than adding a constant amount — which matches how trading activity actually behaves, scaling up multiplicatively rather than by fixed increments.

Try raising the rate parameter in the explorer above and notice the distribution stays non-negative and right-skewed at low rates — exactly the shape a Poisson GLM is built to predict, and exactly the shape ordinary linear regression cannot respect.

What this means in practice

• Logistic regression for fill probabilities, default probabilities, up/down classification — anywhere the outcome is a probability of an event.
• Poisson or negative-binomial regression for trade counts, order arrivals, defaults per period — anywhere the outcome is a non-negative count.
• Coefficients are on the link scale, not the outcome scale. In logistic regression, $\beta$ is a change in log-odds, not in probability; you must convert through the inverse link before quoting an effect in probability terms, and the size of that effect depends on where on the curve you start.
• Diagnostics differ from OLS. Residuals aren't expected to look Gaussian; instead use deviance residuals and compare nested models with a likelihood ratio test.