Multinomial and Ordinal Logistic Regression

Ordinary logistic regression predicts one of two outcomes — up or down, default or no default. Real classification problems are often not binary: a trade signal might be "sell / hold / buy," or a bond rating might be "AAA / AA / A / BBB / ...", where the categories have a natural order. Multinomial logistic regression handles the first kind of problem — more than two unordered categories. Ordinal logistic regression handles the second — more than two categories that come in a meaningful sequence — and it does so more efficiently by using the ordering rather than throwing it away.

The analogy: sorting mail into bins versus into a queue

Multinomial regression is like sorting mail into unlabeled bins by destination city — Chicago mail doesn't need to be "between" New York mail and LA mail in any sense; the categories are just different, with no inherent order. Ordinal regression is like sorting customers into a priority queue — bronze, silver, gold, platinum — where gold is not just "different" from silver, it's better, and a model that ignores this ranking is throwing away real information. Multinomial regression estimates a separate boundary between every pair of unordered categories; ordinal regression instead assumes a single underlying continuous "score" and just cuts it at different thresholds to produce the ordered categories.

The two models, formally

Multinomial logistic regression picks a baseline category (say, category 0) and models the log-odds of every other category $k$ against it with its own coefficient vector $\beta_k$:

$$\log \frac{P(y=k \mid x)}{P(y=0 \mid x)} = \beta_k \cdot x, \qquad k = 1, \dots, K-1$$

Plain English: each non-baseline category gets its own independent linear "score" relative to the baseline, and $K-1$ separate coefficient vectors are fit.

Ordinal logistic regression (proportional-odds model) instead assumes one coefficient vector $\beta$ shared across all categories, and models cumulative probabilities with category-specific intercepts $\theta_k$:

$$\log \frac{P(y \le k \mid x)}{P(y > k \mid x)} = \theta_k - \beta \cdot x$$

Plain English: there's a single latent "score" $\beta \cdot x$, and the category you fall into is just which threshold $\theta_k$ that score crosses — moving $x$ shifts the whole distribution over ordered categories consistently in one direction, rather than reshuffling categories independently.

Worked example 1: multinomial — predicting sell/hold/buy

A model predicts trade action from a momentum feature $x$. With baseline "hold" and fitted $\beta_{\text{sell}} = -0.8$, $\beta_{\text{buy}} = 1.2$ (both times $x$), at $x=1$: $\log\text{-odds}_{\text{sell vs hold}} = -0.8$, so $P(\text{sell})/P(\text{hold}) = e^{-0.8}=0.449$; $\log\text{-odds}_{\text{buy vs hold}} = 1.2$, so $P(\text{buy})/P(\text{hold}) = e^{1.2}=3.32$. Setting $P(\text{hold})=p$, then $P(\text{sell})=0.449p$, $P(\text{buy})=3.32p$, and $p + 0.449p+3.32p=1$ gives $p \approx 0.209$, so $P(\text{hold})\approx21\%$, $P(\text{sell})\approx9\%$, $P(\text{buy})\approx69\%$ — two entirely separate coefficients drove sell and buy probabilities independently.

Worked example 2: ordinal — predicting a credit rating tier

A model predicts one of four ordered tiers (1=worst to 4=best) from a leverage ratio $x$, with fitted $\beta = 1.5$ and thresholds $\theta_1=-2, \theta_2=0, \theta_3=2.5$. At $x=-1$ (low leverage, good), the latent score contribution is $-\beta x = 1.5$. Cumulative log-odds for tier $\le 1$: $\theta_1 - (-\beta x)$... more directly, $P(y\le k \mid x) = \text{logistic}(\theta_k - \beta x) = \text{logistic}(\theta_k + 1.5)$. For $k=1$: $\text{logistic}(-2+1.5)=\text{logistic}(-0.5)=0.378$. For $k=2$: $\text{logistic}(0+1.5)=\text{logistic}(1.5)=0.818$. For $k=3$: $\text{logistic}(2.5+1.5)=\text{logistic}(4)=0.982$. So $P(y=1)=0.378$, $P(y=2)=0.818-0.378=0.440$, $P(y=3)=0.982-0.818=0.164$, $P(y=4)=1-0.982=0.018$ — one coefficient, shared across all cuts, shifted the whole ordered distribution toward the better tiers.

Multinomial regression fits a separate curve per category, unordered; ordinal regression instead slides one underlying score along a line and cuts it at thresholds — watch how shifting a single parameter here moves probability mass consistently across ordered bins, the ordinal model's core mechanism.

What this means in practice

Use multinomial regression when the outcome categories genuinely have no order (asset class chosen, sector classification); use ordinal regression when they do (credit rating tier, signal strength bucket, analyst rating) because it needs far fewer parameters — one shared $\beta$ instead of $K-1$ separate ones — and produces predictions that respect the natural ranking, which usually improves both statistical efficiency and interpretability.