Reduced-Rank Regression

You want to predict the returns of 50 stocks next month, using 20 predictor characteristics for each. The direct approach fits a separate regression for every stock — 50 independent sets of 20 coefficients, a thousand numbers total, each estimated from only however much historical data you have. That's a lot of freedom, and with limited history it invites overfitting: many of those thousand coefficients are noise. But there's often reason to expect the 50 regressions are related — if the true mechanism is that all 50 stocks' returns respond to just a few common combinations of the 20 characteristics (say, "value" and "momentum" composite scores), estimating 50 separate relationships is wasteful; you should instead estimate a handful of shared combinations and route every stock's prediction through them. That's what reduced-rank regression forces.

An analogy: mixing paint colors from a limited palette

Imagine 50 wall colors need reproducing, and you have 20 base pigments available. The unconstrained approach lets each color use its own independent mix of all 20 pigments — maximally flexible, but needing 50 separate, fully-specified recipes, and with only a little trial-and-error data per color, some recipes will be based on noise (a pigment that happened to look right by luck once). Reduced-rank regression instead first mixes just 2 or 3 "master blends" from the 20 pigments, then gets every final color as a simple combination of only those master blends. If the true palette genuinely reduces to a few master blends, this uses vastly fewer independently-estimated numbers, is far more stable, and recovers the true structure better than fitting each color separately ever could.

The mechanics, one symbol at a time

Let $Y$ be the matrix of outcomes ($T$ time periods by $M$ outcome variables — here, $M=50$ stock returns) and $X$ be the matrix of predictors ($T$ periods by $P$ predictors — here, $P=20$ characteristics). Ordinary multivariate regression estimates a full $P \times M$ coefficient matrix $B$ with no restriction: $Y = XB + \varepsilon$, using $P \times M$ free parameters. Reduced-rank regression instead constrains $B$ to have rank $r$, much smaller than $\min(P,M)$, by factoring it as

$$B = A C, \qquad A \text{ is } P \times r, \quad C \text{ is } r \times M,$$

so that $Y = X(AC) + \varepsilon = (XA)C + \varepsilon$. In plain English: $XA$ computes $r$ shared combinations of the predictors (the "master blends" — analogous to composite factor scores), and $C$ then expresses every one of the $M$ outcomes as a simple linear combination of just those $r$ shared scores, rather than as an independent function of all $P$ original predictors. Instead of $P \times M$ free numbers, you now estimate only $r\times(P+M)$ — for $r$ much smaller than $M$, a dramatic reduction whenever the outcomes really do share common structure.

The rank $r$ is a tuning choice, typically selected by checking how much of $Y$'s variance successive ranks explain (similar in spirit to choosing the number of components in PCA) or by cross-validation on held-out data — too small an $r$ underfits real structure that exists across outcomes, too large approaches the unconstrained, overfitting-prone full regression.

Worked example 1: recovering shared structure with less data

Suppose the true mechanism is that all 50 stocks' returns depend only on two composite scores built from the 20 characteristics — a value score and a momentum score — with each stock having its own sensitivity to each score. An unconstrained regression, with limited history (say $T=100$ months), must estimate $20\times 50 = 1{,}000$ coefficients from 100 observations per stock — badly underdetermined, and the fitted coefficients will mostly reflect sampling noise, giving poor out-of-sample predictions. A rank-2 reduced-rank regression instead estimates $2\times(20+50)=140$ numbers total, using the combined information from all 50 stocks' histories to pin down the two shared master combinations — a far better-posed problem given the same 100 months of data, and one that matches how the data was actually generated.

Worked example 2: choosing rank by explained variance

Running reduced-rank regression on the 50-stock, 20-characteristic panel and checking successive ranks, suppose rank 1 explains 40% of $Y$'s cross-sectional variance, rank 2 explains 68%, rank 3 explains 71%, and rank 4 explains 72%. The big jump from rank 1 to rank 2, followed by near-flat gains afterward, signals that the true underlying structure is well described by 2 shared factors — adding a 3rd or 4th shared combination captures almost nothing beyond noise, and a practitioner would settle on $r=2$ rather than chasing the last few percentage points of in-sample fit, which likely reflect overfitting.

What this means in practice

Reduced-rank regression is common wherever many related outcomes (returns of many stocks, yields across many maturities, spreads across many credit names) are believed to be driven by a small number of shared underlying factors — it is closely related to factor models and to principal components regression, but built directly around predicting outcomes from predictors rather than just summarizing predictors alone. It's most valuable exactly when the unconstrained multivariate regression would need more parameters than the data can reliably support.