Gram-Schmidt Orthogonalization

A new trading signal is built, but it turns out to be 70% correlated with the market factor already in the model — some of its apparent predictive power is just repackaged market exposure. Before crediting the new signal with any edge, you need to strip out the part it shares with the market, keeping only what's genuinely new. Gram-Schmidt orthogonalization is the systematic recipe for doing exactly this: given any set of vectors, however overlapping, it produces a new set that point at perfect right angles to each other while still describing the same overall space.

An analogy: peeling off the shared part

Imagine two overlapping circles of light on a wall from two flashlights. To isolate what the second flashlight uniquely contributes, you could measure how much of its light falls exactly where the first flashlight already shines, subtract exactly that overlapping amount, and what's left is the second flashlight's unique contribution — the part with zero overlap with the first. Gram-Schmidt does this arithmetically to vectors: take each new vector, subtract out its "shadow" onto every previous vector, and what remains is orthogonal to all of them by construction.

The mathematics, one symbol at a time

Given vectors $v_1, v_2, \ldots, v_n$, Gram-Schmidt builds an orthogonal set $u_1, u_2, \ldots, u_n$ one at a time. Start with $u_1 = v_1$. For each subsequent vector, subtract its projection onto every $u_i$ already built:

$$u_k = v_k - \sum_{i=1}^{k-1} \frac{\langle v_k, u_i \rangle}{\langle u_i, u_i \rangle} u_i,$$

in words: to build $u_k$, start with the original $v_k$ and subtract off, one at a time, whatever piece of it points along each already-orthogonalized direction — the fraction $\langle v_k,u_i\rangle/\langle u_i,u_i\rangle$ is exactly how much of $v_k$ "lies along" $u_i$ (see Inner Products and Orthogonality). What's left, $u_k$, is guaranteed orthogonal to all of $u_1,\ldots,u_{k-1}$ by construction — it's the part of $v_k$ with zero overlap with anything already accounted for. Normalizing each $u_i$ to unit length gives an orthonormal basis, and stacking the process into matrix form is exactly how QR decomposition (see QR Decomposition) is built.

Worked example 1: orthogonalizing two vectors by hand

Let $v_1 = (3, 1)$, $v_2 = (2, 2)$. Set $u_1 = v_1 = (3,1)$. Project $v_2$ onto $u_1$: $\langle v_2,u_1\rangle = 2(3)+2(1)=8$, $\langle u_1,u_1\rangle=9+1=10$, so the projection coefficient is $8/10=0.8$. Then $u_2 = v_2 - 0.8\,u_1 = (2,2) - (2.4, 0.8) = (-0.4, 1.2)$. Check orthogonality: $\langle u_1,u_2\rangle = 3(-0.4)+1(1.2)=-1.2+1.2=0$ ✓. The pair $(3,1)$ and $(-0.4,1.2)$ now point at exactly 90° to each other, while together they still span the same plane the original pair did.

Worked example 2: stripping the market out of a signal

A new signal's daily values (mean-centered) are $v_2 = (2,-1,-1,0)$, and the market factor's are $v_1 = (1,1,-1,-1)$. Projection coefficient: $\langle v_2,v_1\rangle = 2(1)+(-1)(1)+(-1)(-1)+0(-1)=2-1+1+0=2$; $\langle v_1,v_1\rangle=1+1+1+1=4$; coefficient $=2/4=0.5$. The market-neutral (orthogonalized) signal is $u_2 = v_2 - 0.5v_1 = (2,-1,-1,0)-(0.5,0.5,-0.5,-0.5)=(1.5,-1.5,-0.5,0.5)$. This $u_2$ is, by construction, completely uncorrelated with the market factor — any predictive power it shows in a backtest can no longer be attributed to simply repackaging market beta.

What this means in practice

Gram-Schmidt is the standard tool for orthogonalizing a candidate signal against known factors before crediting it with independent predictive power (see Orthogonalising a Signal Against Known Factors), and it's the constructive method underlying QR decomposition, which in turn underlies numerically stable least-squares regression.