Vector Spaces and Bases

A factor model describes a stock's return as a combination of market, size, and value exposures — three numbers that, together, pin down the stock's behavior. Why three, and why those three? The answer lives in the idea of a vector space and its basis: the minimal set of "directions" needed so that every object of interest can be written as a unique combination of them, with no redundancy and nothing missing.

An analogy: giving directions on a grid

To describe any location in a flat city laid out on a grid, you only need two directions — say, "blocks east" and "blocks north." Every address is some combination of these two: "3 east, 2 north." You don't need a third direction like "northeast" because it's already expressible as a mix of the first two, and you can't get by with just one direction because some addresses aren't purely east or purely north. Two independent directions are exactly enough to reach anywhere on the grid, and every address has one, and only one, correct pair of coordinates in that system.

The mathematics, one symbol at a time

A vector space is a set of objects (vectors) that can be added together and scaled by numbers, with the results staying inside the set — combinations never "escape" the space. A basis is a set of vectors $\{v_1, \ldots, v_n\}$ in that space such that every vector $x$ in the space can be written as exactly one combination

$$x = c_1 v_1 + c_2 v_2 + \cdots + c_n v_n,$$

for some unique numbers $c_1, \ldots, c_n$ called the coordinates of $x$ in that basis. In words: the basis vectors are a minimal toolkit — mixing them in the right proportions builds any vector in the space, and for each vector there's only one right recipe of proportions. The number of vectors in a basis, $n$, is called the dimension of the space; every basis for the same space has exactly the same number of vectors, so dimension is a well-defined property of the space itself, not of any particular basis you happened to choose.

Worked example 1: two bases, same vector, different coordinates

In the plane, take the standard basis $e_1 = (1,0)$, $e_2 = (0,1)$. The vector $x = (5, 3)$ has coordinates $(5,3)$ in this basis, trivially: $x = 5e_1 + 3e_2$. Now use a different basis, $v_1 = (1,1)$, $v_2 = (1,-1)$. Solve $x = c_1 v_1 + c_2 v_2$: $5 = c_1 + c_2$ and $3 = c_1 - c_2$, giving $c_1 = 4$, $c_2 = 1$. Check: $4(1,1) + 1(1,-1) = (5,3)$. Same underlying vector, same dimension (2), but different coordinate numbers — a reminder that "coordinates" are always relative to a chosen basis, not an intrinsic property of the vector.

Worked example 2: a three-factor return basis

A stock's monthly excess return is modeled as $r = \beta_{MKT} \cdot f_{MKT} + \beta_{SMB} \cdot f_{SMB} + \beta_{HML} \cdot f_{HML}$, where $f_{MKT}, f_{SMB}, f_{HML}$ are the market, size, and value factor returns treated as a basis for the space of explainable returns, and the betas are the stock's unique coordinates in that basis. Suppose a stock has $\beta_{MKT}=1.1$, $\beta_{SMB}=0.3$, $\beta_{HML}=-0.2$, and in a given month $f_{MKT}=2\%$, $f_{SMB}=1\%$, $f_{HML}=0.5\%$. Its factor-explained return is $1.1(2) + 0.3(1) - 0.2(0.5) = 2.2 + 0.3 - 0.1 = 2.4\%$ — one unique combination of the three basis directions, exactly as the definition requires.

Drag the transform above and notice that the two arrows shown are always enough to describe where every other point in the plane lands — they form a basis for the 2-dimensional space the transform acts on, and their new positions alone determine the whole picture.

What this means in practice

Factor models, PCA components, and the "features" fed into a regression are all, formally, bases (or approximate bases) for some space of interest — choosing a basis is choosing which combinations of underlying directions you'll use to describe every observation. A model with too few basis directions can't represent everything it needs to (underfitting the structure); one with redundant, non-independent directions (see Linear Independence and Rank) wastes effort and can make coordinates non-unique or numerically unstable to compute.