Linear Algebra for Quants

Linear algebra is the operating system of quantitative finance. A portfolio is a vector, a covariance is a matrix, a factor model is a low-rank approximation, a regression is a projection, and a risk number is a quadratic form. Fluency here is not optional literacy, it is what lets you see that seemingly different problems (hedging, PCA, least squares, optimisation) are the same geometric object viewed from different angles. This is the foundation the whole matrix track (Eigenvalues & Eigenvectors, Singular Value Decomposition, Positive Semidefinite Matrices) builds on.

Vector spaces, subspaces, span

A vector space over $\mathbb{R}$ is a set closed under addition and scalar multiplication. In finance the ambient space is $\mathbb{R}^n$, $n$ assets, $n$ observations, $n$ factors. The key derived objects:

• Span: all linear combinations $\sum c_i v_i$ of a set of vectors, the subspace they reach. A portfolio's achievable payoffs are the span of its tradable assets.
• Linear independence: no vector is a combination of the others; equivalently $\sum c_i v_i = 0$ forces all $c_i = 0$. Redundant (dependent) assets add no new spanning power.
• Basis and dimension: a maximal independent set; its size, the dimension, is the number of true degrees of freedom.

The four fundamental subspaces of a matrix $A \in \mathbb{R}^{m\times n}$ organise everything: the column space (range, what $Ax$ can produce), the null space (solutions of $Ax = 0$, the redundancy), the row space, and the left null space. Their dimensions are locked together by rank.

Rank: the number that matters most

The rank of $A$ is the dimension of its column space, the number of linearly independent columns (equivalently rows; row rank = column rank always). The rank–nullity theorem ties the pieces together: $\operatorname{rank}(A) + \dim(\operatorname{null}(A)) = n \quad (\text{number of columns}).$ Rank is the single most diagnostic quantity in applied linear algebra. Full rank means invertible/uniquely solvable; rank deficiency means redundancy and non-uniqueness. In Ordinary Least Squares (OLS), full column rank of the design matrix $X$ is exactly the "no perfect multicollinearity" assumption that makes $X^\top X$ invertible. In covariance estimation, a rank-deficient sample covariance (more assets than observations, $n > T$) is singular, so it cannot be inverted, the reason naive mean–variance optimisation breaks on large asset universes and needs shrinkage. Rank is where abstract algebra becomes a concrete failure mode.

Norms: measuring size

A norm $\lVert\cdot\rVert$ assigns length, satisfying positivity, homogeneity ($\lVert cv\rVert = |c|\,\lVert v\rVert$), and the triangle inequality. The ones that recur:

• $\ell_2$ (Euclidean): $\lVert v\rVert_2 = \sqrt{\sum v_i^2} = \sqrt{v^\top v}$. Induces the inner product and hence angles and orthogonality; the geometry of least squares and volatility.
• $\ell_1$: $\lVert v\rVert_1 = \sum |v_i|$. Its "pointy" unit ball drives sparsity, the reason Lasso ($\ell_1$-penalised regression) zeroes out coefficients.
• $\ell_\infty$: $\max_i |v_i|$, the worst case, used in robust/minimax bounds.

The inner product $\langle u, v\rangle = u^\top v = \lVert u\rVert\lVert v\rVert\cos\theta$ makes "correlation" geometric: standardised return vectors have inner product equal to their correlation, so uncorrelated is literally orthogonal. For matrices, the spectral norm (largest singular value) and Frobenius norm ($\sqrt{\sum a_{ij}^2}$) measure operator size and are central to low-rank approximation.

Orthogonal projection: the geometry of everything

Projection is the most useful single idea in the quant's linear-algebra toolkit. Given a subspace $S$ (say the column space of $X$), the orthogonal projection of a vector $y$ onto $S$ is the closest point in $S$ to $y$, and the residual $y - \hat y$ is orthogonal to $S$. For $S = \operatorname{col}(X)$ with $X$ full column rank, the projection matrix is $P = X(X^\top X)^{-1}X^\top, \qquad \hat y = Py.$ $P$ is symmetric ($P^\top = P$) and idempotent ($P^2 = P$), the algebraic fingerprint of a projection. This is Ordinary Least Squares (OLS): fitting is projecting $y$ onto the space spanned by the regressors, and the normal equations $X^\top(y - \hat y) = 0$ are just the orthogonality of the residual. The same construction is hedging (project the thing you hold onto the space of hedging instruments; the residual is unhedgeable basis risk) and Gram–Schmidt factor orthogonalisation. See projection once and you see it everywhere.

Quadratic forms: the language of risk

A quadratic form is $Q(x) = x^\top A x$ with $A$ symmetric, a scalar built from a vector and a matrix. This is the shape of portfolio variance: $\operatorname{Var}(w^\top r) = w^\top \Sigma\, w,$ with $w$ the weights and $\Sigma$ the covariance. Because variance is never negative for any weights, $\Sigma$ must satisfy $w^\top \Sigma w \ge 0$ for all $w$, it is positive semidefinite Positive Semidefinite Matrices, the constraint that governs which matrices can even be covariances and why Cholesky and eigen-decompositions behave. The quadratic form also defines the objective in mean–variance optimisation and the exponent of the multivariate normal density. Its curvature, encoded in the eigenvalues of $A$, tells you the directions of maximum and minimum risk, the bridge to Eigenvalues & Eigenvectors.

Worked example, projecting a return onto a factor

Suppose a stock's return vector $y \in \mathbb{R}^T$ over $T$ days, and you want its market beta by projecting onto the market return vector $m \in \mathbb{R}^T$ (a one-dimensional subspace). The projection formula collapses to a scalar: $\hat\beta = \frac{m^\top y}{m^\top m} = \frac{\langle m, y\rangle}{\lVert m\rVert^2}.$ This is exactly the OLS slope $\widehat{\operatorname{Cov}}(y,m)/\widehat{\operatorname{Var}}(m)$ once returns are demeaned, regression is projection, and beta is the length of $y$'s shadow on $m$ per unit of $m$. The residual $y - \hat\beta m$ is the idiosyncratic return, orthogonal to the market by construction. One formula, and you have decomposed systematic from specific risk.

Failure modes and subtleties

• Near-singularity is worse than singularity. A matrix that is almost rank-deficient (high condition number, the ratio of largest to smallest singular value) makes $A^{-1}$ enormous and error-amplifying. Correlated factors give an ill-conditioned $X^\top X$, so OLS coefficients become unstable and sign-flippy, Multicollinearity. Always check conditioning, not just invertibility.
• Never invert to solve. Computing $A^{-1}b$ by forming $A^{-1}$ is numerically inferior to solving $Ax = b$ directly (LU/Cholesky/QR). Explicit inverses lose precision and are slower; production code factorises.
• Dimension mismatches hide bugs. $n > T$ covariance matrices are singular by construction; treating them as invertible silently produces garbage optimal portfolios.
• Orthogonality depends on the inner product. "Uncorrelated" is orthogonality under the covariance inner product, not the raw Euclidean one; conflating them misinterprets factor structure.
• Basis choice is not neutral numerically. The same subspace in a poorly-scaled basis (mixing units, e.g. rates in bps and equities in %) inflates the condition number; standardise before decomposing.

In interviews

Linear-algebra questions on quant desks are rarely rote computation; they test whether you see the geometry. Expect "what is the rank of this matrix and why does it matter for regression?" (invertibility of $X^\top X$). Expect projection: "derive the least-squares solution", set up the orthogonality of the residual. Know that a projection matrix is symmetric and idempotent, and be able to state portfolio variance as the quadratic form $w^\top\Sigma w$ and explain why $\Sigma$ must be positive semidefinite. Condition number and multicollinearity are favourite follow-ups. The strongest signal is connecting these threads, regression, hedging, PCA, and optimisation are all projections and quadratic forms, which sets up Eigenvalues & Eigenvectors and Singular Value Decomposition as the tools that diagonalise them.