Sparse Matrices in Practice

A dense matrix stores every entry, zero or not, and every operation on it touches all of them. Many matrices a quant actually works with are mostly zero — a factor-exposure matrix where each stock loads on only a handful of sectors, a graph of which pairs of instruments actually trade against each other, or a large correlation matrix after shrinkage has zeroed out the weak entries. Storing and multiplying these as dense matrices burns memory and CPU on entries that contribute nothing.

Sparse formats store only the nonzero entries and their positions instead of a full grid. The two common ones are CSR (compressed sparse row), which is efficient for row-wise access and matrix-vector multiplication, and CSC (compressed sparse column), its column-wise mirror; both need roughly $O(\text{nonzeros})$ storage rather than $O(n^2)$, and their multiplication cost scales with nonzeros too, not with the full matrix size.

The payoff is concrete: a 10,000-by-10,000 matrix with only 1% of entries nonzero has 100 million cells but only 1 million nonzero ones. Stored densely at 8 bytes per entry that is roughly 800MB; stored sparsely it is closer to 8MB of values plus a similar amount of index bookkeeping — two orders of magnitude smaller, and a matrix-vector multiply is roughly 100 times fewer operations too, since zero entries contribute nothing to the sum and a sparse format simply skips them.

The catch is that sparse structure has to survive the operations you do — multiplying two sparse matrices together, or inverting one, can produce a much denser result (fill-in), which is why sparse solvers use reordering heuristics to keep fill-in low, and why a quant should check sparsity is preserved before assuming a sparse pipeline stays fast end to end.