Non-Negative Matrix Factorization

PCA is the standard way to compress a big table of numbers into a few underlying components, but its components are allowed to be negative and to cancel each other out, which makes them mathematically efficient and often humanly unreadable — a "component" that's part positive weight on tech stocks, part negative weight on bonds, doesn't correspond to anything a person would naturally point to. Non-negative matrix factorization (NMF) trades some of that mathematical efficiency for components you can actually interpret as ingredients being added together, never subtracted.

The analogy: recipes, not adjustments

Think of a grocery receipt showing quantities of ingredients bought, and imagine trying to explain many different receipts as combinations of a handful of "typical meals" — a taco-night bundle, a pasta-night bundle, a breakfast bundle. Each receipt is some non-negative amount of each bundle: you can buy 2 taco-nights' worth and 0.5 pasta-nights' worth of ingredients, but you can never buy negative-1 breakfasts to "cancel out" other purchases — that's not how groceries work. NMF finds bundles ("parts") and non-negative amounts of each that reconstruct the original data, respecting the fact that in many real datasets — pixel intensities, word counts, trading volumes — negative amounts are physically meaningless.

The maths: factor into two non-negative pieces

Given a data matrix $V$ with $n$ rows and $m$ columns, all entries non-negative, NMF finds two smaller non-negative matrices $W$ (size $n \times k$) and $H$ (size $k \times m$) such that

$$V \approx WH, \qquad W \ge 0, \ H \ge 0$$

In words: each row of $V$ (say, one document's word counts, or one day's trading volumes across assets) is approximated as a non-negative combination of $k$ "parts" — the rows of $H$ are the parts themselves, and the corresponding row of $W$ says how much of each part that observation uses. Because nothing is allowed to go negative, the only way to build up a data point is by adding parts together, never by adding a part and then subtracting another to cancel it — which is exactly why NMF parts tend to look like recognisable, additive building blocks rather than PCA's abstract combinations.

Worked example 1: two topics, four documents

Suppose word counts for the words {stock, bond, yield, dividend} across four short documents give $V = \begin{pmatrix}5&0&0&4\\0&6&5&0\\4&0&0&3\\0&5&6&0\end{pmatrix}$. With $k=2$ parts, NMF might discover $H$ close to $\begin{pmatrix}1&0&0&0.8\\0&1&0.85&0\end{pmatrix}$ — one part heavy on "stock/dividend," the other heavy on "bond/yield." Then $W$ would be close to $\begin{pmatrix}5&0\\0&6\\4&0\\0&5\end{pmatrix}$, meaning documents 1 and 3 are almost entirely made of part 1 ("equity-flavoured"), documents 2 and 4 almost entirely part 2 ("fixed-income-flavoured"). Multiplying $W$ row 1 by $H$: $5 \times (1,0,0,0.8) = (5,0,0,4)$, matching the original row exactly — the reconstruction is exact here because the topics happen to be cleanly separated in this toy example.

Worked example 2: why non-negativity forces different answers than PCA

Take a simpler matrix, $V = \begin{pmatrix}4&2\\2&4\\3&3\end{pmatrix}$. PCA's first component here would run along the direction $(1,1)$, capturing the shared "overall size" of each row and allowing negative loadings on the second component to capture the $(2,-2)$ vs $(-2,2)$-style differences. NMF, forbidden from using any negative numbers anywhere, can only express each row as a non-negative sum of two non-negative basis vectors — it can't cancel anything out, so it's forced to represent even the "shared size" and the "difference" as two separate strictly-additive parts, for instance something like part 1 $\approx (1, 0.5)$ and part 2 $\approx (0.5, 1)$, with each row a non-negative blend. The two methods can land on genuinely different decompositions of the identical data, because they're solving differently-constrained problems.

The explorer above shows a matrix reshaping a circle into an ellipse along two directions — NMF's $W$ and $H$ play an analogous role of finding two "directions" (parts) that reconstruct the data, but constrained to only stretch, never flip through a negative sign.

NMF is typically fit by iteratively rescaling $W$ and $H$ toward a better fit — a multiplicative update rule whose stability, like the power-law shape shown here, depends on staying strictly on the positive side of zero throughout.

What this means in practice

NMF shows up wherever the data is inherently non-negative and additive: decomposing trading volume across venues into a handful of "typical volume profiles," topic modelling in news or filing text, or separating an order book's shape into recurring liquidity patterns. Unlike PCA, NMF's components generally aren't orthogonal and there's no single unique solution — different initialisations of the iterative fitting procedure can converge to different, equally valid factorizations.