Robust PCA
Robust PCA splits a data matrix into a low-rank "normal structure" part and a sparse "outlier" part, so a handful of corrupted entries or extreme events don't distort the whole decomposition the way they would in ordinary PCA.
Prerequisites: Probabilistic PCA
Ordinary PCA is fragile: a small number of extreme, corrupted, or outlying values can dominate the variance calculation and drag every principal component off in the wrong direction. Robust PCA decomposes a data matrix into two pieces solved simultaneously — a low-rank matrix capturing the smooth, repeating structure, and a sparse matrix capturing a small number of large, isolated corruptions — so outliers get absorbed into the sparse part instead of contaminating the structure everyone actually wants.
Robust PCA splits a matrix as data = low-rank + sparse, recovering the underlying structure (low-rank part) even when a modest fraction of entries are wildly corrupted (sparse part), because those corruptions are assumed to be few and large rather than spread evenly like ordinary noise.
Why "low-rank + sparse" instead of "signal + noise"
Ordinary PCA implicitly assumes noise is small and spread across every entry (Gaussian-like). Robust PCA instead assumes corruption is large but rare — a handful of pixels blown out in an image, a handful of days with data-feed errors in a price series — which is a very different failure mode, and one that a small-noise assumption handles badly. Solving for both matrices at once (typically via convex optimization, minimizing the low-rank matrix's nuclear norm plus the sparse matrix's L1 norm) can recover both pieces exactly under fairly general conditions.
Worked example
A background-subtraction task takes a security camera's video frames, stacked as columns of a matrix. The static background repeats across frames and is well approximated by a low-rank matrix; a person walking through the scene shows up only in a few pixels per frame and only in some frames, which is exactly the sparse, large-magnitude pattern robust PCA is built to isolate. Running the decomposition separates the unchanging background (low-rank) from the moving person (sparse) automatically, without labeling a single frame.
Robust PCA needs corruption to be sparse, not merely large — if a large fraction of every column is corrupted (rather than a small fraction of entries scattered across the matrix), the low-rank/sparse split becomes ambiguous and the method can fail to separate the two parts cleanly.
Related concepts
Practice in interviews
Further reading
- Candès et al., 'Robust Principal Component Analysis?'