UMAP

UMAP solves the same problem t-SNE does — compress a high-dimensional dataset into a 2D scatter plot that a human can read for clusters — and for a while it has largely displaced t-SNE as the default choice, for three practical reasons: it runs noticeably faster, it scales comfortably to datasets with hundreds of thousands of points where t-SNE grinds to a crawl, and it tends to preserve more of the large-scale layout — how far apart different clusters are from each other — rather than only preserving fine local neighbor structure and scrambling everything else.

Same spirit, different machinery

Like t-SNE, UMAP starts by building a notion of "who is near whom" in the original high-dimensional space — for each point, it looks at its nearest neighbors and builds a graph connecting points to their neighbors, weighted by how confident it is that two points are truly close (not just close in raw Euclidean distance, but close relative to how spread out each point's own local neighborhood is, which lets UMAP handle regions of the data that are naturally denser or sparser than others). It then looks for a low-dimensional layout whose own neighbor-graph is structurally as similar as possible to the original one, optimized directly rather than by matching two probability distributions the way t-SNE does. The practical upshot: UMAP's underlying graph-based optimization scales roughly linearly with dataset size, where t-SNE's pairwise-probability approach scales much worse, which is the main reason UMAP handles a dataset of 100,000 stock-days where t-SNE would take hours.

A worked comparison: three factor groups

Suppose 500 stocks are described by exposures to three underlying style factors — value, growth, and quality — with each stock's 20-feature vector really just a noisy function of which "corner" of factor space it sits closest to. Run both t-SNE and UMAP on the same 20-dimensional feature set. Both will typically show three separated blobs, one per style. Where they tend to diverge is in the space between the blobs: t-SNE often renders three roughly equal, evenly spaced islands regardless of how similar value stocks actually are to quality stocks versus growth stocks in the original 20 dimensions, while UMAP is more likely to place the value blob closer to the quality blob than to the growth blob, if that relative closeness genuinely existed in the original feature space. This is UMAP's better preservation of large-scale (inter-cluster) structure in action — a real, useful difference, though still not something to read as calibrated distance.

Both methods still rely on a fast-decaying weighting of distance to decide who counts as a "near" neighbor, similar in shape to a curve like this one — the difference is in how the resulting neighbor graph gets laid out in 2D afterward, not in throwing away the idea of locality altogether.

What this means in practice

UMAP has largely replaced t-SNE as the go-to first pass for exploring a new large feature set — running it on tens of thousands of order-book snapshots or company-feature vectors to check, visually, whether obvious groupings exist before committing to a supervised model. Because UMAP (unlike t-SNE) has a defined way to embed a new point into an existing map without recomputing the whole thing, it is also more usable as an actual preprocessing step feeding into a downstream classifier, though this is still a minority use case compared to visualization.