Manifold Learning and Isomap

High-dimensional data often isn't actually spread throughout all those dimensions — it lies on a much lower-dimensional curved surface embedded within them, the way a sheet of paper is a 2D object even when crumpled up and sitting in 3D space. PCA can only find flat low-dimensional structure; fed a crumpled or rolled-up shape, it measures straight-line distances that cut across the curve and badly misjudges which points are truly close together. Isomap is built to respect the curve.

The analogy: the Swiss roll and a hiking trail

Picture a Swiss roll pastry: a flat sheet of dough rolled up into a spiral. Two points that end up physically close together after rolling — one on an inner wind of the spiral, one on an outer wind directly above it — might be inches apart in 3D space but, if you unrolled the dough back to flat, they'd actually be at opposite ends of the sheet. Straight-line 3D distance completely misrepresents how far apart they "really" are on the original flat surface. The right way to measure distance is to trace a path along the dough itself — like measuring a hiking trail's length by walking it, not by drawing a straight line through the mountain it winds around. Isomap estimates exactly this along-the-surface distance using only the data points it has, without ever seeing the true unrolled shape.

The maths: graph distances approximate the true manifold distance

Isomap works in three steps. First, connect each point to its $k$ nearest neighbours, forming a graph — locally, over short distances, straight-line distance is a fine approximation of along-the-surface distance, so this graph's edges are trustworthy. Second, compute the shortest path between every pair of points by hopping along graph edges:

$$d_{\text{geo}}(i,j) = \min_{\text{path } i \to j} \sum_{\text{edges in path}} d_{\text{Euclidean}}(\text{edge})$$

In words: to estimate the true "along the surface" distance between two far-apart points, chain together many short, locally-trustworthy hops rather than trusting one long straight-line measurement that might cut through empty space the data never actually occupies. Third, feed this whole matrix of estimated surface distances into classical multidimensional scaling — a method that finds low-dimensional coordinates whose straight-line distances best reproduce a given distance matrix — to produce the final low-dimensional embedding.

Worked example 1: the crescent, distance the short way versus the long way

Points trace a crescent from $(0,0)$ curving up to $(1,1)$ and back down to $(2,0)$. Consider the two endpoints, $(0,0)$ and $(2,0)$: straight-line Euclidean distance is exactly $2$. But walking along the crescent — through the point at $(1,1)$ — the path length is roughly $\sqrt{2}+\sqrt{2} \approx 2.83$, meaningfully longer. Isomap, built from nearest-neighbour hops along the curve, would estimate something close to $2.83$ for these two points, correctly reflecting that they're far apart along the shape, whereas raw Euclidean distance of $2$ understates it by ignoring the detour the curve forces.

Worked example 2: a 4-point shortest-path chain by hand

Four points on a bent path: $A=(0,0)$, $B=(1,0.2)$, $C=(2,0.5)$, $D=(0.3,3)$, where $D$ is geometrically close to $A$ in raw straight-line terms (distance $\approx 3.02$) but only reachable along the true winding shape by going $A \to B \to C \to D$. Nearest-neighbour edges (each point linked to its closest one or two others) give edge lengths: $A$–$B \approx 1.02$, $B$–$C \approx 1.03$, and suppose $C$–$D \approx 2.55$ is the shortest available hop to $D$, since no direct short edge exists from $A$ to $D$. The Isomap graph distance from $A$ to $D$ is the sum along the only connecting path: $1.02+1.03+2.55 = 4.60$ — more than the raw straight-line $3.02$, correctly signalling that $A$ and $D$, despite looking close in raw coordinates, are actually far apart along the manifold's true shape.

The explorer above shows a linear transform of a circle into an ellipse — a useful contrast: Isomap is built precisely because a single linear transform like this cannot straighten out a curved shape such as a rolled-up sheet, which needs the graph-hopping trick instead.

Shortest-path distances that chain many short hops together, as in worked example 2, tend to grow roughly in proportion to how far you travel along a curve — similar in spirit to how the logarithmic growth shown here compounds many small steps into a larger total.

What this means in practice

Manifold methods like Isomap matter when high-dimensional financial data — say, a full term structure of interest rates, or a wide panel of correlated factor exposures — is suspected to be secretly low-dimensional but curved rather than flat, and a linear method like PCA is visibly failing to compress it without distortion. The nearest-neighbour graph step is the fragile part: too few neighbours and the graph fractures into disconnected pieces with no path between them; too many and short-cut edges sneak in that jump across the curve, defeating the whole point.