Distance Metrics and Metric Learning

Any method that asks "which points are similar to this one?" — k-NN, clustering, nearest-neighbor search over historical market states — first has to define what "similar" means. The default, plain Euclidean distance on raw features, treats every feature as equally important and comparably scaled. That's almost never true, and getting it wrong quietly corrupts results: "closest" points end up being whichever have the largest-scaled feature, not whichever are genuinely most alike.

The analogy: comparing apartments by mismatched units

Comparing apartments by square footage (hundreds or thousands) and bedroom count (1 to 5), Euclidean distance treats a 200-square-foot gap and a 2-bedroom gap as comparable just because the raw numbers are similar magnitude — but a 200-square-foot gap might be trivial while a 2-bedroom gap is a different kind of home entirely. Distance only means something once you've decided how much a unit of each feature is worth relative to the others.

The mechanics: from a fixed formula to a learned one

Plain Euclidean distance is $d(x,z) = \sqrt{(x-z)^\top (x-z)}$ — every coordinate contributes equally. A more general family replaces the implicit identity matrix with a learned matrix $M$:

$$d_M(x,z) = \sqrt{(x-z)^\top M (x-z)} .$$

In plain English: $M$ reweights (and can rotate) the feature space before measuring distance — stretching directions that matter and compressing ones that don't, so two points that differ hugely in an irrelevant feature but agree on what matters are correctly judged close. $M$ is typically learned so that same-class points end up closer than different-class points; a cheap common special case is just standardizing each feature to unit variance first, a diagonal $M$ built from feature scale rather than fully learned.

Worked example: why raw scale wrecks a naive distance

Matching trading days by VIX (typically 12–40) and 1-day return (typically −0.03 to 0.03): two days with VIX 15 vs. 35 (a real 20-point gap) but identical returns differ by Euclidean distance $\sqrt{20^2+0^2}=20$. Two days with identical VIX but returns −0.02 vs. 0.02 differ by only $\sqrt{0^2+0.04^2}=0.04$ — 500 times "closer," purely because returns are numerically tiny compared to VIX, even though a 4-point return swing is arguably just as significant. Standardizing both features to unit variance first fixes this: both gaps come out as several standard deviations, correctly registering both as large.

Drag the transform above and watch the unit circle stretch into an ellipse along certain directions — that stretching is exactly what a learned $M$ does to distance: some directions in feature space end up counting far more than others.

What this means in practice

Never run k-NN or clustering on raw, unscaled features without thinking first — at minimum standardize each feature to comparable scale. Full metric learning is worth the extra machinery when you have enough labeled examples and the right notion of similarity is genuinely task-specific rather than a matter of unit conversion.