Random Forest Proximities

A random forest is usually trained to predict a label, but the same trained trees can be repurposed to measure how similar any two data points are to each other, using nothing more than how the trees already grouped points into leaves.

How it's built

For every pair of points, count how many of the forest's trees route both points to the exact same leaf node, then divide by the total number of trees. Because each tree only saw a bootstrap sample of rows and a random subset of features at each split, two points that consistently end up together despite all that randomization are genuinely similar according to whatever patterns the forest learned — not just similar on one arbitrarily chosen feature.

The resulting proximity matrix (points × points) can feed into standard tools built for distance data: hierarchical clustering, multidimensional scaling for visualization, or flagging outliers as points with low proximity to everything else in the dataset.

Worked example

In a forest of 500 trees trained on a customer dataset, two customers who end up in the same leaf in 480 of the 500 trees get a proximity of 0.96 — very similar according to the forest — while a customer who lands in the same leaf as almost nobody else, say an average proximity below 0.05 to every other point, is flagged as an outlier the forest struggles to group with any cluster of typical customers.