Cluster Assignments as Features

Clustering algorithms like k-means group similar rows together based on their features, without using the target variable at all. Cluster assignments as features takes that unsupervised output — which cluster each row landed in — and feeds it back into a supervised model as a new categorical feature (or one-hot encoded set of features).

This is useful when there's meaningful group structure in the data that isn't captured well by any single existing column, but emerges from the combination of several. A model given the raw features might struggle to reconstruct this structure on its own, especially with a shallow tree or linear model; handing it the cluster label directly as a feature can shortcut that discovery. The main risk is leakage: the clustering must be fit only on training data, never on data that includes the validation or test set, or the cluster boundaries will have "seen" information they shouldn't have.

Worked example

A retailer clusters customers into 5 groups using purchase-frequency and average-basket-size features via k-means, fit only on the training set. Each customer's cluster label (1 through 5) is added as a new categorical feature to a churn-prediction model. The model, which previously struggled to find this pattern on its own from the raw two columns, now uses the cluster label directly and improves its accuracy, since customers in the "low frequency, high basket size" cluster turn out to churn at a distinctly different rate than the others.