Maximum Mean Discrepancy Drift Tests

Maximum Mean Discrepancy, or MMD, is a way of measuring the distance between two probability distributions using only samples drawn from each — exactly the situation a drift monitor is in, comparing a batch of live production features against the training set's features. It works by mapping each sample into a high-dimensional space (via a kernel function) and comparing the average position of each batch in that space; if the two batches came from the same underlying distribution, their average positions should be close, and MMD returns something near zero. If the underlying distribution has shifted, the averages pull apart and MMD rises.

The appeal over simpler tests is that MMD makes no assumption about the shape of the distribution — it isn't limited to comparing means and variances the way a t-test is, and it can pick up on distributional changes that don't show up in any single summary statistic. In practice, a production monitoring job runs an MMD test on each feature (or a joint test across several) on a rolling window of live data against a fixed reference window from training, and treats a statistically significant MMD value as a drift alert worth investigating, often triggering a model retraining check.