The Population Stability Index

You want one number that says "has this feature's distribution shifted since the model was trained?" without needing any labels, computable daily on a dashboard. The Population Stability Index (PSI) is the industry-standard answer: bucket a variable into bins, compare what fraction of observations fell in each bin during a reference period versus a current period, and combine the differences into a single score.

The formula, one piece at a time

Split the variable's range into $k$ bins (deciles are common). For each bin $i$, let $R_i$ be the fraction of the reference-period data in that bin, and $C_i$ be the fraction of the current-period data in that bin. The PSI is:

$$PSI = \sum_{i=1}^{k} (C_i - R_i) \times \ln\left(\frac{C_i}{R_i}\right)$$

In plain English: for each bin, take the difference in the two periods' proportions, multiply by the log of their ratio, and sum across all bins. Both factors are zero when the two periods agree exactly in that bin, and both grow (in the same direction) as a bin's share changes, so the product is always non-negative and larger swings contribute disproportionately more — a bin whose share doubled contributes far more than one that nudged up 10%. Conventional thresholds: PSI below 0.10 means no meaningful shift; 0.10 to 0.25 means a moderate shift worth watching; above 0.25 means a major shift that typically warrants investigation or retraining.

Worked example

A model's key feature is a stock's 20-day momentum, binned into deciles at training time. Six months later, comparing the current month's momentum distribution to the training-period reference:

Bin	Reference share $R_i$	Current share $C_i$	$(C_i - R_i)\ln(C_i/R_i)$
Bottom decile	0.10	0.22	$0.12 \times \ln(2.2) = 0.094$
Middle deciles (8 bins)	~0.80 total	~0.62 total	≈ 0.055 combined
Top decile	0.10	0.16	$0.06 \times \ln(1.6) = 0.028$

Summing all ten bins gives $PSI \approx 0.18$ — a moderate shift. Reading the bins directly: far more observations have piled into the bottom decile (strongly negative momentum) than during training, at the expense of the middle deciles, telling the desk the current market has many more stocks in sharp downtrends than the training period did, well before any model-accuracy metric would confirm a problem.

What this means in practice

PSI is cheap, requires no labels, and is standard enough that "PSI > 0.25" is a common automated alert or even a kill-criterion trigger. It should be computed for every important feature and for the model's own output distribution, not just once at deployment but on a rolling basis, since drift accumulates gradually and a single check at launch won't catch it months later.