Entity-Level Features and Group Leakage

A model that learns "Apple tends to drift up on Fridays" isn't learning a tradeable pattern about markets in general — it's memorizing a quirk of one specific ticker's history, and if Apple happens to appear in both the training set and the test set (just on different dates), the model gets to "cheat" by recognizing the entity itself rather than the underlying relationship the features are supposed to capture. This is group leakage: information about a specific entity leaking across the train/test boundary even when no individual row is literally duplicated.

The idea: an entity, not just a row, must be held out

Ordinary random cross-validation splits rows into folds without regard to which entity each row belongs to. If a stock, account, or counterparty has many rows spread across the dataset (one per day, say), a random split will almost certainly put some of that entity's rows in the training fold and others in the test fold. Any feature that encodes entity identity — directly through an entity ID, or indirectly through an entity-level aggregate like "this stock's average past return" — lets the model partially identify the test-fold entity from what it saw of the same entity in training, inflating validation performance in a way that won't generalize to genuinely new entities.

The fix is group-aware splitting: assign every row belonging to the same entity to the same fold, so an entity is either entirely in training or entirely in test, never split across the boundary — often written as

$$\forall\, i, j : g(i) = g(j) \implies \text{fold}(i) = \text{fold}(j)$$

In plain English: if two rows $i$ and $j$ belong to the same group (entity) $g$, they must be assigned to the same fold — never one to training and one to test.

Worked example: leakage from an entity-average feature

A dataset has 500 rows: 5 stocks, 100 days each. One feature is "this stock's mean return over the entire dataset." A random 80/20 row split puts roughly 80 of each stock's 100 rows in training and 20 in test. The model learns that whenever the entity-average-return feature reads, say, 2.1%, that corresponds to "this is stock C" — a value baked in using the very data (including some test rows' contribution to the mean, since the mean was computed over the whole dataset before splitting) that the model is later evaluated on. Cross-validated accuracy looks strong because the model has effectively learned to recognize five specific stocks, not a generalizable return-predicting relationship — and it would collapse on stock 6, which the model has never touched in any form.

What this means in practice

Group leakage is one of the most common ways cross-sectional equity models look brilliant in validation and mediocre live: the universe of tradeable names rarely turns over that fast, so a model can appear to generalize while actually just memorizing behavior specific to the handful of tickers it has already seen. The practical fix is always the same — split by entity (grouped k-fold, or leave-one-entity-out), and be equally suspicious of any feature computed as an aggregate over an entity's full history if that history includes rows that end up in the test fold.