Class Imbalance and Resampling
When one outcome vastly outnumbers the other — most trades don't blow up, most days aren't crashes — a classifier can score well on accuracy while never once predicting the rare class, and resampling is the standard patch.
Prerequisites: The Confusion Matrix and Classification Metrics, Logistic Regression
Train a classifier to flag which trades will result in a large loss, where only 2% of historical trades qualify, and a model that predicts "no large loss" for every single row is already 98% accurate. It is also completely useless. This is the class imbalance problem: when one label vastly outnumbers the other, standard training procedures — which minimize average error across all rows — have almost nothing to gain from ever predicting the rare class, so they often just don't.
Resampling rebalances the training set before the model ever sees it: either shrink the majority class down, grow the minority class up, or manufacture new synthetic minority examples — so the loss function is no longer free to ignore the rare outcome.
Three ways to rebalance
Random undersampling throws away majority-class rows until the two classes are closer in count. It's simple and fast, but with a 2%-minority problem it can mean discarding 96% of your data, which is wasteful when every row costs money to collect.
Random oversampling duplicates minority-class rows instead. It keeps all the data, but a model can start memorizing the exact duplicated rows rather than a general pattern, since it now sees identical minority examples many times over.
SMOTE (Synthetic Minority Over-sampling Technique) avoids exact duplication: for each minority-class row, it picks one of that row's nearest minority-class neighbors and creates a new synthetic point somewhere on the line between them. In words: instead of copying a rare example, it invents plausible new examples that live in the same neighborhood of feature space, which gives the model more to generalize from than a duplicate would.
Worked example
A dataset of 50,000 historical trades has 1,000 labeled "large loss" (2%) and 49,000 "normal" (98%). Training a logistic regression directly yields a model that never predicts "large loss" at all — 98% accuracy, 0% recall on the class that matters. Applying SMOTE to bring the minority class up to 10,000 synthetic-plus-real examples (a 1:5 ratio instead of 1:49) and retraining, the model now predicts "large loss" for 640 of the true 1,000 large-loss trades in a held-out test set (64% recall), at the cost of also flagging 2,200 normal trades as false alarms. Whether that trade-off is worth it depends entirely on what a missed large loss costs versus what a false alarm costs — resampling changes what the model is willing to predict, not the underlying economics of the decision.
What this means in practice
Resampling only ever touches the training set — the validation and test sets must stay at the real, original class balance, or the reported recall and precision will be meaningless for live performance. It is also not the only lever: adjusting the classification threshold, or using a loss function like focal loss that already downweights easy majority-class examples, can achieve a similar effect without touching the data at all.
Never apply SMOTE (or any resampling) before splitting into train and test — if synthetic points are generated using neighbor information that spans both sets, or oversampled duplicates land on both sides of the split, the test set effectively leaks information from training and reported performance will be optimistic.
Related concepts
Practice in interviews
Further reading
- Chawla et al., 'SMOTE: Synthetic Minority Over-sampling Technique' (2002)
- He & Garcia, 'Learning from Imbalanced Data' (IEEE TKDE, 2009)