One-Class SVM and Novelty Detection
A one-class SVM learns the boundary of "normal" from a single class of training data, so anything falling outside that boundary at prediction time can be flagged as novel or anomalous.
A standard support vector machine separates two labeled classes. A one-class SVM has only one class to work with — "normal" examples — and instead learns a boundary that wraps tightly around where most of that normal data lives. Anything a new point falls outside that boundary is flagged as novel or anomalous, without ever having seen an example of what "anomalous" looks like.
A one-class SVM finds the smallest region (in a possibly transformed feature space) that contains most of the training data, then treats anything falling outside that region at test time as an outlier — it's trained on normal data alone, which is what makes it useful when abnormal examples are rare or don't exist yet.
Where it's used
This is exactly the shape of many real fraud and fault-detection problems: you have plenty of examples of normal transactions or normal machine operation, but almost no labeled examples of the specific new fraud pattern or failure mode you actually care about catching. Training a two-class classifier is impossible with no labeled anomalies; a one-class SVM sidesteps that by only needing the normal examples.
Worked example
A payments platform trains a one-class SVM on a year of transactions from an account, all of which were legitimate. The model learns a boundary encompassing typical amounts, times of day, and merchant categories for that account. A new transaction for 50 times the account's usual spend, at 3am, at a merchant category never used before, falls well outside the learned boundary and gets flagged for review — even though the model was never shown a single fraudulent transaction during training.
The nu parameter (roughly, the expected fraction of training points allowed to fall outside the boundary) has to be set deliberately — too small and the model treats ordinary variation as anomalous, too large and it misses real anomalies by drawing too loose a boundary.
Related concepts
Practice in interviews
Further reading
- Schölkopf et al., 'Estimating the Support of a High-Dimensional Distribution'