Mixture Discriminant Analysis

Linear and quadratic discriminant analysis both assume each class forms one single, roughly elliptical cloud of points. That assumption breaks down when a class is really a combination of distinct subgroups — "defaulted borrowers," for instance, might actually contain both borrowers who defaulted from sudden job loss and borrowers who defaulted from long-term over-leverage, two subgroups with quite different characteristics lumped under one label. Fitting a single elliptical shape to a class that's actually two separate clusters produces a poor fit to either.

Mixture discriminant analysis (MDA) handles this by modeling each class as a mixture of several Gaussian sub-clusters rather than one. Instead of estimating a single mean and covariance per class, it estimates several component distributions per class and lets the data decide how many points belong to each sub-cluster, then classifies a new point based on how well it matches the combined mixture for each class, letting the model capture multimodal structure within what accounting or labels treat as a single category.

Worked example. A model classifying loan applicants as "default" or "no default" performs poorly under standard LDA because the default class actually contains two very different subgroups: young borrowers with thin credit files, and older borrowers with high existing debt loads, each clustering in a different part of feature space. Fitting mixture discriminant analysis with two sub-clusters for the default class instead of one lets the model draw a more accurate decision boundary around each subgroup separately, improving classification accuracy on a held-out test set compared to plain LDA.