The Maximal Information Coefficient
MIC searches over grids of different resolutions to find the binning that best reveals any relationship between two variables — linear, curved, or even periodic — and reports how strong that best-revealed relationship is, on a scale comparable across relationship types.
Prerequisites: Distance Correlation and Nonlinear Dependence
Ordinary correlation only measures linear association, so two variables locked in a strong but curved or cyclical relationship can show a Pearson correlation near zero even though one almost perfectly determines the other. The Maximal Information Coefficient (MIC) tries to catch relationships of any shape, not just linear ones, by an exhaustive search: it overlays grids of different numbers of bins on the scatter of the two variables, computes the mutual information for each grid, normalizes each score to account for how many bins that grid used (finer grids have more room to appear informative just by chance), and then takes the best normalized score across all grids tried.
The resulting statistic ranges from 0 (no detectable association at any resolution) to 1 (a noiseless functional relationship, of any shape), which makes MIC scores roughly comparable across relationships of very different shapes — a strong sine-wave relationship and a strong linear relationship can both score close to 1, something Pearson correlation cannot do. The tradeoff is that MIC needs meaningfully more data than Pearson correlation to estimate reliably, since it is implicitly fitting a grid resolution rather than a single coefficient, and it is more expensive to compute.
MIC searches across grid resolutions to find whichever binning best reveals a relationship between two variables, then normalizes and reports that best score — catching nonlinear and non-monotonic dependence that Pearson correlation misses entirely, at the cost of needing more data.
Related concepts
Practice in interviews
Further reading
- Reshef et al., Detecting Novel Associations in Large Data Sets (2011, Science)