MM-Estimators for Regression

High-breakdown methods like S-estimators can survive nearly 50% data contamination, but that safety costs statistical efficiency: on data that turns out to be mostly clean, their coefficient estimates are noisier than a well-tuned M-estimator's would be. Biweight M-estimators, on the other hand, are efficient on clean data but can be led astray by a poor starting point when the data really is heavily contaminated. MM-estimators are the resolution: run a high-breakdown method first purely to get a safe anchor, then refine from that anchor using a more efficient method, in a way explicitly designed so the refinement step can never move far enough to lose the anchor's breakdown protection.

An analogy: a rough draft locked in, then polished carefully

Think of writing an important document under a strict rule: draft one, written quickly, is guaranteed structurally sound (nothing embarrassing slipped in) even if it's not beautifully written; you're then allowed to polish the prose in a second pass, but you're not allowed to make edits so large that the document could regress into something structurally unsound again. The first pass buys safety; the second pass buys quality, without undoing the safety the first pass established. MM-estimation is exactly this two-pass discipline applied to a regression fit.

The two stages, one symbol at a time

Stage 1 (breakdown): compute an S-estimator (or LTS estimator) $\hat\beta_0$ and its associated robust scale $\hat s_0$ — this stage has a high breakdown point (up to 50%) but possibly lower efficiency. Stage 2 (efficiency): holding the scale $\hat s_0$ fixed from stage 1, refine the coefficients by solving

$$\hat\beta_{MM} = \arg\min_\beta \sum_{i=1}^n \rho_1\!\left(\frac{y_i - x_i^\top\beta}{\hat s_0}\right),$$

using a psi function $\rho_1$ chosen for high statistical efficiency (often a biweight with a different tuning constant than stage 1), typically solved via IRLS started from $\hat\beta_0$. In plain English: the second stage is allowed to move the coefficients to a more efficient fit, but only within the neighborhood that the fixed, already-robust scale $\hat s_0$ permits — because the scale, not the coefficients, is what encodes the contamination protection, and freezing it stops the refinement from wandering back toward a contaminated solution.

Worked example 1: efficiency gain on clean data

Simulating 200 clean (uncontaminated) data points from a known linear model repeatedly, an S-estimator alone recovers the true slope with a standard error of about 0.14 across simulation runs, while OLS achieves 0.10 (OLS is fully efficient here since there's no contamination to guard against). An MM-estimator built on the same S-estimator's scale, but refined in stage 2 for efficiency, achieves a standard error of about 0.11 — nearly matching OLS's efficiency on clean data, unlike the S-estimator alone, which paid a real efficiency cost for its breakdown protection that MM recovers almost fully.

Worked example 2: breakdown protection retained under contamination

Now contaminate 25% of a 200-point sample with extreme outliers. OLS's slope estimate is badly biased, moving from a true value of 1.5 to roughly 0.4. The S-estimator, unaffected by contamination below its 50% breakdown point, recovers approximately 1.48. The MM-estimator, refined from that same S-estimator start, also recovers approximately 1.49 — essentially matching the S-estimator's robustness while (as example 1 showed) being more efficient when contamination happens to be absent. This is the MM-estimator's core selling point demonstrated directly: robustness from stage 1, efficiency from stage 2, at the same time.

What this means in practice

MM-estimation is the default recommendation in most modern robust-regression software precisely because it removes the tradeoff that plagued earlier methods: you no longer have to choose between "safe but inefficient" (LTS, S-estimators) and "efficient but fragile" (plain biweight M-estimators started from OLS). It's the standard choice for fitting factor models or betas on return series that might contain data errors or short-lived regime breaks, where you want both protection against bad data and a fit that doesn't waste information when the data is actually clean.