Least Trimmed Squares and Least Median of Squares

Ordinary least squares (OLS) fits a line by minimizing the sum of all squared residuals, so a single wild outlier — with its huge squared residual — can single-handedly drag the fitted line toward itself. Tukey's biweight softens this by downweighting extreme points, but it still uses every point in some form. Least trimmed squares (LTS) and least median of squares (LMS) go further: they decide in advance to completely discard the worst-fitting points, and fit the line using only the ones that fit well. The line is chosen precisely to make that "well-fitting majority" as tight as possible.

An analogy: judging a diving competition by dropping the outlier scores

In Olympic diving, judges' scores are combined by throwing out the highest and lowest scores before averaging the rest, so one biased or wildly generous judge can't swing the result. LTS and LMS apply the same idea to a regression fit: instead of trusting every data point's residual equally (as OLS does) or gently discounting extreme ones (as biweight does), they explicitly decide that some fraction of the worst-fitting points will simply not count toward judging the fit at all — and then search for the line that makes the remaining, trusted majority fit as tightly as possible.

The two methods, one symbol at a time

For a candidate fitted line with residuals $u_1, \dots, u_n$, sort the squared residuals from smallest to largest: $u_{(1)}^2 \le u_{(2)}^2 \le \dots \le u_{(n)}^2$. Least trimmed squares chooses the line minimizing

$$\sum_{i=1}^{h} u_{(i)}^2 ,$$

where $h < n$ (often around $n/2$) is fixed in advance. In plain English: sort the squared residuals, keep only the smallest $h$ of them, and pick the line that makes that sum as small as possible — the worst $n-h$ points never enter the objective at all, no matter how bad their fit is. Least median of squares instead minimizes the single median squared residual, $\operatorname{median}_i(u_i^2)$: it picks the line that makes the typical (middle-ranked) point fit as well as possible, ignoring entirely how badly the worst half of the points miss. Both methods can tolerate close to 50% of the data being arbitrarily corrupted (their "breakdown point") before the fit is compromised — far higher than OLS's breakdown point of essentially 0%, since even one point moved far enough can move an OLS line arbitrarily.

Worked example 1: LTS by hand on five points

Fit a robust mean (intercept only) to $10, $11, $9, $12, $500 using LTS with $h=4$ (trim the single worst point). For a candidate value $\mu$, the objective sums the four smallest squared deviations. Trying $\mu = 10.5$: deviations are $-0.5, 0.5, -1.5, 1.5, 489.5$; squared and sorted, the four smallest are $0.25, 0.25, 2.25, 2.25$ (dropping the $500 point's huge $489.5^2$), summing to $5.0$. Searching over nearby $\mu$ values shows this is close to optimal — the LTS fit essentially equals the mean of the four sensible points, $(10+11+9+12)/4 = 10.5$, automatically identifying and excluding the $500 corruption.

Worked example 2: comparing breakdown under increasing contamination

Simulate fitting a slope to 20 clean points plus a growing number of extreme outlier points. With 0 outliers, OLS and LTS agree closely (slope $\approx 2.0$ for both). Add 3 outliers (15% contamination): OLS's slope swings to $\approx 0.6$, badly biased, while LTS (with $h$ around 60% of the sample) still reports $\approx 1.95$, barely moved. Push contamination to 9 outliers (about 45% of a 20-point sample): OLS's slope is now meaningless (near 0 or even wrong-signed), while LTS — as long as $h$ was set to protect against up to roughly 50% contamination — still recovers something close to $2.0$. This illustrates the breakdown point concept directly: LTS keeps working right up near the 50% contamination mark, where OLS has long since failed.

What this means in practice

LTS and LMS are used when a dataset may contain a substantial block of bad data — mis-recorded trades, a stale data feed mixed in with a live one, or a regime break contaminating a chunk of history — and you need a fit that survives even when close to half the sample is untrustworthy. LMS is mainly of theoretical interest (its statistical efficiency on clean data is poor and it converges more slowly than LTS), while LTS is the more commonly implemented method and is often used as the robust starting point that feeds into an MM-estimator's efficient second stage.