Bounded-Influence Regression and Leverage Downweighting

Not all outliers hurt a regression equally. A point with an extreme $y$ value but ordinary $x$ values just produces one large residual — the fitted line barely moves, because that point sits in the "crowded" part of the predictor space where lots of other points anchor the fit. A point with an extreme $x$ value — one sitting far out on its own, away from every other observation — is far more dangerous: that single point can pivot the entire regression line toward itself almost regardless of its $y$ value, since there's nothing else nearby to counteract it. This second kind of point is said to have high leverage.

Ordinary least squares has no defense against high-leverage points because it minimizes squared residuals given the fit those points helped create — by the time you look at the residual, the damage to the fit is already baked in. Bounded-influence regression methods address this directly by downweighting observations based on their leverage before or during fitting, not just based on how badly they fit afterward. A common approach assigns each point a weight that shrinks as its leverage (distance from the bulk of the $x$ data) grows, so a single far-flung point can contribute at most a bounded amount to the fitted coefficients, no matter how extreme it is.

Concretely, if one day's trading volume was 50 standard deviations from typical — a strong candidate for a data error or a one-off event like an index rebalance — a bounded-influence fit caps that day's pull on the slope estimate, whereas an unweighted OLS fit could have its slope determined almost entirely by that single day.