Fused Lasso and Total-Variation Penalties

Ordinary lasso regression penalizes the size of coefficients to push unimportant ones to exactly zero, which is useful for selecting variables but doesn't do anything about how coefficients relate to each other. When the coefficients or data points have a natural order — like a sequence of trading days, or ratings across adjacent credit tiers — it's often more realistic to assume that neighboring values shouldn't jump around independently, but should be equal or close to equal unless there's real evidence of a genuine shift.

The fused lasso adds a second penalty on top of the usual one: it penalizes the sum of absolute differences between consecutive coefficients (this is also called a total-variation penalty). Minimizing that penalty pushes the model toward solutions that are piecewise constant — flat stretches punctuated by a small number of sharp jumps — rather than smoothly wiggling or independently noisy from one point to the next.

Worked example. A model estimates a firm's credit risk level for each of 24 consecutive months. Fit without any ordering penalty, the monthly risk estimates jump around noisily even though the firm's true underlying risk barely changed most months. Applying a fused lasso penalty smooths this into a handful of stable plateaus — say, low risk for months 1–14, then a jump to elevated risk for months 15–24 — which better matches the reality that credit conditions shift in discrete regimes rather than fluctuating every single month.