Lagrange Multipliers & KKT Conditions

Almost every optimization a quant actually solves is constrained, weights sum to one, leverage is capped, positions are long-only, tracking error is bounded. Lagrange multipliers and their inequality-handling generalization, the Karush–Kuhn–Tucker (KKT) conditions, are the exact characterization of a constrained optimum: they tell you the equations the solution must satisfy and, as a bonus, the price of each constraint. This is the machinery that turns the Convex Optimization promise of "solvable" into an actual solution, and it is the derivation behind the entire mean–variance efficient frontier.

Equality constraints: the geometry of Lagrange

Consider minimizing $f(x)$ subject to $g(x) = 0$. At an optimum $x^\star$ on the constraint surface, you cannot decrease $f$ by moving along the surface, so the descent direction $-\nabla f$ must have no component tangent to the surface, i.e. $\nabla f$ must be parallel to the constraint normal $\nabla g$. That is the entire geometric content: $\nabla f(x^\star) = \lambda\, \nabla g(x^\star),$ for some scalar $\lambda$, the Lagrange multiplier. Package this by forming the Lagrangian $\mathcal{L}(x, \lambda) = f(x) - \lambda\, g(x),$ and the stationarity conditions $\nabla_x \mathcal{L} = 0$ and $\partial\mathcal{L}/\partial\lambda = 0$ recover both the gradient-parallelism and the constraint $g(x) = 0$. With multiple equality constraints $g_i(x) = 0$, the gradient of $f$ must lie in the span of the constraint gradients: $\nabla f = \sum_i \lambda_i \nabla g_i$. The multiplier is not just algebraic bookkeeping, $\lambda^\star = \partial f^\star/\partial c$ is the sensitivity of the optimal value to relaxing the constraint level $c$, its shadow price.

Inequality constraints: the KKT conditions

Real problems mix equalities with inequalities ($h_j(x) \le 0$: leverage caps, no-short constraints). The KKT conditions are the complete first-order characterization. For $\min_x f(x)\quad\text{s.t.}\quad h_j(x)\le 0,\ \ g_i(x)=0,$ with Lagrangian $\mathcal{L} = f + \sum_j \mu_j h_j + \sum_i \lambda_i g_i$, a point $x^\star$ (with multipliers $\mu^\star, \lambda^\star$) is optimal only if:

1. Stationarity: $\nabla f(x^\star) + \sum_j \mu_j^\star \nabla h_j(x^\star) + \sum_i \lambda_i^\star \nabla g_i(x^\star) = 0.$
2. Primal feasibility: $h_j(x^\star) \le 0,\ g_i(x^\star) = 0.$
3. Dual feasibility: $\mu_j^\star \ge 0$ (inequality multipliers are non-negative, the constraint can only push back, not pull).
4. Complementary slackness: $\mu_j^\star\, h_j(x^\star) = 0$ for every $j$.

Condition 4 is the conceptual heart. For each inequality constraint, either the multiplier is zero or the constraint is active (tight, $h_j = 0$): $\mu_j^\star > 0 \Rightarrow h_j(x^\star) = 0$, and $h_j(x^\star) < 0 \Rightarrow \mu_j^\star = 0$. In words, an inactive constraint has zero shadow price, and only binding constraints carry a price. A leverage cap you're not hitting doesn't affect your optimal portfolio; a cap you're pressed against has a positive multiplier equal to the marginal return you'd gain by loosening it. Complementary slackness is what lets you solve KKT systems by guessing the active set (which constraints bind), solving the resulting equality-constrained problem, and checking the sign conditions.

Sufficiency under convexity. For a general (non-convex) problem KKT is only necessary (and needs a constraint qualification like Slater's or LICQ). But for a convex problem, the KKT conditions are both necessary and sufficient: any point satisfying them is a global optimum. This is the precise sense in which convexity makes constrained problems solvable, you solve a system of equations and inequalities and you are done, with a global certificate. It is also exactly the strong-duality statement from Convex Optimization: the optimal KKT multipliers are the optimal dual variables.

Worked example, mean–variance with a return target

The efficient frontier is a KKT (here, pure-equality Lagrange) problem. Minimize variance subject to a target return and full investment: $\min_w\ \tfrac12 w^\top\Sigma w \quad\text{s.t.}\quad \mu^\top w = r,\ \ \mathbf{1}^\top w = 1.$ Form $\mathcal{L} = \tfrac12 w^\top\Sigma w - \lambda(\mu^\top w - r) - \gamma(\mathbf{1}^\top w - 1)$. Stationarity in $w$: $\nabla_w\mathcal{L} = \Sigma w - \lambda\mu - \gamma\mathbf{1} = 0 \;\Longrightarrow\; w^\star = \Sigma^{-1}(\lambda\mu + \gamma\mathbf{1}).$ The optimal portfolio is a linear combination of $\Sigma^{-1}\mu$ (the The Kelly Criterion / tangency direction) and $\Sigma^{-1}\mathbf{1}$ (the minimum-variance direction), the two-fund separation theorem falls straight out of the two multipliers. Substituting $w^\star$ back into the two constraints gives a $2\times2$ linear system for $\lambda$ and $\gamma$ in terms of the scalars $A = \mathbf{1}^\top\Sigma^{-1}\mathbf{1}$, $B = \mathbf{1}^\top\Sigma^{-1}\mu$, $C = \mu^\top\Sigma^{-1}\mu$, whose solution traces the entire frontier as $r$ varies. Because the problem is convex ($\Sigma \succeq 0$), this stationary point is the global minimum, the multipliers are the answer, not a candidate. See MPT (Harry Markowitz) for the frontier this generates. Adding a long-only constraint $w \ge 0$ turns it into a genuine inequality-KKT problem where complementary slackness decides which names sit at the zero boundary.

Failure modes and subtleties

• KKT is only necessary without convexity. For non-convex $f$, a KKT point can be a local min, a saddle, or a max; you must check second-order conditions or global structure. Convexity is what promotes KKT to sufficient.
• Constraint qualifications matter. KKT presumes the constraint gradients are well-behaved at $x^\star$ (LICQ, or Slater's for convex problems). At degenerate points (redundant or tangentially-meeting constraints) multipliers can fail to exist or be non-unique, and naive solvers stall.
• Sign conventions bite. Inequality multipliers must be $\ge 0$ with the constraints written as $h_j \le 0$; flipping a constraint's sign flips the required multiplier sign. A multiplier coming out negative usually means you mislabeled an active constraint or the "constraint" isn't actually binding.
• Guessing the active set can be combinatorial. With many inequalities, the number of possible active sets is exponential; this is why real solvers use interior-point or active-set algorithms rather than brute-forcing complementary slackness. KKT tells you what to verify, not how to search efficiently.
• Shadow prices assume smoothness and local validity. $\lambda^\star = \partial f^\star/\partial c$ is a marginal, valid for small perturbations; large constraint changes can shift the active set and invalidate the linear sensitivity.

In interviews

The mean–variance derivation is the canonical test: set up the Lagrangian, take the gradient, and produce $w^\star = \Sigma^{-1}(\lambda\mu + \gamma\mathbf{1})$, ideally naming two-fund separation. Be able to give the geometric intuition for Lagrange multipliers, gradients parallel at the optimum, not just the mechanics. For inequality problems, state all four KKT conditions and explain complementary slackness in plain terms (inactive constraints have zero price), since that is the piece candidates most often miss. Know that KKT is necessary-and-sufficient only under convexity, tying back to Convex Optimization. A strong close is interpreting the multiplier as a shadow price, the marginal value of a risk budget or leverage limit, which shows you understand constrained optimization as economics, not just calculus.