Non-Convex Optimization Landscapes

Every convex-optimization guarantee — gradient descent converges, Newton's method finds the minimum — quietly assumes there's only one dip in the landscape to find. Real objectives in quant finance and machine learning are frequently not convex: a strategy's Sharpe ratio as a function of its parameters, or a neural network's loss as a function of its weights, can have many separate dips, ridges, and flat stretches. Non-convexity isn't a minor technicality — it changes what "solving" the optimization problem even means, because there may be no single best answer to converge to, only a landscape of local answers.

An analogy: hiking blind in real mountains, not a single bowl

A convex problem is like a single smooth bowl — wherever you start walking downhill, you eventually reach the one lowest point, guaranteed. A non-convex problem is like real, rugged mountain terrain in fog: you can walk downhill from where you stand and confidently reach the bottom of a valley, but there's no guarantee it's the deepest valley in the whole range — a much lower valley might sit just over a ridge you never crossed, because crossing it would have required walking briefly uphill, which pure downhill-walking never does.

The landscape's features, one at a time

A local minimum is a point lower than everywhere nearby but not necessarily lower than everywhere overall — gradient descent, which only sees the immediate slope, cannot tell a local minimum from the true global minimum once it's standing at one, since the gradient is zero at both. A saddle point also has zero gradient, but the surface curves up in some directions and down in others, like the middle of a horse's saddle — not a minimum at all, yet gradient descent slows to a crawl near one anyway. A plateau is a broad, nearly flat region where the gradient is tiny everywhere, so progress stalls from lack of signal alone. In very high dimensions, research shows saddle points vastly outnumber true local minima — a random critical point needs positive curvature in every direction simultaneously to be a genuine minimum, far rarer than having at least one bad direction.

Worked example 1: a simple two-dip function

Consider $f(x) = x^4 - 5x^2 + 2x$ on the real line. Its derivative $f'(x) = 4x^3 - 10x + 2$ has roots near $x\approx -1.66$, $x\approx 0.20$, and $x\approx 1.46$ (found numerically). Evaluating $f$ at each: $f(-1.66)\approx -3.62$, $f(0.20)\approx 0.29$ (a local max, not min), $f(1.46)\approx -3.85$. So $x\approx -1.66$ is a local minimum but $x\approx 1.46$ is the true global minimum, slightly lower. Starting gradient descent from $x_0 = -3$ converges to the local minimum at $-1.66$ and stops there — the gradient is genuinely zero, so the algorithm reports success, even though a better solution sits on the other side of the local max at $0.20$ that it never crossed.

Worked example 2: escaping via random restarts

A quant calibrating a five-parameter volatility model with a non-convex fit error runs gradient descent from one starting guess and lands on a fit with sum-of-squared-error 0.045. Suspicious this may be a local minimum, she reruns the same optimizer from 20 different random starting points. Eighteen of the twenty runs converge to error values between 0.040 and 0.048 (clustered near the same local minimum found first), but two runs converge to a markedly better error of 0.012 — revealing a substantially better region of parameter space that a single run from one starting point would never have found. This is the standard, low-tech remedy for non-convexity: since no single run can guarantee global optimality, running many independent restarts and keeping the best result is a practical (if not theoretically guaranteed) way to reduce the risk of reporting a mediocre local optimum as if it were the answer.

Try starting from different points on a bumpy loss surface: notice how some starting points slide into a shallow local dip and stop, while others reach the true deepest valley — the starting point alone determines the outcome on non-convex terrain.

What this means in practice

Non-convexity is the norm, not the exception, for strategy-parameter tuning, neural network training, and any calibration with several interacting parameters. In practice quants manage it with random or informed multi-start restarts, stochastic methods that add noise to help escape shallow traps (see simulated annealing), or by deliberately choosing convex approximations to the real problem when possible, since a convex problem's guarantees are worth a great deal.