Optimization

Read these first

1. 01Convex Sets and Convex Functions

   Convexity is the property that guarantees a local best answer is the global best answer, it is the single dividing line between optimisation problems you can solve reliably and ones you generally can't.

2. 02Duality and the Dual Problem

   Every constrained optimisation problem has a shadow twin, built from the price of relaxing each constraint, solving that twin problem often turns out to be easier, and it always gives a certificate of how close your answer is to optimal.

3. 03Gradient Descent

   Walking downhill in the direction that drops fastest, taking a step, and repeating is almost the entire training algorithm behind modern machine learning, the whole method fits in one line, and every subtlety is in choosing the step size.

4. 04Quadratic Programming

   Minimum-variance portfolio construction is a quadratic program: minimise a bowl-shaped risk function subject to straight-line constraints, a problem structured enough to be solved fast, reliably, and exactly.

Then the rest

1. 05Linear Programming and the Simplex Method
2. 06ADMM: The Alternating Direction Method of Multipliers
3. 07Coordinate Descent
4. 08Integer and Mixed-Integer Programming
5. 09Minimax and Worst-Case Optimization
6. 10Newton's Method for Optimization
7. 11Non-Convex Optimization Landscapes
8. 12Optimal Transport
9. 13Proximal Gradient Methods
10. 14Quasi-Newton Methods and BFGS
11. 15Second-Order Cone Programming
12. 16Simulated Annealing
13. 17Stochastic Programming and Scenario Trees
14. 18Subgradient Methods
15. 19The Conjugate Gradient Method
16. 20Interior-Point Methods
17. 21Semidefinite Programming