Evolutionary and Genetic Algorithms

Gradient descent needs a smooth objective it can take a derivative of, but plenty of real optimization problems don't offer one: tuning a trading strategy's rule-based logic (a set of discrete if-then thresholds), searching a combinatorial space of feature subsets, or optimizing something whose objective involves running a whole backtest — a function with no clean formula to differentiate. Genetic algorithms optimize these problems by imitating natural selection rather than calculus.

The mechanics: start with a population of many random candidate solutions (say, a hundred different parameter sets for a strategy). Score every candidate using the objective (backtest Sharpe ratio, say) — this is called its "fitness." Keep the best-scoring candidates and let them "reproduce": combine pairs of good candidates by mixing their parameters (crossover), and occasionally flip a parameter to a random new value (mutation) to keep some diversity in the population. Repeat this cycle of scoring, selecting, and recombining for many generations, and the population tends to drift toward better and better solutions, the same way natural selection drifts a species toward better-adapted traits over many generations — without anyone ever computing a derivative of "fitness."

The trade-off is efficiency: because the search is undirected relative to gradient descent — it doesn't know which direction improves the objective, only which candidates happened to score well — genetic algorithms typically need to evaluate far more candidate solutions to converge than a gradient-based method would need on a differentiable problem. That makes them attractive exactly where gradients aren't available (discrete, rule-based, or simulation-based objectives) and a poor choice when a clean gradient already exists, since gradient descent would find the same answer with a fraction of the evaluations.