Nesterov Acceleration

Plain momentum-based gradient descent accumulates a running "velocity" from past gradients and uses it to keep moving in a consistent direction, which speeds up convergence along shallow, consistent slopes but can overshoot past a minimum, since the momentum term is computed from the gradient at the current position, blind to where that momentum is about to carry the point next. Nesterov acceleration fixes this by evaluating the gradient not at the current position, but at a "look-ahead" position — where the point would land if it simply followed its existing momentum with no further correction — and using that look-ahead gradient to update the actual step.

The intuition is a runner correcting course mid-stride rather than after landing: ordinary momentum is like checking your direction only when your foot hits the ground, so a mistake gets baked into a whole stride before you notice it; Nesterov's look-ahead is like glancing slightly ahead of your current foot placement and adjusting before you fully commit to the step, catching an overshoot a little earlier and damping it before it compounds.

This look-ahead correction is what gives Nesterov's method its improved theoretical convergence rate for convex problems compared to plain gradient descent or ordinary momentum, and the same idea (evaluate the gradient at a shifted, momentum-anticipated point rather than the literal current point) has been folded into several modern optimizers used in deep learning training, including variants of Adam.