Tag: gradient-descent
Concepts
- Batch, Mini-Batch and Online Training
- Convex vs Non-Convex Loss Surfaces
- Gradient Descent
- Ill-Conditioning and the Hessian Spectrum
- Implicit Regularization
- L-BFGS and Quasi-Newton Methods in ML
- Line Search and Trust-Region Methods
- Momentum and Nesterov Acceleration
- Nesterov Acceleration
- Non-Convex Optimization Landscapes
- Saddle Points and Plateaus
- Sigmoid Saturation and Logit Scaling