Machine Learning & Training

What Is Momentum Optimization?

Momentum optimization is a technique that improves gradient descent by adding a fraction of the previous update to the current one. This accumulated velocity helps the optimizer move faster through shallow regions and dampens oscillations. It often leads to faster and more stable convergence.