Machine Learning & Training

What Is Gradient Clipping?

Gradient clipping is a technique that caps the magnitude of gradients during training to keep them from becoming too large. This helps prevent the exploding gradient problem, which can destabilize learning. It is especially useful when training recurrent networks and deep architectures.