method · first seen 7 Oct 2026
gradient clipping
Also: global-norm clipping
Capping gradient norms during training to limit the effect of heavy-tailed noise.
Mentions per week
PapersCodeDiscussion
Earliest sources
- Decentralized SGD under Heavy-Tailed Noise: Optimal Convergence Rates and the Role of Gradient ClippingPapers · 7 Oct 2026 · Aleksandar Armacki, Haoyuan Cai, Ali H. Sayed
- On the Last-Iterate Convergence of Clipped Gradient MethodsPapers · 14 Oct 2026 · Zijian Liu
- Weak-$L^α$ Noise, Strong Guarantees: Gradient Clipping under Affine Heavy TailsPapers · 20 Oct 2026 · Alyona Nekrasova, Makar Sidorov, Savelii Chezhegov, Alexander Gasnikov, Margarit…
- Understanding Practical Optimization in Machine LearningPapers · 26 Oct 2026 · Egor Shulgin
- Privately Clipping Heavy-Tailed DataPapers · 26 Oct 2026 · Matthew Joseph, Alex Kulesza
- Near-Zero Token Embeddings Silently Stall Training in Gated Sequence ModelsPapers · 3 Nov 2026 · Xin Liu, Boyuan Sun, Yuxin Wang, Haoran Song, Mingjian Zhao, Mengyu Liu
Latest sources
- Near-Optimal Methods for Generalized Lipschitz Convex OptimizationPapers · 15 Nov 2026 · Dmitry Kovalev, Grigory Malinovsky
- Near-Zero Token Embeddings Silently Stall Training in Gated Sequence ModelsPapers · 3 Nov 2026 · Xin Liu, Boyuan Sun, Yuxin Wang, Haoran Song, Mingjian Zhao, Mengyu Liu
- Privately Clipping Heavy-Tailed DataPapers · 26 Oct 2026 · Matthew Joseph, Alex Kulesza
- Understanding Practical Optimization in Machine LearningPapers · 26 Oct 2026 · Egor Shulgin
- Weak-$L^α$ Noise, Strong Guarantees: Gradient Clipping under Affine Heavy TailsPapers · 20 Oct 2026 · Alyona Nekrasova, Makar Sidorov, Savelii Chezhegov, Alexander Gasnikov, Margarit…
- On the Last-Iterate Convergence of Clipped Gradient MethodsPapers · 14 Oct 2026 · Zijian Liu
- Decentralized SGD under Heavy-Tailed Noise: Optimal Convergence Rates and the Role of Gradient ClippingPapers · 7 Oct 2026 · Aleksandar Armacki, Haoyuan Cai, Ali H. Sayed