method · first seen 5 Oct 2026
normalized stochastic gradient descent
Also: normalized SGD
Gradient descent where update magnitude is fixed by normalizing the gradient, decoupling step size from gradient norm.
Mentions per week
PapersCodeDiscussion
Earliest sources
- A Solvable Model of Adaptive Learning Rate Rescaling: Acceleration, Stability & ScalingPapers · 5 Oct 2026 · Itay Lavie, Clarissa Lauditi, Cengiz Pehlevan
Latest sources
- A Solvable Model of Adaptive Learning Rate Rescaling: Acceleration, Stability & ScalingPapers · 5 Oct 2026 · Itay Lavie, Clarissa Lauditi, Cengiz Pehlevan