method · first seen 12 Oct 2026
learning rate schedule
Also: LRD, optimizer tuning, stepsize schedule, learning rate decay, learning rate scheduling
A prescribed plan for how the learning rate and related training settings change over the course of training.
Mentions per week
PapersCodeDiscussion
Rank 200 on 2 Nov 2026 · 9 mentions in 28 days · 9 independent groups · papers · first seen 3 weeks ago · acceleration -0.83
Earliest sources
- Asymptotic Risk of Learning-Rate Schedules for Least-Squares Regression with Discretely Distributed FeaturesPapers · 12 Oct 2026 · Noé Colomban, Francis Bach, Fabian Schaipp
- A Refresh Interval Does Not Fully Specify a Refresh Schedule: Learning-Rate-Dependent Placement in Kronecker-Preconditioned Language Model TrainingPapers · 13 Oct 2026 · Xinyu Wang, Zhiwei Zhai, Yu Gu, Zijun Yu, Sicheng Lyu, Jingrui Tian, Lei Ding, Z…
- Large Diffusion Models Need More Than EMAPapers · 16 Oct 2026 · Chuyu Hu, Huanran Chen, Shitong Shao, Yinpeng Dong
- MuPACE: Learning to Pace Muon for LoRA Fine-TuningPapers · 20 Oct 2026 · Jan Kobiolka, Christian Frey, Josif Grabocka
- Data Schedules and Learning Rate Schedules Should Be Designed TogetherPapers · 21 Oct 2026 · Haodong Wen, Kairong Luo, Tingkai Yan, Wenguang Chen, Kaifeng Lyu
- RSI-Opt: Recursive Self-Improvement Optimizer Agents For Model TrainingPapers · 23 Oct 2026 · Binghang Lu, Yige Sun, Guang Lin
Latest sources
- Learning Rate Decay Can Exponentially Accelerate SGD for Global Optimization of Nonconvex FunctionsPapers · 9 Nov 2026 · Guneykan Ozgul, Dylan Herman, Junhyung Lyle Kim, Shree Hari Sureshbabu, Shouvani…
- Stepsize modulation for cross entropy from local quadratic optimizationPapers · 3 Nov 2026 · Bruno Bauwens, Viacheslav Krivodub, Bingyu Zhang
- Exact Theory and Optimal Stepsizes for Stochastic Gradient Descent in Smooth Convex MinimizationPapers · 30 Oct 2026 · Benjamin Grimmer, Qining Shen
- Your LR Schedule May Not Be Principled: Effective Learning Rate Scheduling in Weight-Normalized TrainingPapers · 29 Oct 2026 · Qifan Yu, Xinyu Ma, Xingyan Bin, Di He
- UTA: Universal Training Adaptive Learning Rate DecayPapers · 26 Oct 2026 · Anda Tang, Junjie Wang, Xuanlin Yang, Cong Fang, Zhouchen Lin
- RSI-Opt: Recursive Self-Improvement Optimizer Agents For Model TrainingPapers · 23 Oct 2026 · Binghang Lu, Yige Sun, Guang Lin
- Data Schedules and Learning Rate Schedules Should Be Designed TogetherPapers · 21 Oct 2026 · Haodong Wen, Kairong Luo, Tingkai Yan, Wenguang Chen, Kaifeng Lyu
- MuPACE: Learning to Pace Muon for LoRA Fine-TuningPapers · 20 Oct 2026 · Jan Kobiolka, Christian Frey, Josif Grabocka
- Large Diffusion Models Need More Than EMAPapers · 16 Oct 2026 · Chuyu Hu, Huanran Chen, Shitong Shao, Yinpeng Dong
- A Refresh Interval Does Not Fully Specify a Refresh Schedule: Learning-Rate-Dependent Placement in Kronecker-Preconditioned Language Model TrainingPapers · 13 Oct 2026 · Xinyu Wang, Zhiwei Zhai, Yu Gu, Zijun Yu, Sicheng Lyu, Jingrui Tian, Lei Ding, Z…
- Asymptotic Risk of Learning-Rate Schedules for Least-Squares Regression with Discretely Distributed FeaturesPapers · 12 Oct 2026 · Noé Colomban, Francis Bach, Fabian Schaipp