method · first seen 13 Oct 2026
Kronecker-factored preconditioning
Also: Kronecker-preconditioned optimizer
Optimizers approximating curvature with Kronecker-structured factors, periodically recomputing their eigendecompositions.
Mentions per week
PapersCodeDiscussion
Earliest sources
- A Refresh Interval Does Not Fully Specify a Refresh Schedule: Learning-Rate-Dependent Placement in Kronecker-Preconditioned Language Model TrainingPapers · 13 Oct 2026 · Xinyu Wang, Zhiwei Zhai, Yu Gu, Zijun Yu, Sicheng Lyu, Jingrui Tian, Lei Ding, Z…
Latest sources
- A Refresh Interval Does Not Fully Specify a Refresh Schedule: Learning-Rate-Dependent Placement in Kronecker-Preconditioned Language Model TrainingPapers · 13 Oct 2026 · Xinyu Wang, Zhiwei Zhai, Yu Gu, Zijun Yu, Sicheng Lyu, Jingrui Tian, Lei Ding, Z…