method · first seen 6 Oct 2026
Shampoo optimizer
Also: SOAP, Shampoo, Matrix Shampoo, matrix-structured optimizer
A preconditioned stochastic optimizer that uses Kronecker-factored second-order statistics of gradients.
Mentions per week
PapersCodeDiscussion
Earliest sources
- How Bregman Divergences Shape ShampooPapers · 6 Oct 2026 · Bing Liu, Wenjie Zhou, Chengcheng Zhao, Hongtao Zhang, Boao Kong, Felix Dangel,…
- Efficient Preconditioned Optimization Methods with Frequent DirectionsPapers · 12 Oct 2026 · Sifan Yang, Yuanyu Wan, Yibo Wang, Lijun Zhang
- Minimax Optimality of Shampoo for Nonstationary Stochastic OptimizationPapers · 12 Oct 2026 · Abir Sarkar, Nirmalya Mandal, Martin T. Wells
- Stochastic optimization methods through statistical whiteningPapers · 17 Oct 2026 · Tetiana Parshakova, Robert M. Gower, Paulo Orenstein
- Breaking Precision Walls in First-, Second-, and Zeroth-Order Optimizers: One Constant, Set in AdvancePapers · 7 Nov 2026 · Lngel Kyo
- Factored One-sided Adam-Moment for Practical and Scalable SOAPPapers · 12 Nov 2026 · Jaemyung Yu, Byeongho Heo, Sangdoo Yun, Dongyoon Han
Latest sources
- Do Pretraining Optimizer Gains Transfer to KL Distillation?Papers · 14 Nov 2026 · Lavinia Ghita, Dhruv Desai
- Factored One-sided Adam-Moment for Practical and Scalable SOAPPapers · 12 Nov 2026 · Jaemyung Yu, Byeongho Heo, Sangdoo Yun, Dongyoon Han
- Breaking Precision Walls in First-, Second-, and Zeroth-Order Optimizers: One Constant, Set in AdvancePapers · 7 Nov 2026 · Lngel Kyo
- Stochastic optimization methods through statistical whiteningPapers · 17 Oct 2026 · Tetiana Parshakova, Robert M. Gower, Paulo Orenstein
- Minimax Optimality of Shampoo for Nonstationary Stochastic OptimizationPapers · 12 Oct 2026 · Abir Sarkar, Nirmalya Mandal, Martin T. Wells
- Efficient Preconditioned Optimization Methods with Frequent DirectionsPapers · 12 Oct 2026 · Sifan Yang, Yuanyu Wan, Yibo Wang, Lijun Zhang
- How Bregman Divergences Shape ShampooPapers · 6 Oct 2026 · Bing Liu, Wenjie Zhou, Chengcheng Zhao, Hongtao Zhang, Boao Kong, Felix Dangel,…