method · first seen 8 Oct 2026
muP
Also: μP, maximal update parametrization, maximal update parameterization, feature learning in neural networks
A parametrization scheme that allows hyperparameters to transfer across model width so optimal settings remain stable at scale.
Mentions per week
PapersCodeDiscussion
Earliest sources
- $σ$Transfer: Uncertainty Transfer from Small to Large Networks under $μ\mathrm{P}$Papers · 8 Oct 2026 · Richard Bergna, Fernando Ruiz Mazo, Nicolò Felicioni, José Miguel Hernández-L…
- On the Depth Transfer of Neural Optimizers via L-SmoothnessPapers · 9 Oct 2026 · Maximilian Wong, Chenyu Xu, Jiajin Li, Yiping Lu
- ShellElastic: Scalable Elastic Pretraining with Simultaneous Maximal Updates and Hyperparameter TransferPapers · 10 Oct 2026 · Can Jin, Yan Kang, Yifan Gong, Chongjian Ge, Mingze Xu, Yuanjun Xiong, Dimitris…
- Towards Hyperparameter Transfer for Differentially Private OptimizationPapers · 16 Oct 2026 · Tianze Wang, Zhiqi Bu, Linjun Zhang
- Width and Depth Limits Commute During Training in Linear Residual NetworksPapers · 22 Oct 2026 · Ruijia Zhang, Soufiane Hayou
- Sliding Networks: A Family of Residual Structures Preserving Feature Learning and Feature DiversityPapers · 24 Oct 2026 · Datong Zhou, Blake Bordelon
Latest sources
- Sliding Networks: A Family of Residual Structures Preserving Feature Learning and Feature DiversityPapers · 24 Oct 2026 · Datong Zhou, Blake Bordelon
- Width and Depth Limits Commute During Training in Linear Residual NetworksPapers · 22 Oct 2026 · Ruijia Zhang, Soufiane Hayou
- Towards Hyperparameter Transfer for Differentially Private OptimizationPapers · 16 Oct 2026 · Tianze Wang, Zhiqi Bu, Linjun Zhang
- ShellElastic: Scalable Elastic Pretraining with Simultaneous Maximal Updates and Hyperparameter TransferPapers · 10 Oct 2026 · Can Jin, Yan Kang, Yifan Gong, Chongjian Ge, Mingze Xu, Yuanjun Xiong, Dimitris…
- On the Depth Transfer of Neural Optimizers via L-SmoothnessPapers · 9 Oct 2026 · Maximilian Wong, Chenyu Xu, Jiajin Li, Yiping Lu
- $σ$Transfer: Uncertainty Transfer from Small to Large Networks under $μ\mathrm{P}$Papers · 8 Oct 2026 · Richard Bergna, Fernando Ruiz Mazo, Nicolò Felicioni, José Miguel Hernández-L…