phenomenon · first seen 6 Oct 2026
hyperparameter transfer
Also: proxy model search transfer
Reusing hyperparameters tuned on small proxy models for larger models by scaling rules that keep training dynamics stable.
Mentions per week
PapersCodeDiscussion
Earliest sources
- $α$Transfer: Coefficient Transfer for Efficient Model MergingPapers · 6 Oct 2026 · Shih-Cheng Huang, Zhi Rui Tam, Chieh-Yen Lin, Yun-Nung Chen, Hung-yi Lee, Shao-H…
- $σ$Transfer: Uncertainty Transfer from Small to Large Networks under $μ\mathrm{P}$Papers · 8 Oct 2026 · Richard Bergna, Fernando Ruiz Mazo, Nicolò Felicioni, José Miguel Hernández-L…
- On the Depth Transfer of Neural Optimizers via L-SmoothnessPapers · 9 Oct 2026 · Maximilian Wong, Chenyu Xu, Jiajin Li, Yiping Lu
- Improving Data-Constrained Scaling with Hyperparameter Transfer and EnsemblingPapers · 27 Oct 2026 · Ethan Baron, Shmuel Berman, Anirudh Satheesh, Akshay Vegesna, Samip Dahal, Andre…
- Learning Rate Transfer and Feature Learning Across Depth for Constrained Spectral Optimizers: Complete ScionPapers · 11 Nov 2026 · Eline Pot, Laurent Lefebvre, Mathieu Segond, Antonio Silveti-Falls
Latest sources
- Learning Rate Transfer and Feature Learning Across Depth for Constrained Spectral Optimizers: Complete ScionPapers · 11 Nov 2026 · Eline Pot, Laurent Lefebvre, Mathieu Segond, Antonio Silveti-Falls
- Improving Data-Constrained Scaling with Hyperparameter Transfer and EnsemblingPapers · 27 Oct 2026 · Ethan Baron, Shmuel Berman, Anirudh Satheesh, Akshay Vegesna, Samip Dahal, Andre…
- On the Depth Transfer of Neural Optimizers via L-SmoothnessPapers · 9 Oct 2026 · Maximilian Wong, Chenyu Xu, Jiajin Li, Yiping Lu
- $σ$Transfer: Uncertainty Transfer from Small to Large Networks under $μ\mathrm{P}$Papers · 8 Oct 2026 · Richard Bergna, Fernando Ruiz Mazo, Nicolò Felicioni, José Miguel Hernández-L…
- $α$Transfer: Coefficient Transfer for Efficient Model MergingPapers · 6 Oct 2026 · Shih-Cheng Huang, Zhi Rui Tam, Chieh-Yen Lin, Yun-Nung Chen, Hung-yi Lee, Shao-H…