method · first seen 15 Oct 2026
load balancing in distributed training
Also: workload balancing, data-parallel load imbalance
Distributing work across data-parallel ranks so that each step's compute cost is roughly equal and idle time is minimised.
Mentions per week
PapersCodeDiscussion
Earliest sources
- Coordination-Free Workload Regularization for Variable-Cost Distributed TrainingPapers · 15 Oct 2026 · Tao Huang, Chen Zhu, Antonio Alonso, Daming Li, Adarsh Goura, Jaideep Ray, Yiton…
Latest sources
- Coordination-Free Workload Regularization for Variable-Cost Distributed TrainingPapers · 15 Oct 2026 · Tao Huang, Chen Zhu, Antonio Alonso, Daming Li, Adarsh Goura, Jaideep Ray, Yiton…