method · first seen 7 Oct 2026
all-to-all communication
Also: A2A communication
Collective communication pattern where every device exchanges data with every other device, used in distributed MoE training.
Mentions per week
PapersCodeDiscussion
Earliest sources
- Expert Coupling in MoE Pretraining: Reducing All-to-All Overhead with Correlated Placement and Token ShufflingPapers · 7 Oct 2026 · Radha Gulhane, Quentin Anthony, Beren Millidge
- Accelerating MoE Training through Expert Placement and Replica CoordinationPapers · 10 Oct 2026 · Zhiqiang Tan, Maoxin Wang, Wenxiang Lin, Sijie Wang, Qiang Wang, Shaohuai Shi, B…
Latest sources
- Accelerating MoE Training through Expert Placement and Replica CoordinationPapers · 10 Oct 2026 · Zhiqiang Tan, Maoxin Wang, Wenxiang Lin, Sijie Wang, Qiang Wang, Shaohuai Shi, B…
- Expert Coupling in MoE Pretraining: Reducing All-to-All Overhead with Correlated Placement and Token ShufflingPapers · 7 Oct 2026 · Radha Gulhane, Quentin Anthony, Beren Millidge