method · first seen 9 Oct 2026
quantized all-reduce
Also: low-bit communication compression
Compressing collective communication values to low bit widths to reduce distributed training and inference communication cost.
Mentions per week
PapersCodeDiscussion
Earliest sources
- Echo: Error-Compensated 4-Bit Tensor-Parallel Communication for LLM Training and InferencePapers · 9 Oct 2026 · Xingyu Xie, Xiaoyong Xie, Zhouchen Lin, Kim-Chuan Toh
Latest sources
- Echo: Error-Compensated 4-Bit Tensor-Parallel Communication for LLM Training and InferencePapers · 9 Oct 2026 · Xingyu Xie, Xiaoyong Xie, Zhouchen Lin, Kim-Chuan Toh