method · first seen 10 Oct 2026
sequence parallelism
Also: SP
Splitting long input sequences across devices so that activations for a single sequence are distributed.
Mentions per week
PapersCodeDiscussion
Earliest sources
- BSIPipe: Bidirectional Sequence-Level Interleaved Pipeline Parallelism for Long-Sequence Large Language Model TrainingPapers · 10 Oct 2026 · Yuzhou Ni, Ling Chen
- Layerwise Repacking: Variable-Length Hybrid-Attention Training without an Extra Redistribution PassPapers · 26 Oct 2026 · Jiahao Zhang, Junwei Feng, Dongliang Huang, Xiaoling Zhou, Yiqiao Liao, Chengru…
- CHARM+: Cross-Hardware Attention with Re-Merge Multistream MechanismPapers · 4 Nov 2026 · Xinhang Zhang, Boning Zhang, Chengchun Liu, Chunpu Li, Lei Sun, Weifeng Zhang, L…
Latest sources
- CHARM+: Cross-Hardware Attention with Re-Merge Multistream MechanismPapers · 4 Nov 2026 · Xinhang Zhang, Boning Zhang, Chengchun Liu, Chunpu Li, Lei Sun, Weifeng Zhang, L…
- Layerwise Repacking: Variable-Length Hybrid-Attention Training without an Extra Redistribution PassPapers · 26 Oct 2026 · Jiahao Zhang, Junwei Feng, Dongliang Huang, Xiaoling Zhou, Yiqiao Liao, Chengru…
- BSIPipe: Bidirectional Sequence-Level Interleaved Pipeline Parallelism for Long-Sequence Large Language Model TrainingPapers · 10 Oct 2026 · Yuzhou Ni, Ling Chen