method · first seen 13 Oct 2026
context parallelism
Also: prefill context parallelism
Distributing long sequences across devices by splitting the sequence dimension for attention computation.
Mentions per week
PapersCodeDiscussion
Earliest sources
- FlashMask-3: Efficient and Expressive Mask-Aware Distributed AttentionPapers · 13 Oct 2026 · Guoxia Wang, Qianyue He, Siming Wu, Haoyang Xie, Qiwen Bao, Jinle Zeng, Jiabin Y…
- When the KV Cache Is Slower Than the GPU: Scaling Remote KV Tiers for Context-Parallel PrefillPapers · 15 Oct 2026 · Muhammad Ibrahim
- LiquidCP: Decoupling Attention Tasks from Communication and Kernel Boundaries in Context ParallelismPapers · 3 Nov 2026 · Hongyu Chen, Yikang Ruan, Yuhai Long, Ziyu Huang, Yiyu Luo, Liukun Yu, Yangjie Z…
- Split-KV: Efficient Context Parallelism for Training Diffusion DraftersPapers · 4 Nov 2026 · Geng Zhang, Fangzhao Zhang, Cong Xie, Tiantian Fan, Yang You
Latest sources
- Split-KV: Efficient Context Parallelism for Training Diffusion DraftersPapers · 4 Nov 2026 · Geng Zhang, Fangzhao Zhang, Cong Xie, Tiantian Fan, Yang You
- LiquidCP: Decoupling Attention Tasks from Communication and Kernel Boundaries in Context ParallelismPapers · 3 Nov 2026 · Hongyu Chen, Yikang Ruan, Yuhai Long, Ziyu Huang, Yiyu Luo, Liukun Yu, Yangjie Z…
- When the KV Cache Is Slower Than the GPU: Scaling Remote KV Tiers for Context-Parallel PrefillPapers · 15 Oct 2026 · Muhammad Ibrahim
- FlashMask-3: Efficient and Expressive Mask-Aware Distributed AttentionPapers · 13 Oct 2026 · Guoxia Wang, Qianyue He, Siming Wu, Haoyang Xie, Qiwen Bao, Jinle Zeng, Jiabin Y…