method · first seen 6 Oct 2026
latent attention
Also: MLA, multi-head latent attention
Attention variant that compresses older key-value entries into compact latent vectors to reduce KV cache size.
Mentions per week
PapersCodeDiscussion
Rank 42 on 16 Nov 2026 · 6 mentions in 28 days · 6 independent groups · papers → code · first seen 6 weeks ago · acceleration +1.02
Earliest sources
- Hybrid Latent Attention for Looped Language ModelsPapers · 6 Oct 2026 · Yuhan Chen, Siyuan Zhang, Nan Wang, Feiyang Kang, Ruoxi Jia
- MLAfication: Efficiently Enabling High-Fidelity MLA for Modern LLMsPapers · 12 Oct 2026 · Zile Wang, Zhide Xie, Tao Ji
- RoLaQ: Joint Rotation and Range Calibration for 2-Bit KV Caches in Multi-Head Latent AttentionPapers · 14 Oct 2026 · Shiyu Ji, Peng Ling, Chen Lin, Shupeng Li, Yixuan Wang, Qingfu Zhu, Wanxiang Che
- When the KV Cache Is Slower Than the GPU: Scaling Remote KV Tiers for Context-Parallel PrefillPapers · 15 Oct 2026 · Muhammad Ibrahim
- RAQ-MLA: Role-Aware Quantization for Multi-Head Latent Attention CachesPapers · 24 Oct 2026 · Tianqi Chen, Xunchao Li, Linchuan Xie, Guanghua Yu, Jianchen Zhu, Peisong Wang,…
- Where Position Matters: Profiling-Guided Layer-Wise RoPE Allocation in TransformersPapers · 29 Oct 2026 · Zhiwei Liu, Boao Kong, Bing Liu
Latest sources
- Vaibhaav-Tiwari/deepseek-v3-jaxCode · 13 Nov 2026 · Vaibhaav-Tiwari · 238 stars
- NVFP4-MLA: Unified FP4 KV-Cache Storage and Attention Computation for Multi-Head Latent AttentionPapers · 13 Nov 2026 · Chao Ni, Qi Zhang, Xiao Wang, Zongfei Jing, Mingyang Hao, Binghan Chen, Kefeng D…
- KVPath: Connecting Key-Value Representations Across Transformer LayersPapers · 11 Nov 2026 · Xuan Luo, Zhiyuan Wang, Hanzhong Cao, Xifeng Yan
- Translating KV Caches Between Hybrid-Attention Language Models: MLA Latents, Linear-Attention States and Sparse-Attention IndexersPapers · 9 Nov 2026 · Povilas Kvedaras, Tomas Rasymas
- Where Position Matters: Profiling-Guided Layer-Wise RoPE Allocation in TransformersPapers · 29 Oct 2026 · Zhiwei Liu, Boao Kong, Bing Liu
- RAQ-MLA: Role-Aware Quantization for Multi-Head Latent Attention CachesPapers · 24 Oct 2026 · Tianqi Chen, Xunchao Li, Linchuan Xie, Guanghua Yu, Jianchen Zhu, Peisong Wang,…
- When the KV Cache Is Slower Than the GPU: Scaling Remote KV Tiers for Context-Parallel PrefillPapers · 15 Oct 2026 · Muhammad Ibrahim
- RoLaQ: Joint Rotation and Range Calibration for 2-Bit KV Caches in Multi-Head Latent AttentionPapers · 14 Oct 2026 · Shiyu Ji, Peng Ling, Chen Lin, Shupeng Li, Yixuan Wang, Qingfu Zhu, Wanxiang Che
- MLAfication: Efficiently Enabling High-Fidelity MLA for Modern LLMsPapers · 12 Oct 2026 · Zile Wang, Zhide Xie, Tao Ji
- Hybrid Latent Attention for Looped Language ModelsPapers · 6 Oct 2026 · Yuhan Chen, Siyuan Zhang, Nan Wang, Feiyang Kang, Ruoxi Jia