architecture · first seen 13 Oct 2026
grouped-query attention
Also: GQA
An attention variant where groups of query heads share a single key and value head to reduce KV cache size.
Mentions per week
PapersCodeDiscussion
Earliest sources
- Virtual Head AttentionPapers · 13 Oct 2026 · Xibo Ding, Guoxia Wang, Jinle Zeng, Jiabin Yang, Sirui Tao, Chao Yang, Dianhai Y…
- Joint Subspace Sparse Attention for Efficient Long-Context DecodingPapers · 13 Oct 2026 · Jiwon Song, Yoongon Kim, Jae-Joon Kim
- HobbyLM: A Sparse Mixture-of-Experts Model Family on a Hobby BudgetPapers · 25 Oct 2026 · Harish Santhanalakshmi Ganesan, Hariharan M. S., Prabhurajhan R
- Parallelism-Aligned Pruning and Round-Aware Context Reuse for On-Device Multimodal AgentsPapers · 2 Nov 2026 · Peipei Jiao, Mingkang Liu, Weiming Meng, Lichen Pang, Tianyuan Liu, Junyan Li
- The SIMPLE Law for Energy Cost of LLM InferencePapers · 2 Nov 2026 · Hiari Pizzini Cavagna, Andrea Proia, Daniele Cesarini, Zeynep Kiziltan, Andrea B…
- TailDART: Compute-Efficient Foundation Models for Financial Time-Series ForecastingPapers · 6 Nov 2026 · Chiang Qin Zhi, Ranjan Satapathy, Erik Cambria, Joyjit Chattoraj
Latest sources
- KVPath: Connecting Key-Value Representations Across Transformer LayersPapers · 11 Nov 2026 · Xuan Luo, Zhiyuan Wang, Hanzhong Cao, Xifeng Yan
- TailDART: Compute-Efficient Foundation Models for Financial Time-Series ForecastingPapers · 6 Nov 2026 · Chiang Qin Zhi, Ranjan Satapathy, Erik Cambria, Joyjit Chattoraj
- The SIMPLE Law for Energy Cost of LLM InferencePapers · 2 Nov 2026 · Hiari Pizzini Cavagna, Andrea Proia, Daniele Cesarini, Zeynep Kiziltan, Andrea B…
- Parallelism-Aligned Pruning and Round-Aware Context Reuse for On-Device Multimodal AgentsPapers · 2 Nov 2026 · Peipei Jiao, Mingkang Liu, Weiming Meng, Lichen Pang, Tianyuan Liu, Junyan Li
- HobbyLM: A Sparse Mixture-of-Experts Model Family on a Hobby BudgetPapers · 25 Oct 2026 · Harish Santhanalakshmi Ganesan, Hariharan M. S., Prabhurajhan R
- Joint Subspace Sparse Attention for Efficient Long-Context DecodingPapers · 13 Oct 2026 · Jiwon Song, Yoongon Kim, Jae-Joon Kim
- Virtual Head AttentionPapers · 13 Oct 2026 · Xibo Ding, Guoxia Wang, Jinle Zeng, Jiabin Yang, Sirui Tao, Chao Yang, Dianhai Y…