phenomenon · first seen 23 Oct 2026
retrieval heads
Attention heads specialized for locating and copying relevant tokens from long contexts.
Mentions per week
PapersCodeDiscussion
Earliest sources
- Head Dimension Matters in Language Models for Long-Context RetrievalPapers · 23 Oct 2026 · Seiya Tokui, Shoichiro Yamaguchi, Sosuke Kobayashi
- A Broader Look into MLLM's Retrieval HeadsPapers · 23 Oct 2026 · Jia Wang, Yuqing Li, Jiangnan Li, Zheng Lin, Mo Yu
- Re$^3$Rank: Listwise Reranking with Reasoning-Conditioned Retrieval HeadsPapers · 2 Nov 2026 · Ruiyi Yan, Zhuoyuan Mao, Yugo Murawaki, Yiwen Guo
Latest sources
- Re$^3$Rank: Listwise Reranking with Reasoning-Conditioned Retrieval HeadsPapers · 2 Nov 2026 · Ruiyi Yan, Zhuoyuan Mao, Yugo Murawaki, Yiwen Guo
- A Broader Look into MLLM's Retrieval HeadsPapers · 23 Oct 2026 · Jia Wang, Yuqing Li, Jiangnan Li, Zheng Lin, Mo Yu
- Head Dimension Matters in Language Models for Long-Context RetrievalPapers · 23 Oct 2026 · Seiya Tokui, Shoichiro Yamaguchi, Sosuke Kobayashi