phenomenon · first seen 9 Nov 2026
attention head specialization
Emergence of distinct, interpretable roles among attention heads in transformer models.
Mentions per week
PapersCodeDiscussion
Earliest sources
- Training Vision Transformers to Focus: Emergence of Spatial Attention Head SpecializationPapers · 9 Nov 2026 · Yuji Nozawa, Yu-Chieh Lin, Youyang Ng
Latest sources
- Training Vision Transformers to Focus: Emergence of Spatial Attention Head SpecializationPapers · 9 Nov 2026 · Yuji Nozawa, Yu-Chieh Lin, Youyang Ng