phenomenon · first seen 10 Nov 2026
linear attention sink
Concentration of queries, keys and values of linear attention layers in a stable low-dimensional subspace.
Mentions per week
PapersCodeDiscussion
Earliest sources
- SinkState: On-the-Fly SVD of Recurrent States for Efficient Hybrid LLM ServingPapers · 10 Nov 2026 · Haolin Jiang, Weihang Li, Zhongzhu Zhou, Jinwen Pan, Santosh Pandey, Hang Liu
Latest sources
- SinkState: On-the-Fly SVD of Recurrent States for Efficient Hybrid LLM ServingPapers · 10 Nov 2026 · Haolin Jiang, Weihang Li, Zhongzhu Zhou, Jinwen Pan, Santosh Pandey, Hang Liu