trends.barnas.net

phenomenon · first seen 10 Nov 2026

linear attention sink

Concentration of queries, keys and values of linear attention layers in a stable low-dimensional subspace.

Mentions per week

0119 May11 Aug10 NovPapersCodeDiscussion

PapersCodeDiscussion

Earliest sources

Latest sources

Mentioned together with

hybrid attention modelslow-rank state compression