architecture · first seen 5 Oct 2026
linear attention
Also: linear transformers, linear recurrent neural network
Attention mechanism that replaces softmax with a kernel so cost scales linearly with sequence length.
Mentions per week
PapersCodeDiscussion
Rank 63 on 16 Nov 2026 · 24 mentions in 28 days · 24 independent groups · papers → code · first seen 6 weeks ago · acceleration -0.18
Earliest sources
- HLA-WM: Hybrid Linear Attention for Long-Horizon Video World ModelsPapers · 5 Oct 2026 · Zhuokun Chen, Feng Chen, Xi Lin, Xiyu Wu, Jiahao He, Jianfei Cai, Bohan Zhuang
- HLA: Expressive Hybrid Linear Attention via Chunk-Wise Dynamic MixingPapers · 5 Oct 2026 · Zhuokun Chen, Xi Lin, Xiyu Wu, Jiahao He, Jianfei Cai, Bohan Zhuang
- Mechanics of Long-Context Hybrid Models Part 1.1: From Hybrid Attention to Hybrid PositionPapers · 7 Oct 2026 · Xiaoran Liu, Ziwei He, Xipeng Qiu
- What can linear attention learn from nonlinear teachers in-context?Papers · 7 Oct 2026 · Mary Letey, Arman Rysmakhanov, Yue M. Lu, Cengiz Pehlevan, Jacob Zavatone-Veth
- Bridging KV-Cache Quantization and Linear Attention: From Theory to Pretrained Weight MigrationPapers · 8 Oct 2026 · Kaicheng Xiao, Liran Dong, Haotian Li, Guoliang Xing
- V-CoLA: Vision Token Compression with Linear AttentionPapers · 8 Oct 2026 · Hao Jiang, Yiru Mao, Tianpeng Bu, Hao Zhou, Hongtao Duan, Wang Jing, Bowen Xu, X…
Latest sources
- Bilateral Log-Linear AttentionPapers · 13 Nov 2026 · Kazi Ashik Islam, Guojing Cong
- leapquant/LeapQuantCode · 11 Nov 2026 · leapquant · 59 stars
- Translating KV Caches Between Hybrid-Attention Language Models: MLA Latents, Linear-Attention States and Sparse-Attention IndexersPapers · 9 Nov 2026 · Povilas Kvedaras, Tomas Rasymas
- Polynomial Denoisers on the Image Manifold: Volterra Networks for Self-Contained Pixel DiffusionPapers · 7 Nov 2026 · Siddharth Roheda, Rohit Chowdhury, Amit Satish Unde, Aniruddha Bala
- LEMMO: A Large Electromagnetic Model leveraging Pretrained LLMs for Million-Sample Signal UnderstandingPapers · 6 Nov 2026 · Mingkai Li, Jiajun Wu, Yihan Mao, Zilin Sang, Taosong Yu, Yuan Ma, Minghe Liu, Q…
- Muir: Linear Attention with Sensitivity-Allocated Gating and Matched-Filter ReadoutPapers · 5 Nov 2026 · Tushaar Gangavarapu, Gil Tabak, Clemens JS Schaefer
- SPARE: Fast Linear Attention Decoding with Compact Recurrent StatesPapers · 5 Nov 2026 · Ke Wan, Zaiyi Zheng, Chen Chen
- Systolic Linear Attention: Co-Designing Tile-Grouped Gated Delta Attention for TPUsPapers · 4 Nov 2026 · Gagik Amirkhanyan, Nuojin Cheng
- Shallow Encoding Emits Deep MemoriesPapers · 4 Nov 2026 · Mingyuan Ma, Yangmin Li, Harris Nover, Zheyu Fu, Po-Han Huang, Alex Settle, Ashw…
- LiSA: Linear Stochastic Attention for Long-Range Visual ModelingPapers · 3 Nov 2026 · Sahar Dastani, Fereshteh Shakeri, Samuel Barbeau, Kunal Mahatha, Ismail Ben Ayed…
- Who Asks, Who Answers: Linear Attention Builds the Query in Hybrid Language ModelsPapers · 1 Nov 2026 · Wenqi Zeng, Chen Feng, YuAng Chen
- FlyWheel: Attention Kernels Built for the TPUPapers · 1 Nov 2026 · Zishuo Bao, Zeshen Zhang, Linkun Chen, Yucheng Lu