architecture · first seen 6 Oct 2026
DeltaNet
Also: delta rule linear attention
A linear-attention recurrent architecture that updates a memory matrix with a delta rule.
Mentions per week
PapersCodeDiscussion
Earliest sources
- DeltaTTT: Layerwise Optimization for Nonlinear Recurrent MemoryPapers · 6 Oct 2026 · Yining Li, Dongchen Han, Jie Fu, Gao Huang
- DeltaNet-PC: Test-Time Memory Without Backpropagation Through TimePapers · 10 Oct 2026 · Todd Morrill, Richard Zemel
- ReWY: Faster Backpropagation for Delta-Rule Attention - From 7 to 3 Matrix MultiplicationsPapers · 1 Nov 2026 · Yi Zhang, Zhixin Zhao, Weiwei Sun, Yitao Hu, Sheng Chen, Keqiu Li
Latest sources
- ReWY: Faster Backpropagation for Delta-Rule Attention - From 7 to 3 Matrix MultiplicationsPapers · 1 Nov 2026 · Yi Zhang, Zhixin Zhao, Weiwei Sun, Yitao Hu, Sheng Chen, Keqiu Li
- DeltaNet-PC: Test-Time Memory Without Backpropagation Through TimePapers · 10 Oct 2026 · Todd Morrill, Richard Zemel
- DeltaTTT: Layerwise Optimization for Nonlinear Recurrent MemoryPapers · 6 Oct 2026 · Yining Li, Dongchen Han, Jie Fu, Gao Huang