architecture · first seen 8 Oct 2026
attention residuals
Also: AttnRes
Residual mechanism performing softmax attention over network depth to retrieve earlier layer representations.
Mentions per week
PapersCodeDiscussion
Earliest sources
- jianke0604/MSARCode · 8 Oct 2026 · jianke0604 · 43 stars
- Multi-Stream Attention ResidualsPapers · 9 Oct 2026 · Xiaohan Qin, Xiangdong Zhang, Tuo Dai, Sunan Zou, Huaijin Wu, Yebin Yang, Zhuo X…
- Depth MemoryPapers · 13 Oct 2026 · David González-Martínez, Philipp Nazari, T. Konstantin Rusch, Shiwei Liu
- ARLoop: Fully-Open Pre-trained Looped Transformers with Attention ResidualsPapers · 3 Nov 2026 · Yuekun Yao, Yupei Du, Dawei Zhu, Huan Sun
Latest sources
- ARLoop: Fully-Open Pre-trained Looped Transformers with Attention ResidualsPapers · 3 Nov 2026 · Yuekun Yao, Yupei Du, Dawei Zhu, Huan Sun
- Depth MemoryPapers · 13 Oct 2026 · David González-Martínez, Philipp Nazari, T. Konstantin Rusch, Shiwei Liu
- Multi-Stream Attention ResidualsPapers · 9 Oct 2026 · Xiaohan Qin, Xiangdong Zhang, Tuo Dai, Sunan Zou, Huaijin Wu, Yebin Yang, Zhuo X…
- jianke0604/MSARCode · 8 Oct 2026 · jianke0604 · 43 stars