architecture · first seen 4 Oct 2026
attention mechanism
Also: self-attention, cross-attention, cross-image attention, transformer attention, cross-representation attention
Neural operation that weights and aggregates sequence elements by learned query-key relevance scores.
The attention mechanism weights and combines sequence elements using learned query-key relevance scores, and it has been the core of transformer models since 2017. The sources here use it as standard machinery rather than introduce it. Papers apply self- or cross-attention to video and audio generation, time-series forecasting, seismic velocity inversion, recommendation, medical VQA and video segmentation. Others study its theory or variants: a mathematical analysis of transformers as measure-to-measure maps, a critique of QK-normalization, an oscillator-based reformulation, and memorization localized in a few cross-attention heads. Discussion posts and a Chinese course repo treat it as educational material.
Attention is already a foundational building block across deep learning, so the apparent rise reflects ordinary publication volume rather than new adoption. The more specific threads, such as the QK-norm critique and memorization in attention heads, are better tracked as their own concepts.
Analyst note (established), 17 Nov 2026.
Mentions per week
PapersCodeDiscussion
Rank 58 on 14 Nov 2026 · 34 mentions in 28 days · 34 independent groups · papers → discussion → code · first seen 6 weeks ago · acceleration +0.04
Earliest sources
- Kandinsky 6.0 Video: Foundation Models for Synchronized Video and Audio GenerationPapers · 4 Oct 2026 · Team Kandinsky, Julia Agafonova, Bulat Akhmatov, Mikhail Aksyutin, Grigorii Alek…
- When Attention Does Not Explain the Peak: Temporal Reference vs. Forecast Output in Attention-Based Time-Series ForecastingPapers · 5 Oct 2026 · Yuji Akamatsu, Takao Yamanaka
- Stability of Measure-to-Measure Transformers on Sub-Gaussian DataPapers · 6 Oct 2026 · Frank Cole, Nicholas H. Nelsen, Takashi Furuya
- Self-attention summary networks for subsurface velocity-model building from common-image gathersPapers · 7 Oct 2026 · Shiqin Zeng, Yunlin Zeng, Abhinav Prakash Gahlot, Zijun Deng, Felix J. Herrmann
- SPD-MetaFormer is what you need for small-data brain decodingPapers · 7 Oct 2026 · Zhida Wang, Wei Lyu, Guo Yu, Sui Tang
- 4-Tensor Attention Model for Semantic Physical RealityPapers · 8 Oct 2026 · Jongwook Kim, Sangheon Yun
Latest sources
- Wildfire-SSR: Geolocation-Aware Multimodal Fusion for Structure-Level Wildfire Risk AssessmentPapers · 16 Nov 2026 · Yu-Hsuan Ho, Yiming Xiao, Hao-En Lai, Ali Mostafavi, Samuel D. Brody
- Self-attention is a Dyson equation for many-body machine learningPapers · 16 Nov 2026 · Jarvist Moore Frost
- Memorization Lives in a Few Attention Heads: Mitigating Memorization in Diffusion Models via Projection-Based SuppressionPapers · 14 Nov 2026 · Puwei Lian, Yujun Cai, Songze Li, Yuezu Lv, Guanghui Wen
- Unlocking Cross-Attention in Label-Scarce Regimes: A Weakly Supervised Approach for Video Semantic SegmentationPapers · 13 Nov 2026 · Andrea Marelli, Alessandro Ardenghi, Giacomo Boracchi, Luca Magri
- QCAP-VQA: Question-Conditioned Capped Prior-Mass Supervision for Medical Visual Question AnsweringPapers · 13 Nov 2026 · Yining Zhao, Catarina Moreira, Mukesh Prasad, Ali Braytee
- Everything in its right place: attention is a radio headPapers · 9 Nov 2026 · Lluís Arola-Fernández, Aissam Khadraoui, Alex Arenas
- An Adaptive, Context-Aware, and Stacked Attention Network-Based Recommendation System to Capture Users' Temporal PreferencePapers · 8 Nov 2026 · Jung-Hsien Chiang, Chung-Yao Ma, Chi-Shiang Wang, Pei-Yi Hao
- Large Language Models without QK-NormalizationPapers · 7 Nov 2026 · Yongding Tao, Yitong Qiao, Tianqi Shao, Tian Wang, Zhixuan Chu, Xiaoming Wu, Yih…
- Scaling Optimization-Oriented Hypernetworks for Implicit Neural RepresentationsPapers · 7 Nov 2026 · Lulu Cai, Hanxiang Ren, Siyan Dong, Li Sun, Jiefeng Wu, Youyi Zheng, Yi Ma, Yanc…
- Guided Co-Attention with Adaptive Gating for Multimodal Medical Data FusionPapers · 7 Nov 2026 · Muntaqim Ahmed Raju, Priyanka Siddappa, Md Shifat Haider Al Amin, Ruizhe Ma, Md…
- MSR-ABC: Complementary Feature Fusion for Clear Cell Renal Cell Carcinoma ClassificationPapers · 6 Nov 2026 · Yuan Liang, Sourav Bhattacharjee, Abraham Campbell
- KineCouple: Substrate-conditioned residual readout for enzyme kinetic predictionPapers · 6 Nov 2026 · Chun Kang, Yunbo Zhang, Fanhu Zeng, Jian Liu, Siwei Deng, Lei Sha