trends.barnas.net

architecture · first seen 7 Oct 2026

multi-head self-attention

Also: MHA, MHSA, multi-head attention

Attention mechanism running several attention heads in parallel over projected query, key and value representations.

Mentions per week

0219 May11 Aug10 NovPapersCodeDiscussion

PapersCodeDiscussion

Earliest sources

Latest sources

Mentioned together with

Gaussian equivalenceintrinsic dimensionLean theorem proverlong-context language modelingretrieval heads