trends.barnas.net

architecture · first seen 24 Oct 2026

mixture of bidirectional and autoregressive attention

Also: MoBA

An attention mechanism that mixes bidirectional and causal attention so a model can be made autoregressive without losing pretrained priors.

Mentions per week

0119 May11 Aug10 NovPapersCodeDiscussion

PapersCodeDiscussion

Earliest sources

Latest sources

Mentioned together with

distribution matching distillationinteractive world modelteacher forcing