architecture · first seen 24 Oct 2026
mixture of bidirectional and autoregressive attention
Also: MoBA
An attention mechanism that mixes bidirectional and causal attention so a model can be made autoregressive without losing pretrained priors.
Mentions per week
PapersCodeDiscussion
Earliest sources
- Mix Forcing: Causal Pre-Training via Mixture Attention for Interactive World ModelsPapers · 24 Oct 2026 · Zelin Gao, Hao Ouyang, Jiapeng Zhu, Qiuyu Wang, Jingye Chen, Zichen Liu, Xing Zh…
Latest sources
- Mix Forcing: Causal Pre-Training via Mixture Attention for Interactive World ModelsPapers · 24 Oct 2026 · Zelin Gao, Hao Ouyang, Jiapeng Zhu, Qiuyu Wang, Jingye Chen, Zichen Liu, Xing Zh…