architecture · first seen 5 Oct 2026
hybrid attention-recurrent language models
Also: hybrid LLM, hybrid attention, interleaved attention, hybrid language models, hybrid linear-softmax attention, recurrent-attention hybrid models, hybrid attention-state space model, hybrid attention-recurrent architecture
Language models interleaving attention layers with recurrent layers to combine efficiency and recall.
Mentions per week
PapersCodeDiscussion
Earliest sources
- Balancing Memory Pathways: Analyzing and Improving Memory Utilization in Hybrid LMsPapers · 5 Oct 2026 · Hyunji Lee, Joykirat Singh, Zaid Khan, Justin Chih-Yao Chen, Elias Stengel-Eskin…
- PHBA: Prefix-State Hybrid Block AttentionPapers · 6 Oct 2026 · Ruijie Li, Jiaxi Hu, Shiyu Wang, Yuxuan Liang
- HART: Hybrid Attention-RWKV Time series Foundation ModelPapers · 9 Oct 2026 · Qingwen Lin, Boyan Xu, Shifeng Xie, Zhifeng Hao, Keli Zhang, Ruichu Cai
- GateSight: Recurrent Memory Anticipates Attention Memory for Query-Agnostic KV Eviction in Hybrid LLMsPapers · 10 Oct 2026 · Junjie Li, Jiong Lou, Jie Li
- RadixFlow: Filling Execution Bubbles in Prefix-Cached Hybrid LLM ServingPapers · 13 Oct 2026 · Huihao Jing, Wenbin Hu, Shaojin Chen, Haochen Shi, Haoyu Huang, Sirui Zhang, Yux…
- MixPT: Point Transformer with Interleaved Window and Global AttentionPapers · 22 Oct 2026 · Minkyun Seo, Hyungtae Lim, Jaesik Park
Latest sources
- UniSpark: Exploring Hybrid Attention for Multimodal Understanding and GenerationPapers · 12 Nov 2026 · Jiancheng Pan, Yuzheng Liu, Hengyuan Cao, Qiyuan Zhang, Yangyan Li, Biao Gong, Z…
- Shallow Encoding Emits Deep MemoriesPapers · 4 Nov 2026 · Mingyuan Ma, Yangmin Li, Harris Nover, Zheyu Fu, Po-Han Huang, Alex Settle, Ashw…
- Who Asks, Who Answers: Linear Attention Builds the Query in Hybrid Language ModelsPapers · 1 Nov 2026 · Wenqi Zeng, Chen Feng, YuAng Chen
- Request-Adaptive Execution for Efficient Hybrid LLM InferencePapers · 27 Oct 2026 · Fahao Chen, Xuzhe Yan, Qi Zhang, Peng Li, Dongxiao Yu, Song Guo
- Learning from Points and Patches: A Pre-trained Hybrid Attention Transformer for Time Series AnalysisPapers · 27 Oct 2026 · Peiwang Tang, Cheng HE, Yupeng Ye, Weitai Zhang, Qinghua Sun, Defu Lian
- Layerwise Repacking: Variable-Length Hybrid-Attention Training without an Extra Redistribution PassPapers · 26 Oct 2026 · Jiahao Zhang, Junwei Feng, Dongliang Huang, Xiaoling Zhou, Yiqiao Liao, Chengru…
- MixPT: Point Transformer with Interleaved Window and Global AttentionPapers · 22 Oct 2026 · Minkyun Seo, Hyungtae Lim, Jaesik Park
- RadixFlow: Filling Execution Bubbles in Prefix-Cached Hybrid LLM ServingPapers · 13 Oct 2026 · Huihao Jing, Wenbin Hu, Shaojin Chen, Haochen Shi, Haoyu Huang, Sirui Zhang, Yux…
- GateSight: Recurrent Memory Anticipates Attention Memory for Query-Agnostic KV Eviction in Hybrid LLMsPapers · 10 Oct 2026 · Junjie Li, Jiong Lou, Jie Li
- HART: Hybrid Attention-RWKV Time series Foundation ModelPapers · 9 Oct 2026 · Qingwen Lin, Boyan Xu, Shifeng Xie, Zhifeng Hao, Keli Zhang, Ruichu Cai
- PHBA: Prefix-State Hybrid Block AttentionPapers · 6 Oct 2026 · Ruijie Li, Jiaxi Hu, Shiyu Wang, Yuxuan Liang
- Balancing Memory Pathways: Analyzing and Improving Memory Utilization in Hybrid LMsPapers · 5 Oct 2026 · Hyunji Lee, Joykirat Singh, Zaid Khan, Justin Chih-Yao Chen, Elias Stengel-Eskin…