architecture · first seen 4 Oct 2026
mixture of experts
Also: MoE, MMoE, SMoE, Soft-MoE, Sparse MoE, expert mixture, soft mixture of experts
Neural network architecture that routes each input token to a small subset of specialist expert subnetworks.
Mixture of experts is a neural network design in which a learned router sends each input or token to a small subset of specialist subnetworks, so total parameters grow without a matching rise in per-token compute. The sources show it as standard infrastructure, not a new idea. Papers cover router design (prototype-mediated routing with congestion pricing), on-device and CPU inference (cache-aware routing, matrix-engine kernels), and domain uses in video restoration, anomaly detection, multimodal learning and robotics. Several GitHub repos focus on running large MoE models such as DeepSeek and GLM locally, streaming experts from SSD or running them on CPUs. Weekly mentions are falling after an initial spike.
The concept has been mainstream since sparse MoE language models became common, so the apparent rise most likely reflects ongoing volume rather than new adoption. The active work is now in efficient local inference and better routing, not in the architecture itself.
Analyst note (established), 17 Nov 2026.
Mentions per week
PapersCodeDiscussion
Rank 24 on 14 Nov 2026 · 204 mentions in 28 days · 202 independent groups · papers → code → discussion · first seen 6 weeks ago · acceleration -0.23
Earliest sources
- LoopMoEVR: Loop-Based Degradation-Aware Mixture-of-Experts for Unified UHD Video RestorationPapers · 4 Oct 2026 · Yucheng Xin, Runci Bai, Yongcong Wang, Guangwei Gao, Jiao Liu, Dianjie Lu, Linwe…
- InstMoE: Adaptive Multimodal Routing with Specialized ExpertsPapers · 4 Oct 2026 · Guimin Hu, Xiang He, Yingjian Li, Zheng Lian, Boyan Xu, Ruichu Cai
- HiNa-MoE: High-Performance, Non-Intrusive MoE Inference on CPUs with Matrix EnginesPapers · 4 Oct 2026 · Weiling Yang, Junwen Zhang, Dezun Dong, Jianbin Fang, Enda Yu, Zhe Bai, Xiaopeng…
- RoMod: Temporal Routing Modulation via Mixture-of-Experts for Video Anomaly DetectionPapers · 4 Oct 2026 · Chao Huang, Pengfei Wei, Benfeng Wang, Chengliang Liu, Wei Wang, Li Shen, Wenqi…
- Measuring and Reducing Cross-Vendor Mismatch in Language ModelsPapers · 4 Oct 2026 · Erland Hilman Fuadi, Chong Tian, Xiaosong Ma, Qirong Ho
- CodeForge-MA: Execution-Verified Multi-Agent Learning with Language-Conditioned LoRA for Multilingual Code GenerationPapers · 4 Oct 2026 · Zhizhou Gu, Xianting Wu, Siyu Gu, Tian Zhang, Kejian Tong
Latest sources
- Specificity-Guided Knowledge Diversion for Decomposable Generative ModelsPapers · 16 Nov 2026 · Yucheng Xie, Fu Feng, Jing Wang, Xin Geng, Yong Rui
- TEMPO: Regime-Aware Operator Learning with Locally Adapted POD BasesPapers · 16 Nov 2026 · Rodion Akinzhala, Alexander Terentyev, Alexey Golubev, Yury Maximov
- Sage S2.1 Flash Technical ReportPapers · 16 Nov 2026 · Kun Cai, Haodong Chen, Guangran Cheng, Qingxiu Cheng, Xu Dai, Guanting Dong, Qim…
- K2-Horizon: A Frontier Fleet of Open Source Language ModelsPapers · 16 Nov 2026 · K2 Team, Zhengzhong Liu, Haonan Li, Xuezhe Ma, Alexander Moreno, Subham Sekhar S…
- ProSE: Probing Reconstruction Error for Semantic-guided Expert Skipping in MoE-MLLMsPapers · 16 Nov 2026 · Jiameng Li, Minye Wu, Matthew B. Blaschko
- UNITE: Consolidating Embodied VLM Specialists via Shared-Anchor One-Bit FactorizationPapers · 16 Nov 2026 · Tao Tao, Anke Tang, Enneng Yang, Zhi Hou, Xiaogang Wang, Dacheng Tao
- ParGLU: Partial Gating Improves Language Modeling in MoE TransformersPapers · 16 Nov 2026 · Yuting Ge
- GEAR$^2$: Pruned Experts Live On through Gated Reuse of Activated Experts and a Missing-Mass ResidualPapers · 16 Nov 2026 · Linliang Chen, Yan Zhong, Wang Kang, Sai Li, Ji Wu, Ji Liu
- CECAR: Cache & Expert Co-Aware Routing Accelerates On-Device Inference of MoE LLMsPapers · 15 Nov 2026 · Byeongju Kim, Hoonki Lee, Byungjun Kim, Inyul Ra, Yongjin Kim, Yongseong Lee, Sa…
- Multimodal AI in Healthcare: Review of Vision-Language Foundation Models for Real-World Medical ApplicationsPapers · 15 Nov 2026 · Taha Razzaq, Murtaza Taj, Asim Iqbal
- DiffWAM: Learning World Action Model from Consecutive Visual DifferencePapers · 15 Nov 2026 · Xinheng Zhang, Haoyu Xiong, Chao Jing, Xiaohan Zhang, Linshan Li, Hongbang Fan,…
- Multi-Branch TransformerPapers · 15 Nov 2026 · Gaotang Li, Lingjie Chen, Chunyuan Deng, Zhiwei Xu, Hongzheng Chen, Christopher…