method · first seen 6 Oct 2026
expert offloading
Also: MoE offloading, weight offloading, CPU-GPU offloading, MoE expert offloading
Storing inactive expert weights in CPU memory or streaming them to GPU on demand to run large MoE models on limited VRAM.
Mentions per week
PapersCodeDiscussion
Rank 180 on 9 Nov 2026 · 12 mentions in 28 days · 12 independent groups · code → papers · first seen 5 weeks ago · acceleration +0.77
Earliest sources
- Indras-Mirror/XolotlCode · 6 Oct 2026 · Indras-Mirror · 528 stars
- Wallawalla47/InfernixCode · 8 Oct 2026 · Wallawalla47 · 303 stars
- IbrahimSiddiqui007/moe-cacheCode · 8 Oct 2026 · IbrahimSiddiqui007 · 22 stars
- 1314521gjy/cyber-on-strataCode · 9 Oct 2026 · 1314521gjy · 28 stars
- SpecFit: Not All Draft Tokens Cost the Same in Offloaded MoE InferencePapers · 11 Oct 2026 · Hongliang Cao, Xiuyu Li, Siqi Wang, Ziming Wang, Jingmin Zhu, Hailong Yang
- Efficient CPU-GPU collaborative MoE inference via Adaptive Expert AssignmentPapers · 16 Oct 2026 · Wenhui Lv, Yunming Liao, Hongli Xu, Yang Xu, Junhao Cheng
Latest sources
- CECAR: Cache & Expert Co-Aware Routing Accelerates On-Device Inference of MoE LLMsPapers · 15 Nov 2026 · Byeongju Kim, Hoonki Lee, Byungjun Kim, Inyul Ra, Yongjin Kim, Yongseong Lee, Sa…
- Ostinato: Fast CPU-GPU MoE Inference for Local Agents via Resident Experts in Every Forward PassPapers · 8 Nov 2026 · Junning Zhu
- Multi-GPU MoE Inference with Adaptive Expert OffloadingPapers · 6 Nov 2026 · William X. Zheng, Zhongjie Chen, Shuo Yang, Ziming Mao, Xiaoze Fan, Yi Pan, Kunj…
- Accelerating Offloading-Based MoE Inference by Interleaving Concurrent RequestsPapers · 3 Nov 2026 · Hongseung Yu, In Gim, Kyunghan Lee, Lin Zhong
- ewhacc/tandem.cppCode · 1 Nov 2026 · ewhacc · 79 stars
- Layer-wise CPU-GPU Scheduling for LLM Inference on Memory-Limited Consumer GPUsPapers · 1 Nov 2026 · Naoto Sugiura, Yuta Tokusashi, Hiroki Matsutani
- Betelgeuse-AI/BetelgeuseCode · 31 Oct 2026 · Betelgeuse-AI · 158 stars
- PDScope: Breaking the Prefill-Decode Seesaw for Memory-Bound MoE InferencePapers · 29 Oct 2026 · Enda Yu, Dezun Dong, Zhaoning Zhang, Xiaojian Yang, Weiling Yang, Shaonan Ma, Do…
- CRISP: Co-designing Pruning and Scheduling for Efficient MoE Inference on Edge ServersPapers · 27 Oct 2026 · Congwei Zhang, Youming Tao, Yifei Zou, Yu Yang, Ruirui Zhang, Xiaoyu Zhang, Falk…
- Prefetch-Steered Routing: Deterministic Cache-Aware Expert Selection for Offloaded Mixture-of-Experts DecodingPapers · 26 Oct 2026 · Siming Huang
- COPE: Composable Performance Emulation for Mixture-of-Experts Offloading PoliciesPapers · 24 Oct 2026 · Hanming Zhang, Erhu Feng
- Magnvs7/PromethevsCode · 21 Oct 2026 · Magnvs7 · 193 stars