method · first seen 11 Oct 2026
expert pruning
Also: MoE pruning, MoE expert pruning, Mixture-of-Experts pruning
Removing less useful experts from a mixture-of-experts model after training to reduce its size and cost.
Expert pruning removes routed experts from a trained mixture-of-experts model, usually after one calibration pass, so the model needs less memory to serve. The sources are almost all independent academic papers. They go beyond simply choosing which experts to drop. Some allocate pruning per layer by routing entropy, others treat selection as a reconstruction subset problem or preserve structure among redundant experts. Others separate speech or vision tokens from text tokens, or reuse pruned experts through gating. Several papers test calibration robustness, and two find that English calibration degrades Lithuanian performance. A Show HN project combines pruning with sub-2-bit quantization to run Gemma 4 26B on a 6GB GPU.
MoE models are cheap per token but expensive to hold in memory, so reliable expert pruning could let large open MoE models run on consumer and edge hardware. The calibration-language findings suggest pruned models may quietly lose quality in low-resource languages.
Analyst note (emerging), 18 Nov 2026.
Mentions per week
PapersCodeDiscussion
Rank 15 on 16 Nov 2026 · 9 mentions in 28 days · 8 independent groups · papers → discussion · first seen 5 weeks ago · acceleration +0.56
Earliest sources
- Towards Robust Calibration Dataset for Domain-General Mixture of Experts PruningPapers · 11 Oct 2026 · Luis Frentzen Salim, Chia-Chun Wu, Tran Van Nhiem, Lun-Wei Ku, Yung-Hui Li
- PW-Pruning: Phase-Aware Expert Selection for Speech-Centric Omni-Modal ModelsPapers · 15 Oct 2026 · Junjie Cao, Fengping Tian, Wenjie Zhou, Xuanfan Ni, Chenyang Lyu, Siyou Liu, Lon…
- Reconstruction-Optimized Expert Pruning for Mixture-of-Experts Language ModelsPapers · 19 Oct 2026 · Jelena Markovic-Voronov, Petros Prastakos, Kayhan Behdin, Jincheng Cao, Zhipeng…
- SPARE: Sparsity Allocation via Routing Entropy for Efficient MoEsPapers · 21 Oct 2026 · Yifan Yang, Hanling Yi, Dongquan Yang, Feng Lin, Rong Xiao, Aijun Li, Xiaotian Y…
- CRISP: Co-designing Pruning and Scheduling for Efficient MoE Inference on Edge ServersPapers · 27 Oct 2026 · Congwei Zhang, Youming Tao, Yifei Zou, Yu Yang, Ruirui Zhang, Xiaoyu Zhang, Falk…
- Experts Exhibit Structured Redundancy: Structure-Preserving Expert Pruning for Mixture-of-Experts Large Language ModelsPapers · 30 Oct 2026 · Tian Wang, Yongding Tao, Yihong Dong, Xiaoming Wu, Ying Guo, Ge Li
Latest sources
- GEAR$^2$: Pruned Experts Live On through Gated Reuse of Activated Experts and a Missing-Mass ResidualPapers · 16 Nov 2026 · Linliang Chen, Yan Zhong, Wang Kang, Sai Li, Ji Wu, Ji Liu
- Reasoning Again with a Quarter of the Experts: Retry First, Then TrainPapers · 16 Nov 2026 · Anik Jha
- Prune, Don't Merge: Expert Compression of Mixture-of-Experts LLMs on Multilingual Production TasksPapers · 9 Nov 2026 · Tomas Rasymas, Povilas Kvedaras
- Show HN: Maple – Sub-2-BPW Moe Pruning to Run Gemma 4 26B at 45 TPS on a 6GB GPUDiscussion · 8 Nov 2026 · AmSach · 21 points
- English Calibration Erodes Lithuanian in Expert-Pruned Mixture-of-Experts Language ModelsPapers · 5 Nov 2026 · Tomas Rasymas, Povilas Kvedaras
- Modality-Aware Expert Pruning for MoE-Based Multimodal Large Language ModelsPapers · 4 Nov 2026 · Donggeun Kim
- Experts Exhibit Structured Redundancy: Structure-Preserving Expert Pruning for Mixture-of-Experts Large Language ModelsPapers · 30 Oct 2026 · Tian Wang, Yongding Tao, Yihong Dong, Xiaoming Wu, Ying Guo, Ge Li
- CRISP: Co-designing Pruning and Scheduling for Efficient MoE Inference on Edge ServersPapers · 27 Oct 2026 · Congwei Zhang, Youming Tao, Yifei Zou, Yu Yang, Ruirui Zhang, Xiaoyu Zhang, Falk…
- SPARE: Sparsity Allocation via Routing Entropy for Efficient MoEsPapers · 21 Oct 2026 · Yifan Yang, Hanling Yi, Dongquan Yang, Feng Lin, Rong Xiao, Aijun Li, Xiaotian Y…
- Reconstruction-Optimized Expert Pruning for Mixture-of-Experts Language ModelsPapers · 19 Oct 2026 · Jelena Markovic-Voronov, Petros Prastakos, Kayhan Behdin, Jincheng Cao, Zhipeng…
- PW-Pruning: Phase-Aware Expert Selection for Speech-Centric Omni-Modal ModelsPapers · 15 Oct 2026 · Junjie Cao, Fengping Tian, Wenjie Zhou, Xuanfan Ni, Chenyang Lyu, Siyou Liu, Lon…
- Towards Robust Calibration Dataset for Domain-General Mixture of Experts PruningPapers · 11 Oct 2026 · Luis Frentzen Salim, Chia-Chun Wu, Tran Van Nhiem, Lun-Wei Ku, Yung-Hui Li