Early-Signal Detector

method · first seen 11 Oct 2026

expert pruning

Also: MoE pruning, MoE expert pruning, Mixture-of-Experts pruning

Removing less useful experts from a mixture-of-experts model after training to reduce its size and cost.

Expert pruning removes routed experts from a trained mixture-of-experts model, usually after one calibration pass, so the model needs less memory to serve. The sources are almost all independent academic papers. They go beyond simply choosing which experts to drop. Some allocate pruning per layer by routing entropy, others treat selection as a reconstruction subset problem or preserve structure among redundant experts. Others separate speech or vision tokens from text tokens, or reuse pruned experts through gating. Several papers test calibration robustness, and two find that English calibration degrades Lithuanian performance. A Show HN project combines pruning with sub-2-bit quantization to run Gemma 4 26B on a 6GB GPU.

MoE models are cheap per token but expensive to hold in memory, so reliable expert pruning could let large open MoE models run on consumer and edge hardware. The calibration-language findings suggest pruned models may quietly lose quality in low-resource languages.

Analyst note (emerging), 18 Nov 2026.

Mentions per week

0319 May11 Aug10 NovPapersCodeDiscussion

PapersCodeDiscussion

Rank 15 on 16 Nov 2026 · 9 mentions in 28 days · 8 independent groups · papers → discussion · first seen 5 weeks ago · acceleration +0.56

Earliest sources

Latest sources

Mentioned together with

mixture of expertsquantizationgpt-ossomnimodal large language modelexpert offloadingmultimodal large language model physics reasoninglow-rank adaptation