method · first seen 10 Oct 2026
knowledge distillation with pruning
Compressing a model by pruning and training a smaller student to match a larger teacher.
Mentions per week
PapersCodeDiscussion
Earliest sources
- UltraBERT: Beyond the Limit of BERT Pre-TrainingPapers · 10 Oct 2026 · Jungwoo Park, Taewhoo Lee, Jaewoo Kang
Latest sources
- UltraBERT: Beyond the Limit of BERT Pre-TrainingPapers · 10 Oct 2026 · Jungwoo Park, Taewhoo Lee, Jaewoo Kang