method · first seen 11 Nov 2026
top-k vocabulary truncation for distillation
Also: sparse logit distillation
Distilling only from the teacher's highest-probability vocabulary tokens with renormalization to cut compute and memory cost.
Mentions per week
PapersCodeDiscussion
Earliest sources
- Effective Teacher Support for Reasoning DistillationPapers · 11 Nov 2026 · Lavinia Ghita, Dhruv Desai
Latest sources
- Effective Teacher Support for Reasoning DistillationPapers · 11 Nov 2026 · Lavinia Ghita, Dhruv Desai