trends.barnas.net

method · first seen 11 Nov 2026

top-k vocabulary truncation for distillation

Also: sparse logit distillation

Distilling only from the teacher's highest-probability vocabulary tokens with renormalization to cut compute and memory cost.

Mentions per week

0119 May11 Aug10 NovPapersCodeDiscussion

PapersCodeDiscussion

Earliest sources

Latest sources

Mentioned together with

knowledge distillationreasoning distillation