method · first seen 14 Oct 2026
forward Kullback-Leibler divergence
Also: forward KL
A divergence measuring how well a student distribution covers the teacher's probability mass.
Mentions per week
PapersCodeDiscussion
Earliest sources
- Divergence-Routed On-Policy Distillation for Language ModelsPapers · 14 Oct 2026 · Tiantian Peng, Guannan Lv, Hongyang Li, Qiuhe Hong, Hang Cheng, Hongjian Dou, Xi…
Latest sources
- Divergence-Routed On-Policy Distillation for Language ModelsPapers · 14 Oct 2026 · Tiantian Peng, Guannan Lv, Hongyang Li, Qiuhe Hong, Hang Cheng, Hongjian Dou, Xi…