method · first seen 7 Oct 2026
delta distillation
Transferring a teacher's shift in logits relative to its base model rather than its full endpoint policy.
Mentions per week
PapersCodeDiscussion
Earliest sources
- Composing What Each Teacher Learned: Delta Distillation for Multi-Teacher On-Policy LearningPapers · 7 Oct 2026 · Yuanda Xu, Hejian Sang, Zhengze Zhou, Shayan Mohajer Hamidi, Xiaomin Li, Rohit J…
Latest sources
- Composing What Each Teacher Learned: Delta Distillation for Multi-Teacher On-Policy LearningPapers · 7 Oct 2026 · Yuanda Xu, Hejian Sang, Zhengze Zhou, Shayan Mohajer Hamidi, Xiaomin Li, Rohit J…