method · first seen 4 Oct 2026
on-policy distillation
Also: OPD, on-policy knowledge distillation, Open Publication Distribution System, multi-objective on-policy distillation
Distilling a student model from teacher signals computed on the student's own generated trajectories.
On-policy distillation trains a student model on trajectories it generates itself, with a teacher scoring or supplying token-level distributions over those student-produced sequences, rather than imitating fixed teacher outputs. The sources show it being treated as a standard post-training step for reasoning LLMs and then extended in many directions: sparse teacher interfaces for long traces (ResOPD), multi-teacher merging (Janus-MOPD), block diffusion and few-step image diffusion students, speculative decoding drafts (a Qwen repo), agent memory and ReAct behavior, multimodal perception, and backdoor transfer from teacher to student. Activity is almost entirely arXiv papers from many independent academic and industry groups. A few hits, such as an OPDS e-book server, are acronym collisions.
If it keeps spreading, on-policy distillation could become a default way to compress and merge frontier models into smaller ones. That would make its failure modes practical concerns for anyone shipping distilled models, including teacher errors on student-induced states, multi-domain interference and inherited backdoors.
Analyst note (emerging), 17 Nov 2026.
Mentions per week
PapersCodeDiscussion
Rank 12 on 16 Nov 2026 · 78 mentions in 28 days · 77 independent groups · papers → code · first seen 6 weeks ago · acceleration +0.08
Earliest sources
- ResOPD: Tail Residualization for Sparse On-Policy DistillationPapers · 4 Oct 2026 · Penghui Yang, Long Xing, Xuanlang Dai, Ziyu Liu, Kai Chen, Yuhang Zang
- How Should Teachers Be Prepared? RL on Student-Induced States for On-Policy DistillationPapers · 4 Oct 2026 · Xiaoyu Ma, Haoyue Liu, Zhichao Wang, Jionghao Zhu, Xiaoying Tang
- Towards Unbiased On-Policy Distillation for Block Diffusion Language ModelsPapers · 4 Oct 2026 · Zaiquan Yang, Fei Wei, Yong Wang, Yudong Han, Yiyu Li, Zhuofan Zong, Gerhard Pet…
- Beyond In-Distribution Preservation: Recovering Generalization in Quantized VLAs via Vulnerability-Oriented TuningPapers · 5 Oct 2026 · Shen Ruan, Wenchang Gao, Jin Wang, Siao Liu, Zhaxizhuoma, Dongchun Ren, Xin Zhen…
- ThunderSyncRL: Lossless Acceleration of Agentic Reinforcement LearningPapers · 5 Oct 2026 · Seil Kang, Hangoo Kang, Tarun Suresh, Youngeun Kim, Shreyas Pimpalgaonkar, Seong…
- Transfer-Stratified On-Policy Distillation for RL-Improved Reasoning TeachersPapers · 5 Oct 2026 · Xiaoyu Chen, Bo Shao, Tiangang Zhu, Bintao Wu, Linjun Shou, Fengge Wu, Feng Sun,…
Latest sources
- ernie-research/SWE-OPDCode · 16 Nov 2026 · ernie-research · 13 stars
- Conflict-Aware Multi-Teacher On-Policy Distillation in Aligned Action Spaces for Cross-Game GeneralizationPapers · 16 Nov 2026 · Jing Yi, Cehao Yang, Hualei Zhang, Haitao Li
- SWE-OPD: On-Policy Distillation for Agentic CodingPapers · 16 Nov 2026 · Yuhang He, Yanyu Chen, Chuanzhe Guo, Jie Zhu, Zichen Tian, Hongyu Ge, Zixuan Ren…
- Rethinking Supervision Scaling in On-Policy Distillation: From Dense Supervision to Stochastic HorizonsPapers · 16 Nov 2026 · Xinhao Yao, Zhenxing Dou, Yong Liu
- LEAF-OPD: Learning to Forget from Privileged Contexts via On-Policy DistillationPapers · 16 Nov 2026 · Zirui Pang, Zhijie Deng, Yuancheng Wang, Qihan Yu, Xin Li, Jiaheng Wei, Zixin Zh…
- Agentic On-Policy Distillation Implicitly Reshapes LLMs' ReAct PatternsPapers · 14 Nov 2026 · Deyang Kong, Zhaoyi Li, Sijian Ren, Yuan Wei, Ying Wei, Defu Lian, Shikun Zhang,…
- On-policy distillation as a Backdoor Propagation ChannelPapers · 14 Nov 2026 · Zane Fu, Yanzhao Zheng, Ziting Qin, Qianshan Wei, Yi Yu, Gongli Xi, Jihuai Zhu,…
- Teach with Blinks: Breaking Linguistic Momentum in On-Policy Distillation for Fine-Grained Multimodal PerceptionPapers · 12 Nov 2026 · Tangzheng Lian, Hang Zhang, Guanyu Hu, Renjiu Hu, Zhensong Zhang, Jiankang Deng,…
- Recover, Don't Just Continue: Learning from Feedback for Physical Failure Understanding and CorrectionPapers · 11 Nov 2026 · Miao Pan, Kaixiang Yao, Xiaohe Zhou, Xu Wang, Qinfeng Li, Wenqi Zhang, Xuhong Zh…
- Janus-MOPD: Tracing the Student Learning Dynamics in Multi-Teacher On-Policy DistillationPapers · 11 Nov 2026 · Zongji Yu, Ming Zhang, Dongcheng Zhang, Jiawei Zhou, Haotian Pang, Yiliu Sun, Xi…
- Coding Agent Memory Post-Training II: Unlocking the Memory Potential of Pre-trained File Operations for Long-Horizon Tasks via On-Policy DistillationPapers · 10 Nov 2026 · Lirui Luo, Kelong Mao, Xinwei Yang, Rongqing Li, Luyu Chen, Kieran Wong, Yudong…
- Beyond Passive Imitation: On-Policy Mid-Training Distillation for Language ModelsPapers · 10 Nov 2026 · Ruize Zheng, Xuyang Liu, Siyu Ren, Xiaobing Tu, Yinggui Wang, Jinkui Ren, Xianta…