method · first seen 13 Oct 2026
attention distillation
Transferring attention patterns from a teacher model to a student model to guide its internal correspondences.
Mentions per week
PapersCodeDiscussion
Earliest sources
- GAD: Geometry Attention Distillation for Consistent Video GenerationPapers · 13 Oct 2026 · Junfeng Ni, Ziyu Zhu, Fan Jiang, Yu Liu, Chiyu Wang, Jiawei Zhang, Zhaoxu Sun, M…
- ABNAT: Attention-Based Noise-Aware Training for Robust Transformers on Analog Compute-in-Memory SystemsPapers · 15 Oct 2026 · Wenshuai Yao, Wenyong Zhou, Hanyong Shao, Zhiyuan Ning, Yizhe Chen, Yuannuo Feng…
Latest sources
- ABNAT: Attention-Based Noise-Aware Training for Robust Transformers on Analog Compute-in-Memory SystemsPapers · 15 Oct 2026 · Wenshuai Yao, Wenyong Zhou, Hanyong Shao, Zhiyuan Ning, Yizhe Chen, Yuannuo Feng…
- GAD: Geometry Attention Distillation for Consistent Video GenerationPapers · 13 Oct 2026 · Junfeng Ni, Ziyu Zhu, Fan Jiang, Yu Liu, Chiyu Wang, Jiawei Zhang, Zhaoxu Sun, M…