Machine Learning Algorithms & Natural Language Processing

Oct 10, 2026 · Artificial Intelligence

Relay-OPD: Teacher Steps In at Critical Moments to Fix Student Reasoning Errors in Online Distillation

Zhejiang University and Alibaba propose Relay-OPD, an online distillation method that detects student reasoning errors via teacher-student token divergence and briefly hands control to the teacher for targeted correction, achieving +5.73% accuracy on math benchmarks while halving training trajectory length.