Tagged articles

math reasoning benchmarks

3 articles · Page 1 of 1

Machine Learning Algorithms & Natural Language Processing

Aug 25, 2026 · Artificial Intelligence

When Online Distillation Goes Off‑Track: Relay‑OPD Lets the Teacher Take the Baton

The paper identifies the “prefix failure” problem in on‑policy distillation, proposes Relay‑OPD—a handoff‑triggered, budget‑controlled teacher takeover mechanism that corrects early mistakes, and demonstrates across eight math‑reasoning benchmarks that it improves accuracy by up to 7.3% while halving training trajectory length.

Large Language ModelsOn-Policy DistillationPrefix Failure

0 likes · 13 min read

When Online Distillation Goes Off‑Track: Relay‑OPD Lets the Teacher Take the Baton

Machine Learning Algorithms & Natural Language Processing

Aug 24, 2026 · Artificial Intelligence

When Online Distillation Goes Off‑Track: How Relay‑OPD Lets the Teacher Take Over at Critical Moments

The article analyzes the prefix‑failure problem in on‑policy distillation, introduces Relay‑OPD with a handoff trigger that lets a teacher model intervene locally, and shows through eight math‑reasoning benchmarks that this approach improves accuracy by up to 7.3% while cutting training trajectory length by more than half.

Large Language ModelsOn-Policy DistillationPrefix Failure

0 likes · 13 min read

When Online Distillation Goes Off‑Track: How Relay‑OPD Lets the Teacher Take Over at Critical Moments

Machine Heart

Jul 9, 2026 · Artificial Intelligence

Can Your Self‑Distillation Model Do Without Reference Solutions? Introducing d‑OPSD for Diffusion LLMs

The paper presents d‑OPSD, the first on‑policy self‑distillation framework for diffusion large language models that eliminates reference solutions and extra teacher models, using only one‑tenth of RL steps while achieving equal or superior reasoning performance and markedly higher training efficiency, as demonstrated on multiple math‑reasoning benchmarks.

Diffusion Language ModelsOPSDSelf-Distillation

0 likes · 7 min read

Can Your Self‑Distillation Model Do Without Reference Solutions? Introducing d‑OPSD for Diffusion LLMs