Machine Learning Algorithms & Natural Language Processing
Jul 15, 2026 · Artificial Intelligence
Breaking OPD’s Teacher Ceiling with MAD‑OPD: Small Models Learn Debated Answers
MAD‑OPD replaces the single‑teacher supervision of On‑Policy Distillation with a multi‑teacher debate that produces a weighted consensus, yielding significant gains on agentic and code benchmarks—e.g., a 4B student surpasses a 14B teacher by 4.26 % on LiveCodeBench v6—and demonstrates the importance of confidence‑weighted debate and divergence selection.
Agentic TasksBenchmarkLarge Language Models
0 likes · 9 min read