BestHub
Sign in
Tagged articles

On-Policy Distillation

25 articles · Page 1 of 1
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Oct 26, 2026 · Artificial Intelligence

On-Policy Distillation Explained: White-Box, Self-Distillation & Black-Box Methods

This article provides a comprehensive analysis of On-Policy Distillation (OPD), covering white-box distillation with token-level probability matching, self-distillation using privileged context, black-box distillation via reward modeling, and industrial applications from MiMo, DeepSeek, and Qwen, while discussing OPD's limitations and generalization capabilities.

Black-Box DistillationGKDLarge Language Models
0 likes · 56 min read
On-Policy Distillation Explained: White-Box, Self-Distillation & Black-Box Methods
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Oct 19, 2026 · Artificial Intelligence

AdaFlash: Adaptive Speculative Decoding with Online Self-Improvement for High-Concurrency LLM Inference

Nanjing University and Huawei's Foundation Model team introduce AdaFlash, an adaptive speculative decoding framework that enables diffusion draft models to self-improve online via on-policy distillation and dynamic verification length prediction, achieving up to 5.32x end-to-end speedup on Qwen3-8B and 66% higher throughput under high concurrency.

AdaFlashLLM inference accelerationOn-Policy Distillation
0 likes · 17 min read
AdaFlash: Adaptive Speculative Decoding with Online Self-Improvement for High-Concurrency LLM Inference
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Oct 10, 2026 · Artificial Intelligence

Relay-OPD: Teacher Steps In at Critical Moments to Fix Student Reasoning Errors in Online Distillation

Zhejiang University and Alibaba propose Relay-OPD, an online distillation method that detects student reasoning errors via teacher-student token divergence and briefly hands control to the teacher for targeted correction, achieving +5.73% accuracy on math benchmarks while halving training trajectory length.

Large Language ModelsMathematical ReasoningOn-Policy Distillation
0 likes · 17 min read
Relay-OPD: Teacher Steps In at Critical Moments to Fix Student Reasoning Errors in Online Distillation
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Aug 25, 2026 · Artificial Intelligence

When Online Distillation Goes Off‑Track: Relay‑OPD Lets the Teacher Take the Baton

The paper identifies the “prefix failure” problem in on‑policy distillation, proposes Relay‑OPD—a handoff‑triggered, budget‑controlled teacher takeover mechanism that corrects early mistakes, and demonstrates across eight math‑reasoning benchmarks that it improves accuracy by up to 7.3% while halving training trajectory length.

Large Language ModelsOn-Policy DistillationPrefix Failure
0 likes · 13 min read
When Online Distillation Goes Off‑Track: Relay‑OPD Lets the Teacher Take the Baton
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Aug 24, 2026 · Artificial Intelligence

When Online Distillation Goes Off‑Track: How Relay‑OPD Lets the Teacher Take Over at Critical Moments

The article analyzes the prefix‑failure problem in on‑policy distillation, introduces Relay‑OPD with a handoff trigger that lets a teacher model intervene locally, and shows through eight math‑reasoning benchmarks that this approach improves accuracy by up to 7.3% while cutting training trajectory length by more than half.

Large Language ModelsOn-Policy DistillationPrefix Failure
0 likes · 13 min read
When Online Distillation Goes Off‑Track: How Relay‑OPD Lets the Teacher Take Over at Critical Moments
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Aug 3, 2026 · Artificial Intelligence

OPD Evolution: From CoT SFT to Self‑Distillation and Preference Optimization

Since 2026, On‑Policy Distillation (OPD) has rapidly become a focal research area, evolving from offline teacher‑generated data to online student‑driven supervision, with advances such as OPD+, Direct OPD, weak‑to‑strong OPD, self‑distillation techniques, and preference‑optimization signals reshaping post‑training for large language models.

Large Language ModelsNLPOPD
0 likes · 7 min read
OPD Evolution: From CoT SFT to Self‑Distillation and Preference Optimization
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Jul 15, 2026 · Artificial Intelligence

Breaking OPD’s Teacher Ceiling with MAD‑OPD: Small Models Learn Debated Answers

MAD‑OPD replaces the single‑teacher supervision of On‑Policy Distillation with a multi‑teacher debate that produces a weighted consensus, yielding significant gains on agentic and code benchmarks—e.g., a 4B student surpasses a 14B teacher by 4.26 % on LiveCodeBench v6—and demonstrates the importance of confidence‑weighted debate and divergence selection.

Agentic TasksLarge Language ModelsMulti-Agent Debate
0 likes · 9 min read
Breaking OPD’s Teacher Ceiling with MAD‑OPD: Small Models Learn Debated Answers
DataFunSummit
DataFunSummit
Jul 10, 2026 · Artificial Intelligence

UI-MOPD Enables Cross‑Platform GUI Agents to Retain Skills Without Forgetting

The article analyzes why GUI agents trained on both desktop (mouse‑click) and mobile (touch) interactions suffer from behavior collapse and catastrophic forgetting, introduces the UI‑MOPD framework that assigns platform‑specific teachers through on‑policy distillation, and shows an 8B model surpassing a 235B baseline on OSWorld and MobileWorld benchmarks while preserving general GUI understanding.

Continual LearningGUI agentsOn-Policy Distillation
0 likes · 8 min read
UI-MOPD Enables Cross‑Platform GUI Agents to Retain Skills Without Forgetting
Xiaohongshu Tech REDtech
Xiaohongshu Tech REDtech
Jul 10, 2026 · Artificial Intelligence

PIPO: Pair‑In Pair‑Out Architecture Cuts Input Length in Half and Doubles Token Output for LLMs

The PIPO (Pair‑In, Pair‑Out) architecture compresses two input tokens into one latent vector and expands each hidden state to emit an extra draft token, halving effective input length and doubling per‑step output, achieving up to +7.15 pass@4 improvement and 2.64× TTFT / 2.07× TPOT speedups on benchmarks such as AIME 2025, GPQA‑Diamond, LiveCodeBench v6 and LongBench v2 with Qwen3.5‑4B/9B backbones.

EfficiencyMulti-Token PredictionOn-Policy Distillation
0 likes · 14 min read
PIPO: Pair‑In Pair‑Out Architecture Cuts Input Length in Half and Doubles Token Output for LLMs
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Jun 21, 2026 · Artificial Intelligence

xOPD Evolution: Mapping Recent OPD Improvements – Rephrased Same Problems vs. New Modules

This article surveys the latest on‑policy distillation (OPD) research, categorizing each work as either a reinterpretation of an existing problem or a modification of a different module, and highlights the experimental findings, design choices, and trade‑offs reported across the papers.

LLMModel EfficiencyOPD
0 likes · 31 min read
xOPD Evolution: Mapping Recent OPD Improvements – Rephrased Same Problems vs. New Modules
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Jun 18, 2026 · Artificial Intelligence

From Imitation to Optimization: Recent Advances in On-Policy Distillation

This article surveys the latest research on On-Policy Distillation for large language models, covering methods that improve training stability, self‑distillation frameworks, and detailed analyses of when and why OPD succeeds or fails, with concrete experimental results and practical insights.

Entropy-AwareLarge Language ModelsOn-Policy Distillation
0 likes · 19 min read
From Imitation to Optimization: Recent Advances in On-Policy Distillation
ThinkingAgent
ThinkingAgent
Jun 17, 2026 · Artificial Intelligence

Why a 7B Model Can Outperform a 70B Model: The Power of Knowledge Distillation

The article explains how knowledge distillation lets a small LLM learn from a much larger teacher model, achieving near‑teacher performance while cutting inference cost, latency and memory, and provides a step‑by‑step guide, benchmark results, advanced on‑policy techniques, common pitfalls and best‑practice recommendations.

LLMLLaMA-3On-Policy Distillation
0 likes · 13 min read
Why a 7B Model Can Outperform a 70B Model: The Power of Knowledge Distillation
Machine Heart
Machine Heart
May 29, 2026 · Artificial Intelligence

DiffusionOPD: A New Online Policy Distillation Paradigm for Multi‑Task Diffusion Models

DiffusionOPD introduces a unified on‑policy distillation framework for diffusion models that decouples single‑task online policy exploration from multi‑task capability integration, training expert teachers per task and distilling their skills into a single student model, achieving faster convergence and higher performance across composition, OCR, and aesthetic tasks.

KL DivergenceOn-Policy DistillationPPO
0 likes · 8 min read
DiffusionOPD: A New Online Policy Distillation Paradigm for Multi‑Task Diffusion Models
Baobao Algorithm Notes
Baobao Algorithm Notes
May 26, 2026 · Artificial Intelligence

How On-Policy Distillation (OPD) Solves Core Challenges in Large-Model Post-Training

The article explains how On-Policy Distillation (OPD) combines on‑policy sampling with dense teacher feedback via reverse KL to address low signal density, distribution shift, and capability interference in large‑model post‑training, and compares implementations by Qwen3, GLM‑5, MiMo‑V2 and DeepSeek‑V4.

Large Language ModelsOPDOn-Policy Distillation
0 likes · 20 min read
How On-Policy Distillation (OPD) Solves Core Challenges in Large-Model Post-Training
Network Intelligence Research Center (NIRC)
Network Intelligence Research Center (NIRC)
May 25, 2026 · Artificial Intelligence

What Does On-Policy Distillation Really Teach Large Language Models?

On-Policy Distillation (OPD) trains large language models by letting the student generate its own inference paths while the teacher supplies token‑level guidance, offering denser signals than RL but sometimes failing when teacher and student reasoning diverge, as detailed by THUNLP’s recent study.

Distillation MetricsLarge Language ModelsOn-Policy Distillation
0 likes · 8 min read
What Does On-Policy Distillation Really Teach Large Language Models?
Machine Heart
Machine Heart
May 25, 2026 · Artificial Intelligence

Breaking the Reward Trade‑off: Flow‑OPD Brings Multi‑Teacher OPD to Image Generation

Flow‑OPD introduces on‑policy distillation into flow‑matching diffusion models, using a multi‑teacher online rollout framework and manifold‑anchor regularization to resolve the seesaw effect of single and mixed rewards, achieving superior multi‑task performance and surpassing specialist models in image generation.

Flow-OPDManifold Anchor RegularizationOn-Policy Distillation
0 likes · 9 min read
Breaking the Reward Trade‑off: Flow‑OPD Brings Multi‑Teacher OPD to Image Generation
AIWalker
AIWalker
May 20, 2026 · Artificial Intelligence

AnyFlow: Generate High‑Quality Video in 4 Steps and Keep Improving with More Sampling

AnyFlow introduces a flow‑map distillation framework that enables video diffusion models to produce high‑quality results in just four sampling steps while still gaining quality as the number of steps increases, supporting both causal and bidirectional architectures and scaling up to 14 B parameters.

On-Policy DistillationVideo Diffusionbidirectional video
0 likes · 14 min read
AnyFlow: Generate High‑Quality Video in 4 Steps and Keep Improving with More Sampling
Machine Heart
Machine Heart
May 13, 2026 · Artificial Intelligence

Why Bigger Teachers Don’t Teach Better: Tsinghua’s On‑Policy Distillation Study

Recent research by Tsinghua and collaborators dissects On‑Policy Distillation for large language models, revealing that higher‑scoring teachers often fail to improve students unless their thinking patterns align, detailing token‑level overlap dynamics, failure cases, and two practical remedies to rescue ineffective distillation.

Large Language ModelsOn-Policy DistillationRL post-training
0 likes · 9 min read
Why Bigger Teachers Don’t Teach Better: Tsinghua’s On‑Policy Distillation Study
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
May 1, 2026 · Artificial Intelligence

What DeepSeek V4’s Multi‑Expert On‑Policy Distillation Reveals About Human Learning

The article analyzes DeepSeek V4’s post‑training pipeline, explains how multi‑expert on‑policy distillation (OPD) differs from traditional teacher‑forcing, compares reverse‑KL and forward‑KL objectives, and uses analogies to human learning to illustrate the benefits and limits of OPD.

DeepSeek-V4LLM trainingMulti-Expert Models
0 likes · 11 min read
What DeepSeek V4’s Multi‑Expert On‑Policy Distillation Reveals About Human Learning
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Apr 29, 2026 · Artificial Intelligence

Dual Engine for Training and Inference: How Princeton’s SD‑ZERO and AggAgent Redefine Complex Reasoning

The article reviews two recent Princeton papers—SD‑ZERO, which introduces self‑revision training and on‑policy self‑distillation to turn a model’s own error traces into dense supervision, and AggAgent, which actively aggregates parallel long‑horizon trajectories—showing how internal trajectory mining can cut compute costs and boost accuracy on challenging math and code benchmarks.

AggAgentComplex ReasoningLarge Language Models
0 likes · 10 min read
Dual Engine for Training and Inference: How Princeton’s SD‑ZERO and AggAgent Redefine Complex Reasoning
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Apr 14, 2026 · Artificial Intelligence

Revisiting On-Policy Distillation (OPD): Typical Failures and a More Stable Fix

On‑Policy Distillation (OPD) is widely used for post‑training large language models, but the sampled‑token variant often becomes unstable due to token‑level reward imbalance, teacher‑student signal mismatch on student‑generated prefixes, and tokenizer mismatches; this article analyses the bias‑variance trade‑off, identifies three root failure modes, and proposes a teacher‑top‑K local‑support‑set objective with top‑p rollout and special‑token masking that yields more stable training and better performance on both math and agentic benchmarks.

Large Language ModelsOPDOn-Policy Distillation
0 likes · 32 min read
Revisiting On-Policy Distillation (OPD): Typical Failures and a More Stable Fix
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Feb 22, 2026 · Artificial Intelligence

What Is On-Policy Distillation? A Deep Dive into On-Policy and Self-Distillation

The article explains On-Policy Distillation, derives its forward and reverse KL gradients, introduces Self‑Distillation where the policy serves as its own teacher, discusses practical implementation tricks such as extra‑knowledge injection, EMA or trust‑region teacher stabilization, and highlights benefits like reduced catastrophic forgetting, fewer Aha moments, and a narrower train‑test gap, especially for larger models.

EMAKL DivergenceOn-Policy Distillation
0 likes · 6 min read
What Is On-Policy Distillation? A Deep Dive into On-Policy and Self-Distillation
HyperAI Super Neural
HyperAI Super Neural
Jan 9, 2026 · Artificial Intelligence

How HY-MT1.5 Achieves 1 GB Mobile Translation with a 1.8B Model

The article explains how Tencent's open‑source HY‑MT1.5 tackles the high‑cost, large‑parameter barrier of neural machine translation by offering a 1.8 B‑parameter model that runs on roughly 1 GB of RAM, processes 50 tokens in 0.18 s, supports 33 languages, and uses on‑policy distillation to retain top‑tier accuracy, while providing a step‑by‑step online demo and free compute credits for new users.

HY-MT1.5Large Language ModelsOn-Policy Distillation
0 likes · 5 min read
How HY-MT1.5 Achieves 1 GB Mobile Translation with a 1.8B Model
DataFunTalk
DataFunTalk
Oct 30, 2025 · Artificial Intelligence

How On-Policy Distillation Cuts LLM Training Cost by 90%

Thinking Machines Lab introduces On-Policy Distillation, a post‑training technique that matches reinforcement‑learning performance while reducing compute cost by up to tenfold, and demonstrates its effectiveness through extensive experiments on reasoning, personalization, and catastrophic‑forgetting mitigation.

Model EfficiencyOn-Policy Distillationknowledge distillation
0 likes · 15 min read
How On-Policy Distillation Cuts LLM Training Cost by 90%