Tagged articles

on-policy distillation

18 articles · Page 1 of 1

Machine Learning Algorithms & Natural Language Processing

Jul 15, 2026 · Artificial Intelligence

Breaking OPD’s Teacher Ceiling with MAD‑OPD: Small Models Learn Debated Answers

MAD‑OPD replaces the single‑teacher supervision of On‑Policy Distillation with a multi‑teacher debate that produces a weighted consensus, yielding significant gains on agentic and code benchmarks—e.g., a 4B student surpasses a 14B teacher by 4.26 % on LiveCodeBench v6—and demonstrates the importance of confidence‑weighted debate and divergence selection.

Agentic TasksModel CompressionMulti-Agent Debate

0 likes · 9 min read

Breaking OPD’s Teacher Ceiling with MAD‑OPD: Small Models Learn Debated Answers

DataFunSummit

Jul 10, 2026 · Artificial Intelligence

UI-MOPD Enables Cross‑Platform GUI Agents to Retain Skills Without Forgetting

The article analyzes why GUI agents trained on both desktop (mouse‑click) and mobile (touch) interactions suffer from behavior collapse and catastrophic forgetting, introduces the UI‑MOPD framework that assigns platform‑specific teachers through on‑policy distillation, and shows an 8B model surpassing a 235B baseline on OSWorld and MobileWorld benchmarks while preserving general GUI understanding.

GUI agentsUI‑MOPDcontinual learning

0 likes · 8 min read

UI-MOPD Enables Cross‑Platform GUI Agents to Retain Skills Without Forgetting

Xiaohongshu Tech REDtech

Jul 10, 2026 · Artificial Intelligence

PIPO: Pair‑In Pair‑Out Architecture Cuts Input Length in Half and Doubles Token Output for LLMs

The PIPO (Pair‑In, Pair‑Out) architecture compresses two input tokens into one latent vector and expands each hidden state to emit an extra draft token, halving effective input length and doubling per‑step output, achieving up to +7.15 pass@4 improvement and 2.64× TTFT / 2.07× TPOT speedups on benchmarks such as AIME 2025, GPQA‑Diamond, LiveCodeBench v6 and LongBench v2 with Qwen3.5‑4B/9B backbones.

Multi-Token PredictionSpeculative Decodingefficiency

0 likes · 14 min read

PIPO: Pair‑In Pair‑Out Architecture Cuts Input Length in Half and Doubles Token Output for LLMs

Machine Learning Algorithms & Natural Language Processing

Jun 21, 2026 · Artificial Intelligence

xOPD Evolution: Mapping Recent OPD Improvements – Rephrased Same Problems vs. New Modules

This article surveys the latest on‑policy distillation (OPD) research, categorizing each work as either a reinterpretation of an existing problem or a modification of a different module, and highlights the experimental findings, design choices, and trade‑offs reported across the papers.

LLMOPDReinforcement Learning

0 likes · 31 min read

xOPD Evolution: Mapping Recent OPD Improvements – Rephrased Same Problems vs. New Modules

Machine Learning Algorithms & Natural Language Processing

Jun 18, 2026 · Artificial Intelligence

From Imitation to Optimization: Recent Advances in On-Policy Distillation

This article surveys the latest research on On-Policy Distillation for large language models, covering methods that improve training stability, self‑distillation frameworks, and detailed analyses of when and why OPD succeeds or fails, with concrete experimental results and practical insights.

Entropy-AwareModel CompressionReinforcement Learning

0 likes · 19 min read

From Imitation to Optimization: Recent Advances in On-Policy Distillation

ThinkingAgent

Jun 17, 2026 · Artificial Intelligence

Why a 7B Model Can Outperform a 70B Model: The Power of Knowledge Distillation

The article explains how knowledge distillation lets a small LLM learn from a much larger teacher model, achieving near‑teacher performance while cutting inference cost, latency and memory, and provides a step‑by‑step guide, benchmark results, advanced on‑policy techniques, common pitfalls and best‑practice recommendations.

LLMLlama-3Model Compression

0 likes · 13 min read

Why a 7B Model Can Outperform a 70B Model: The Power of Knowledge Distillation

Machine Heart

May 29, 2026 · Artificial Intelligence

DiffusionOPD: A New Online Policy Distillation Paradigm for Multi‑Task Diffusion Models

DiffusionOPD introduces a unified on‑policy distillation framework for diffusion models that decouples single‑task online policy exploration from multi‑task capability integration, training expert teachers per task and distilling their skills into a single student model, achieving faster convergence and higher performance across composition, OCR, and aesthetic tasks.

KL divergencePPOReinforcement Learning

0 likes · 8 min read

DiffusionOPD: A New Online Policy Distillation Paradigm for Multi‑Task Diffusion Models

Baobao Algorithm Notes

May 26, 2026 · Artificial Intelligence

How On-Policy Distillation (OPD) Solves Core Challenges in Large-Model Post-Training

The article explains how On-Policy Distillation (OPD) combines on‑policy sampling with dense teacher feedback via reverse KL to address low signal density, distribution shift, and capability interference in large‑model post‑training, and compares implementations by Qwen3, GLM‑5, MiMo‑V2 and DeepSeek‑V4.

Model CompressionOPDReverse KL

0 likes · 20 min read

How On-Policy Distillation (OPD) Solves Core Challenges in Large-Model Post-Training

Network Intelligence Research Center (NIRC)

May 25, 2026 · Artificial Intelligence

What Does On-Policy Distillation Really Teach Large Language Models?

On-Policy Distillation (OPD) trains large language models by letting the student generate its own inference paths while the teacher supplies token‑level guidance, offering denser signals than RL but sometimes failing when teacher and student reasoning diverge, as detailed by THUNLP’s recent study.

Distillation MetricsPost-TrainingToken-level Supervision

0 likes · 8 min read

What Does On-Policy Distillation Really Teach Large Language Models?

Machine Heart

May 25, 2026 · Artificial Intelligence

Breaking the Reward Trade‑off: Flow‑OPD Brings Multi‑Teacher OPD to Image Generation

Flow‑OPD introduces on‑policy distillation into flow‑matching diffusion models, using a multi‑teacher online rollout framework and manifold‑anchor regularization to resolve the seesaw effect of single and mixed rewards, achieving superior multi‑task performance and surpassing specialist models in image generation.

Flow-OPDManifold Anchor Regularizationdiffusion models

0 likes · 9 min read

Breaking the Reward Trade‑off: Flow‑OPD Brings Multi‑Teacher OPD to Image Generation

AIWalker

May 20, 2026 · Artificial Intelligence

AnyFlow: Generate High‑Quality Video in 4 Steps and Keep Improving with More Sampling

AnyFlow introduces a flow‑map distillation framework that enables video diffusion models to produce high‑quality results in just four sampling steps while still gaining quality as the number of steps increases, supporting both causal and bidirectional architectures and scaling up to 14 B parameters.

Video Diffusionbidirectional videocausal video

0 likes · 14 min read

AnyFlow: Generate High‑Quality Video in 4 Steps and Keep Improving with More Sampling

Machine Heart

May 13, 2026 · Artificial Intelligence

Why Bigger Teachers Don’t Teach Better: Tsinghua’s On‑Policy Distillation Study

Recent research by Tsinghua and collaborators dissects On‑Policy Distillation for large language models, revealing that higher‑scoring teachers often fail to improve students unless their thinking patterns align, detailing token‑level overlap dynamics, failure cases, and two practical remedies to rescue ineffective distillation.

RL post-trainingTeacher-Student AlignmentToken Overlap

0 likes · 9 min read

Why Bigger Teachers Don’t Teach Better: Tsinghua’s On‑Policy Distillation Study

Machine Learning Algorithms & Natural Language Processing

May 1, 2026 · Artificial Intelligence

What DeepSeek V4’s Multi‑Expert On‑Policy Distillation Reveals About Human Learning

The article analyzes DeepSeek V4’s post‑training pipeline, explains how multi‑expert on‑policy distillation (OPD) differs from traditional teacher‑forcing, compares reverse‑KL and forward‑KL objectives, and uses analogies to human learning to illustrate the benefits and limits of OPD.

DeepSeek V4LLM trainingMulti-Expert Models

0 likes · 11 min read

What DeepSeek V4’s Multi‑Expert On‑Policy Distillation Reveals About Human Learning

Machine Learning Algorithms & Natural Language Processing

Apr 29, 2026 · Artificial Intelligence

Dual Engine for Training and Inference: How Princeton’s SD‑ZERO and AggAgent Redefine Complex Reasoning

The article reviews two recent Princeton papers—SD‑ZERO, which introduces self‑revision training and on‑policy self‑distillation to turn a model’s own error traces into dense supervision, and AggAgent, which actively aggregates parallel long‑horizon trajectories—showing how internal trajectory mining can cut compute costs and boost accuracy on challenging math and code benchmarks.

AggAgentComplex ReasoningSelf-Distillation

0 likes · 10 min read

Dual Engine for Training and Inference: How Princeton’s SD‑ZERO and AggAgent Redefine Complex Reasoning

Machine Learning Algorithms & Natural Language Processing

Apr 14, 2026 · Artificial Intelligence

Revisiting On-Policy Distillation (OPD): Typical Failures and a More Stable Fix

On‑Policy Distillation (OPD) is widely used for post‑training large language models, but the sampled‑token variant often becomes unstable due to token‑level reward imbalance, teacher‑student signal mismatch on student‑generated prefixes, and tokenizer mismatches; this article analyses the bias‑variance trade‑off, identifies three root failure modes, and proposes a teacher‑top‑K local‑support‑set objective with top‑p rollout and special‑token masking that yields more stable training and better performance on both math and agentic benchmarks.

OPDlarge language modelson-policy distillation

0 likes · 32 min read

Revisiting On-Policy Distillation (OPD): Typical Failures and a More Stable Fix

Machine Learning Algorithms & Natural Language Processing

Feb 22, 2026 · Artificial Intelligence

What Is On-Policy Distillation? A Deep Dive into On-Policy and Self-Distillation

The article explains On-Policy Distillation, derives its forward and reverse KL gradients, introduces Self‑Distillation where the policy serves as its own teacher, discusses practical implementation tricks such as extra‑knowledge injection, EMA or trust‑region teacher stabilization, and highlights benefits like reduced catastrophic forgetting, fewer Aha moments, and a narrower train‑test gap, especially for larger models.

EMAKL divergenceSelf-Distillation

0 likes · 6 min read

What Is On-Policy Distillation? A Deep Dive into On-Policy and Self-Distillation

HyperAI Super Neural

Jan 9, 2026 · Artificial Intelligence

How HY-MT1.5 Achieves 1 GB Mobile Translation with a 1.8B Model

The article explains how Tencent's open‑source HY‑MT1.5 tackles the high‑cost, large‑parameter barrier of neural machine translation by offering a 1.8 B‑parameter model that runs on roughly 1 GB of RAM, processes 50 tokens in 0.18 s, supports 33 languages, and uses on‑policy distillation to retain top‑tier accuracy, while providing a step‑by‑step online demo and free compute credits for new users.

HY-MT1.5Machine TranslationMobile AI

0 likes · 5 min read

How HY-MT1.5 Achieves 1 GB Mobile Translation with a 1.8B Model

DataFunTalk

Oct 30, 2025 · Artificial Intelligence

How On-Policy Distillation Cuts LLM Training Cost by 90%

Thinking Machines Lab introduces On-Policy Distillation, a post‑training technique that matches reinforcement‑learning performance while reducing compute cost by up to tenfold, and demonstrates its effectiveness through extensive experiments on reasoning, personalization, and catastrophic‑forgetting mitigation.

Reinforcement Learningknowledge distillationmodel efficiency

0 likes · 15 min read

How On-Policy Distillation Cuts LLM Training Cost by 90%