Tagged articles

OPD

8 articles · Page 1 of 1

Machine Learning Algorithms & Natural Language Processing

Aug 3, 2026 · Artificial Intelligence

OPD Evolution: From CoT SFT to Self‑Distillation and Preference Optimization

Since 2026, On‑Policy Distillation (OPD) has rapidly become a focal research area, evolving from offline teacher‑generated data to online student‑driven supervision, with advances such as OPD+, Direct OPD, weak‑to‑strong OPD, self‑distillation techniques, and preference‑optimization signals reshaping post‑training for large language models.

NLPOPDOn-Policy Distillation

0 likes · 7 min read

OPD Evolution: From CoT SFT to Self‑Distillation and Preference Optimization

PaperAgent

Jul 5, 2026 · Artificial Intelligence

Uncovering the Privilege Illusion in OPD Distillation and How DOPD Solves It

The article identifies the hidden “privilege illusion” that degrades on‑policy distillation when privileged information is injected, and introduces Dual On‑policy Distillation (DOPD), a dynamic two‑stream approach that separates true ability gaps from information gaps, achieving superior performance and stability across LLM and VLM benchmarks.

DOPDOPDVision-Language Models

0 likes · 13 min read

Uncovering the Privilege Illusion in OPD Distillation and How DOPD Solves It

Machine Learning Algorithms & Natural Language Processing

Jun 28, 2026 · Artificial Intelligence

Why the Log‑Ratio Reward in OPD Is Fundamentally Flawed and Should Be Replaced

The paper reveals that the unbounded log‑ratio reward used in vanilla On‑Policy Distillation causes extreme gradient variance, early‑stage instability, and poor final performance, and demonstrates that replacing the log with a bounded Box‑Cox power transform (PowerOPD) resolves these issues while improving accuracy, efficiency, and memory usage.

Box-CoxOPDlarge language models

0 likes · 16 min read

Why the Log‑Ratio Reward in OPD Is Fundamentally Flawed and Should Be Replaced

Machine Learning Algorithms & Natural Language Processing

Jun 21, 2026 · Artificial Intelligence

xOPD Evolution: Mapping Recent OPD Improvements – Rephrased Same Problems vs. New Modules

This article surveys the latest on‑policy distillation (OPD) research, categorizing each work as either a reinterpretation of an existing problem or a modification of a different module, and highlights the experimental findings, design choices, and trade‑offs reported across the papers.

LLMOPDOn-Policy Distillation

0 likes · 31 min read

xOPD Evolution: Mapping Recent OPD Improvements – Rephrased Same Problems vs. New Modules

Machine Learning Algorithms & Natural Language Processing

Jun 16, 2026 · Artificial Intelligence

SFT, DAgger, Offline RL, and OPD: Four Methods Mapped onto a Single 2×2 Grid

The paper shows that SFT, DAgger, offline RL and OPD are the four orthogonal combinations of prefix source (teacher vs. student) and KL direction (forward vs. reverse), exposing three hidden trade‑offs—KL direction, prefix source, and training length—and proposes KL‑mixing and entropy‑gated length curricula that boost Avg@k by 3.6 points, raise Pass@k by up to 5.8 points, and cut response length by three‑fold.

DAggerKL divergenceLLM distillation

0 likes · 17 min read

SFT, DAgger, Offline RL, and OPD: Four Methods Mapped onto a Single 2×2 Grid

Baobao Algorithm Notes

May 26, 2026 · Artificial Intelligence

How On-Policy Distillation (OPD) Solves Core Challenges in Large-Model Post-Training

The article explains how On-Policy Distillation (OPD) combines on‑policy sampling with dense teacher feedback via reverse KL to address low signal density, distribution shift, and capability interference in large‑model post‑training, and compares implementations by Qwen3, GLM‑5, MiMo‑V2 and DeepSeek‑V4.

OPDOn-Policy DistillationReverse KL

0 likes · 20 min read

How On-Policy Distillation (OPD) Solves Core Challenges in Large-Model Post-Training

Machine Learning Algorithms & Natural Language Processing

Apr 14, 2026 · Artificial Intelligence

Revisiting On-Policy Distillation (OPD): Typical Failures and a More Stable Fix

On‑Policy Distillation (OPD) is widely used for post‑training large language models, but the sampled‑token variant often becomes unstable due to token‑level reward imbalance, teacher‑student signal mismatch on student‑generated prefixes, and tokenizer mismatches; this article analyses the bias‑variance trade‑off, identifies three root failure modes, and proposes a teacher‑top‑K local‑support‑set objective with top‑p rollout and special‑token masking that yields more stable training and better performance on both math and agentic benchmarks.

OPDOn-Policy Distillationlarge language models

0 likes · 32 min read

Revisiting On-Policy Distillation (OPD): Typical Failures and a More Stable Fix

Machine Heart

Apr 14, 2026 · Artificial Intelligence

Why Binary Success Rate Is Obsolete: Introducing PRM-as-a-Judge for Dense Evaluation of Embodied Tasks

The article critiques binary success rate for long‑horizon robotic tasks, proposes the PRM-as-a-Judge framework with a potential‑based progress signal and the three‑layer OPD metric suite, validates it on the RoboPulse benchmark, and shows how it yields fine‑grained, diagnostic insights into policy performance.

OPDRoboPulsedense metrics

0 likes · 20 min read

Why Binary Success Rate Is Obsolete: Introducing PRM-as-a-Judge for Dense Evaluation of Embodied Tasks