Why the Log‑Ratio Reward in OPD Is Fundamentally Flawed and Should Be Replaced

The paper reveals that the unbounded log‑ratio reward used in vanilla On‑Policy Distillation causes extreme gradient variance, early‑stage instability, and poor final performance, and demonstrates that replacing the log with a bounded Box‑Cox power transform (PowerOPD) resolves these issues while improving accuracy, efficiency, and memory usage.

Box-CoxLarge language modelsOPD

0 likes · 16 min read

Why the Log‑Ratio Reward in OPD Is Fundamentally Flawed and Should Be Replaced

Model Perspective

Sep 6, 2023 · Fundamentals

How Box‑Cox Transformation Turns Skewed Data Into Normal Distributions

Box‑Cox transformation, introduced by Box and Cox in 1964, corrects skewed data to approximate normality by optimizing a λ parameter via maximum likelihood, enabling more accurate statistical modeling and machine‑learning predictions, as demonstrated with a crime‑rate dataset and Shapiro‑Wilk tests.

Box-CoxNormalityPython

0 likes · 8 min read

How Box‑Cox Transformation Turns Skewed Data Into Normal Distributions