Skip to content
Tagged articles

Inverse Probability Weighting

1 articles · Page 1 of 1
Machine Heart
Machine Heart
Nov 18, 2026 · Artificial Intelligence

STOPD: Lossless OPD Acceleration via Random Truncation and Inverse Probability Weighting

STOPD introduces random truncation with inverse probability weighting and block-wise stopping to accelerate on-policy distillation, reducing generated tokens by 25-48% and achieving 1.3-2.2x training speedup while maintaining accuracy comparable to vanilla OPD, with theoretical guarantees of unbiased gradient estimation and error bounds for SGD, Adam, and Muon optimizers.

Gradient EstimationInverse Probability WeightingLarge Language Models
0 likes · 14 min read
STOPD: Lossless OPD Acceleration via Random Truncation and Inverse Probability Weighting