KL constraint — 2 Technical Articles

Jun 13, 2025 · Artificial Intelligence

How GVPO Improves LLM Fine‑Tuning: Stable, Sample‑Rich Policy Optimization

The article introduces GVPO, a Group Variance Policy Optimization method that uniquely achieves KL‑constrained reward maximization, supports diverse sampling distributions, and resolves instability and inefficiency issues found in GRPO and traditional policy‑gradient approaches for large language model post‑training.

GVPOKL constraintpolicy optimization

0 likes · 9 min read

How GVPO Improves LLM Fine‑Tuning: Stable, Sample‑Rich Policy Optimization

Baobao Algorithm Notes

Feb 10, 2025 · Artificial Intelligence

Why Base‑Model RL Beats Traditional SFT‑RL: Theory, Practice, and Zero‑RL Insights

The article analyzes how applying reinforcement learning directly on base LLMs offers theoretical advantages, practical guidance, and experimental evidence that surpasses conventional cold‑start SFT‑RL pipelines, while also exploring zero‑RL approaches, KL constraints, and scaling considerations.

KL constraintbase-model RLzero-shot RL

0 likes · 11 min read

Why Base‑Model RL Beats Traditional SFT‑RL: Theory, Practice, and Zero‑RL Insights