Boosting Large Model Training: Optimizing Performance with the Verl Framework

On July 12, 09:00‑17:00, the DataFun Summit 2025 will host a session on large‑model capability leap, featuring Tencent FinTech senior researcher Gong Dihong.

Speaker bio: Gong holds a Ph.D. in Computer Science from the University of Florida, has published over 40 papers with 5,000+ citations, and serves as a reviewer for top AI venues (TIP, PAMI, ICLR, ICML, KDE, IJCV). He holds 13 AI‑related patents, six granted.

Talk title: Training Performance Optimization Based on the Verl Framework

Abstract: Verl is an open‑source large‑model fine‑tuning framework supporting SFT and RL. To address its low training efficiency in reinforcement learning, the team redesigned the training system architecture, improving distributed training with the Megatron engine, integrating Sglang as the RL rollout engine, proposing an efficient Megatron‑Sglang parameter‑synchronization algorithm, and optimizing model offloading and data load balancing. Experiments on the QwQ‑32B model demonstrated significant speed gains.

Outline: 1) Overview of current large‑model training principles and challenges; 2) Introduction to the widely used Verl framework; 3) Detailed extensions and optimizations applied.

Audience takeaways: Learn the core principles of RL training systems and acquire concrete optimization techniques that can be transferred to other training frameworks to improve efficiency.

Implementation challenges:

The solution builds on the latest versions of Verl, Sglang, and Megatron, requiring manual updates when upstream code changes.

Because it relies on the Megatron engine, models need custom adaptation.