Tagged articles
GRPO
87 articles · Page 1 of 1
Nanjing University's Skill Adapter framework separates general skills from personalized adapters and uses online routing with GRPO optimization to dynamically select skills during agent execution, achieving 11.43% average success rate improvement across ALFWorld, WebShop, and SkillsBench benchmarks while reducing execution time.
ALFWorldAdaptive SkillsAgent Skill Routing
0 likes · 7 min read

FireFlash, a 300B-parameter terminal and coding agent trained via full-parameter RL on 64 NVIDIA B300 GPUs in under five days, achieves Terminal-Bench 4.0 Pass@1 of 46.2%—matching Fable 5 and GPT-6 Sol—while reducing per-task cost by over 30x, with gains transferring to DeepSWE and SWE-Bench benchmarks.
FireFlashGRPOLong-Horizon Agents
0 likes · 22 min read

Researchers introduce FireFlash, a 300B-parameter terminal and coding agent trained via full-parameter RL from GLM-5.3-Flash on 64 B300 GPUs in under five days, achieving Terminal-Bench 4.0 Pass@1 of 46.2%—matching Fable 5 and GPT-6 Sol—while reducing per-task cost by over 30x through optimized infrastructure, task filtering, reward shaping, harness design, and a corrected GRPO objective.
GRPOLong-Horizon AgentsOpen Recipe
0 likes · 21 min read

Tsinghua researchers show that training diffusion language models with left-to-right token unmasking and GRPO significantly improves math and code reasoning performance, reducing entropy at logical branching points and outperforming flexible decoding strategies on GSM8K and MATH-500 benchmarks.
Entropy ReductionGRPOICML 2026
0 likes · 11 min read

A 26-page survey from HUST, Tsinghua, and other institutions reviews how reinforcement learning integrates with diffusion models across language, vision, robotics, and scientific computing, covering methods like RLHF, DPO, GRPO, and challenges in reward design, exploration, and sparse rewards.
DPOGRPOMultimodal Generation
0 likes · 13 min read

HKU researchers propose RaPO, a reinforcement fine-tuning method that adds a retention reward based on trajectory-level drift to prevent catastrophic forgetting in multimodal large language models during visual continual learning, achieving state-of-the-art results on five benchmarks with minimal forgetting.
Catastrophic ForgettingContinual LearningGRPO
0 likes · 16 min read

Researchers from Fudan University, CUHK MMLab, and Shanghai Jiao Tong University introduce ReTerm, a recursive self-improvement framework where code agents learn to diagnose and recover from their own terminal failures, turning self-generated error trajectories into training data that boosts 9B model performance on Terminal-Bench, SWE-bench, and OpenThoughts-TBLite by over 13 percentage points.
Code AgentsGRPOReTerm
0 likes · 9 min read

Shanghai Jiao Tong University researchers introduce ZwZ2 and the EvidenceConnect framework, which synthesizes 20.4K single-image training samples requiring models to connect evidence across multiple regions, yielding 4B, 9B, and 27B models that surpass base models on 25 benchmarks and transfer gains to multi-image, spatial reasoning, and agent tasks.
Data SynthesisEvidenceConnectGRPO
0 likes · 14 min read

This article reviews DeepSeek's 10 most impactful open-source projects, spanning math reasoning (GRPO), MoE communication (DeepEP), distributed storage (3FS), Hopper-optimized kernels (FlashMLA), production inference engine (DeepInfer), optical document compression (DeepSeek-OCR), code generation (Coder-V2), pure RL reasoning (R1), efficient Mixture-of-Experts training (V3), and next-gen agent infrastructure (Harness).
DeepSeekGRPOInference Optimization
0 likes · 25 min read

FinATOM introduces a unified head-free interface where a LoRA-adapted Llama 3.2 1B model directly generates financial predictions and portfolio allocations via constrained token generation, using supervised fine-tuning with a causal mean-variance teacher and DAPO-enhanced GRPO policy optimization to improve Sharpe ratios on ETF and stock benchmarks.
DAPOETFFinATOM
0 likes · 17 min read

Meituan and Peking University propose ClawLoop, a Harness-native framework closing the loop of task generation, model training, and diagnostic feedback, enabling agents to self-improve without stronger teacher models; experiments on Qwen3.6-27B show consistent gains across three benchmarks after three iterations, with SkillsBench rising from 46.3 to 57.2.
Agent Self-ImprovementClawLoopGRPO
0 likes · 11 min read

Researchers from Georgia Tech and MIT propose MASCOT, a two-stage training framework that teaches AI agents distinct personas and a director agent to coordinate turn-taking, enabling diverse, coherent multi-agent conversations for emotional support and meeting scenarios.
Conversational AIEMNLP 2026GRPO
0 likes · 16 min read

Xiaohongshu and Fudan University propose DR-Rubric, a framework where models perform deep research to gather evidence and synthesize atomic constraints as rubrics for reinforcement learning rewards, achieving competitive results on six benchmarks with only 1,000 training samples while revealing non-monotonic bootstrap dynamics and reward polarization as a stopping signal.
DR-RubricDeep ResearchGRPO
0 likes · 21 min read

SPADER introduces a step-wise GRPO framework with diversity-aware exploration rewards to improve long-tail answer coverage in multi-answer question answering, achieving state-of-the-art results across four benchmarks and two base models by refining credit assignment and incentivizing rare entity discovery.
Diversity-Aware RewardsEMNLP 2026GRPO
0 likes · 16 min read

A large-scale empirical study comparing GRPO with 10 self-distillation methods across 4 tasks and 3 model sizes finds no method consistently beats GRPO; some methods show high final scores but suffer training instability, and compute-equivalent comparisons often reverse apparent gains.
GRPOReinforcement LearningReproducibility
0 likes · 10 min read

Meta released Muse Spark Open, an 816B parameter multimodal model for coding and long-horizon agents, accompanied by a complete post-training ecosystem including TorchTitan, SGLang, vLLM, Miles, and verl, with experimental results showing GRPO training improvements on math and geometry benchmarks, though Chinese models like Qwen, GLM, and Kimi already surpass it on coding benchmarks and have similar training infrastructure.
Chinese AI modelsGRPOLoRA
0 likes · 8 min read

Tsinghua researchers propose Jev-LDE, a 1.7B model that performs a single local edit (Keep, Delete, Replace) on retrieved demonstrations for in-context learning, improving average 1-shot classification accuracy from 81.2% to 88.1% across multiple datasets and target LLMs with only 11% inference overhead.
Demonstration SelectionFew-Shot ClassificationGRPO
0 likes · 17 min read

This article chronicles DeepSeek's paper timeline from 2023 to 2026, analyzing how each model generation optimized training compute, KV cache, and throughput via MoE, MLA, sparse attention, and conditional memory, shifting long-context competition from maximum length to per-token byte cost.
DeepSeekGRPOKV cache optimization
0 likes · 37 min read

HOMURA introduces a reinforcement learning framework with KL regularization and dynamic syllable-ratio rewards to achieve precise length control in LLM translation for subtitles and dubbing, outperforming baselines on the Sand-Glass benchmark and deployed at scale on Bilibili.
EMNLP 2026GRPOHOMURA
0 likes · 21 min read

The article details a rigorous agent evaluation framework for after-sales operations using GRPO, revealing why a 93% overall success rate masks critical failures in specific scenarios and safety guardrails, and how to structure evaluations to withstand interviewer scrutiny.
Agent EvaluationBucket AnalysisGRPO
0 likes · 18 min read

This paper introduces GRC, a generation-reflection-correction framework for generative recommendation that enables models to self-diagnose and correct full draft semantic ID sequences via structured reflection labels, GRPO reinforcement learning, and entropy-guided scheduling, achieving state-of-the-art results on public and industrial datasets with minimal latency overhead.
CIKM 2026GRPOReflection
0 likes · 12 min read

This article breaks down six high-frequency GRPO interview questions from top Chinese tech companies, covering GRPO vs PPO trade-offs, group-relative advantage calculation with concrete numbers, handling all-correct/all-wrong sample groups, KL constraint mechanics, convergence monitoring priorities, and reward hacking detection via shadow evaluation.
Advantage EstimationGRPOKL Divergence
0 likes · 19 min read

The article details a rule-based verifier for Agentic RL post-training in after-sales automation, replacing LLM scoring with structured fact-checking across five weighted dimensions and eleven guardrails to prevent reward hacking, achieving 88% human agreement and boosting task success from 8% to 93%.
GRPORLHFReward Design
0 likes · 23 min read

V-Rubrics decomposes 50,248 visual samples into 352,938 atomic rubrics across visual faithfulness, reasoning consistency, and instruction following, enabling fine-grained credit assignment in GRPO training that lifts Overall Avg. to 68.04 on general benchmarks and 62.45 on visual math and logic tasks.
Credit AssignmentEMNLP 2026GRPO
0 likes · 13 min read

This article surveys the 2026 reinforcement learning landscape for large language models, detailing foundational algorithms (PPO, DPO, GRPO), advanced GRPO variants (DAPO, GSPO, GMPO, GFPO), emerging Agentic RL methods (ARPO, Tree-GRPO), and a practical scenario-based selection table.
Algorithm SelectionDPOGRPO
0 likes · 12 min read


Amap TechSep 23, 2026 · Artificial Intelligence
Amap details its system for extracting navigation landmarks from street-view imagery using multimodal large models, covering data annotation standards, a three-stage training pipeline (SFT, DPO, GRPO with a risk-aware reward), bounding-box-free evaluation, offline production, online integration, and A/B tests showing 23.5% faster user departure.
DPOGRPOSFT
0 likes · 26 min read

The article explores how batch size scaling impacts LLM reinforcement learning training efficiency, introducing critical generation and training batch concepts, showing optimal batch size balances sample efficiency and system throughput, with experiments achieving 29% time reduction on fixed hardware.
Batch Size ScalingCritical BatchGRPO
0 likes · 36 min read

FinSMART introduces a market-aligned reinforcement learning framework that trains LLMs for financial sentiment analysis using realized market returns instead of human annotations, employing GRPO with a dual-filter trading reward to achieve 220% higher cumulative returns than the previous state-of-the-art FinDPO.
FinSMARTGRPOLLM
0 likes · 23 min read

The paper introduces BehR, a behavior consistency reward for training text-based world models, showing that optimizing for agent decision alignment rather than text fidelity improves trajectory-level consistency across 16 configurations, reduces false positives in offline evaluation from 42.5% to 9.5%, and enhances lookahead planning for weaker agents.
Behavior ConsistencyEMNLP 2026GRPO
0 likes · 11 min read

AReaL-Ascend v1.0.5 introduces two major innovations: colocated training and inference on shared NPUs via Ray scheduling and AWEX IPC zero-copy, and Multi-Teacher On-Policy Distillation (MOPD) that fuses domain expert models into a single student using token-level teacher signals, demonstrated on Qwen3.6-27B with GRPO.
AReaLAWEXAscend NPU
0 likes · 12 min read

The article presents a from‑scratch LLM training framework called grpo‑llm, implements GRPO with Trio rollout, FSDP and a C++ reward extension, and conducts a controlled experiment comparing GRPO, PPO and DPO on the GSM8K math‑reasoning benchmark, revealing why DPO outperforms the other two under sparse binary rewards.
DPOGRPOGSM8K
0 likes · 10 min read

DeepSeek released the V4‑Flash model with the same architecture as the preview but a revamped post‑training pipeline—SFT, reinforcement learning with GRPO, and distillation—yielding dramatic benchmark jumps and illustrating how post‑training now defines the model's real‑world capabilities.
DeepSeekGRPOLLM training
0 likes · 11 min read

The paper identifies the broadcast‑style credit assignment of GRPO as a bottleneck for RL‑LLM math reasoning, proposes the Outcome‑Grounded Advantage Reshaping (OAR) framework with token‑importance estimation, and demonstrates that its two variants, OAR‑P and OAR‑G, consistently improve accuracy, training efficiency, and stability across multiple math benchmarks.
Credit AssignmentGRPOLLM
0 likes · 15 min read

Teams building LLM‑powered products often hit a wall where 30‑40% of responses are wrong and the model never learns from mistakes; the article explains how modern fine‑tuning using GRPO‑based reinforcement learning and the open‑source ART framework, together with the RULER reward‑free evaluator, lets small open‑source models surpass larger ones in cost, latency, and accuracy.
ART frameworkGRPOLLM fine-tuning
0 likes · 9 min read

GLM‑5.2 replaces the Group Relative Policy Optimization (GRPO) algorithm with a critic‑based PPO approach for long‑horizon tasks, arguing that GRPO’s group comparison breaks down on variable‑length trajectories, a shift that has sparked vigorous debate across the reinforcement‑learning community.
DeepSeekGLM-5.2GRPO
0 likes · 10 min read

This article surveys five years of large‑model reinforcement learning, detailing the evolution from PPO + RLHF to DPO and GRPO, comparing reward‑model‑based and verifiable‑reward approaches, discussing multi‑agent extensions, and evaluating open‑source frameworks for training LLM‑driven agents.
AI AlignmentDPOGRPO
0 likes · 34 min read

This article reviews the five‑year evolution of reinforcement‑learning techniques for large language models, comparing PPO, DPO, GRPO and emerging multi‑agent approaches, analyzing their reward signals, practical trade‑offs, and the open‑source frameworks that support them.
DPOGRPOLLM
0 likes · 34 min read

The article reviews five years of LLM‑centric reinforcement learning, tracing the evolution from early Q‑learning to PPO, then to Direct Preference Optimization, Group Relative Policy Optimization, and finally multi‑agent RL, detailing each method’s mechanics, strengths, failure modes, practical considerations, and emerging open‑source toolchains.
DPOGRPOLLM Alignment
0 likes · 33 min read

The article analyzes why DeepSeek’s large‑model training struggles with Monte‑Carlo Tree Search, explains its use of Chain‑of‑Thought prompting, GRPO entropy‑boosting and rejection‑sampling fine‑tuning, compares these methods with Google’s OmegaPRM and PRM approaches, and proposes a concrete MCTS‑driven data‑generation pipeline to overcome the “chicken and liquor” trade‑off.
DeepSeekGRPOMonte Carlo Tree Search
0 likes · 14 min read

The article examines why DeepSeek’s large‑model training cannot yet leverage Monte‑Carlo Tree Search, detailing its reliance on SFT, GRPO‑driven CoT activation and rejection‑sampling, contrasting this with Google’s PRM‑based approaches, and proposing a MCTS‑powered data‑generation pipeline to overcome the “roast chicken and baijiu” training dilemma.
GRPOMonte Carlo Tree SearchRejection Sampling
0 likes · 14 min read

This article surveys the rapid evolution of reinforcement‑learning algorithms for large‑language‑model inference from early REINFORCE and PPO to newer approaches such as GRPO, RLOO, DAPO, CISPO, DPPO, ScaleRL and MaxRL, highlighting their design motivations, mathematical formulations, empirical trade‑offs and open research challenges.
GRPOLLMMaxRL
0 likes · 27 min read

This article explains why interviewers use seemingly rote questions to probe a candidate's true grasp of concepts, contrasts memorization with genuine understanding using PPO vs GRPO, and provides a practical three‑question framework and dialogue examples to help candidates answer technical principle questions confidently.
Answering TechniquesGRPOInterview
0 likes · 12 min read

This article systematically reviews the core post‑training techniques for large language models—including supervised fine‑tuning, RLHF, PPO, GRPO, DPO, RLVR and Agentic RL—explains their evolution, compares their trade‑offs, and highlights the most promising research directions for 2025‑2026.
AI AlignmentGRPOLLM
0 likes · 20 min read

This article explains the full lifecycle of large language models in 2026, covering pretraining fundamentals, the limits of classic Scaling Laws, data‑centric advances, fine‑tuning strategies, RLHF, DPO, and the emerging post‑training methods GRPO, DAPO and RLVR, with concrete benchmarks and cost analyses.
DAPODPOGRPO
0 likes · 17 min read

This article systematically explains the post‑training pipeline for large language models, covering supervised fine‑tuning, RLHF, PPO, GRPO, RLVR, DPO and emerging Agentic RL, while illustrating each method with analogies, detailed workflows, tables, and recent research findings.
DPOGRPOLLM
0 likes · 24 min read

A new MIT paper reveals that pretrained large models already contain many hidden expert submodels, and that a simple one‑step Gaussian perturbation (RandOpt) can locate and ensemble these experts to achieve performance comparable to or better than traditional GRPO/PPO tuning, especially as model size grows.
GRPOModel ScalingPPO
0 likes · 9 min read

This article walks through the fundamental derivation of policy‑based reinforcement learning, explains how traditional RL concepts extend to large‑language‑model RL, and details engineering enhancements such as GRPO memory reduction, asynchronous rollout, importance‑sampling corrections, and token‑flow management for stable industrial‑scale training.
GRPOImportance SamplingRLHF
0 likes · 11 min read

The article walks through the fundamentals of traditional policy‑gradient reinforcement learning, derives the Reinforce objective, maps its concepts to large‑language‑model RL, and then discusses practical engineering solutions such as GRPO, async rollout, importance‑sampling corrections, and token‑flow management for industrial‑scale training.
Async RolloutGRPOImportance Sampling
0 likes · 10 min read

After DPO, the typical research trajectory moves through GRPO, DAPO, GSPO, and SAPO, each introducing new optimization objectives, sampling strategies, and reward‑shaping techniques that aim to reduce memory usage, improve gradient stability, and enhance the efficiency of large‑model reinforcement learning.
DAPOGRPOGSPO
0 likes · 6 min read

This article systematically reviews the main reinforcement‑learning algorithms—PPO, GRPO, DAPO, GSPO, and SAPO—used for fine‑tuning large language models, explaining why supervised fine‑tuning precedes RL, how each method improves training efficiency and stability, and what trade‑offs they entail.
GRPOPPORL
0 likes · 15 min read

Alibaba's new AgeMem framework turns long‑term and short‑term memory management for large language model agents into a learnable reinforcement‑learning task, replacing handcrafted rules with a three‑stage training process and achieving significant benchmark gains.
AgeMemGRPOLLM
0 likes · 9 min read

The article explains that the real difficulty of RLHF lies in designing and curating high‑quality preference data, building robust reward models through bad‑case rewriting, human‑in‑the‑loop labeling, and inference‑based reward modeling, while algorithmic details like PPO are secondary concerns.
Data EngineeringGRPORLHF
0 likes · 9 min read

This article provides a detailed, easy‑to‑understand analysis of DeepSeek‑Math‑V2’s self‑verification training method and DeepSeek‑V3.2’s GRPO framework, sparse‑attention DSA mechanism, massive agent data pipeline, and benchmark results that place both models among the world’s top open‑source large language models.
DeepSeekGRPOLLM
0 likes · 19 min read

This article analyzes the RETuning framework, which addresses LLMs' bias toward analyst opinions and lack of evidence weighting in stock movement prediction by introducing a two‑stage cold‑start fine‑tuning and reinforcement learning pipeline, evaluating it on the large Fin‑2024 dataset and demonstrating significant F1 gains, inference‑time scaling, and out‑of‑distribution robustness.
Fin-2024GRPOInference Scaling
0 likes · 12 min read

DeepSeekMath‑V2, released open‑source on 27 Nov 2025, attains gold‑level results on IMO 2025, scores 118 out of 120 on the Putnam 2024 competition, introduces a generator‑verifier self‑verification architecture, uses GRPO training, and outperforms leading closed‑source models on IMO‑ProofBench.
DeepSeekMath-V2GRPOLLM
0 likes · 7 min read

The article introduces the TLM large‑model development platform, details its fine‑tuning options, explains reinforcement learning fundamentals and key algorithms such as PPO and the newer GRPO, describes the architecture of a reinforcement evaluator, and shows how to configure RL training on the platform.
AI PlatformGRPOLLMOps
0 likes · 10 min read

This article examines how reinforcement‑learning techniques such as PPO, DPO, and GRPO are integrated into the Baixiaosheng QA system to improve answer stability, deepen domain knowledge understanding, and accelerate response generation, and it evaluates the impact of Reinforcement Fine‑Tuning (RFT) on real‑world performance.
AIDPOGRPO
0 likes · 16 min read

GSPO (Group Sequence Policy Optimization) is a reinforcement‑learning algorithm for LLMs that replaces token‑level GRPO with sequence‑level optimization, addressing instability in ultra‑large model training, especially for long‑sequence and MoE architectures, by aligning reward granularity and reducing variance.
GRPOGSPOReinforcement Learning
0 likes · 11 min read

This article details how Bilibili’s multimedia lab leveraged a multimodal training pipeline combining data‑compressed SFT and the GRPO reinforcement‑learning algorithm to achieve a 13.5% metric boost and secure second place in the ICCV MIPI Detailed Image Quality Assessment competition.
GRPOMIPI competitionReinforcement Learning
0 likes · 15 min read

Shopify’s engineering team explains how they transformed the Sidekick AI assistant from a simple tool‑calling system into a robust, production‑grade AI agent platform, sharing architectural, evaluation and training lessons to help others avoid common pitfalls.
AI agentsGRPOJust-in-Time instructions
0 likes · 12 min read

DeepSeek‑V3.2‑Exp, the latest experimental large‑language model, is open‑sourced with a paper, featuring a million‑token context window, a new sparse attention mechanism, GRPO‑enhanced reasoning, and detailed cost‑analysis showing up to ten‑fold inference savings.
DeepSeekGRPOInference Optimization
0 likes · 5 min read

The article explains the fundamental principles of PPO and GRPO reinforcement‑learning algorithms, compares their architectures and training workflows, highlights why GRPO is gaining traction in large‑model fine‑tuning, discusses associated risks, and offers practical guidance on group size selection for engineers preparing for interviews.
GRPOPPORLHF
0 likes · 9 min read

DeepSeek R1 replaces traditional supervised fine‑tuning with a pure reinforcement‑learning pipeline, introducing the GRPO algorithm and a four‑stage training regime that dramatically lowers cost, boosts reasoning and code‑generation performance, and raises important ethical, privacy, and societal considerations for large language models.
AI ReasoningDeepSeekGRPO
0 likes · 14 min read

DeepSeek‑R1, the first peer‑reviewed large language model, landed on Nature’s cover after a novel reinforcement‑learning‑only training pipeline that dramatically boosted reasoning performance while keeping training costs surprisingly low.
DeepSeekGRPOModel Training
0 likes · 14 min read

DeepSeek‑R1, the first peer‑reviewed large language model, leveraged a pure reinforcement‑learning framework and the novel GRPO algorithm to achieve breakthrough reasoning performance, low training cost, and widespread acclaim, culminating in a Nature magazine cover story.
AI ReasoningDeepSeekGRPO
0 likes · 14 min read

This article explains the GRPO algorithm, an improvement over PPO for large language model training that eliminates the value network, uses group‑relative advantage estimation, and offers flexible supervision, resulting in higher efficiency, stability, and performance on tasks such as mathematical reasoning.
GRPOLLM trainingPPO
0 likes · 16 min read

This article provides a comprehensive technical analysis of PPO‑based reinforcement learning methods for large language models, detailing the evolution from the original PPO algorithm through GRPO, DAPO, and GSPO, and explaining their motivations, mathematical formulations, advantages, and practical challenges such as entropy collapse and importance‑sampling variance.
DAPOGRPOGSPO
0 likes · 30 min read

This article explains the motivation, mathematical foundations, implementation details, advantages, experimental results, and future directions of Group Relative Policy Optimization (GRPO), a novel reinforcement‑learning algorithm that replaces PPO’s value network with efficient group‑wise relative evaluation for large language models.
GRPOLLMPPO
0 likes · 17 min read

Perception‑R1, a post‑training framework that applies rule‑based reinforcement learning to existing multimodal LLMs, dramatically improves visual perception tasks such as grounding, OCR, counting and object detection, as demonstrated by extensive benchmarks and ablation studies.
GRPOPerception PolicyReinforcement Learning
0 likes · 10 min read

The paper identifies that GRPO’s token‑level importance weighting introduces high‑variance noise causing instability in large‑scale language model RL training, and proposes GSPO, a sequence‑level importance sampling method that aligns with reward definitions, improves gradient stability, and yields higher training efficiency and better performance across benchmarks.
GRPOGSPORL
0 likes · 8 min read

This article explains post‑training concepts, compares SFT, DPO, and GRPO fine‑tuning methods, and provides step‑by‑step guidance for using Alibaba Cloud's PAI platform—including Model Gallery and DSW—to fine‑tune large language models with code examples and practical tips.
DPOGRPOLLM
0 likes · 14 min read

This article explains the GRPO reinforcement‑learning algorithm, shows its core idea of internal group competition without a separate evaluator model, and provides a complete, step‑by‑step code walkthrough—including environment setup, dataset preparation, reward‑function design, training configuration, and evaluation—using the Qwen2.5‑0.5B‑Instruct model on the GSM8K math dataset.
GRPOGSM8KQwen2.5
0 likes · 23 min read

The article explains GRPO, a reinforcement‑learning algorithm that extends PPO with group sampling, no value network, dual penalties and KL regularisation, showing how it improves efficiency and stability when fine‑tuning large language models such as DeepSeek‑Math and DeepSeek‑R1.
DeepSeekGRPOPPO
0 likes · 6 min read

This article critically examines R1‑Zero‑style training by analyzing foundation models and reinforcement learning, uncovering pre‑training and optimization biases, proposing an unbiased Dr. GRPO method, and demonstrating a minimalist 7B‑model recipe that achieves new state‑of‑the‑art performance on AIME 2024.
GRPOLLM EvaluationR1-Zero
0 likes · 20 min read

The article analyzes the DAPO technical report, showing how dynamic‑sampling pipelines and token‑level loss handling in SFT and RL training outperform ad‑hoc algorithm tricks, and compares the training dynamics of reinforce_baseline and GRPO with concrete code examples.
Dynamic SamplingGRPOLLM
0 likes · 8 min read

The article explains why the GRPO loss in OpenR1 and trl starts at zero and then rises, detailing the underlying KL‑divergence formulation, the single‑step update mechanism, and how gradients are preserved despite a zero scalar loss, with code examples from the trl implementation.
GRPOLoss InitializationOpenR1
0 likes · 5 min read


ArchitectMar 17, 2025 · Artificial Intelligence
This article details a reinforcement‑learning experiment that teaches 7B‑ and 3B‑parameter language models to solve Sudoku, covering data preparation, GRPO‑based reward design, training configurations, performance comparisons, key insights, and future research directions.
GRPOModel ScalingReinforcement Learning
0 likes · 15 min read

DeepSeek‑R1 replaces the traditional PPO‑based RLHF approach with GRPO, reducing reliance on human‑labeled data by using pure reinforcement learning environments and carefully designed reward mechanisms; the article explains reinforcement learning fundamentals, compares PPO, DPO and GRPO, and offers practical application recommendations.
AI AlignmentDPOGRPO
0 likes · 14 min read


ArchitectMar 16, 2025 · Artificial Intelligence
This article walks through the complete lifecycle of building a small large‑language model, covering token‑level inference, pre‑training, post‑training steps such as supervised fine‑tuning, reward‑model creation, and reinforcement‑learning methods like DPO, PPO and GRPO, culminating in a practical 0.5B model fine‑tuned for chain‑of‑thought reasoning.
GRPOLLM trainingReinforcement Learning
0 likes · 22 min read

This article details how a 7B parameter language model, fine‑tuned with DeepSeek's GRPO reinforcement‑learning algorithm and a carefully crafted multi‑component reward system, learned to solve Sudoku puzzles without any cold‑start data, outperforming a comparable 3B model and revealing key insights for structured reasoning tasks.
AI TrainingGRPOQwen
0 likes · 15 min read


ArchitectMar 10, 2025 · Artificial Intelligence
This article analyzes DeepSeek’s latest large‑model breakthroughs, covering the MLA attention compression, GRPO alignment algorithm, MoE load‑balancing redesign, multi‑stage training pipelines, reinforcement‑learning tricks, and performance comparisons with GPT‑4o‑Mini and Llama 3.1, highlighting both strengths and remaining challenges.
AI TrainingDeepSeekGRPO
0 likes · 19 min read

This tutorial explains how to build a distributed reinforcement‑learning pipeline using the GRPO algorithm, covering data preparation, evaluation and reward functions, multi‑GPU DataParallel implementation, and full fine‑tuning of the Qwen2.5‑1.5B‑Instruct model with PyTorch, FlashAttention2 and Weights & Biases.
AIGRPOPyTorch
0 likes · 10 min read

The article reviews reinforcement-learning fundamentals and the progression from policy-gradient to PPO, then introduces Group Relative Policy Optimization (GRPO)—a critic-free method that normalizes rewards across multiple sampled outputs to compute group-relative advantages—and shows how DeepSeek-R1 leverages GRPO with rule-based rewards to achieve strong reasoning performance.
GRPOPPOPolicy Optimization
0 likes · 16 min read


DevOpsFeb 23, 2025 · Artificial Intelligence
This article explains how DeepSeek‑R1‑Zero uses group‑relative policy optimization (GRPO) to enhance inference without labeled data, introduces reinforcement learning with human feedback (RLHF) and its components, and compares the PPO and GRPO algorithms, highlighting their suitable engineering scenarios and practical implications for AI applications.
AI model trainingGRPOPPO
0 likes · 15 min read

DeepSeek‑R1 demonstrates that large‑scale reinforcement learning, especially with the novel Group Relative Policy Optimization and a rule‑based reward scheme, can markedly boost reasoning in LLMs without heavy supervised fine‑tuning, while a brief cold‑start SFT phase, two‑stage alignment, and knowledge distillation further improve performance and efficiency, despite remaining challenges such as language mixing.
DeepSeek-R1GRPOLLM reasoning
0 likes · 21 min read

This article explains how to replicate Deepseek RI's slow‑thinking inference using the GRPO reinforcement‑learning algorithm on the Qwen2.5‑7B model in a free Colab notebook, covering the underlying COT concept, reward‑function design, data preparation, training configuration, and observed results.
GRPOLLMPython
0 likes · 14 min read

DeepSeek AI’s new open‑source model DeepSeek‑R1 leverages a novel Group‑Related Policy Optimization (GRPO) reinforcement‑learning framework and multi‑stage training to dramatically boost complex reasoning performance, achieving AIME 2024 Pass@1 scores comparable to OpenAI’s o1 model.
AIDeepSeekGRPO
0 likes · 4 min read
