Tagged articles
policy gradient
18 articles · Page 1 of 1
The article analyzes why large language models cannot simply adopt AlphaGo’s Monte‑Carlo Tree Search, highlighting credit‑assignment difficulties, gradient‑variance explosion in multi‑step RL, and how AlphaGo’s tight integration of value and policy networks amortizes search in a way LLMs cannot replicate.
AlphaGoCredit AssignmentLLM
0 likes · 6 min read

The SIGGRAPH 2026 paper “Generative 3D Gaussians with Learned Density Control” introduces Density‑Sampled Gaussians (DeG), a differentiable framework that lets a model learn where to place Gaussian splats by sampling from a learned spatial density, enabling arbitrary‑budget, non‑uniform 3D representations with higher quality per cost.
3D Gaussian splattingSIGGRAPH 2026Tsinghua
0 likes · 14 min read

The article reviews the recent Intentional Updates framework—co‑authored by Turing laureate Richard Sutton—that redefines step‑size in streaming reinforcement learning using a 1967 NLMS‑style formula, details its algorithmic design, experimental validation, and remaining challenges.
Suttonintentional updatespolicy gradient
0 likes · 11 min read

This article provides a concise overview of a ten‑chapter reinforcement‑learning textbook, outlining the progression from basic concepts such as states and rewards to advanced algorithms like policy gradients and actor‑critic methods, and explains how each chapter builds on the previous ones.
Bellman equationMonte Carloactor-critic
0 likes · 11 min read

The article walks through the fundamentals of traditional policy‑gradient reinforcement learning, derives the Reinforce objective, maps its concepts to large‑language‑model RL, and then discusses practical engineering solutions such as GRPO, async rollout, importance‑sampling corrections, and token‑flow management for industrial‑scale training.
Async RolloutGRPOImportance Sampling
0 likes · 10 min read

This article walks through the fundamentals of reinforcement learning, builds a custom drone‑landing simulation, defines state and action spaces, designs reward functions, implements a neural‑network policy with Bernoulli sampling, and trains it using REINFORCE with baseline techniques, while exposing common pitfalls such as reward‑cheating.
OpenAI GymPythonReward Shaping
0 likes · 22 min read

The SPG algorithm introduces a sandwiched policy gradient that uses computable lower and upper evidence bounds to align reinforcement learning for discrete diffusion language models, achieving faster convergence, higher peaks, and lower variance on four major reasoning benchmarks.
EUBOSPGbenchmark
0 likes · 9 min read

This article analyzes why standard reinforcement learning can degrade Pass@K metrics after fine‑tuning large language models, introduces a risk‑sensitive RL objective that reshapes the advantage estimator, and demonstrates through bandit and mathematical‑reasoning experiments that the RS‑GRPO method consistently boosts diversity and overall Pass@K scores across multiple LLMs.
Exploration-ExploitationLLM fine-tuningRS-GRPO
0 likes · 12 min read


Amap TechMay 19, 2025 · Artificial Intelligence
The article introduces Group Policy Gradient (GPG), a reinforcement‑learning framework that eliminates surrogate loss functions and critic models, directly optimizes the original objective, reduces bias and variance, and achieves state‑of‑the‑art performance on both single‑modal and multimodal tasks.
AI researchLLM fine-tuningbias reduction
0 likes · 7 min read

This comprehensive guide examines large‑scale deep reinforcement learning, detailing policy‑gradient fundamentals, the mathematics of PPO and GAE, practical implementation tricks, reward and observation normalization, network initialization, and the newer Phasic Policy Gradient method, all supported by code snippets and key research references.
Deep RLGAEPPG
0 likes · 19 min read

The article explains why the GRPO loss in OpenR1 and trl starts at zero and then rises, detailing the underlying KL‑divergence formulation, the single‑step update mechanism, and how gradients are preserved despite a zero scalar loss, with code examples from the trl implementation.
GRPOLoss InitializationOpenR1
0 likes · 5 min read

The article chronicles how Andrew Barto and Richard Sutton’s four‑decade partnership transformed reinforcement learning from a theoretical curiosity into a cornerstone of modern AI, earning them the 2024 Turing Award and inspiring breakthroughs from AlphaGo to large‑language models.
AI historyBartoSutton
0 likes · 13 min read

This article provides a thorough, step‑by‑step explanation of reinforcement‑learning theory—covering policy‑based objectives, value‑function definitions, the derivation of policy gradients, actor‑critic architecture, advantage estimation, importance sampling, GAE, and the PPO algorithm—aimed at readers with little prior RL knowledge.
Advantage EstimationPPOactor-critic
0 likes · 31 min read

This article provides a comprehensive overview of reinforcement learning, covering fundamental concepts, differences from supervised learning, algorithm families, policy gradient methods, practical tricks like baselines and reward‑to‑go, and detailed explanations of TRPO and PPO variants with illustrative diagrams.
PPOactor-criticmachine learning
0 likes · 19 min read


HomeTechNov 16, 2022 · Artificial Intelligence
This article introduces the basic concepts of reinforcement learning, derives model‑based and model‑free policy gradient methods—including vanilla policy gradient and Actor‑Critic—explains their mathematical foundations, and demonstrates their use in scene text recognition and image captioning tasks.
AIAttention Mechanismactor-critic
0 likes · 22 min read

This article introduces deep reinforcement learning by explaining its Markov decision process foundations, then categorizes the main algorithm families—value‑based methods like DQN, policy‑based approaches such as PG/DPG/DDPG, and actor‑critic techniques including A3C, PPO, and DDPG—detailing their architectures, training procedures, and key advantages.
DQNMDPactor-critic
0 likes · 14 min read

This article introduces reinforcement learning by defining agents, environments, rewards, and policies, explains key concepts such as Markov Decision Processes and Bellman equations, and surveys major algorithms—including dynamic programming, Monte‑Carlo, TD learning, policy gradients, Q‑learning, DQN, and evolution strategies—while highlighting practical challenges and notable case studies like AlphaGo Zero.
MDPQ-Learningdeep learning
0 likes · 27 min read


Code DAODec 3, 2021 · Artificial Intelligence
This article derives the policy‑gradient objective for discrete actions, implements the Monte‑Carlo REINFORCE algorithm in PyTorch, explains the actor‑critic framework, introduces Advantage Actor‑Critic (A2C) versus A3C, and demonstrates their performance on the OpenAI Gym CartPole‑v0 environment.
A2COpenAI GymPython
0 likes · 13 min read
