Reinforcement Learning: Fundamentals, Classic Algorithms, and Applications in Short Video Recommendation

The article introduces reinforcement learning (RL) as a subfield of machine learning aimed at achieving general artificial intelligence by maximizing cumulative rewards through agent‑environment interactions.

It explains the mathematical basis of RL using the Markov Decision Process (MDP), describing states, actions, rewards, transition probabilities, and the discount factor γ that balances immediate and future rewards.

Comparisons are drawn between RL, supervised learning, and unsupervised learning, highlighting RL’s unique focus on long‑term value optimization and its connections to fields such as economics, neuroscience, and control theory.

Classic RL algorithms are surveyed. Value‑based methods like Q‑learning and Deep Q‑Network (DQN) estimate action‑value functions; DQN incorporates a target network to mitigate the moving‑target problem and uses convolutional neural networks for state representation. Policy‑based methods (e.g., TRPO, PPO) and evolutionary strategies are also mentioned.

The DQN algorithm is detailed: it builds target values via the Bellman equation, updates Q‑values with a learning rate α, and converges under sufficient visitation and appropriate learning‑rate schedules. Extensions to continuous spaces (DDPG, TD3, SAC) and the role of target networks for stability are discussed.

Application to short‑video recommendation is presented. The problem is modeled as an infinite‑horizon MDP where each user session is a step, actions are ranked video sets, and the reward reflects user retention. A novel RL‑based method (RLUR) optimizes the cumulative revisit interval, using an active‑critic framework, TD‑learning, and a Random Network Distillation (RND) module for intrinsic motivation.

Offline experiments on a public dataset derived from the Kuaishou platform compare RLUR with baseline CEM and TD3 methods, showing significant improvements in return time and retention. Online A/B tests confirm higher app check‑in frequency and user retention for RLUR.

Future challenges are outlined: sample efficiency, sparse rewards, generalization across tasks, multi‑agent cooperation, and efficient deployment of RL in large‑scale models.