Tagged articles
Reward Modeling
35 articles · Page 1 of 1
This paper introduces Diffusion Reward Models (DRM) that replace the scalar value head with a Diffusion Transformer to model full reward distributions, capturing multimodal human disagreement, enabling uncertainty-aware decisions, test-time scaling via repeated reward sampling, and improving downstream RLHF performance.
Distribution ModelingHuman Preference LearningMultimodal Distributions
0 likes · 20 min read

Apple and the Hasso Plattner Institute evaluated over 200 multilingual inference training experiments across nine base models and eleven languages, finding that training with Chinese rewards incurs just a 1.1 percentage‑point loss versus English, while low‑resource languages can cause severe performance collapses in specific model‑language combos.
GemmaQwen3Reward Modeling
0 likes · 9 min read

The CAMEL framework introduces a confidence‑gated reflection mechanism that uses the log‑probability margin between verdict tokens to decide whether a single‑token fast judgment suffices or a full generative reflection is needed, achieving 82.9% average accuracy—a 3.2% gain over prior best—while a 14B model outperforms several 70B‑scale reward models and offers a tunable accuracy‑cost trade‑off.
CAMELRLHFReward Modeling
0 likes · 10 min read

The article analyzes the E‑GRM framework's need for both accurate score regression and stable ranking signals, proposes a weighted combination of Huber and hinge losses, and demonstrates through extensive ablations and downstream GRPO experiments that the mixed loss yields superior calibration, ranking, and policy‑learning performance.
E‑GRMHinge LossHuber Loss
0 likes · 10 min read

The article breaks down the exact five‑stage pipeline—data collection, pre‑training, supervised fine‑tuning, reward modeling, and reinforcement learning—that transforms raw internet text into powerful LLMs like GPT and Claude, and explains how understanding each step lets you build a miniature version yourself.
ClaudeGPTLLM
0 likes · 15 min read

This survey examines how rubrics—structured, multi‑dimensional evaluation criteria—are defined, constructed, and applied to train and evaluate large language models, especially for open‑ended, high‑risk and agentic tasks, while highlighting current challenges such as reward hacking and bias.
AI safetyAgentReward Modeling
0 likes · 15 min read


Amap TechJun 23, 2026 · Artificial Intelligence
GrowLoop proposes a self‑evolving loop that uses a few human seed annotations and large‑language‑model meta‑reflection to automatically generate and refine scoring rubrics and test questions for open‑domain dialogue, enabling reliable benchmarking where no fixed standard exists.
LLMReward Modelingbenchmark
0 likes · 23 min read

The Skill‑RM paper reveals that simply appending evaluation resources can degrade large‑model scoring, while structuring those resources into a Reward‑Evaluation Skill boosts performance across benchmarks, best‑of‑N selection, and RL‑based instruction following.
Alibaba QwenEvaluation FrameworkRLHF
0 likes · 7 min read

This article reviews the five‑year evolution of reinforcement‑learning techniques for large language models, comparing PPO, DPO, GRPO and emerging multi‑agent approaches, analyzing their reward signals, practical trade‑offs, and the open‑source frameworks that support them.
DPOGRPOLLM
0 likes · 34 min read

PromptEcho computes a continuous reward for text‑to‑image generation by measuring how well a frozen vision‑language model can reconstruct the original prompt from the generated image, eliminating the need for annotated data or a trained reward model and outperforming prior methods across multiple benchmarks.
PromptEchoReinforcement LearningReward Modeling
0 likes · 10 min read

The article highlights the rapid rise of reinforcement learning across major 2026 conferences, curates 181 RL papers from eight top venues, and provides detailed summaries of innovative works such as MSRL and MedVR, offering free access to the papers and code.
Reinforcement LearningReward Modelingagentic RL
0 likes · 6 min read

LLM judges often produce contradictory scores and non‑transitive preferences; the TrustJudge framework replaces discrete scoring with distribution‑sensitive scoring and likelihood‑aware aggregation, dramatically reducing both score‑comparison and pairwise‑transitivity inconsistencies across multiple model families, improving accuracy and even serving as a reward signal for RL training.
LLM EvaluationReinforcement LearningReward Modeling
0 likes · 12 min read

The article presents CharacterFlywheel, a 15‑generation flywheel methodology that iteratively improves social‑dialogue LLMs in production using data‑driven reward models, rejection sampling, and a mix of SFT, DPO, and RL, with detailed experiments and best‑practice insights.
AI safetyLLM OptimizationReinforcement Learning
0 likes · 12 min read

This article proposes an ASR‑driven pipeline that creates high‑quality AI‑reply vs. human‑like reply pairs, trains a rewrite model and a reward model, and uses GRPO reinforcement learning to generate natural, helpful, and less AI‑sounding responses in digital‑human live streaming, achieving 92% accuracy and 97% helpfulness while improving user experience.
ASR dataLLMQwen
0 likes · 20 min read

Interviewers increasingly ask why modern reward models must go beyond scalar scores to incorporate reasoning, and this article explains the limitations of traditional scalar reward models, the benefits of the RM‑R1 framework, and how reasoning‑based rewards improve alignment, stability, and task performance in large language model training.
AI AlignmentLLMRLHF
0 likes · 11 min read

The article explains that the real difficulty of RLHF lies in designing and curating high‑quality preference data, building robust reward models through bad‑case rewriting, human‑in‑the‑loop labeling, and inference‑based reward modeling, while algorithmic details like PPO are secondary concerns.
Data EngineeringGRPORLHF
0 likes · 9 min read

This article details a new research pipeline that leverages large‑scale human preference data, a multi‑dimensional video reward model, and specialized alignment algorithms to dramatically improve video generation quality, motion fidelity, and text‑video consistency, with open‑source code and benchmarks for reproducibility.
AI AlignmentHuman FeedbackRLHF
0 likes · 10 min read

RIVAL (Reinforcement Learning with Iterative and Adversarial Optimization) introduces an adversarial game between a reward model and a translation LLM, combining qualitative preference rewards with quantitative metrics like BLEU, to overcome distribution shift in RLHF and achieve superior performance on conversational subtitle and WMT translation tasks.
BLEULLMReinforcement Learning
0 likes · 13 min read

The article examines the fundamental similarity between SFT and RL loss functions for large language models, explains why RL training is prone to instability, discusses infrastructure and data quality challenges, and reviews practical tricks and reward‑model considerations for more reliable RL fine‑tuning.
AILLMReinforcement Learning
0 likes · 11 min read

Atom-Searcher introduces an atomic‑thought reinforcement‑learning framework that decomposes complex reasoning into fine‑grained units, uses a Reasoning Reward Model to assign step‑wise rewards, dynamically balances process and result incentives, and achieves state‑of‑the‑art performance on multiple LLM benchmarks.
Agentic ResearchAtomic ThoughtLLM
0 likes · 12 min read

Perception‑R1, a post‑training framework that applies rule‑based reinforcement learning to existing multimodal LLMs, dramatically improves visual perception tasks such as grounding, OCR, counting and object detection, as demonstrated by extensive benchmarks and ablation studies.
GRPOPerception PolicyReinforcement Learning
0 likes · 10 min read

Researchers discovered that trivial symbols such as a colon or the word “Solution” can trigger false‑positive rewards in LLM judge models, causing GPT‑4o, Claude‑4 and LLaMA‑3‑70B to fail, and proposed a robust “Master‑RM” model that eliminates these attacks.
AI robustnessLLM securityReward Modeling
0 likes · 10 min read


AIWalkerJun 18, 2025 · Artificial Intelligence
Large language models are booming, and this article highlights six cutting‑edge research directions—LLM‑plus synthetic data, reward modeling, inference techniques, LLM‑as‑a‑Judge, safety alignment, and long‑context handling—each illustrated with recent papers, experimental results, and links to code repositories.
LLMLong ContextReward Modeling
0 likes · 9 min read

The article analyzes how large language models can acquire true reasoning abilities for hard‑to‑score industry tasks by combining Chain‑of‑Thought prompting with reinforcement learning, addressing vague reward signals, reward hacking, and loyalty, and proposing a toolbox of reward engineering, synthetic data, hierarchical RL and multi‑agent collaboration.
LLMReinforcement LearningReward Modeling
0 likes · 22 min read

Young JD algorithm engineers share how they solve tough AI problems—from optimizing large‑model training and reward‑model design for ad image generation, to building LLM‑based query expansion, agent evaluation, and model pruning with FFT and RDP—illustrating practical breakthroughs and personal growth in cutting‑edge AI research.
AIAlgorithm EngineeringReinforcement Learning
0 likes · 15 min read

Researchers at Bilibili present Index‑Anisora, an open‑source anime video generation framework that builds a 30k‑sample reward dataset, introduces the multi‑dimensional AnimeReward model and a Gap‑Aware Preference Optimization (GAPO) method, and demonstrate through extensive automatic and human evaluations that their approach significantly outperforms baseline video generators.
AIGAPOHuman Feedback
0 likes · 20 min read

JD Retail’s engineering team tackles hard AI problems by replacing a monolithic reward model with specialized small models for ad‑image generation, deploying an LLM‑driven query‑expansion pipeline that lifts conversion rates, and pruning text‑to‑image transformers using FFT and RDP to boost throughput 40% without loss, while building comprehensive evaluation tools and a semantic smart‑assistant.
AIReinforcement LearningReward Modeling
0 likes · 14 min read


ArchitectMar 16, 2025 · Artificial Intelligence
This article walks through the complete lifecycle of building a small large‑language model, covering token‑level inference, pre‑training, post‑training steps such as supervised fine‑tuning, reward‑model creation, and reinforcement‑learning methods like DPO, PPO and GRPO, culminating in a practical 0.5B model fine‑tuned for chain‑of‑thought reasoning.
GRPOLLM trainingReinforcement Learning
0 likes · 22 min read


ArchitectFeb 25, 2025 · Artificial Intelligence
DeepSeek R1 builds on the DeepSeek V3 base model using a multi‑stage reinforcement learning pipeline—including GRPO optimization, rule‑based reward modeling, supervised fine‑tuning, language‑consistency rewards, rejection sampling, and distillation—to produce a high‑performing, aligned LLM capable of accurate reasoning.
DeepSeekLLM trainingReward Modeling
0 likes · 24 min read

This article compiles and analyzes the post‑training pipelines of Llama 3.1, DeepSeek‑V3, TÜLU 3 and Qwen 2.5, detailing their data compositions, SFT, reward modeling, DPO, GRPO, RLVR methods, hyper‑parameters, and practical tricks for large‑language‑model alignment.
DPODeepSeek-V3Llama3.1
0 likes · 22 min read

This article analytically explores the implicit assumptions behind the RLHF optimization objective, examines how they limit DPO and PPO methods, and proposes practical improvements such as rejection sampling and online on‑policy data selection to narrow the gap between theory and practice.
AI AlignmentDPOPPO
0 likes · 22 min read

This article provides a thorough, four‑part overview of RLHF for large language models, covering preference‑optimization algorithms (PPO‑based and offline RL approaches), reward‑model training techniques, inference‑time exploration strategies, and practical implementation details including the OpenRLHF framework and resource‑allocation tricks.
DPOLLM OptimizationOpenRLHF
0 likes · 27 min read


NewBeeNLPSep 5, 2024 · Artificial Intelligence
The article analyzes why supervised fine‑tuning (SFT) cannot replace reinforcement learning from human feedback (RLHF), highlighting SFT's lack of negative feedback and backward‑looking capability, and explains how RLHF’s reward model addresses these fundamental shortcomings.
RLHFReward ModelingSFT
0 likes · 7 min read

This article explains why Reinforcement Learning from Human Feedback (RLHF) is crucial for LLM intelligence, outlines the three-stage training pipeline, details InstructGPT's reward model and PPO optimization, and provides a practical guide to implementing RLHF with deep‑learning frameworks.
PPORLHFReinforcement Learning
0 likes · 17 min read

This article explains Reinforcement Learning with Human Feedback (RLHF), outlining its definition, suitable tasks, advantages over other reward‑model methods, types of algorithms, challenges of human feedback, and practical strategies to mitigate its limitations for building robust AI systems.
AI AlignmentHuman FeedbackReinforcement Learning
0 likes · 14 min read
