Tagged articles
DPO
41 articles · Page 1 of 1
A 26-page survey from HUST, Tsinghua, and other institutions reviews how reinforcement learning integrates with diffusion models across language, vision, robotics, and scientific computing, covering methods like RLHF, DPO, GRPO, and challenges in reward design, exploration, and sparse rewards.
DPOGRPOMultimodal Generation
0 likes · 13 min read

This article surveys the 2026 reinforcement learning landscape for large language models, detailing foundational algorithms (PPO, DPO, GRPO), advanced GRPO variants (DAPO, GSPO, GMPO, GFPO), emerging Agentic RL methods (ARPO, Tree-GRPO), and a practical scenario-based selection table.
Algorithm SelectionDPOGRPO
0 likes · 12 min read


Amap TechSep 23, 2026 · Artificial Intelligence
Amap details its system for extracting navigation landmarks from street-view imagery using multimodal large models, covering data annotation standards, a three-stage training pipeline (SFT, DPO, GRPO with a risk-aware reward), bounding-box-free evaluation, offline production, online integration, and A/B tests showing 23.5% faster user departure.
DPOGRPOSFT
0 likes · 26 min read

The article presents a from‑scratch LLM training framework called grpo‑llm, implements GRPO with Trio rollout, FSDP and a C++ reward extension, and conducts a controlled experiment comparing GRPO, PPO and DPO on the GSM8K math‑reasoning benchmark, revealing why DPO outperforms the other two under sparse binary rewards.
DPOGRPOGSM8K
0 likes · 10 min read

A recent ACL paper from Amazon reveals that injecting user memory into large language models causes significant performance drops and fairness biases, favoring privileged personas across demographics, but shows that targeted DPO fine‑tuning can mitigate these effects.
AmazonDPOLLM
0 likes · 10 min read

This article surveys five years of large‑model reinforcement learning, detailing the evolution from PPO + RLHF to DPO and GRPO, comparing reward‑model‑based and verifiable‑reward approaches, discussing multi‑agent extensions, and evaluating open‑source frameworks for training LLM‑driven agents.
AI AlignmentDPOGRPO
0 likes · 34 min read

This article reviews the five‑year evolution of reinforcement‑learning techniques for large language models, comparing PPO, DPO, GRPO and emerging multi‑agent approaches, analyzing their reward signals, practical trade‑offs, and the open‑source frameworks that support them.
DPOGRPOLLM
0 likes · 34 min read

The article reviews five years of LLM‑centric reinforcement learning, tracing the evolution from early Q‑learning to PPO, then to Direct Preference Optimization, Group Relative Policy Optimization, and finally multi‑agent RL, detailing each method’s mechanics, strengths, failure modes, practical considerations, and emerging open‑source toolchains.
DPOGRPOLLM Alignment
0 likes · 33 min read

Anthropic and MIT researchers reveal that large language models can sense injected steering vectors, a capability that emerges during post‑training (especially DPO), and they present a two‑stage detection circuit whose performance improves by up to 75% when reject directions are ablated or bias vectors are trained.
DPOIntrospective AwarenessLLM
0 likes · 15 min read

The article explains that after task alignment, teams can produce functional demos, but true competitiveness requires preference alignment—optimizing for human comfort across dimensions like brevity, tone, and safety—and discusses how RLHF and DPO address this, especially the additional challenges of generating natural, responsive voice output.
AI AlignmentDPOHuman Feedback
0 likes · 7 min read

The article presents a generative retrieval framework that anchors on a product‑side explicit semantic space, uses single‑sided query alignment, multi‑stage LLM‑driven attribute extraction, schema standardization, and 14B model SFT + DPO training to achieve interpretable, controllable, and high‑performance e‑commerce search.
DPOLLMSFT
0 likes · 29 min read

This article explains the full lifecycle of large language models in 2026, covering pretraining fundamentals, the limits of classic Scaling Laws, data‑centric advances, fine‑tuning strategies, RLHF, DPO, and the emerging post‑training methods GRPO, DAPO and RLVR, with concrete benchmarks and cost analyses.
DAPODPOGRPO
0 likes · 17 min read

This article systematically explains the post‑training pipeline for large language models, covering supervised fine‑tuning, RLHF, PPO, GRPO, RLVR, DPO and emerging Agentic RL, while illustrating each method with analogies, detailed workflows, tables, and recent research findings.
DPOGRPOLLM
0 likes · 24 min read

This guide walks through setting up a Kunlun P800 XPU host, preparing Docker containers, deploying Qwen3‑8B/‑32B/‑VL models with vLLM‑Kunlun, benchmarking performance, and running full‑parameter DPO training using LLaMA‑Factory, providing scripts, configuration files, and troubleshooting tips for AI engineers.
DPOKunlun P800LLaMA-Factory
0 likes · 32 min read


Amap TechNov 19, 2025 · Artificial Intelligence
Gaode transforms its map app into a dynamic, AI‑driven “living map” by fine‑tuning the large Spacetime‑GR model through embedding‑based and generative ranking SFT, DPO alignment, and multimodal augmentation, achieving significant offline CTR‑AUC improvements and online CTR gains in POI recommendation.
AI recommendationDPOSFT
0 likes · 12 min read

This article shares practical experience on deploying private Agentic AI, covering background, architecture design, challenges, data generation, reinforcement learning with DPO, automated multi‑dimensional evaluation, and future plans for open‑source models and richer tool integration.
Agentic AIDPOLLM fine-tuning
0 likes · 16 min read

This article examines how reinforcement‑learning techniques such as PPO, DPO, and GRPO are integrated into the Baixiaosheng QA system to improve answer stability, deepen domain knowledge understanding, and accelerate response generation, and it evaluates the impact of Reinforcement Fine‑Tuning (RFT) on real‑world performance.
AIDPOGRPO
0 likes · 16 min read

This article explains Meta's LLaMA model series, detailing LLaMA 2's architectural innovations—RMSNorm, SwiGLU, rotary positional encoding, and grouped-query attention—and LLaMA 3's post-training pipeline including supervised fine-tuning, reward modeling, rejection sampling, and direct preference optimization.
DPOGrouped-Query AttentionLLM Architecture
0 likes · 10 min read

This comprehensive guide explains the full RLHF training pipeline, the mathematical foundations of reward modeling and PPO, and introduces DPO and KTO algorithms—including their implementations, advantages, limitations, and practical tuning strategies—for building aligned large language models.
DPOHuman FeedbackKTO
0 likes · 32 min read

This guide explains how to use the EasyDistill framework and Alibaba Cloud PAI to distill large language models for high‑quality text generation, covering model deployment, SFT and DPO training data construction, code examples, configuration files, and best practices for achieving resource‑efficient, high‑performance student models.
DPOEasyDistillPAI
0 likes · 14 min read

This article explains post‑training concepts, compares SFT, DPO, and GRPO fine‑tuning methods, and provides step‑by‑step guidance for using Alibaba Cloud's PAI platform—including Model Gallery and DSW—to fine‑tune large language models with code examples and practical tips.
DPOGRPOLLM
0 likes · 14 min read

In an interview at the DA Digital Intelligence Conference, ByteDance AI specialist Cai Conghuai explains how large language models, combined with techniques like SFT, DPO, and RAG, are reshaping Douyin's user‑experience signal detection, root‑cause analysis, and evaluation, while outlining future AI‑agent breakthroughs.
AIDPORAG
0 likes · 12 min read

In a detailed interview, Vivo AI engineer Liang Tianan explains how the BlueHeart Small V assistant leverages large language models, multi‑stage recall, ranking, and reward‑model fine‑tuning (SFT/DPO) to generate high‑quality, diverse post‑dialogue recommendation items while balancing latency, cost, and evaluation challenges.
DPOLLMSFT
0 likes · 15 min read

This article provides a comprehensive overview of large language models (LLMs), covering their transformer architecture, parameter counts, GPU memory and storage requirements, and detailed fine‑tuning methods such as prompt engineering, data construction, LoRA, PEFT, RLHF, and DPO, along with practical deployment and inference acceleration strategies.
DPOLLMLoRA
0 likes · 17 min read

This guide introduces Hugging Face's TRL library, explains how to install it alongside Transformers, and walks through modifying LLaVA's trainer, dataset, and data collator to apply the DPO reinforcement‑learning algorithm for multimodal model fine‑tuning.
DPOHugging FaceLLaVA
0 likes · 4 min read

DeepSeek‑R1 replaces the traditional PPO‑based RLHF approach with GRPO, reducing reliance on human‑labeled data by using pure reinforcement learning environments and carefully designed reward mechanisms; the article explains reinforcement learning fundamentals, compares PPO, DPO and GRPO, and offers practical application recommendations.
AI AlignmentDPOGRPO
0 likes · 14 min read

The paper presents a generative recommendation framework for JD Alliance advertising that combines semantic‑ID modeling, large‑model pre‑training and fine‑tuning, and Direct Preference Optimization (including Softmax‑DPO and β‑DPO) to jointly boost click‑through and conversion rates, achieving +0.6% UCTR and +8% UCVR in online tests while outlining future multi‑objective extensions.
AdvertisingDPOMulti-Objective Optimization
0 likes · 12 min read

We built an LLM‑powered system for Bilibili that automatically creates ad titles from user keywords, employing fluency, style, and quality classifiers, mixed domain data cleaning, and alignment methods such as SFT, DPO and KTO, resulting in a product that now generates about ten percent of daily titles and drives significant ad spend.
AI AlignmentAd Title GenerationBilibili
0 likes · 24 min read

This article compiles and analyzes the post‑training pipelines of Llama 3.1, DeepSeek‑V3, TÜLU 3 and Qwen 2.5, detailing their data compositions, SFT, reward modeling, DPO, GRPO, RLVR methods, hyper‑parameters, and practical tricks for large‑language‑model alignment.
DPODeepSeek-V3Llama3.1
0 likes · 22 min read

This article walks through the various loss functions used in OpenRLHF—including GPTLMLoss, KDLoss, DPOLoss, KTOLoss, and reward model losses—explaining their mathematical foundations, implementation details, and practical considerations for RLHF training.
DPOKTOLoss Functions
0 likes · 23 min read

This guide explains when to rely on prompt engineering versus SFT fine‑tuning for Vision‑Language Models, emphasizing data quality, appropriate dataset sizes, training epochs, hyper‑parameter tuning, and practical steps to build robust VLM pipelines.
AIDPOPrompt engineering
0 likes · 10 min read

This article analytically explores the implicit assumptions behind the RLHF optimization objective, examines how they limit DPO and PPO methods, and proposes practical improvements such as rejection sampling and online on‑policy data selection to narrow the gap between theory and practice.
AI AlignmentDPOPPO
0 likes · 22 min read

This article provides a thorough, four‑part overview of RLHF for large language models, covering preference‑optimization algorithms (PPO‑based and offline RL approaches), reward‑model training techniques, inference‑time exploration strategies, and practical implementation details including the OpenRLHF framework and resource‑allocation tricks.
DPOLLM OptimizationOpenRLHF
0 likes · 27 min read

This article breaks down how Direct Preference Optimization (DPO) mathematically reduces the two‑stage RLHF pipeline into a single‑stage SFT process, explains the underlying loss transformations, and discusses DPO's practical limitations and trade‑offs for large language model alignment.
DPODirect Preference OptimizationRLHF
0 likes · 9 min read

This article provides a thorough, yet concise, overview of Llama 3’s training pipeline, data handling, model architecture, scaling laws, post‑training techniques like SFT and DPO, and inference optimizations such as KV‑Cache, GQA, PagedAttention, and FP8 quantization, highlighting practical insights and benchmark results.
DPOGQAKV Cache
0 likes · 32 min read


NewBeeNLPSep 23, 2024 · Artificial Intelligence
This article analyzes recent post‑training trends in large language models, comparing DPO and PPO, examining the scarcity of open‑source preference data, the iterative training process, the rise of synthetic data pipelines, and emerging methods for improving math and reasoning capabilities.
DPOLLMPPO
0 likes · 12 min read

This article breaks down the mathematical derivation of Direct Preference Optimization (DPO), showing how it replaces the traditional RLHF‑PPO pipeline by directly training an alignment model from human preference data, eliminating the need for a separate reward model and simplifying the overall training process.
DPOLLM AlignmentRLHF
0 likes · 17 min read


NewBeeNLPAug 7, 2024 · Artificial Intelligence
This article analyses the shortcomings of current large language model training methods such as SFT, RLHF and DPO, explains why they incur high data and compute costs, and introduces Intuitive Fine‑Tuning (IFT) with temporal residual connections as a cheaper yet effective alternative that better aligns training objectives with real generation tasks.
DPOIntuitive Fine-TuningLLM
0 likes · 15 min read

This guide explains the Direct Preference Optimization (DPO) algorithm for aligning large language models, demonstrates its advantages over RLHF, and provides a step‑by‑step tutorial on using Alibaba Cloud’s PAI‑QuickStart to fine‑tune the open‑source Qwen2 series, including data preparation, hyper‑parameter settings, training, deployment, and API usage.
AI AlignmentAlibaba CloudDPO
0 likes · 14 min read

This article surveys the current RLHF ecosystem, comparing on‑policy methods like PPO with off‑policy approaches such as DPO, and examines recent variants—including ReMax, GRPO, DPOP, TDPO, and ORPO—highlighting their algorithmic differences, resource trade‑offs, and practical performance insights.
DPOLLMPPO
0 likes · 23 min read


NewBeeNLPMay 13, 2024 · Artificial Intelligence
This article offers a detailed theoretical interpretation of the DPO algorithm, showing how large language models can be viewed as Q‑functions, unifying sequence‑wise and step‑wise decision perspectives, and discussing the resulting implications for reinforcement‑learning‑based alignment research.
DPOLLMQ-Function
0 likes · 14 min read
