Tagged articles
preference optimization
17 articles · Page 1 of 1
Since 2026, On‑Policy Distillation (OPD) has rapidly become a focal research area, evolving from offline teacher‑generated data to online student‑driven supervision, with advances such as OPD+, Direct OPD, weak‑to‑strong OPD, self‑distillation techniques, and preference‑optimization signals reshaping post‑training for large language models.
NLPOPDlarge language models
0 likes · 7 min read

LocalDPO introduces a zero‑annotation, region‑aware DPO framework that uses high‑quality real videos as positive samples and automatically generated locally degraded negatives to align video diffusion models with human preferences, achieving significant gains in visual quality, temporal consistency, and subjective ratings on CogVideoX and Wan2.1.
CVPR 2026LocalDPOartificial intelligence
0 likes · 13 min read

This article analyzes the challenges of generative retrieval for e‑commerce (shared SID prefixes, noisy implicit feedback, and probability squeezing) and presents RAD‑DPO, a robust adaptive denoising direct preference optimization that uses session‑level multi‑label contrast, token‑level gradient detachment, and dynamic reward weighting to improve both effectiveness and training efficiency.
E-Commerce SearchRAD-DPOgenerative retrieval
0 likes · 18 min read

DrPO introduces a drifting‑field based, rank‑only reward mechanism for one‑step text‑to‑image models, enabling reinforcement‑learning‑after‑training without back‑propagating reward gradients; it speeds up training 3.51× versus DRaFT, works with non‑differentiable rewards, and improves generation quality on SD‑Turbo and SDXL‑Turbo.
DrPODrifting ModelHPSv3
0 likes · 11 min read

The paper introduces LCPO, a lightweight preference‑optimization technique that uses only 800 training examples and 50 steps to teach large language models to produce concise, accurate answers, halving inference length while often improving accuracy and reducing training cost by up to two orders of magnitude.
LCPOLow-Resource Trainingefficient inference
0 likes · 8 min read

The ICLR‑2026 paper introduces LCPO, a lightweight preference‑optimization technique that uses only 800 curated examples and 50 training steps to cut large‑model chain‑of‑thought generation length by about 50% while maintaining or even improving answer accuracy, dramatically reducing training and inference costs.
LCPOLow-Resource Trainingchain-of-thought
0 likes · 8 min read

APO (Autonomous Preference Optimization) converts the drift and conflict among multiple teacher multimodal LLMs into dynamic negative constraints while treating consensus as a positive preference, enabling robust concept alignment and superior diagnostic accuracy on the CXR‑MAX benchmark, as demonstrated by extensive ICML‑2026 experiments.
APOICML 2026concept drift
0 likes · 11 min read

The paper introduces AP2O‑Coder, an adaptive progressive preference optimization framework that systematically captures error types, progressively refines LLM code generation, and dynamically adapts training data, achieving up to a 3% pass@k improvement across multiple open‑source models while reducing data requirements.
AP2O-CoderLLMcode generation
0 likes · 11 min read

The paper introduces POPEN, a new framework that uses preference‑based optimization and ensemble methods to reduce hallucinations and improve segmentation accuracy in large visual language models, achieving state‑of‑the‑art results on multiple benchmarks.
Computer VisionLVLMMultimodal Models
0 likes · 14 min read

This article presents a novel paradigm—Latent Reward Model (LRM) and Latent Preference Optimization (LPO)—that repurposes diffusion models as noise‑aware latent reward models for step‑level preference optimization, addressing the shortcomings of pixel‑level reward models, introducing multi‑preference consistent filtering, and demonstrating significant performance and efficiency gains on benchmarks such as PickScore and T2I‑CompBench++.
AI AlignmentImage Generationdiffusion models
0 likes · 9 min read

Meituan’s research team showcases eight ACL 2025 papers spanning generative retrieval, multi‑objective preference alignment, rich‑text image understanding, cross‑language transfer, multimodal math reasoning, and more, offering insights and breakthroughs that can inspire and aid fellow researchers.
ACL 2025Code-Switchinggenerative retrieval
0 likes · 15 min read

This paper proposes CAIG, a novel method for generating high-CTR advertising images using multimodal large language models, combining reinforcement learning and preference optimization to align generated content with product features.
CTR predictionMultimodal Large Language ModelsReinforcement Learning
0 likes · 10 min read

This article presents a comprehensive study of generative retrieval for large‑scale e‑commerce search, detailing background challenges, the advantages of generative methods, two concrete strategies—Lexical‑based and SemanticID‑based—along with task redesign, preference optimization, constrained beam search, extensive experiments, and future research directions.
E-Commerce SearchSemantic IDgenerative retrieval
0 likes · 21 min read

Bilibili’s in‑house role‑playing large language model, built on the Index architecture and refined through pre‑training, supervised fine‑tuning, and preference optimization (PPO and DPO), achieved top scores on the Chinese CharacterEval benchmark, surpassing rivals while incorporating safety alignment and showcasing consistent, personality‑driven dialogue examples.
Pretrainingcontent safetyevaluation benchmark
0 likes · 13 min read

This article breaks down the mathematical derivation of Direct Preference Optimization (DPO), showing how it replaces the traditional RLHF‑PPO pipeline by directly training an alignment model from human preference data, eliminating the need for a separate reward model and simplifying the overall training process.
DPOLLM AlignmentRLHF
0 likes · 17 min read


NewBeeNLPAug 7, 2024 · Artificial Intelligence
This article analyses the shortcomings of current large language model training methods such as SFT, RLHF and DPO, explains why they incur high data and compute costs, and introduces Intuitive Fine‑Tuning (IFT) with temporal residual connections as a cheaper yet effective alternative that better aligns training objectives with real generation tasks.
DPOIntuitive Fine-TuningLLM
0 likes · 15 min read


NewBeeNLPMay 13, 2024 · Artificial Intelligence
This article offers a detailed theoretical interpretation of the DPO algorithm, showing how large language models can be viewed as Q‑functions, unifying sequence‑wise and step‑wise decision perspectives, and discussing the resulting implications for reinforcement‑learning‑based alignment research.
DPOLLMQ-Function
0 likes · 14 min read
