Tagged articles
Offline Evaluation
12 articles · Page 1 of 1
The paper introduces BehR, a behavior consistency reward for training text-based world models, showing that optimizing for agent decision alignment rather than text fidelity improves trajectory-level consistency across 16 configurations, reduces false positives in offline evaluation from 42.5% to 9.5%, and enhances lookahead planning for weaker agents.
Behavior ConsistencyEMNLP 2026GRPO
0 likes · 11 min read

The paper identifies a structural mismatch between probability‑driven beam search and reward‑driven RL fine‑tuning in generative recommendation, proposes V-STAR with value‑guided efficient decoding (VED) and sibling‑wise GRPO to align decoding and optimization, and demonstrates superior offline and online performance through extensive experiments and ablations.
Offline EvaluationV‑STARgenerative recommendation
0 likes · 13 min read

The article explains how AI product managers can avoid costly product failures by designing a comprehensive, multi‑dimensional evaluation framework that guides development, quantifies progress, and builds trust with users and stakeholders.
AI product evaluationMetricsOffline Evaluation
0 likes · 18 min read

This article surveys how large language models reshape recommendation systems, details a generative recommender framework for JD Alliance ads—including item representation, model input, training, and inference—presents extensive offline and online experiments, and discusses future optimization directions.
JD AllianceLLMOffline Evaluation
0 likes · 25 min read

This article surveys how large language models can reshape recommendation systems, describes the four-stage generative pipeline, details item representation techniques such as semantic IDs, presents a JD affiliate advertising use case with offline and online experiments, and outlines future optimization directions.
LLMOffline EvaluationSemantic ID
0 likes · 25 min read

This talk presents Kuaishou's research on combining reinforcement learning with multi‑task recommendation, detailing a two‑stage constrained actor‑critic method for short‑video ranking, a multi‑task RL framework, experimental results on offline and online systems, and practical Q&A insights.
KuaishouOffline Evaluationactor-critic
0 likes · 18 min read

This article presents OPPO's comprehensive advertising recall system, detailing the transition from the old to the new architecture with ANN support, the selection of main‑road recall models, the construction of offline evaluation metrics, sample optimization techniques, model enhancements, multi‑scenario training strategies, and outlook for future improvements.
AdvertisingOffline Evaluationdual-tower model
0 likes · 24 min read

OPPO revamped its advertising recall system by replacing a latency‑prone directional pipeline with an ANN‑based full‑ad personalized architecture, employing a dual‑tower LTR model, multi‑path auxiliary branches, refined offline metrics, price‑sensitive and hard‑negative sampling, and hybrid joint training, which together boosted ARPU by about 15%.
AdvertisingOffline EvaluationRecommendation System
0 likes · 24 min read

This article examines recommendation system problem definitions, differences between academia and industry, offline evaluation pitfalls and data leakage issues, data construction challenges with datasets like MovieLens, and evaluates whether large language models can serve as effective solutions for modern recommendation tasks.
Offline Evaluationdata leakagelarge language models
0 likes · 20 min read

This article examines how large language models fit into recommendation systems by discussing problem definitions, offline evaluation pitfalls such as data leakage, dataset construction issues exemplified by MovieLens, and the practical limits of using LLMs as a universal solution.
MovieLensOffline Evaluationdata leakage
0 likes · 18 min read

The paper models short‑video recommendation as a constrained Markov decision process and introduces a two‑stage constrained actor‑critic algorithm that jointly maximizes watch time while satisfying multiple interaction constraints, demonstrating superior offline and online performance on the KuaiRand dataset and Kuaishou app.
Offline EvaluationOnline A/B Testingactor-critic
0 likes · 7 min read

This article presents a comprehensive overview of hotel search ranking, covering problem definition, the distinction between ranking and probability estimation, handling position bias, detailed feature engineering, the AnyBoost linear boosting model, offline evaluation methods, and observed online performance improvements.
Offline EvaluationPosition Biasfeature engineering
0 likes · 7 min read
