Tagged articles

Offline Evaluation

12 articles · Page 1 of 1

Sep 8, 2026 · Artificial Intelligence

Behavior Consistency Beats State Consistency in Text World Models for Agents

The paper introduces BehR, a behavior consistency reward for training text-based world models, showing that optimizing for agent decision alignment rather than text fidelity improves trajectory-level consistency across 16 configurations, reduces false positives in offline evaluation from 42.5% to 9.5%, and enhances lookahead planning for weaker agents.

Behavior ConsistencyEMNLP 2026GRPO

0 likes · 11 min read

Behavior Consistency Beats State Consistency in Text World Models for Agents

Tencent Advertising Technology

Aug 8, 2026 · Artificial Intelligence

V-STAR: A Value‑Driven Reinforcement Learning Paradigm for Generative Recommendation

The paper identifies a structural mismatch between probability‑driven beam search and reward‑driven RL fine‑tuning in generative recommendation, proposes V-STAR with value‑guided efficient decoding (VED) and sibling‑wise GRPO to align decoding and optimization, and demonstrates superior offline and online performance through extensive experiments and ablations.

Offline EvaluationReinforcement LearningV‑STAR

0 likes · 13 min read

V-STAR: A Value‑Driven Reinforcement Learning Paradigm for Generative Recommendation

PMTalk Product Manager Community

Jul 15, 2026 · Product Management

Why AI Product Managers Need a Robust Evaluation System to Build a Competitive Moat

The article explains how AI product managers can avoid costly product failures by designing a comprehensive, multi‑dimensional evaluation framework that guides development, quantifies progress, and builds trust with users and stakeholders.

AI product evaluationOffline EvaluationOnline A/B Testing

0 likes · 18 min read

Why AI Product Managers Need a Robust Evaluation System to Build a Competitive Moat

JD Retail Technology

Jul 1, 2024 · Artificial Intelligence

Generative Recommendation Systems for JD Alliance Advertising: Design, Implementation, and Evaluation

This article surveys how large language models reshape recommendation systems, details a generative recommender framework for JD Alliance ads—including item representation, model input, training, and inference—presents extensive offline and online experiments, and discusses future optimization directions.

JD AllianceLLMOffline Evaluation

0 likes · 25 min read

Generative Recommendation Systems for JD Alliance Advertising: Design, Implementation, and Evaluation

JD Tech Talk

Jun 13, 2024 · Artificial Intelligence

Generative Recommender Systems for JD Affiliate Advertising: Architecture, Methods, and Experimental Evaluation

This article surveys how large language models can reshape recommendation systems, describes the four-stage generative pipeline, details item representation techniques such as semantic IDs, presents a JD affiliate advertising use case with offline and online experiments, and outlines future optimization directions.

LLMOffline EvaluationSemantic ID

0 likes · 25 min read

Generative Recommender Systems for JD Affiliate Advertising: Architecture, Methods, and Experimental Evaluation

DataFunTalk

Mar 30, 2024 · Artificial Intelligence

Reinforcement Learning and Multi‑Task Recommendation: Two‑Stage Constrained Actor‑Critic and Multi‑Task RL Approaches at Kuaishou

This talk presents Kuaishou's research on combining reinforcement learning with multi‑task recommendation, detailing a two‑stage constrained actor‑critic method for short‑video ranking, a multi‑task RL framework, experimental results on offline and online systems, and practical Q&A insights.

KuaishouOffline EvaluationReinforcement Learning

0 likes · 18 min read

Reinforcement Learning and Multi‑Task Recommendation: Two‑Stage Constrained Actor‑Critic and Multi‑Task RL Approaches at Kuaishou

DataFunSummit

Mar 9, 2024 · Artificial Intelligence

OPPO Advertising Recall Algorithm: Architecture, Model Selection, Offline Evaluation, Sample Optimization, and Future Directions

This article presents OPPO's comprehensive advertising recall system, detailing the transition from the old to the new architecture with ANN support, the selection of main‑road recall models, the construction of offline evaluation metrics, sample optimization techniques, model enhancements, multi‑scenario training strategies, and outlook for future improvements.

AdvertisingOffline Evaluationdual-tower model

0 likes · 24 min read

OPPO Advertising Recall Algorithm: Architecture, Model Selection, Offline Evaluation, Sample Optimization, and Future Directions

Sohu Tech Products

Jan 3, 2024 · Artificial Intelligence

OPPO Advertising Recall Algorithm: Architecture, Model Selection, Evaluation, and Optimization

OPPO revamped its advertising recall system by replacing a latency‑prone directional pipeline with an ANN‑based full‑ad personalized architecture, employing a dual‑tower LTR model, multi‑path auxiliary branches, refined offline metrics, price‑sensitive and hard‑negative sampling, and hybrid joint training, which together boosted ARPU by about 15%.

AdvertisingOffline Evaluationlarge-scale classification

0 likes · 24 min read

OPPO Advertising Recall Algorithm: Architecture, Model Selection, Evaluation, and Optimization

DataFunTalk

Dec 12, 2023 · Artificial Intelligence

Challenges and Considerations of Recommendation Systems: Evaluation, Data Leakage, and the Role of Large Models

This article examines recommendation system problem definitions, differences between academia and industry, offline evaluation pitfalls and data leakage issues, data construction challenges with datasets like MovieLens, and evaluates whether large language models can serve as effective solutions for modern recommendation tasks.

Offline Evaluationdata leakagelarge language models

0 likes · 20 min read

Challenges and Considerations of Recommendation Systems: Evaluation, Data Leakage, and the Role of Large Models

DataFunSummit

Oct 23, 2023 · Artificial Intelligence

Large Models in Recommendation Systems: Evaluation Challenges, Data Leakage, and Practical Considerations

This article examines how large language models fit into recommendation systems by discussing problem definitions, offline evaluation pitfalls such as data leakage, dataset construction issues exemplified by MovieLens, and the practical limits of using LLMs as a universal solution.

MovieLensOffline Evaluationdata leakage

0 likes · 18 min read

Large Models in Recommendation Systems: Evaluation Challenges, Data Leakage, and Practical Considerations

Kuaishou Tech

Apr 27, 2023 · Artificial Intelligence

Two-Stage Constrained Actor‑Critic (TSCAC) for Short‑Video Recommendation

The paper models short‑video recommendation as a constrained Markov decision process and introduces a two‑stage constrained actor‑critic algorithm that jointly maximizes watch time while satisfying multiple interaction constraints, demonstrating superior offline and online performance on the KuaiRand dataset and Kuaishou app.

Offline EvaluationOnline A/B TestingReinforcement Learning

0 likes · 7 min read

Two-Stage Constrained Actor‑Critic (TSCAC) for Short‑Video Recommendation

Qunar Tech Salon

Aug 21, 2016 · Artificial Intelligence

Hotel Search Ranking: Problem Definition, Model Construction, Feature Engineering, and Offline Evaluation

This article presents a comprehensive overview of hotel search ranking, covering problem definition, the distinction between ranking and probability estimation, handling position bias, detailed feature engineering, the AnyBoost linear boosting model, offline evaluation methods, and observed online performance improvements.

Offline Evaluationfeature engineeringhotel ranking

0 likes · 7 min read