How Content-Based Video Relevance Prediction Advances Personalized Streaming

Challenge Background

The ACM MM 2019 conference in Nice, France (Oct 21‑25, 2019) featured the Hulu‑sponsored CBVRP (Content‑Based Video Relevance Prediction) challenge, aiming to explore how content understanding can compute relevance between videos—a key step for high‑quality personalized recommendations in online streaming services.

Why Content‑Based Recommendation?

Most streaming services rely on user‑based collaborative filtering, which suffers from cold‑start problems for new or rarely viewed videos. Content‑based recommendation extracts features from titles, images, audio, text, and metadata to compute video similarity, addressing cold‑start and improving personalization.

Challenge Tasks and Timeline

Registration opened on March 15, 2019; final results were due July 1, final papers July 8, and paper presentations at ACM MM 2019 in October. Participants received Hulu’s video and audio feature data (no raw videos) along with millions of real user watch logs.

The main task was to predict click‑through probability for new TV series or movies based on user history and video metadata, with separate tracks for TV dramas and movies, each providing training, validation, and test sets.

Participation and Results

More than 90 teams from various countries and over 200 participants entered. Twelve teams submitted results for both tracks; the TV‑drama track champion (UESTC_cfm) achieved an AUC of 0.67, while the movie‑track champion (USTC_I_Know_U) achieved an AUC of 0.65. Detailed rankings and prize information are shown in the figures below.

Selected Papers Included in ACM MM 2019

BERT4SessRec: Content‑Based Video Relevance Prediction with Bidirectional Encoder Representations from Transformer – Xusong Chen et al.

Cold‑Start Representation Learning: A Recommendation Approach with Bert4Movie and Movie2vec – Xinran Zhang et al.

Content‑Based Video Relevance Prediction with Multi‑view Multi‑level Deep Interest Network – Zheyuan Chen et al.

Exploring Content‑based Video Relevance for Video Click‑Through Rate Prediction – Xun Wang et al.

Time‑aware Session Embedding for Click‑Through‑Rate Prediction – Qidi Xu et al.

The TV‑drama champion team’s paper introduced a Time‑aware Session Embedding (TSE) framework, improving over DIN, VBPR, and AMR by about 25%.

Technical Highlights of the Winning Papers

The TSE paper reduced heterogeneous multimedia features to a common length via dimensionality reduction, visualized with t‑SNE, and used audio features as the primary signal. Item2Vec generated item embeddings (ID features), which were decay‑weighted by position to model session influence, similar to positional embeddings in Transformers.

Transformers were employed to aggregate session embeddings, followed by a multilayer perceptron classifier. The approach combined pre‑trained ID features, fixed audio embeddings, and position‑aware vectors, achieving superior performance on the cold‑start recommendation task.

Historical Context

The CBVRP challenge has been held three times: 2017 (ICIP), 2018 (ACM MM), and 2019 (ACM MM). Each edition refined the task and data, moving from limited retrieval datasets to large‑scale multimedia feature sets and real user watch logs.

All challenge code repositories remain publicly available on GitHub.

Future Outlook

Hulu continues to support multimedia research, encouraging more scholars to tackle cold‑start video recommendation and advance the field of personalized streaming.