Boost Short-Video Recommendations: Multi-Goal Optimization with Weighted Logistic Regression
Alibaba's short‑video recommendation team details how they enhance both click‑through rate and viewing duration by applying sample reweighting, weighted logistic regression, and playback‑completion‑rate normalization, achieving over 6% offline AUC gains and more than 10% increase in average user watch time.
Short videos dominate traffic due to fragmented viewing time, strong interaction, rich content, and good experience, making recommendation a core competitive edge. Platforms optimize click‑through rate (CTR) and watch duration to increase user consumption.
Background
Current feed‑short video ranking relies on a Wide&Deep CTR prediction model. Various enhancements—relevance signals, multi‑scene sample fusion, multimodal learning, tree models—have yielded solid gains.
Optimization is split into two parts:
Perceived relevance optimization – click model (CTR) as the target.
True relevance optimization – multi‑objective duration optimization (RDTM/PCR).
While click‑based models capture perceived relevance well, they may narrow user interests and exacerbate long‑tail issues. Watch duration, a key metric across platforms, demands a multi‑objective approach to improve true relevance.
RDTM Reweighting
We adopt a weighted logistic regression method, inspired by YouTube’s duration modeling, to reweight samples on the click model. Positive samples receive weights proportional to their watch time, while negative sample weights stay unchanged, allowing longer‑watch samples to be trained more heavily for the duration objective.
The weighted logistic regression learns an expected value:
where N is the total sample count, K the number of positive samples, and T_i the watch duration. The expected value approximates E(T)*(1+P), with P as click probability and E(T) the long‑term expected watch time. In sparse click scenarios, this weighting aligns the model with duration bias while preserving CTR stability.
Key differences from YouTube:
Label : YouTube uses duration as the label; we keep the click label to maintain perceived relevance.
Task : YouTube treats it as a regression problem; we treat it as a classification problem, optimizing log‑loss with duration bias.
Weighting form : We apply a piecewise function that smooths the relationship between video length and watch time, whereas YouTube uses direct duration weighting.
These choices ensure CTR remains stable while improving true relevance through duration bias.
PCR_Norm Reweighting
After achieving gains with duration reweighting, we focus on playback‑completion‑rate (PCR) optimization. High‑PCR videos often have lower CTR and receive lower scores, especially short videos. By introducing quantile‑based PCR normalization and Wilson confidence interval smoothing, we mitigate length‑induced scoring bias.
We discretize videos by length, compute quantile‑based PCR, and apply Wilson interval smoothing to make PCR comparable across length segments, reducing the natural advantage of short videos.
Offline Evaluation Metrics
AUC remains the primary offline metric for the binary click prediction task, ensuring model admission criteria are met.
AVG_RDTM (average watch duration) evaluates the duration objective by averaging watch times of top‑k scored positive samples per batch, reflecting duration gains under stable CTR.
Results
Phase 1 (duration reweighting) and Phase 2 (PCR normalization) together improved offline AUC by over 6% and increased average user watch time by more than 10% in online experiments.
Conclusion
Short‑video multi‑objective optimization is still exploratory. The roadmap progresses from sample reweighting to point‑wise/list‑wise duration modeling and finally to multimodal joint learning. Future work will shift toward adaptive duration modeling and balance between click and duration objectives to achieve major breakthroughs in consumption time.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
Alibaba Cloud Developer
Alibaba's official tech channel, featuring all of its technology innovations.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
