Boosting Short Video Recommendations with Multi‑Objective Weighted Logistic Regression
This article explains how short‑video platforms enhance recommendation quality by combining click‑through‑rate models with multi‑objective optimization of watch time and completion rate, using sample reweighting and weighted logistic regression to balance perceived and real relevance while improving offline AUC and online user engagement.
Background
Short videos have become a traffic storm due to fragmented viewing time, strong video interaction, rich content, and good experience, prompting platforms to focus on recommendation. Current feed short‑video ranking relies on a Wide&Deep CTR model, with subsequent optimizations such as relevance and tactile signals, multi‑scene sample fusion, multimodal learning, and tree models, all yielding notable gains.
Two Optimization Directions
Perception relevance optimization – click model (CTR/Click) as target
Real relevance optimization – multi‑objective duration optimization (RDTM/PCR)
While perception relevance (CTR) captures user‑item relevance with high weight, it can narrow user interests and exacerbate long‑tail issues. Therefore, incorporating duration‑based multi‑objective optimization is essential to improve real relevance and overall watch time.
Sample Reweighting for Duration Optimization
We keep the click label unchanged and use watch time as a strong bias to influence the duration target, ensuring perception relevance while optimizing real relevance. Ongoing research explores more adaptive duration modeling (point‑wise, list‑wise).
RDTM Reweighting
We adopt a weighted logistic regression method inspired by YouTube’s duration modeling (RecSys 2016). Positive samples are reweighted by their watch time, while negative sample weights remain unchanged, allowing longer‑watch samples to receive stronger training for the duration objective.
Assuming N total samples, K positive samples, and T_i watch time, the weighted logistic regression learns an expectation E(T)*(1+P), where P is click probability and E(T) is the expected watch time. In sparse click scenarios, this approximates the true expectation, enabling the model to learn a partial order of items by watch time.
PCR_NORM Reweighting
Phase 1 achieved solid gains with watch‑time reweighting. Phase 2 focuses on optimizing playback completion rate (PCR). High‑completion videos often have low CTR, especially short videos, leading to insufficient exposure. We introduce quantile‑based PCR weighting and Wilson confidence interval smoothing to balance scores across video lengths, reducing bias from video duration.
Offline Evaluation Metrics
AUC : Standard offline metric for binary classification; ensures baseline model quality before assessing duration improvements.
AVG_RDTM : Average predicted watch time for top‑k positive samples in each batch; larger values indicate better duration performance while maintaining CTR.
Conclusion
Short‑video multi‑objective optimization is in an exploratory stage, progressing from sample reweighting to point‑wise/list‑wise duration modeling and multimodal joint learning. The current approach yields over 6% cumulative offline AUC improvement and more than 10% increase in average user watch time online. Future work will shift toward adaptive duration models that balance click and duration objectives to achieve major breakthroughs in consumption time.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
Alibaba Cloud Developer
Alibaba's official tech channel, featuring all of its technology innovations.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
