Deep Learning Ranking System and Model for NetEase News Feed Personalization
This article presents the design, optimization, and deployment of a deep‑learning based ranking pipeline and model for NetEase News, covering offline and online pipelines, feature‑processing enhancements, custom TensorFlow operators, modular model architectures, and performance improvements demonstrated in production.
The talk introduces NetEase News' personalized feed recommendation, focusing on the deep‑learning ranking system and model used in both the headline and short‑video channels after recall, ranking, and re‑ranking stages.
Online optimization metrics such as CTR, dwell time, retention, and refresh count are discussed, with CTR being the most straightforward to model.
The ranking system consists of three parts: the pipeline, the ranking model, and the model serving component. In the offline stage, the pipeline aggregates real‑time feedback logs, matches exposure and click labels, fills features, and generates training samples; the online stage mirrors this process but uses real‑time client feedback and recall candidates.
To address consistency, performance, and scalability challenges, the pipeline was refactored, focusing on feature preprocessing and the model itself. Feature preprocessing was moved from CPU‑bound TensorFlow jobs to a Hadoop‑Streaming cluster, and a custom operation library was created to handle sample reading and data transformation, ensuring online‑offline parity.
Custom ops replace tf.data and feature_column pipelines, providing higher throughput and supporting multi‑value weighted features and operations not natively supported by TensorFlow.
Feature processing is described via DAG‑style configuration files, where each feature (e.g., age, Doc_POI, User_POI) passes through a series of operators such as correction, CDF/Bucket, or similarity computation before being fed to the ranking model.
Images illustrating the pipeline flow, DAG configurations, and operator graphs are included throughout the article.
For the ranking model, several deep learning architectures are examined, including DNN, FNN, PNN, Wide&Deep, DCN (Deep & Cross Network), and DIEN (Deep Interest Evolution Network). All share common building blocks: linear, cross, and deep modules, with feature representation, cross, and fully‑connected layers.
A generic model framework was built to support these architectures, allowing configuration‑driven model definition. Input features such as click history and target docid are embedded, passed through attention, pooled, and finally processed by a fully‑connected network.
After the pipeline and model optimizations, the system achieved consistent online‑offline feature handling, improved overall performance, and significant gains over baseline models across recommendation tasks, as shown by experimental results.
Supplementary materials, including a PPT and community links, are provided for further reference.
DataFunTalk
Dedicated to sharing and discussing big data and AI technology applications, aiming to empower a million data scientists. Regularly hosts live tech talks and curates articles on big data, recommendation/search algorithms, advertising algorithms, NLP, intelligent risk control, autonomous driving, and machine learning/deep learning.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.