Inside 今日头条's Recommendation Engine: Architecture, Features, and Evaluation

System Overview

Recommendation can be formalized as fitting a function that predicts user satisfaction based on three dimensions: content, user, and environment. The system must handle diverse content types (articles, videos, UGC, Q&A) and adapt to mobile scenarios where user context constantly changes.

Feature Dimensions

Content features include text, images, and video attributes; each type requires specific extraction methods. User features cover explicit interests, demographics, and implicit interests derived from models. Environment features capture location, time, and device context, reflecting the mobile‑first nature of the platform.

Unmeasurable Objectives

Beyond quantifiable metrics like click‑through rate, reading time, likes, comments, and shares, the system must incorporate non‑directly measurable goals such as promoting community‑generated answers, controlling the frequency of special content, and enforcing content‑ecosystem policies (e.g., suppressing low‑quality or pornographic material).

Model Choices and Implementation

The core formula y = F(X_i, X_u, X_c) is a classic supervised learning problem. Implementations range from traditional collaborative‑filtering and logistic regression to deep learning models, factorization machines, and GBDT. An industrial‑grade recommendation platform must support flexible algorithm experimentation and model composition, as no single architecture fits all scenarios.

Feature Types

Relevance features : keyword, category, source, and topic matches; implicit matches are derived from vector distances.

Environment features : geographic location and time, used as bias or matching features.

Popularity features : global, category, and topic hotness, crucial for cold‑start.

Collaborative features : similarity of user behaviors (click, interest, topic, vector similarity) to alleviate filter‑bubble effects.

Training Pipeline

Real‑time training is employed to capture fresh user actions quickly. Data flows from Storm clusters that process click, impression, collection, and share events. A custom high‑performance parameter server replaces open‑source solutions that cannot meet the scale of billions of raw features and tens of billions of vector features.

The online training loop records real‑time features, pushes them to a Kafka queue, consumes them via Storm, constructs labeled samples, and updates model parameters almost instantly, with the main latency coming from user‑action feedback delay.

Recall Strategy

Because the content pool is massive, a multi‑stage recall pipeline selects a few thousand candidates per request. The offline inverted index (keyed by category, topic, entity, source) is sorted by hotness, freshness, and user actions. Online recall quickly truncates this index based on user interest tags, achieving sub‑50 ms latency.

Content Analysis

Text analysis is vital for user‑interest modeling. Articles are annotated with explicit semantic tags (pre‑defined taxonomy) and implicit semantic features (topic distributions, keyword vectors). Semantic similarity helps reduce duplicate recommendations, while spatio‑temporal relevance ensures location‑specific content is not shown to unrelated users.

Hierarchical Classification

The online classifier uses a hierarchical taxonomy: root → top‑level categories (e.g., technology, sports) → sub‑categories (e.g., football, basketball). Different algorithms (SVM, CNN, RNN) are applied at various levels to handle data skew.

User Tag Engineering

User tags are the other pillar of the system. They include explicit interests (categories, topics, keywords), implicit clusters, and vertical interests (car models, sports teams, stocks). Demographic signals (gender, age, location) are derived from third‑party login data, model predictions, and GPS‑based clustering.

Simple tags such as recently viewed content are filtered for noise (short dwell time), penalized for hot‑spot overexposure, decayed over time, and punished if shown without clicks.

From Batch to Stream

Initially, daily Hadoop jobs processed two months of activity for millions of users, causing resource contention. In late 2014, a Storm‑based streaming system was introduced, updating tags in near‑real‑time with an 80 % CPU reduction and supporting tens of millions of daily updates on a few dozen machines.

Evaluation and Experimentation

Effective evaluation requires a multi‑metric framework (clicks, dwell time, conversion, ecosystem health) and a robust A/B testing platform. Experiments are bucketed offline, assigned traffic online, and monitored hourly, with daily aggregation for stability.

The platform automatically generates traffic allocation, statistical confidence, result summaries, and optimization suggestions, but human analysis remains essential for interpreting long‑term user experience impacts.

Content Safety

Content safety is a top priority. A dedicated moderation team reviews PGC content, while UGC passes a risk‑model filter before a secondary human review. Models for pornography, profanity, and low‑quality detection are trained on massive multimodal datasets, favoring high recall (≥95 %) at the expense of some precision.

Low‑Quality Detection

Detecting fake news, clickbait, and mismatched titles requires extensive feedback loops and human‑in‑the‑loop verification. Current models achieve 95 % recall but still need manual thresholds and ongoing research collaborations (e.g., with Michigan University on rumor detection).

Author: 曹欢欢 – Senior Algorithm Architect at 今日头条