Architecture and Evaluation of Toutiao's Large-Scale Recommendation System

1. System Overview

The recommendation system is modeled as a function fitting user satisfaction based on three dimensions: content, user attributes, and environmental context. It processes diverse media types (text, images, videos, UGC) and incorporates both explicit and implicit features.

2. Content Analysis

Text analysis provides essential user interest signals through semantic tags, topics, and keywords. Additional features include relevance, environment, popularity, and collaborative signals. The platform also handles special content such as Q&A cards and advertorials, requiring tailored mixing and frequency control.

3. Modeling Approaches

Various algorithms are employed, from classic collaborative filtering and logistic regression to deep learning models, factorization machines, and GBDT. An industrial‑grade experiment platform supports flexible model composition, allowing combinations like LR + DNN or LR + GBDT.

4. Feature Types

Relevance features (keyword, category, source matching)

Environment features (location, time)

Popularity features (global, category, topic hotness)

Collaborative features (user‑user similarity, click patterns, vector similarity)

5. Model Training & Real‑Time Updates

Training is performed in real time using a Storm‑based pipeline that ingests click, impression, and interaction events, updates parameters on a custom high‑performance parameter server, and maintains low latency (≈50 ms) for online inference.

6. Recall Strategies

Given billions of items, an inverted‑index based recall selects a few thousand candidates per request, ranking them by freshness, popularity, and user interest. The recall must meet strict latency constraints.

7. User Tagging

User profiles include explicit interests, demographics, location, and implicit behavior signals. Tag generation transitioned from daily Hadoop batch jobs to a Storm‑based streaming system, reducing CPU usage by 80 % and enabling near‑real‑time updates for tens of millions of users.

8. Evaluation & Experimentation

A comprehensive evaluation framework combines short‑term metrics (CTR, dwell time) with long‑term user and ecosystem health indicators. Experiments are managed by an A/B testing platform that automatically allocates traffic, collects real‑time logs, and provides statistical confidence and actionable insights.

9. Content Safety

Multi‑layered moderation includes pre‑publish risk models, post‑publish monitoring, and human review. Deep‑learning classifiers detect pornographic, abusive, low‑quality, and misinformation content, achieving high recall while balancing precision.