Intelligent UGC Content Moderation with User Safety Rating at iQIYI

This article introduces the use of artificial intelligence (AI) for large‑scale moderation of user‑generated content (UGC) on the iQIYI platform. It explains how AI can improve the efficiency of manual review by evaluating both individual content items and the users who produce them.

Background : Current industry practice combines AI models (deep‑learning classifiers for images, text, audio) with human review to detect prohibited content such as political, pornographic, or violent material. Building such AI services is costly because it requires massive, high‑quality labeled data and expensive GPU resources.

UGC Intelligent Moderation : The proposed solution adds a user‑level safety rating to the traditional content‑level detection. Users are assigned a safety grade based on historical upload pass rates, high‑risk rates, and other behavioral signals. High‑grade users may bypass AI content checks, reducing latency and resource consumption.

User Safety Rating Design : An unsupervised labeling pipeline automatically generates safety grades for users. The grade is computed from weighted pass‑rate metrics across multiple time windows (e.g., past 1 day, 7 days, 30 days). The formula aggregates the number of passed items p_i and total items t_i in each window with weight w_i to produce a composite score.

Feature Engineering & Model Training : Features include recent pass rates, upload volumes, high‑risk counts, registration info, user profile, and report statistics. Training data are constructed by merging these features with the unsupervised safety labels. Two models are trained:

GBDT (Gradient Boosting Decision Tree) using XGBoost for its efficiency and robustness.

DeepFM, a deep learning model that handles both dense and sparse features; categorical features are one‑hot encoded and multi‑hot features are supported.

The predictions of both models are fused to obtain the final safety grade.

High‑Risk User Detection : A separate binary classifier predicts whether a user will publish high‑risk content in the near future. It uses the same feature set and is also built with GBDT and an enhanced DeepFM model. This model improves recall of dangerous users.

Model Inference & Deployment : The platform’s big‑data and ML pipelines automate data collection, model training, evaluation, and online serving. When a user uploads content, the system retrieves the user’s features, queries both the safety‑grade model and the high‑risk model, and merges the results to decide the appropriate moderation path (e.g., fast‑track for trusted users, full AI check for others).

Intelligent Moderation Strategy : Users are mapped to three risk tiers (high, medium, low) based on their safety grade and high‑risk prediction. Corresponding moderation policies (e.g., priority, resource allocation) are defined in a table (Table 3). Content that passes the combined checks is published, while ongoing monitoring of exposure, reports, and other metrics can trigger re‑review.

Results : After deployment, AI model compute consumption dropped by 25 %, and the average review time for high‑quality videos decreased by 80 %. The approach also reduces manual labeling effort and improves overall user experience while maintaining content safety.

The article concludes with future directions, such as refining labeling schemes, extending the models to image‑text and live‑stream moderation, and further optimizing prediction accuracy.