How ByteDance’s Creation Quality Platform Automates Massive Content Testing

Introduction

Creation resources such as props, templates, and mini‑programs are produced by designers and used in many ByteDance apps (Douyin, Kuaishou, CapCut, etc.). As user‑generated content grows, the volume of resources rises sharply, leading to safety and quality issues that manual inspection cannot handle efficiently. To improve efficiency, coverage, and creator experience, the Creation Quality Platform (CQ) was built as a professional vertical testing‑as‑a‑service platform.

Platform Overview

Background

Traditional detection involves user submission, content checking, manual quality inspection, and release or rejection. Manual quality inspection suffers from low efficiency, missed detections, and errors, which can cause online incidents or discourage creators. CQ addresses this by combining automation (template generation, performance testing, policy tiering) with algorithmic solutions (classification tagging, content detection) and providing re‑check and feedback loops to continuously improve models.

Task Flow

Tasks are abstracted as Jobs, which can represent any type of detection. Each Job contains one or more Tasks, the smallest unit of automated execution. Business apps (e.g., template detection, effect performance detection) define specific detection types. The task management module handles generic task logic.

Quality Detection Types

Template Detection

Targets include templates from CapCut, WakeMap, LightFace, etc., with daily volumes around 35 k+. The workflow uploads a template, registers a CQ task, dispatches based on priority, runs automated steps (material application, video synthesis, export), then applies algorithmic checks and tagging before returning results to the workflow platform.

Algorithm Classification Tagging

Beyond detection, template tagging is highly labor‑intensive. Using multimodal feature extraction (titles, videos, audio) fused by a Gate Multimodal Unit, the system achieves 91% accuracy on theme classification, close to human performance.

Impact

After integration, daily template detection rose to 38 k+, filtering over 2 k problematic templates, with a cumulative total of 3.5 M detections. Accuracy and stability remain above 99%, and detection latency dropped from three days to under four hours, dramatically improving efficiency.

Mini‑Program Detection

Targets ByteDance mini‑programs, which have many pages and over 20 detection points (name, icon, description, content safety, page anomalies, text anomalies, inducement info, service category, etc.). Creators upload mini‑programs, a screenshot service collects samples, CQ registers a task, classifies screenshots, runs CV/NLP algorithms, visualizes results, and feeds back to the workflow platform for continuous model improvement.

Algorithm Empowerment

CV and NLP algorithms provide over 20 detection points covering name, icon, description, content safety, page anomalies, text anomalies, inducement information, and service categories. Detected issues are highlighted directly on the samples to aid operators.

Sampling Mechanism

Historical accuracy: lower accuracy yields higher score.

Time since last sampled: longer interval yields higher score.

Random factor: ensures diversity.

Top‑scoring detection points are sampled for manual labeling, feeding back into the model.

Impact

Daily processing of over 500 mini‑programs, latency reduced from 24 h to under 5 h, and accuracy exceeding 95%. Third‑party mini‑programs now enjoy a fully automated submit‑detect‑deploy pipeline without human intervention.

Special‑Effect Performance Detection

Targets effect resources (props, styles, makeup, stickers, filters, transitions, animations) across multiple apps. Creators upload effects, CQ registers a task, dispatches appropriate device pools based on business and resource type, runs automated tests, and generates analysis reports, visualizations, and alerts.

Trigger‑Prop Detection

For effects that require specific actions (e.g., blink, heart gesture), CQ simulates algorithmic data instead of real hardware. It parses the effect package, extracts required trigger actions (expressions, gestures, touch), constructs a protobuf file, and passes it to the SDK, which then triggers the corresponding algorithmic effect.

Graded Testing Process

Effects are tested on representative low, mid, and high‑end devices. Testing proceeds from low‑end upwards; only if a low‑end device fails does the test continue to mid‑end, and so on, ultimately recommending the device tier range for deployment.

Impact

The system now serves Douyin, Kuaishou, CapCut, LightFace, FaceU, WakeMap, and other products. Stability exceeds 99% and issue discovery rate is around 22%, enabling fully automated performance validation without manual intervention.

Conclusion

The Creation Quality Platform provides an efficient, accurate, fully automated resource detection pipeline, enhancing user experience and creator enthusiasm. By integrating AI‑driven algorithms, it addresses detection gaps that automation alone cannot cover, and will continue to expand algorithmic coverage to align with ByteDance product scenarios and safeguard company‑wide product quality.