Inside Tencent Cloud's Scalable Face‑Fusion: Architecture & AI Techniques

Face‑fusion, a popular entertainment‑oriented AI product, gained massive attention during events like the 2017 National Day collaboration between People's Daily and Tencent, which attracted over 800 million visits in three days.

Typical Use Cases

The technology is applied in brand promotions (e.g., Jiangxiaobai’s "Find the Other Me" campaign), cultural events (2019 New Year "Pre‑see 2019·National Destiny 70"), exam‑related activities, internal Tencent anniversaries, and media collaborations such as the May 4th Youth Day and recent Two‑Session promotions.

Technical Workflow

The end‑to‑end process starts with a user‑uploaded photo and a template (e.g., a military uniform image). First, a face‑detection model locates the face and key points, then the system aligns the input photo to the template. After extracting facial features, the algorithm blends the user’s face onto the template, followed by image correction to handle lighting and angle variations.

Algorithms

The service relies on Youtu Lab’s face‑detection and key‑point localization algorithms, which identify up to 90 facial landmarks (including eyebrows, eyes, mouth, pupils, and contour). These landmarks enable precise feature extraction and high‑quality fusion.

System Architecture & Scaling

To support massive traffic, the architecture uses static‑file acceleration, load balancing, cloud virtual machines (CVM), and object storage (COS). Global CDN and cloud components ensure users connect to the nearest node. Zookeeper synchronizes API configuration across regions, enabling seamless failover. The backend is built on micro‑services (logging, audit, face‑fusion services) with stateless clusters, Redis caching, message queues (MQ), and COS for photo storage.

Monitoring & High Availability

Multi‑layer monitoring covers infrastructure (CPU, memory, network), automatic scaling (up/down based on metrics), and health checks that demote or remove unhealthy instances from load balancers. API‑layer metrics capture request success rates, latency, and per‑user statistics. Each request receives a unique ID, allowing end‑to‑end tracing via Elasticsearch logs for rapid fault isolation.

Content Moderation

The face‑fusion API integrates automatic content‑review capabilities that filter pornographic, violent, or politically sensitive images, ensuring non‑compliant media are blocked before they can be shared.

SDKs & Integration

Various language SDKs (e.g., Java, Python, Go) enable developers to call the face‑fusion API with minimal code. A visual console can generate sample code automatically, and online testing tools help fine‑tune parameters before deployment.

Q&A Highlights

Video face‑fusion uses a tracking algorithm to follow multiple moving faces.

Load balancing is provided by Tencent Cloud's CLB, both internal and external.

High‑availability is achieved through multi‑region Zookeeper clusters and CDN/Redis caching for read‑heavy scenarios.

Video processing is asynchronous; a 900 MB video may take 3–4 minutes for decoding, frame extraction, and fusion, making sub‑20 MB videos the practical limit for real‑time sharing.

Future Directions

Upcoming features include multi‑person fusion (combining several faces into one template), video‑level fusion with real‑time face tracking, and further improvements to speed and accuracy.