Kuaishou 3D Digital Human Live Streaming and Interactive Solution

The talk by Jian Weihua, visual interaction technology lead at Kuaishou, delivered at the Datafun Digital Human Forum, introduces Kuaishou's comprehensive 3D digital‑human live‑streaming and interactive solution.

Given Kuaishou's massive user base, the company seeks new content‑creation models that turn consumers into producers by leveraging 3D digital humans as personal avatars.

The solution consists of two parts: a platform side called Kuaishou Metaverse Interactive Platform (KMIP) built on PaaS technology, and a software side comprising the Kuaishou Virtual Studio (KVS) and various SDKs that enable cloud rendering and gameplay extensions.

KMIP offers multi‑terminal access (PC, mobile, XR), low‑latency RTC transmission of structured data (motion capture, voice, control parameters), state synchronization for multi‑user interaction, support for thousands of concurrent users, and heterogeneous computing with edge‑cloud collaboration.

Architecturally, KMIP is organized into three layers: an infrastructure layer managing compute, network, and storage; a compute layer containing an AI engine, graphics engine, audio‑video capabilities, and an asset library; and an access layer handling authentication, intelligent scheduling, and edge‑cloud coordination.

The asset library integrates AI‑driven generation tools for custom avatars, facial reconstruction, and AIGC research, while establishing material, lighting, and rendering standards to ensure consistency across engines.

Motion‑capture capabilities range from high‑precision setups to low‑cost camera‑only solutions, supporting facial, body, and finger capture, and a non‑wearable algorithm that drives 3D avatars with real‑time physics and collision handling.

Cloud rendering enables large‑scale multiplayer virtual‑world interactions using GPU clusters, supporting MMO‑level sessions and integration with popular game engines (UE, Unity, Cocos). The KMIP Game SDK adds AI‑enhanced gameplay and personalized content.

Key advantages include high‑quality optical and inertial capture, low‑threshold camera capture, a rich library of effects, high‑performance heterogeneous computing, and AI integration for personalized, thousand‑person‑per‑screen experiences.

Deployment examples feature the official avatar Guan Xiaofang, singing avatar Zhang Fengqin, a virtual employee for Mengniu, virtual World Cup events, IP‑driven concerts, and interactive games that blend entertainment with social interaction.

Developers can integrate via the PC‑based KVS software, KMIP SDK for app‑level access, or KMIP Game SDK for AI‑augmented experiences, following a documented cloud‑rendering workflow.

In summary, with over 600 million monthly active users, Kuaishou sees vast potential for virtual‑world interaction beyond live streaming, invites partners to join, and anticipates large‑scale adoption of 3D digital‑human experiences within the next year.