Kuaishou 3D Digital Human Live Streaming and Interactive Solution

The presentation, delivered by Jian Weihua, head of visual interaction technology at Kuaishou, outlines the company's efforts since 2021 to develop 2D and 3D digital humans, focusing on the 3D side for live streaming and interactive scenarios.

It introduces the overall solution, which consists of a platform side (Kuaishou Metaverse Interactive Platform, KMIP) built on PaaS technology, and a software integration side that includes the Kuaishou Virtual Studio (KVS) and KMIP SDKs for cloud rendering and gameplay extensions.

The KMIP platform supports multi‑terminal access (PC, mobile, XR), low‑latency RTC transmission of motion capture, voice, text and control data, synchronized multi‑user state, thousand‑person simultaneous rendering, and heterogeneous cloud‑edge computing to handle large‑scale interactive workloads.

Technical architecture is layered into a foundation layer (network, storage), a compute layer (AI engine, graphics engine, audio‑video capabilities, asset library), and an access layer (authentication, scheduling, edge‑cloud coordination). Asset management integrates AI‑driven generation, facial modeling, and standardized material and lighting pipelines.

Key innovations include high‑quality optical and inertial capture, low‑cost camera‑only motion capture, extensive special‑effects integration, cloud‑rendered multi‑user virtual worlds, AI‑enhanced personalization, and a unified meta‑computing framework that leverages KwaiNN for performance gains.

Real‑world deployments showcase virtual avatars such as the popular “Guan Xiaofang” and other IP‑driven virtual hosts, virtual concerts, esports‑style interactions, and collaborative experiences with external partners, demonstrating significant user engagement and data metrics.

The talk concludes with a summary and outlook, emphasizing Kuaishou's massive user base, the scalability of the virtual world interaction framework, and an invitation to external developers to contribute content, AI capabilities, assets, or hardware to accelerate the metaverse ecosystem.