How Alibaba’s Taobao Life Renders Realistic Virtual Avatars on the Web

Industry Background

The talk begins by noting the growing demand for high‑fidelity human rendering in games and movies, citing examples like "Alita: Battle Angel" and the 2018 Siren virtual human demo.

Cross‑Domain Applications

Virtual idols and VTubers illustrate how avatar technology creates new business value, with millions of creators on platforms such as Bilibili and YouTube.

Rendering Basic Process

When a user enters Taobao Life, the engine loads a shared base model, textures, and materials, then applies user‑specific configuration data (clothing, beauty, facial adjustments) to generate a personalized avatar before rendering.

Rendering Pipeline Basics

The system follows a classic 3D real‑time rendering pipeline where CPU and GPU work together to pass data through successive stages, repeatedly drawing geometry until the final image is composed.

Clothing & Beauty Implementation

Clothing assets are loaded dynamically as independent models and attached to the appropriate skeleton joints. To avoid rendering hidden body parts, the pipeline masks out geometry covered by clothing, reducing draw calls and preventing visual artifacts.

Beauty effects are applied via dynamic texture merging: pre‑made texture patches for foundation, eye‑shadow, lipstick, etc., are composited on‑GPU into a single facial texture (Render‑to‑Texture) that is then used in the second rendering pass.

Architecture Overview

A hierarchical renderer structure is used: CharacterRenderer at the top, with HeadRenderer and BodyRenderer underneath. The body renderer handles dynamic clothing, while the head renderer manages both texture merging and accessory loading.

Effect Optimizations

The engine adopts PBR materials and adds custom effects such as Sub‑Surface Scattering for skin translucency and anisotropic shading for hair and fabrics, enhancing realism without excessive performance cost.

Facial Capture & AR Integration

Real‑time AR camera captures the user’s face, a deep‑learning model extracts skeletal and morph (vertex‑deformation) data, and this data drives the avatar’s head, enabling live facial animation.

Same‑Layer Rendering

Because the AR camera runs natively, a "same‑layer rendering" bridge component transfers the camera feed into the Web canvas, allowing Web‑based avatar rendering to combine seamlessly with native AR capabilities.

Why Web?

Taobao Life runs inside the mobile Taobao app, so a Web‑based solution avoids bundling large asset packages in the native binary, enables rapid feature rollout, and leverages existing Web 3D engines (LayaAir, now migrating to Hilo3D).

Web Advantages

Fast iteration and deployment without app store releases.

Flexibility to experiment with new features.

Ability to reach a massive user base instantly.

Web Disadvantages

Limited pre‑loading compared to native apps.

WebGL performance and feature gaps versus OpenGL/Metal.

Higher GPU overhead due to browser abstraction layers.

Conclusion

The presentation summarizes the dual approaches of bone‑skinning and vertex morphing for avatar customization, the integration of AR facial capture, and the pragmatic choice of Web technologies to deliver a rich, interactive virtual‑human experience within Taobao.