Automating High-Fidelity Digital Human Creation: Scanning, Driving, and Remaining Challenges

YINGMOU explains how the industry traditionally spends weeks on digital‑human creation, heavily relying on manual work, but their automated pipeline can produce a fully driven, PBR‑textured digital human in just 3–5 days.

The core of this achievement is a product‑grade facial asset database (PFA) containing hundreds of high‑resolution assets with personalized muscle blendshapes, pore‑level geometry, complete PBR materials, dynamic maps, and independence from third‑party generators, usable across Unity, UE, Maya, and Blender.

The scanning technology evolved from mold casting to structured‑light, multi‑view capture, and finally to a light‑field system. Starting in 2017 with a volumetric capture rig, they added photometric techniques in 2018 (CVPR 2018), single‑photo reconstruction in 2019 (ICCV 2019), and a dome‑light field stage in 2021 that records facial appearance under varying illumination, enabling sub‑micron skin detail and high‑precision expression capture.

Because high‑detail models contain millions of triangles, manual re‑topology is impractical. YINGMOU developed an automatic re‑topology pipeline that registers low‑poly scans to high‑poly references, then uses neural‑network‑based volume reconstruction to generate consistent low‑poly topology with detailed normal maps, reducing a full scan to fifteen minutes.

The driving system combines tracking (capturing performer facial motion) and retargeting (applying motion to new characters). Existing methods—generic bone‑skin rigs, blendshape‑based rigs, and muscle simulation—failed to meet their requirements, so they created an ML‑rigging solution that automatically splits facial muscles into 56 physics‑based blendshapes, adds components (eyes, lashes, teeth), and generates dynamic maps for wrinkles and blood flow.

Remaining challenges include realistic hair rendering and cloth simulation. Future work aims to extend the dome‑light field to capture hair strands, and to integrate capture and binding into a single neural network, a solution already accepted as a SIGGRAPH Asia Technical Paper.

Overall, YINGMOU has built a comprehensive, AI‑driven pipeline that dramatically shortens digital‑human production while delivering photorealistic assets, yet continues to address hair, cloth, and fully integrated capture‑to‑animation workflows.