How Adaptive Skinning Model Boosts Low-Cost High-Quality 3D Face Reconstruction

Research Challenges

Low‑cost, high‑precision 3D face reconstruction remains difficult because reconstructing a 3D model from one or few 2D images is an under‑determined problem with infinitely many solutions.

ASM Method

To overcome the limited expressive power of traditional 3D Morphable Models (3DMM), Tencent AI Lab proposes an Adaptive Skinning Model (ASM) that redesigns the bone‑skinning pipeline used in games. ASM introduces two key innovations:

Gaussian‑Mixture Skinning Weights (GMM Skinning Weights) : The skinning weight matrix is modeled as a Gaussian mixture based on the distance between mesh vertices and bones in UV space, drastically reducing the number of parameters needed.

Dynamic Bone Binding : Bone positions are also parameterized in UV space, allowing runtime adjustment of bone locations without manual rigging.

Both components are learned without any training data, using only a small set of parameters derived from facial priors.

Technical Details

The traditional Linear Blend Skinning (LBS) formula is extended in ASM by replacing fixed skinning weights with the GMM‑based representation and by allowing bone positions to be dynamically bound in UV space. This unified parameterization enables the model to capture fine facial details while keeping the parameter count low.

Experimental Results

ASM was evaluated on the LYHM and FaceScape datasets, achieving state‑of‑the‑art expression capability and multi‑view reconstruction accuracy, surpassing PCA‑based 3DMMs (BFM, FLAME), neural‑network‑based 3DMMs (CoMA, ImFace), and industry solutions such as MetaHuman.

On the Florence MICC multi‑view benchmark, ASM reached SOTA reconstruction precision for both indoor and outdoor scenarios. Additional experiments on FaceScape showed that with as few as five input images, ASM consistently outperformed competing models.

Conclusion and Future Work

ASM demonstrates a significant step toward affordable high‑fidelity 3D face modeling, with potential applications in game character creation, automatic avatar generation, and AR/VR. Future research will focus on stronger multi‑view consistency constraints to further improve reconstruction accuracy.