How Motion Retargeting Powers Real‑Time Virtual Humans: Techniques & Insights

Background

From traditional CG animation, special effects, and games to today’s virtual idols and metaverse, high‑quality virtual characters are a core component of immersive content. Motion retargeting (or motion adaptation) is a key technology that enables a source motion to drive any virtual character while preserving semantic features and natural fluidity.

Motion Retargeting Overview

The goal is to map motion captured data onto arbitrary virtual avatars, ensuring that the driven character’s motion matches the source semantics and remains smooth. Challenges include diverse character morphologies and complex interaction dynamics.

Traditional Geometry‑Based Methods

These methods abstract motion as point sets in 3‑D space and model geometric properties mathematically.

Interaction Mesh : Simplifies the character with capsules, represents motion as a point cloud of skeletal joints, builds connections via Delaunay tetrahedralization, and uses Laplacian coordinates to measure similarity.

Relation Descriptors : Uses sampled environment points as references and encodes offsets between skeletal joints and these points to capture relative positions.

Egocentric Mesh : Samples points on the body surface, encodes offsets between end‑effectors and sampled points, normalizes by skeletal structure to describe similarity across models.

Aura Mesh : Directly models mesh deformation to preserve inter‑body contacts and interactions.

Deep‑Learning Approaches

Recent works employ neural networks to learn implicit motion representations.

NKN : Uses an RNN encoder‑decoder to map source skeletal rotations to target rotations, trained with adversarial cycle‑consistency.

SAN : Introduces skeleton‑aware convolution and pooling operators to map different skeletal topologies to a canonical structure before decoding.

CAR : Encodes mesh vertices with PointNet, combines RNN‑based motion encoding, and jointly embeds motion and geometry in a latent space, explicitly preserving contact relationships.

Technical Solution by Kuaishou Y‑Tech

Kuaishou Y‑Tech improves the Interaction Mesh, incorporates body proportion information, and retains contact cues. The pipeline builds a sparse point‑cloud model for each character, computes geometric features with the enhanced Interaction Mesh, and drives the avatar via either inverse kinematics or forward kinematics.

Results

The optimized algorithm achieves high‑quality retargeting for single‑person motions, preserves limb contacts, and handles large body‑proportion differences, outperforming commercial tools such as MotionBuilder in visual fidelity and runtime efficiency.

Conclusion and Future Work

Kuaishou’s lightweight retargeting algorithm effectively transfers single‑person motions to diverse avatars while preserving contact information, and its real‑time capability supports numerous virtual‑human applications. Future research will explore multi‑person interactions, environmental context adaptation, and further optimization of generalization.

References

https://zh.wikipedia.org/wiki/动作捕捉

Ho E S L, Komura T, Tai C L. Spatial relationship preserving character motion adaptation. ACM SIGGRAPH 2010.

Lee D T, Schachter B J. Two algorithms for constructing a Delaunay triangulation. IJCS, 1980.

Al‑Asqhar R A, Komura T, Choi M G. Relationship descriptors for interactive motion adaptation. SIGGRAPH/Eurographics 2013.

Molla E, Debarba H G, Boulic R. Egocentric mapping of body surface constraints. IEEE TVCG, 2017.

Jin T, Kim M, Lee S H. Aura mesh: Motion retargeting to preserve spatial relationships. CGF, 2018.

Villegas R, Yang J, Ceylan D, et al. Neural kinematic networks for unsupervised motion retargetting. CVPR, 2018.

Aberman K, Li P, Lischinski D, et al. Skeleton‑aware networks for deep motion retargeting. TOG, 2020.

Villegas R, Ceylan D, Hertzmann A, et al. Contact‑Aware Retargeting of Skinned Motion. ICCV, 2021.

Qi C R, Su H, Mo K, et al. PointNet: Deep learning on point sets for 3D classification and segmentation. CVPR, 2017.