Encoding‑Alignment‑Interaction (EAI) Framework for Full‑Body Human Motion Forecasting

The paper introduces the Encoding‑Alignment‑Interaction (EAI) framework, a novel approach for predicting full‑body human motion, including fine‑grained hand movements. EAI consists of three core stages: Encoding, which extracts spatio‑temporal features using Discrete Cosine Transform (DCT) and Graph Convolutional Networks (GCNs); Alignment, which employs Cross‑Context Alignment (XCA) to neutralize heterogeneity among body parts; and Interaction, which uses Cross‑Context Interaction (XCI) to capture semantic and physical interactions between body components.

The authors propose a full‑body motion prediction task that jointly forecasts the 25 body joints, 15 left‑hand joints, and 15 right‑hand joints (total 55 joints). They argue that existing methods focus only on major joints and ignore the crucial hand motions needed for realistic human‑computer interaction in VR, gaming, and robotics.

Key components of the framework:

Cross‑Context Alignment (XCA) : feature neutralization via learnable factors and Maximum Mean Discrepancy (MMD), followed by ring‑shaped neutralization to align body‑to‑hand features and an inconsistency constraint to reduce distribution gaps.

Cross‑Context Interaction (XCI) : semantic interaction using cross‑attention and physical interaction via a split‑and‑merge strategy that treats the wrist as a bridge between body and hands.

The training loss combines four terms: joint loss (MPJPE), physical loss (wrist error), bone‑length loss, and alignment loss, weighted by \(\lambda_1, \lambda_2, \lambda_3\).

Experiments are conducted on the large‑scale GRAB dataset (over 1.6 M frames, 10 actors, 29 actions). Evaluation metrics include Mean Per Joint Position Error (MPJPE) and MPJPE‑AW (wrist‑aligned). Two training strategies are compared: separate (D) training for each component and unified (U) training with a 55‑node graph. Results show that EAI achieves state‑of‑the‑art performance across all metrics, especially improving hand‑joint accuracy.

Ablation studies demonstrate the contribution of each module: removing XCA/XCI, or their sub‑components (neutralization, inconsistency constraint, semantic/physical interaction) leads to noticeable performance drops, confirming their importance.

The authors conclude that EAI effectively handles heterogeneity and interaction in full‑body motion prediction, opening new possibilities for immersive VR, human‑robot collaboration, and fine‑grained motion synthesis. Future work includes extending the model to incorporate object interactions.