May 10, 2024 · Artificial Intelligence

SIF3D: Sense‑Informed Forecasting of 3D Human Motion with Multimodal Attention

SIF3D is a scene‑aware 3D human motion forecasting framework that fuses observed motion, 3D point‑cloud scenes, and gaze through novel ternary intention‑aware and semantic‑coherence‑aware attention mechanisms, encoding with PointNet++ and Transformers, and decoding with a graph‑convolutional network, achieving state‑of‑the‑art results on GIMO and GTA‑1M benchmarks.

3D scene understandingCVPR2024Computer Vision

0 likes · 15 min read

SIF3D: Sense‑Informed Forecasting of 3D Human Motion with Multimodal Attention

Xiaohongshu Tech REDtech

Jan 31, 2024 · Artificial Intelligence

Encoding‑Alignment‑Interaction (EAI) Framework for Full‑Body Human Motion Forecasting

The Encoding‑Alignment‑Interaction (EAI) framework predicts full‑body human motion—including detailed hand joints—by extracting spatio‑temporal features with DCT and GCNs, aligning heterogeneous body‑hand representations via Cross‑Context Alignment, and modeling semantic and physical interactions through Cross‑Context Interaction, achieving state‑of‑the‑art accuracy on the GRAB dataset.

Computer VisionEAI frameworkcross-context alignment

0 likes · 15 min read

Encoding‑Alignment‑Interaction (EAI) Framework for Full‑Body Human Motion Forecasting