Data In Motion: The Embodied Intelligence Data Battle – Human Video vs Robot Data

Embodied Intelligence Data Battle: Core Questions

The Triple T summit at Zhangjiang AI Innovation Town focused on how to turn embodied data into verified robot capabilities. With training data reaching millions of hours, the central challenge is choosing and combining data sources — human first-person videos, real-robot interaction logs, and simulation-generated data — to maximize capability gains per unit cost across model architectures, training stages, and deployment targets.

Dual Expert Debate: Pre-training Data Priorities

Huang Yongtao (Ant Lingbo Chief Data Scientist): Prioritize Real-Robot Data

Argument: For embodied pre-training, real-robot data should come first because robots need direct physical interaction to build reliable manipulation skills. Human video data supports task understanding and high-level planning but lacks precision in action trajectories, force control, and contact feedback.

Tactile integration: Tactile sensing should be incorporated early in base capability learning, not just as a post-training add-on. Low-cost tactile collection via body-agnostic acquisition and electronic skin is worth exploring.

R&D practice: Teams should first use real-robot data to understand tasks and failure modes, then scale. Pre-training validation must go beyond success rates on known tasks to measure learning speed on new tasks, adaptation data volume, and cross-scene generalization via multi-task evaluation.

Liu Miao (Tsinghua AI Institute Assistant Professor): Context-Dependent Choice

Argument: No universal answer; data choice depends on robot application and training stage. Industrial precision grasping/assembly relies heavily on real-robot data, while open-environment general embodied intelligence benefits from the scale of human experience data.

Stage differences: Pre-training values data coverage; post-training uses higher-quality data for specific task gains. However, structural and control differences between human and robot bodies mean exact human motion replication does not guarantee transfer.

Human data value: Includes active perception, task planning, hand-eye coordination, and long-horizon behavior chaining.

Tactile trade-offs: Must weigh collection cost, quality, and coverage. Whether small amounts of specific-configuration tactile data improve pre-training remains unproven.

Evaluation gap: Current embodied pre-training lacks reliable independent evaluation; its true contribution is often masked by post-training results. Disentangling the two is a key challenge.

Technical Open Mic Sessions

Ma Daolin (Qianjue Robotics Founder / SJTU Professor): From Multimodal Collection to Physical Interaction Data

Tactile richness: Tactile signals encompass force, geometry, slip, and motion — critical for fine manipulation.

System requirements: High-quality, consistent sensors; standardized large-scale collection protocols; co-developed model training and verification infrastructure.

Collection strategy: Human-robot decoupled collection to extract tactile experience from human demonstrations; proprietary tactile simulator for low-cost, large-scale, high-quality simulation data and evaluation testbeds.

Demonstration: VTLA model (vision, tactile, language, action) shown on real-robot tasks: automatic shoelace tying and cardboard box folding.

Li Yonglu (SJTU Associate Professor / RhOS.ai Lab Lead): Data Recipes and Evaluation

Recipe design: Data mixtures should start from target tasks, jointly determined with model architecture, training objectives, and evaluation methods.

Validation loop: New data effectiveness verified by training and downstream task evaluation, feeding back into next collection/selection round.

Scaling costs: As data grows, costs shift to processing, training, and storage — demanding continuous ROI checks.

Co-design: Hardware, model, and data infrastructure should be co-designed and co-iterated to form a continuous feedback loop between collection and training, improving data investment efficiency under total cost constraints.

Li Minglei (DeepRobotics Co-founder / VP Technology): From Experience to Intelligence — Scaling and Continuous Learning

Framework: High-fidelity experience for pre-training, then continuous learning to expand capability boundaries.

HiFi-UMI: Explores reducing policy cold-start dependence on real-robot demonstrations.

Iteration loop: 3D supervision, physical evaluation, and error-feedback drive policy updates.

LLM collaboration: Integrates models like GPT-Astra with embodied policies so real deployment serves as both testbed and data source for ongoing learning in physical interaction.

Industry Panel: Diversity, Multimodality, and Scale

Chen Tianxing (HKU MMLab PhD / Lumina Community Founder)

Data value judged by distribution, quality, model architecture, and pre-training capability.

Generalization gains from new data require evaluation on long-horizon tasks, complex interactions, and scene changes, validated via cross-comparison and ablation.

For tactile, task design must ensure contact information actually participates in decision-making to test real contribution.

Zhou Hang (Mogan Technology Co-founder / CTO)

Whole-body tactile supports physical world cognition for the "brain" and closed-loop control for the "cerebellum".

Data must be stable, accurate, temporally synchronized, and filtered for motion interference.

Cost reduction via mass production, reduced customization, and AI-assisted cleaning.

Advocates unified industry data standards.

Zhao Shihao (New Intelligence Embodiment CEO)

Different training stages demand different data quantity, quality, and precision.

Effective tactile use requires first aligning data representations across hardware, then exploring fusion.

In experiments, early visuo-tactile fusion sometimes degraded performance.

Data production must emphasize operator selection and skill to improve source quality and investment efficiency.

Conclusion

The summit decomposed "data scale" into concrete problems tied to tasks, models, and deployment conditions. Discussions highlighted that data investment must target explicit capability goals and be continuously validated through evaluation. Ongoing research-industry collaboration on data production, model evaluation, and real-world application aims to turn accumulated data into verifiable robot capabilities.