LAS Video Operator Drives Physical AI Breakthroughs in Robotics & Autonomous Driving

A Shared Challenge: Video Data "Too Slow to Label, Too Hard to Get Right, Too Hard to Retrieve"

As robots enter factories, retail, and homes, and intelligent vehicles tackle more complex roads, large models are moving from screen-based content understanding to real-world perception, judgment, and execution. Investment in embodied intelligence surged — H1 2026 domestic financing reached 93.5 billion RMB , a 5× year-on-year increase — yet training data remains critically scarce. Training a general embodied foundation model requires tens of millions of hours of high-quality physical interaction data; as of early 2026, global available data is under 500,000 hours , a 95%+ gap . Intelligent driving faces a similar "data hunger": hundreds of thousands of production vehicles return massive video daily, but the long-tail scenarios that actually drive model evolution account for only a tiny fraction.

Whether for embodied robots or autonomous driving, making models truly understand and reason about the physical world depends on world-model training. Facing tens-of-millions-of-hours data demand, data preparation is no longer a pre-training auxiliary step but a foundational engineering pillar of world-model construction . The efficiency of acquiring, understanding, and structurally supplying high-quality video data has become the key dividing line in physical AI competition.

One Operator, Two Customized Adaptations

General capabilities cannot solve industry-depth problems. A single operator supporting two industry needs hinges on adapting to different data paradigms: embodied videos are short , demanding high-precision action-temporal depiction; autonomous-driving videos are long , demanding precise localization of rare clips from hours of footage.

Embodied Operation Videos: Three-Step "Object–Context–Action" Pipeline

For embodied operation videos, Volcano Engine's multimodal data lake LAS builds a three-step chain around "Object–Context–Action":

Step 1: LAS video fine-grained understanding operator first identifies interacted objects, generating an object list with category, name, and key attributes.

Step 2: Constructs a unified semantic context so the same object maintains consistent reference across a long task.

Step 3: Combined with first-person video, understands action start/end points, outputs task summaries and time-bounded action descriptions. Objects, actions, and task relationships are thus converted into structured Caption data directly usable by models.

This is not simple "image captioning." Around dual-arm coordination, left/right hand distinction, repetitive actions, and timeline continuity , the team continuously optimizes prompts, output structures, and evaluation rules. In a phased test on 20 public first-person operation videos, the LAS operator achieved superior results on 19 videos, with dual-arm overall understanding accuracy reaching 94.41% , 38% higher than overseas mainstream models .

Autonomous Driving: "Front-End Funnel" for Hard-Case Labeling

For autonomous driving, the LAS operator handles the hard-case "labeling" stage, acting as the data pipeline's "front-end funnel": it combines industry rules to establish a road-obstacle label taxonomy, outputs structured tags traceable to source video and prompt version, and filters clips worth human refinement and model training. For long-video splitting before labeling, two paths are provided:

Path 1: When full processing of long videos is needed and split clips must be reused elsewhere, use EMR Ray to batch-read files from TOS paths, split by preset duration while keeping segment naming continuous, then hand off to LAS.

Path 2: When the effective time window is known and independent clips are not required, directly use LAS operator's start/end time parameters to target only the relevant interval, avoiding useless video processing.

Frame-rate and resolution adaptation strategies are also optimized to avoid one-size-fits-all auto-downgrading: high frame rates increase cost, but in embodied scenes, tiny objects like earphones or clips may lose key actions after downsampling. Customers can use AutoPE to adjust parameters per their data characteristics, then call LAS to strike a better business-fit balance among recognition accuracy, processing efficiency, and cost.

Industrial-Grade Validation: Data Supply Enters "Hour-Level", Hard Cases Automatically "Surface"

In an embodied intelligence customer's real scenario, the automated pipeline now delivers 100,000 hours/day processing capacity, supporting million-hour-scale Caption labeling tasks. Label data usability exceeds 90% , freeing teams from repetitive rework to focus on data strategy, training validation, and scenario expansion.

In an autonomous driving customer's production validation, recognition accuracy across ten-plus core operating conditions improved significantly. Daily savings exceed 200 person-days of hard-case manual review; data iteration cycle shrank from weeks to days ; task success rate reached 98%+ . This "long-video localization → hard-case understanding → structured labeling" path has a foundation for replication across more driving scenarios.

In the physical AI era, the data pipeline is the R&D pipeline. Only when video is understood faster, finer, and more consistently can robot learning speed and intelligent vehicle evolution truly accelerate. For multimodal data-intensive scenarios like embodied intelligence and autonomous driving, Volcano Engine will continue deepening multimodal data processing technology, working with industry partners to move physical AI from lab verification into real-world deployment across thousands of industries.