How ABot-PhysWorld Achieves Physical Consistency in Embodied Video Generation

Background and Motivation

Recent advances in video generation models (e.g., Sora, Veo, Wanx‑2.5) have produced impressive visual fidelity, yet they fail when applied to embodied AI scenarios because the generated frames often violate physical laws. Common failure modes include gripper penetration, object disappearance, temporal incoherence, and mis‑identifying targets.

Problem Statement

Existing models prioritize visual plausibility over physical realism, learning pixel continuity without mastering real‑world dynamics. The goal is to generate video sequences that are not only visually coherent but also obey the underlying physics required for robot execution.

ABot‑PhysWorld Overview

ABot‑PhysWorld is the first sub‑project of the ABot‑World series focused on physical‑consistency video generation. It achieves a comprehensive PAI‑Bench score of 0.8491 and a domain‑specific score of 0.9306, surpassing baselines such as GigaWorld and Wanx‑2.5.

Core Design Principles

Physical Plausibility : Generated content must obey real‑world dynamics.

Task Execution Accuracy : Action sequences should be directly usable by robots.

Evaluation Benchmark – PAI‑Bench

PAI‑Bench (Physical AI Benchmark) evaluates two dimensions: physical plausibility and task execution precision. It uses datasets from BridgeData V2, AgiBot, and Open X‑Embodiment, covering 174 high‑complexity videos with diverse manipulation tasks.

Metrics include:

Robot Domain Score – binary visual Q&A accuracy over 886 domain‑specific questions using Qwen2.5‑VL‑72B‑Instruct.

Quality Score – visual‑condition consistency (I2V Subject, I2V Background, Overall Consistency, Aesthetic Quality, Imaging Quality, Motion Smoothness, Subject/Background Consistency) computed with models such as DINO, DreamSim, ViCLIP, LAION aesthetic head, MUSIQ, etc.

Four‑Dimensional Generalization Data

Data pipeline reduces 3 M raw samples to 1.6 M, then samples 300 k high‑quality SFT data covering:

Entity Generalization : multiple robot morphologies (arm, humanoid, wheeled).

Task Generalization : 50+ tasks (grasp, push, pull, assemble).

Scene Generalization : >10 environments (kitchen, lab, warehouse).

Object Generalization : >1 000 object categories with varied materials and deformability.

Automated High‑Quality Text Annotation Pipeline

Step 1 – Qwen‑VLM Structured Perception : Input video frames, output structured descriptions (subject, action, object, spatial relation, physical state).

Step 2 – Qwen‑LLM Text Summarization : Combine VLM output with original prompts to generate natural‑language instruction tags (e.g., “robot arm picks up the red cup from the left and places it on the table”).

Quality Control : Human sampling + consistency filtering to achieve >95 % annotation accuracy.

DPO Preference Alignment

Using a VLM‑as‑Judge approach, 10 k preference pairs (physically correct vs. incorrect) are constructed. Direct Preference Optimization (DPO) maximizes the probability of selecting physically correct samples, reducing penetration and deformation errors.

Dense Action Map for Fine‑Grained Control

11 k action‑control data points encode robot motions as dense spatial signals. These are injected via Context Blocks that fuse with video latents, preserving spatio‑temporal consistency. Ablation shows the full model outperforms variants on PSNR (21.09), SSIM (0.813), and DTW (0.852).

Quantitative Results

On PAI‑Bench, ABot‑PhysWorld achieves the highest overall score (0.8491) and domain score (0.9306), outperforming open‑source models (Cosmos‑Predict2.5‑2B, GigaWorld‑0, UnifoLM‑WMA‑0, WOW) and closed‑source models (Veo 3.1, Sora 2, Wan‑2.5).

Key findings:

Visual‑quality‑focused models (Veo 3.1, Sora 2) have lower domain scores (0.8350, 0.7626).

UnifoLM excels in background/subject consistency but scores poorly on physical reasoning (0.6693).

ABot‑PhysWorld attains a competitive quality score of 0.7678 while leading in domain performance, breaking the long‑standing visual‑vs‑physical trade‑off.

Embodied‑ZeroShot (EZS) Benchmark

EZS is a zero‑shot benchmark designed for embodied manipulation. It mixes real and synthetic data, creates diverse initial observations, and uses an AI‑driven questioning mechanism to verify physical compliance. ABot‑PhysWorld reaches a Domain Score of 0.8366, surpassing all baselines.

Qualitative Case Studies

Representative scenarios demonstrate the model’s ability to handle complex tasks:

Case 1 – Flexible Object Manipulation : Dual‑arm folding of a towel with realistic deformation.

Case 2 – Precise Grasping : Multi‑object picking (cups, blocks, knives) with adaptive gripper angles.

Case 3 – Articulated Interaction : Opening cabinet doors while respecting hinge constraints.

Case 4 – Fluid Handling : Coordinated dual‑arm pouring water into a bowl.

Case 5 – Cleaning Tasks : Consistent contact pressure while wiping surfaces.

Case 6 – Multi‑Scene Object Placement : Fruit classification and placement across varied lighting and backgrounds.

Conclusions and Impact

ABot‑PhysWorld demonstrates that large‑scale generalized data, preference‑aligned training, and dense action control can produce video outputs that are both visually high‑quality and physically executable. This bridges the gap between generative perception and actionable robot policies, paving the way for reliable Sim‑to‑Real transfer and broader deployment of embodied AI.