From Pixels to the Physical World: Inside Gaode’s ABot Full‑Stack

ABot Full‑Stack Overview

Gaode, leveraging two decades of spatiotemporal data and large‑scale map engineering, proposes a three‑layer embodied‑intelligence foundation composed of a World Model, navigation models (N series), operation models (M series) and a Harness architecture. The stack runs on a unified ABot core, closing the loop from environment understanding through path planning to fine‑grained actuation, and is designed to be deployed on wheeled, quadruped and humanoid robots with consistent robustness.

ABot General EAI System

The system consists of a Data Infra Layer, a Foundation Model Layer and an Agent Layer, forming a closed loop “real‑world data → sustainable algorithm iteration → long‑term physical feedback”.

Data Infra Layer

Core is the ABot‑World simulation engine, which batch‑generates Video, Depth, Point Cloud and Trajectory data. Combined with an RL Training Engine, it defines rewards and performs trial‑and‑error in virtual environments, effectively replacing costly real‑world data collection.

Foundation Model Layer

ABot‑M handles manipulation, ABot‑N handles navigation. The two models are trained separately and combined through a Model‑Skill mechanism to accomplish long‑range, complex tasks.

Agent Layer

ABot‑Claw provides task planning, multimodal memory and closed‑loop error correction, explicitly assuming that execution errors will occur.

ABot‑World: Interactive Embodied World Model

Unlike conventional visual‑only world models, ABot‑World embeds physical laws throughout the pipeline, becoming a differentiable, evolvable physics engine for robots.

Embodied‑native architecture : 14B DiT model that takes observation and action as input and directly generates future state sequences in latent space, trained on tens of millions of navigation and manipulation trajectories.

3DGS cold‑start base : From sparse inputs (phone photos, aerial maps) to high‑fidelity 3D reconstruction via “coarse‑model → high‑fidelity repair → distillation loop”.

Physical hard‑constraint training : Diffusion‑DPO framework aligns physical preferences, using VLM‑generated rule lists and a discriminator to create preference pairs; each frame includes mass, friction, contact forces.

Dual‑engine self‑evolution : Parallel “training engine + data engine”. Leveraging high‑precision maps and real trajectories, 3DGS produces centimeter‑level reconstructions and lighting‑consistent scenes, generating millions of 3D scenes, billions of inference data and tens of millions of training trajectories covering 99 % of typical life scenarios. VLA closed‑loop enables “predict‑as‑train, rehearsal‑as‑learn”. Cross‑modal action mapping unifies control of single‑arm, dual‑arm and dexterous hands.

Benchmark Performance

ABot‑World ranks first in industry‑authoritative embodied AI leaderboards (WorldScore, WorldArena, Agibot World Challenge) and outperforms OpenAI Sora 2.0, Google DeepMind Veo 3.1, NVIDIA Cosmos and emerging models on physical realism, controllability and 3D consistency, surpassing Veo 3.1 by 10 % on WorldArena.

ABot‑N (Navigation)

Unified Architecture : Hierarchical “Brain‑Action” design. Cognitive Brain handles high‑level semantics and spatial reasoning; Dream Module performs future rollout; Action Expert generates continuous multimodal trajectories via Flow Matching, enabling a single model to cover five core navigation tasks.

Heterogeneous Target Encoder : Unified multimodal encoding for panoramic/monocular vision, text commands, object categories, POI names and geometric coordinates; temporal memory maintains context in POMDP environments.

RL Alignment : SAFE‑GRPO framework teaches the model that “physically traversable ≠ socially compliant”, avoiding lane intrusion, stepping on grass, etc., through a three‑stage curriculum (Cognitive Warm‑up → Sensorimotor SFT → Value Alignment).

ABot‑N sets new SOTA on seven benchmarks (VLN‑CE, HM3D‑OVON, EVT‑Bench, etc.) in navigation accuracy, social compliance and zero‑shot generalisation.

ABot‑M (Manipulation)

Action Manifold Learning : 14B DiT predicts continuous feasible action trajectories, shifting learning from denoising to manifold projection, improving stability and decoding efficiency for high‑DOF whole‑body control.

Semantic × Spatial Experts : Dual‑stream perception; semantic stream inherits VLM multimodal understanding, spatial stream injects 3D perception and multi‑view implicit fusion, fused via cross‑attention to boost fine‑grained operation precision.

Unified Action Representation & Sustainable Generalisation : Actions expressed as incremental EEF‑frame deltas, enabling parameter sharing across single‑ and dual‑arm tasks; padding mechanism allows a single network to model multiple morphologies; two‑stage training (pre‑train + spatial‑aware fine‑tune) supports continual addition of new forms while preserving existing skills.

Benchmark Leadership : SOTA on four embodied manipulation benchmarks (LIBERO, RoboCasa, RoboTwin 2.0, LIBERO‑Plus), surpassing GR00T‑N1 by 11 % success on RoboCasa and π0.5 by 44 % on RoboTwin.

ABot‑Claw: General Harness Architecture

Addresses long‑range task closure, weak multi‑robot coordination and knowledge sharing. Provides centralized scheduling, shared spatial memory anchored to global coordinates, hierarchical cloud‑brain / edge‑claw control, and social behaviour alignment via multi‑agent RL that learns elevator avoidance and pedestrian courtesy.

Demonstrated on real robots: quadruped fetching coffee, humanoid greeting visitors, and multi‑arm kitchen operation, showing robust long‑term task execution and cross‑morphology collaboration.

Ecosystem and Vision

Gaode runs a “Physical AI data flywheel”: billions of daily real‑world travel data feed algorithmic iteration, whose outputs return to products, generating further data. The closed loop continuously improves model accuracy and creates a self‑valuing asset. Gaode open‑sources parts of the stack (ABot‑World, ABot‑M, ABot‑N, ABot‑Claw) to foster a collaborative spatial‑intelligence ecosystem, inviting developers, partners and hardware vendors to build on a common foundation.

For more details, see the Amap CV Lab website and the GitHub repositories listed.