Breaking Scene Binding: Adaptive Diffusion Policy (DADP) Boosts Robot Generalization

Motivation

Diffusion‑based policies generate actions by iteratively denoising a noise vector. Real‑world robots operate under non‑fixed dynamics: ground friction, body parameters, mass and damping vary, and the same task may require different motion modes across dynamics.

Problem with Environment‑Bound Policies

In reinforcement and imitation learning, policies are tightly coupled to the training environment. When deployment conditions shift, learned motion patterns often fail. Prior pipelines first extract a domain representation from interaction history and then concatenate it to the policy network, but both steps can be error‑prone.

Transient cues : recent velocity, gait phase, and higher‑order motion trends are mixed into the representation, causing it to drift within the same domain.

Limited use of domain information : standard diffusion policies start from a pure Gaussian prior; the domain label is only an auxiliary input, so the denoising process itself is not adapted to the domain.

Domain‑Adaptive Diffusion Policy (DADP)

Core Action 1 – Lagged Context Dynamical Prediction

DADP introduces a time offset Δt that separates the context from the prediction point. The extreme setting Δt→∞ samples context from a different episode of the same domain, forcing the model to answer “what physical world does the robot live in?” rather than “which leg is moving now.”

Core Action 2 – Domain‑Aware Diffusion Injection

Standard diffusion policies start from a pure Gaussian noise distribution. DADP replaces this prior with a mixture of Gaussians biased by the learned domain embedding z. The diffusion prior becomes domain‑aware, and the model predicts both the noise and a domain offset at each denoising step.

Experimental Evaluation

Three evaluation settings are used:

Seen : domains present during training.

Unseen : new parameter combinations within the training range (interpolation).

OOD : domains outside the training factor space (extrapolation).

Benchmarks include four MuJoCo agents (Ant, HalfCheetah, Walker2d, Hopper) with variations in friction, damping, leg length, torso length, and mass, plus two Adroit manipulation tasks (Door, Relocate). Zero‑shot performance is reported.

Walker2d results illustrate the advantage:

Seen: DADP 3999 vs. Meta‑DT 1304

Unseen: DADP 2834 vs. Meta‑DT 889

OOD: DADP 2197 vs. Meta‑DT 954

Similar gains are observed on Ant, Hopper, and HalfCheetah; on Adroit Door DADP is competitive in Unseen and Relocate.

Ablation Studies

Effect of Lagged Context (Δt)

Increasing Δt improves representation quality. Linear‑probe accuracy on Walker2d rises from 27.9 % (Δt=1) to 99.3 % (Δt→∞), and reconstruction loss drops from 476.1 to 3.2, confirming that lagged context yields cleaner domain clusters.

How to Use the Representation

Three variants are compared:

Concatenating the representation to the policy input – yields unstable gains.

Biasing only the diffusion prior – helps but is insufficient.

Full DADP – biases the prior and predicts a composite target containing the representation offset; achieves the highest scores on Walker2d and HalfCheetah.

Additional Findings

Under non‑stationary friction changes, a single Walker2d checkpoint maintains high performance, indicating that online context can track friction variations.

When DDIM sampling is compressed to a single step, standard diffusion policies collapse, whereas DADP retains most of its performance because the diffusion start point is already close to the target action manifold.

Limitations

The current method assumes static or piecewise‑stable dynamics. Extending DADP to continuously evolving domains while preserving stable domain factors remains an open challenge.

Conclusion

Integrating domain information into the diffusion prior, rather than treating it as a peripheral label, enables robust cross‑domain control and brings robot generalization closer to the underlying physics of motion.

Code example

来源：新智元
本文
约2500字
，建议阅读
5
分钟
本文介绍 DADP 策略，通过表征解耦与扩散注入，提升机器人跨域自适应控制能力。