PhyFilter: Physics-Based Filtering Enables Robot Generalization Beyond Data Scaling

Background: Is Data Scaling the Only Path to Robot Generalization?

The past decade's progress in large language models has made scaling data the default route to general intelligence. However, robot training data must be collected in the physical world, making this path nearly infeasible. Teleoperation is expensive and slow; simulation and video generation suffer from sim-to-real gaps, high compute costs, and lack of standardization. The authors turn to biology: organisms generalize not just through scale but through innate physical structures and real-time feedback loops that minimize energy and adapt to unseen environments.

Core Method: Treating Learning Residuals as Filterable Signals

PhyFilter starts from the learning residual — the difference between the true mapping and the learned model — which arises from sensor noise, external disturbances, or unmodeled interactions in unseen scenarios. Since residuals cannot be measured directly during deployment (no labels), the team draws inspiration from low-pass filtering in human sensory systems. They formulate the correction as a high-order low-pass filter that converges to capture low-frequency residual features, with the filter's convergence process itself representing the robot's adaptation.

The key technical challenge is that the learning residual is unmeasurable. The solution uses the robot's real-time state feedback and known prior differential-equation structure to rewrite the filter into an equivalent form that depends only on measurable signals, avoiding direct high-order differentiation of the nominal model.

Key Properties of PhyFilter

Plug-and-play: Model-agnostic lightweight module deployable on pre-trained models without retraining.

Degradable and extensible: First-order setting reduces to a feedback disturbance observer; higher orders handle more complex residuals.

Auto-learned parameters: Filter parameters, traditionally hand-tuned via pole placement, are reinterpreted as control inputs of a dynamical system. The tuning becomes an optimal control problem solved with Lagrange multipliers and the adjoint method (reverse-mode differentiation), optimized with Adam, mini-batches, and early stopping. Learned parameters match carefully hand-tuned ones.

Low compute: Runs at 500 Hz on an STM32F765 microcontroller in drone and aerial-manipulator experiments, proving suitability for resource-constrained platforms.

Evaluation: Four Real Robot Systems

Experiments cover both reinforcement learning (RL) and supervised learning (SL) paradigms.

Quadruped Locomotion: Sim-to-Real Across Terrains

Policy trained only on flat ground in simulation with standard parameter randomization. With PhyFilter, training return is consistently higher across five seeds, and gait is more coordinated. Generalization test: robot runs at 2.83 m/s (outside training distribution) carrying 15%–60% extra body weight (11.86 kg), up to 137.33% beyond training limit. Success rates on five unseen terrains:

Stairs: 70% (baseline 0%, RL2AC 0%)

Horizontal rough: 80% (baseline 0%, RL2AC 20%)

Slope rough: 50% (baseline 10%, RL2AC 10%)

Wavy: 80% (baseline 20%, RL2AC 50%)

Gravel: 40% (baseline 0%, RL2AC 0%)

Real-world deployment without any tuning: baseline fails on sand and gravel; PhyFilter succeeds on all terrains. Notably, even with the correction term disabled at deployment, the policy still transfers — PhyFilter shapes a fundamentally better policy, not just a runtime patch.

Drone Agile Flight: Plug-and-Play Generalization Boost

PhyFilter added on top of SEER-I (offline basis-function learning + online adaptive control). SEER-I handles mass uncertainty but degrades under wind disturbances. On a 2 m/s circular trajectory, PhyFilter reduces mean absolute tracking error by 30.22% vs SEER-I and 50.17% vs baseline. Uncertainty estimation shows SEER-I captures only mass uncertainty, while PhyFilter identifies both wind-induced disturbances and inherent dynamics uncertainty.

Aerial Manipulator: Centimeter-Level Pick-and-Place Under 5 m/s Wind

Task: pick medicine from a tripod and place in target zone, max allowed tracking error 2.5 cm. A 380 W fan generates 5 m/s wind; extra 0.3 kg unmodeled load added. Baseline and SEER-I fail; PhyFilter completes all four stages, improving tracking error by 23.99% in mean and 55.04% in variance over baseline.

Acceleration Perception: Reliable Neural Differentiation Under Distribution Shift

For lightweight drones lacking high-precision sensors, acceleration estimation is hard. A two-hidden-layer neural network maps historical velocity to acceleration, trained on 0–2 m/s, tested on 0–3 m/s. Pure learning model fails; PhyFilter infused with kinematic structure knowledge achieves highest accuracy, outperforming robust differentiator (RD), tracking differentiator (TD), and discrete differentiator (DD) across sampling intervals 0.002–0.018 s.

Unified Insight: Generalization and Precision Without Excessive Domain Randomization

Current mainstream relies on domain randomization — essentially seeking average performance across random environments, trading precision for robustness, akin to classic robust control. PhyFilter unifies generalization and precision in one framework: it activates in out-of-distribution scenarios to enhance generalization, while its intervention adaptively decays in familiar conditions to preserve accuracy. This reduces the engineering effort spent crafting effective randomization parameters.

Limitations and Future Directions

Output-only enhancement: Currently PhyFilter only corrects the learning output. Future work could feed filtered knowledge back to refine the original network's parameters or structure via continual learning.

Parameter learning efficiency: Each optimization iteration requires a full forward trajectory rollout, limiting large-scale data efficiency. Truncated or parallelized rollouts are promising improvements.

Residual disentanglement: Learning residuals mix neural prediction errors with other uncertainties (e.g., modeling mismatch in quadruped case). Disentangling these components, possibly via meta-learning, remains open.

Conclusion

PhyFilter offers a complementary path to "data scaling": robots are not image generators lacking physical priors; they already possess well-structured dynamics models and continuous real-time feedback. Leveraging these existing physical structures may be closer to the essence of biological generalization than endlessly expanding datasets.

Key References and Resources

Paper: https://www.nature.com/articles/s44182-026-00114-y DOI: 10.1038/s44182-026-00114-y Project Website: https://scoardyy.github.io/PhyFilter/ Code:

https://github.com/JIAjindou/PhyFilter