How Xiaomi’s Tactile‑Enabled Robot Graduates from Lab to Automotive Assembly Line

Previously we introduced the tactile‑refinement model TacRefineNet and the Vision‑Language‑Action (VLA) large model Xiaomi‑Robotics‑0, which together give robots a knowledgeable brain, agile body, and the missing piece of tactile perception.

Moving from the lab to an automotive factory reveals a huge reality gap: production cadence and pass rate. In the lab thousands of failure iterations are acceptable, but the factory demands precise, reliable actions timed to the second. The robot must evolve from an "apprentice" to a "full‑time worker".

In a real factory self‑tapping‑nut pick‑and‑place station the robot ran autonomously for three hours, achieving a dual‑side installation success rate of 90.2% and satisfying the fastest 76‑second line cadence.

The task requires the robot to repeatedly grasp self‑tapping nuts from an automatic feeder, place them on a positioning fixture, and cooperate with a slide table and automatic locking of the work station to automate the fastening of floor‑panel nuts after integrated casting.

Key technical highlights:

General VLA base model: Unified action‑space design and cross‑embodiment pre‑training dramatically improve generalization in task understanding, spatial perception, and motion execution.

VLA + RL joint training framework: Introducing reinforcement learning reduces dependence on real‑robot tele‑operation data and enhances cross‑embodiment and cross‑environment generalization.

Tactile information fusion: Incorporating tactile feedback in dense‑contact factory scenarios significantly boosts stability and robustness.

To support planning and provide online RL rewards, we fuse vision, tactile, and joint proprioception for collaborative perception, markedly lowering state‑misrecognition under complex conditions. For example, relying solely on vision leads to uncertainty under lighting changes or occlusion, while tactile alone can be disturbed by unintended contacts.

Whole‑body motion control: A hybrid architecture combines optimization control and reinforcement learning. The optimizer uses quadratic programming with null‑space projection to enforce four strict priority levels—balance, safety, task, and other metrics—solving each instance in under 1 ms. The RL controller is trained on a large‑scale parallel simulation platform, exposing billions of random disturbances and failure scenarios to teach the robot balance‑preserving strategies that transfer zero‑shot to the real robot.

Typical failure cases: Misalignment of the nut’s spline with the pin leads to jamming and incomplete installation; variable grasp pose and magnetic pull further increase difficulty, resulting in poor spline‑pin fit or missed placement.

Beyond the self‑tapping‑nut station, other pilot stations (e.g., material box handling, badge installation) are being deployed. Scaling to broader industrial use still requires breakthroughs in production cadence, pass rate, whole‑body coordination, and dexterous hand efficiency.

All earlier technical details, experiment videos, and code are openly available:

Project page: https://sites.google.com/view/hil-daft/

arXiv paper: https://arxiv.org/abs/2509.13774

GitHub repository: https://github.com/XiaomiRobotics/Xiaomi-Robotics-0