Why Vision Alone Isn't Enough: Adding Tactile Sensing to VLA Models

From Seeing to Acting: What Is VLA?

Traditional robot pipelines separate perception, planning, and control. Vision-Language-Action (VLA) models unify these stages: they take camera images, natural-language instructions, and the robot's proprioceptive state, and directly output the next action for the arm and hand. Instead of answering "What do I see?" VLA answers "What should I do to complete the task?" This end-to-end approach has become a leading paradigm for general-purpose robot manipulation.

Why Seeing Does Not Equal Touching

Vision excels at global scene understanding — object location, approach direction, distance to target. However, once the hand nears the object, vision becomes unreliable for contact-rich questions: Has a finger actually touched the cup? Which finger made contact? Where exactly is the contact point? Is the object slipping? Is the grasp stable? These are physical interaction properties that cameras cannot measure directly. Humans can grasp a cup with eyes closed because our hands are both actuators and high-resolution sensory organs. This gap motivates tactile sensing for robots.

From Vision to Vision + Tactile

If cameras are the robot's eyes, tactile sensors are its skin. Vision observes the scene from a distance; tactile sensing activates at the moment of contact. The two modalities are strongly complementary: before contact, vision tells "where the object is"; after contact, tactile tells "how my hand is interacting with it." This expands the VLA input from Vision + Language to Vision + Language + Tactile, allowing the model to perceive ongoing physical interaction.

Recent work integrates both modalities into manipulation models:

TacVLA jointly encodes vision, language, proprioception, and tactile signals to generate actions.

ViTacFormer and DreamTacVLA similarly explore vision-tactile fusion for robotic control.

Conclusion

Moving from vision to touch extends the robot's perceptual frontier. Vision lets the robot observe the world; VLA lets it act on observations; tactile sensing lets it feel how its actions physically couple with the world. The real world contains not only color, shape, and position but also pressure, friction, contact, and slip. For future robots that must perform fine-grained grasping, dexterous manipulation, and complex assembly, merely "seeing" is insufficient — they must learn to "feel."