A vision-language-action model (VLA model) is an AI model that connects visual observations and language instructions to actions. In robotics, a VLA can look at a scene, interpret a command and produce action outputs that help a robot move, grasp or manipulate objects.
VLA models extend the idea behind vision-language models. A vision-language model can describe what is visible or answer questions about an image. A VLA model adds an action interface, turning perception and language understanding into behavior.
From vision and language to action
Suppose a robot is told, “Pick up the red mug and place it on the tray.” Cameras provide images of the scene, the language instruction specifies the goal, and the VLA model produces actions or action tokens that represent how the robot should move. A low-level controller then converts those outputs into commands for joints, grippers or mobile bases.
The exact action representation differs by system. Some models predict continuous robot commands, some discretize actions into tokens, and others generate higher-level waypoints that another controller follows.
What a VLA model learns
Training data typically pairs observations and instructions with robot trajectories. The model learns statistical relationships among what a robot sees, what a person asks for and which actions lead to successful outcomes. Large datasets may combine demonstrations from multiple robots, teleoperation, simulation and automatically labeled video.
The data problem is difficult because robot trajectories are expensive compared with internet text or images. A robot must physically perform actions, and demonstrations collected on one embodiment may not transfer cleanly to another.
VLA models vs vision-language models
A vision-language model is optimized to understand or generate information about images and text. A VLA model must additionally produce behavior. That difference changes evaluation: a fluent explanation of a scene is not enough if the robot collides with the table or fails to grasp the object.
How VLA models relate to robot foundation models
A VLA can be a type of robot foundation model when it is pretrained broadly and reused across many tasks or robots. “VLA” describes the model’s input-output structure; “robot foundation model” describes breadth, pretraining and reuse. The concepts overlap but are not identical.
Why VLA models matter for physical AI
Physical AI needs systems that can connect high-level goals to real actions. VLA models are attractive because language provides a flexible task interface while vision grounds the command in the current environment. A robot can potentially learn many tasks without a separate hand-coded program for every instruction.
World models and action planning
A VLA can react directly from observations to actions, while a world model focuses on predicting how the environment may change. Some architectures combine these ideas: a model can imagine or score possible futures, then choose an action policy based on those predictions.
Current examples
Google DeepMind’s Gemini Robotics family uses vision-language-action models to combine multimodal understanding with robot control. Xiaomi-Robotics-1 is described as a robot foundation model trained on large-scale real-world manipulation trajectories. These systems illustrate a broader move from models that only interpret the world toward models that can generate physical actions.
Limitations
- Data coverage: rare objects, unusual viewpoints and unfamiliar tools may fall outside training experience.
- Embodiment mismatch: actions learned on one robot may not map directly to another body.
- Long-horizon errors: small mistakes can accumulate across a multi-step task.
- Safety: action generation must remain inside hardware and operational limits.
- Evaluation: benchmark success may not reflect reliability in messy real environments.
