Robots Finally Get Active, Human-Like Vision

The way that a computer analyzes a visual scene is vastly different from how humans process visual information. Modern computer vision algorithms are typically fed a static image, which they then analyze to identify certain types of objects, make a classification, or some other related task. Humans understand visual data in a far different, and much more interactive, way. We don’t just take one quick look, then draw all of our conclusions about a scene. Rather, we look around, zoom in on certain areas, and focus on items of interest to us.

A group led by researchers at the Shanghai Jiao Tong University realized that by adopting a similar approach, artificial systems might be able to improve their performance. For this reason, they developed what they call EyeVLA, a robotic eyeball for active visual perception. Using this system, robots can take proactive measures that help them in better understanding — and interacting with — their surroundings.

In most embodied AI systems, cameras are mounted in fixed positions. These setups work well for acquiring broad overviews of an environment but struggle to capture fine-grained details without expensive, high-resolution sensors. EyeVLA directly tackles this limitation by mimicking the mobility and focusing ability of the human eye. Built on a simple 2D pan-tilt mount paired with a zoomable camera, the system can rotate, tilt, and adjust its focal length to gather more useful visual information on demand.

The team adapted the Qwen2.5-VL (7B) vision-language model and trained it, via reinforcement learning, to interpret both visual scenes and natural-language instructions. Instead of passively analyzing whatever image it receives, EyeVLA predicts a sequence of “action tokens” — discrete commands that correspond to camera movements. These tokens let the model plan viewpoint adjustments in much the same way that language models plan the next word in a sentence.

To guide these decisions, the researchers integrated 2D bounding-box information into the model’s reasoning chain. This allows EyeVLA to identify areas of potential interest and then zoom in to collect higher-quality data. Their hierarchical token-based encoding scheme also compresses complex camera motions into a small number of tokens, making the system efficient enough to run within limited computational budgets.

Experiments in indoor environments showed that EyeVLA can actively acquire clearer and more accurate visual observations than fixed RGB-D camera systems. Even more impressively, the model learned these capabilities using only about 500 real-world training samples, thanks to reinforcement learning and pseudo-labeled data expansions.

By combining wide-area awareness with the ability to zoom in on fine details, EyeVLA gives embodied robots a more human-like awareness of their surroundings. The researchers envision future applications for this technology in areas such as infrastructure inspection, warehouse automation, household robotics, and environmental monitoring. As robotic systems take on increasingly complex tasks, technologies like EyeVLA may become essential for enabling them to perceive — and interact with — the world as flexibly as humans do.