Researchers have introduced a new benchmark, ActiveVision, designed to assess how well artificial intelligence models replicate human active visual observation, a process where gaze is continuously redirected by intermediate hypotheses. The findings, published on arXiv, indicate that even leading multimodal large language models (MLLMs) demonstrate significant limitations in this area.
Active observation is crucial for a wide range of tasks, allowing for continuous refinement of understanding through repeated visual perception rather than a single static description. Current vision-language benchmarks, however, do not adequately measure this capability. The ActiveVision benchmark comprises 17 distinct tasks across three categories: Distributed Scanning for exhaustive coverage of local signals, Sequential Traversal for ordered movement along connected structures, and Visual Attribute Transfer for fine-grained comparison across regions. These tasks are specifically engineered to require iterative visual perception, making it difficult for models to rely on a single linguistic summary of an image.
In evaluations using the ActiveVision benchmark, frontier MLLMs, including OpenAI's GPT-5.5, performed poorly. The highest-scoring model, GPT-5.5, when utilizing its highest reasoning-effort tier, managed to solve only 10.6% of the tasks. Furthermore, GPT-5.5 scored zero on 11 out of the 17 tasks presented. Another advanced model, Claude Fable 5, which often ranks highly on reasoning and coding leaderboards, achieved a score of just 3.5% on ActiveVision. For comparison, three human participants averaged 96.1% across the same tasks.
The research also explored whether models could overcome these limitations by writing and running their own vision code. However, the study found that such code proved unreliable on realistic imagery, and the process of identifying these failures itself demanded the active perception that the models lacked. This suggests that the ability to substitute active vision with tool use is limited and costly.
The ActiveVision benchmark draws upon decades of work in cognitive science and computer vision that posits perception as an inherently active process. It also builds on existing research highlighting MLLMs' difficulties with basic visual perception and existing benchmarks that emphasize vision over language. The tasks are designed to be challenging due to arbitrary positions of items and subtle attribute distinctions, resisting concise language descriptions and demanding continuous interaction with visual data.
GPT-5.5, while recognized for its advanced agentic capabilities, including planning, tool use, and sustained task execution, has shown strong performance in other benchmarks. These include Terminal-Bench 2.0 for complex command-line workflows and OSWorld-Verified for operating a real computer. However, its performance on ActiveVision indicates that its strengths do not directly translate to the nuanced demands of iterative visual perception.
The researchers conclude that current MLLMs lack robust active visual observation capabilities. They advocate for the development of new architectures and training objectives that can better integrate perception and reasoning, effectively closing the perception-reasoning loop.