A new research paper published on arXiv details VBVR-Pro, a closed-loop testbed aimed at improving native visual reasoning through generative processes. The system enables visual states, such as images and videos, to function as substrates for problem-solving that extends beyond language. This approach addresses current limitations in scalable training tasks, reliable feedback mechanisms, and controlled comparisons across generative visual models.
VBVR-Pro features a controlled task space comprising 300 procedurally generated tasks. Models trained using VBVR-Pro have demonstrated transfer capabilities across seven external visual reasoning benchmarks, including RISE-Video, MME-CoF-Pro, and BabyVision. This indicates the testbed's potential to develop models that can apply learned reasoning skills to new, diverse visual challenges.
The testbed also provides verifiable reward scorers for task-grounded evaluation. These scorers use deterministic, task-specific rules to align with human judgments, offering a contrast to the "VLM-as-a-judge" paradigm, which the researchers identified as having recurring failure modes. The verifiable rewards serve as reliable signals for large-scale multi-task reinforcement learning, leading to improved post-reinforcement learning performance across visual reasoning tasks.
A mechanism study conducted with VBVR-Pro involved over 30 image, video, and interleaved generators. The analysis from this study suggests that video generation is most effective for tasks requiring persistent spatiotemporal state tracking. Interleaved generation offers a compute-efficient alternative. The researchers also found evidence of vision-native trajectories, which they describe as crucial to visual reasoning.
The development of VBVR-Pro builds on prior work in video reasoning. The "Very Big Video Reasoning" (VBVR) suite, introduced in earlier research, focused on creating large-scale datasets and verifiable evaluation frameworks for video-based tasks. The VBVR Dataset, for instance, contained over one million video clips across 200 reasoning tasks, significantly larger than previous datasets. Tasks included navigating mazes, sorting shapes, and tracking objects. This earlier suite also introduced VBVR-Bench, an evaluation framework that utilized rule-based, human-aligned scorers to move beyond model-based judging. This framework allowed for reproducible and interpretable diagnosis of video reasoning capabilities.
The creators of VBVR-Pro have made all data, models, scorers, and code publicly available. This release aims to support further research and development in the field of native visual reasoning. The progression from VBVR to VBVR-Pro indicates a continued focus on developing more rigorous and scalable methods for evaluating and training models in visual intelligence.