Visual generation, not language, is the load-bearing structure for machine reasoning

The central claim here is that language is the wrong substrate for many forms of machine reasoning, and that visual generation—images, videos, interleaved sequences—can serve as the actual working medium where problems get solved. VBVR-Pro is the infrastructure built to test whether that claim holds at scale.

The dataset spans 300 procedurally generated tasks organized around five cognitive faculties: perception, spatiality, transformation, abstraction, and knowledge. Half are reworked from an earlier suite; the other 150 are new and measurably harder—the median new task contains 80 distinct connected color regions versus 12 for the reworked ones, and 47% require multi-step reasoning compared with just 7% of the reworked set. Each task generates aligned video, image, and interleaved versions from the same underlying problem, which is what makes cross-modality comparison meaningful rather than confounded by different task distributions.

The evaluation infrastructure is the more consequential contribution. VLM judges fail in consistent, documented ways: they miss fine-grained color errors, overlook violated constraints, and misread task rules in both directions—penalizing correct outputs and rewarding wrong ones. Reproducibility is also a problem; repeated runs on identical inputs produce different scores for a non-trivial fraction of cases. The paper's verifiable scorers replace pixel-level comparison with semantic extraction—HSV segmentation, contour detection, OCR, trajectory tracking—and then apply task-specific weighted or multiplicative checks. Against human preference annotations, the verifiable scorer achieves the highest per-vote agreement while costing less than any VLM judge tested.

The modality comparison yields a clear finding: video generation leads on tasks requiring persistent spatiotemporal tracking, while interleaved generation is competitive on familiar task families at substantially lower compute cost. Single-image generation consistently lags on transformation tasks. Ablations show that removing intermediate visual states hurts more than replacing intermediate reasoning text with uninformative placeholders—visual trajectories are the load-bearing structure, not language.

The reinforcement learning section demonstrates that verifiable rewards produce steadier gains than a VLM judge across 50 tasks. Training with verifiable rewards reaches an overall score of 0.548 versus 0.508 for the VLM-judge variant and 0.503 for supervised fine-tuning alone. Out-of-domain improvement is also larger under verifiable rewards, suggesting the signal generalizes. The paper also documents a practical systems win: on 128 H800 GPUs, verifiable-reward training completes in roughly 5.1 days versus 8.3 days with the VLM judge, a 38.7% reduction in wall-clock time.

Transfer to seven external benchmarks shows gains often exceeding 20 percentage points, and nearest-neighbor analysis argues against memorization as the explanation—retrieved training examples are visually dissimilar to the test cases that improve most.