Scalar progress estimates outperform generated future frames for robot policy conditioning

The central bet in GigaBrain-0.7 is that a robot policy benefits from being told not just what to do next, but what the world should look like after it acts—and whether it is currently making progress toward that state. The architecture divides this labor across three systems. System 1 generates continuous action chunks via flow matching from a PaliGemma2 backbone paired with a 0.5B Action Expert in a Mixture-of-Transformers arrangement. System 2 handles scene interpretation and decomposes long-horizon instructions into subtasks. System 3 is a 5B world value model that predicts future visual states and estimates a scalar task-progress value; both signals condition System 1 during post-training and inference.

The ablation on System 3 is the most instructive section. On gift wrapping, the base policy fails entirely; adding the predicted subgoal image alone brings success to 20%, adding only the value-derived advantage condition brings it to 60%, and combining both reaches 80%. On clothes folding, all variants reach full success, but the combined model cuts average completion time from 107 seconds to 75 seconds and raises average task score from 68.3% to 88.3%. The value signal does more work than the visual subgoal on most tasks, which is a meaningful finding: a scalar progress estimate derived from a world model appears to be a more reliable conditioning signal than a generated future frame, at least at this scale.

The data side is equally ambitious. Pretraining covers more than 37,000 hours of embodied trajectories spanning 16 robot morphologies, plus roughly a quarter-billion vision-language samples. The paper is explicit that adding egocentric human video and UMI wrist-camera demonstrations on top of robot trajectories improves post-training performance, not just zero-shot behavior—suggesting the human data changes the learned prior rather than just padding the distribution.

The experience-reinforcement results are the most striking numbers in the paper. Four real-robot tasks start at an average 30% success after supervised fine-tuning. Offline advantage-weighted regression on collected rollouts raises this to 57.5%. Online reinforcement with human corrections at failure states brings all four tasks to 100%. The jump from offline to online is largest on precision-critical tasks like cable tie insertion, where the policy must thread a slender component through a locking head—exactly the kind of contact-rich behavior that demonstration data alone cannot cover adequately.

The one-stage training recipe, which jointly optimizes language understanding, hierarchical subtask prediction, and continuous action generation in a single pass, is presented as a deliberate departure from staged pipelines that introduce the action expert only after VLM pretraining. The Soft Knowledge Insulation mechanism attenuates rather than blocks action gradients entering the VLM backbone, a middle path between full gradient isolation and unconstrained co-training. Whether this specific mechanism is doing the heavy lifting or whether the data scale is the dominant factor remains somewhat underexplored in the ablations provided.

Training code and pretrained weights are promised for public release, which will matter considerably for anyone trying to reproduce the real-robot numbers.