GE-Act 2.0: Pretraining and Scaling a World-Action Model for Robotic Manipulation

Most robot world models bolt an action head onto a pretrained video generator and call it done. GE-Act 2.0 takes the opposite approach: every trainable component is initialized from scratch on manipulation data, and the paper's central question is what that discipline actually buys at scale.

The architecture has three pieces. A control-oriented autoencoder (CoAE) compresses each frame to just 24 tokens while staying within 13–31% of the best semantic encoders on action-recovery error—using one-sixteenth as many tokens per frame. A single-step visual planner (SVP) produces a complete future in one differentiable forward pass rather than through sequential denoising, which matters because it lets the inverse dynamics model (IDM) be pretrained separately on instruction-free or failed trajectories before the two are connected. That separation is the architectural bet: broader, messier data becomes usable precisely because the IDM doesn't need to be entangled with the generator during its own pretraining.

The third piece, KASO, addresses what the paper calls the validity gap. When a stochastic generator samples a future depicting one valid completion mode and the recorded action reflects a different mode, pairing them trains the IDM to predict the mean of incompatible behaviors. KASO samples multiple candidate futures, scores each by how closely the IDM's action response matches its response to the recorded future, and trains only on the compatible candidates. In a controlled toy system, competing strategies collapse the action distribution to a single mean; KASO recovers both action modes.

The scaling results are the empirical core. Co-training data runs from 300 to 30,000 hours across four nested checkpoints. Suite-level zero-shot success on G1-OP rises from 17.1% to 44.1%; on G2-90D, which contributes under 2% of the co-training mixture, it rises from 13.4% to 31.1%. Neither curve shows signs of saturation. Skill-specific training coverage correlates strongly with zero-shot success across 19 skill groups (Pearson and Spearman correlations are both reported), with a fitted gain of roughly 1.94 logit units per tenfold increase in skill-specific hours—a concrete, falsifiable relationship between data composition and capability.

Instruction grounding holds up on object identity, color, shape, and direct position at 90% follow score or better, but degrades sharply on ordinal references like "second from the left," which appear in only 0.13% of training instructions. The paper is candid that frequency isn't the only factor—shape achieves 90% despite low occurrence—but the tail is real and the gap is large.

The honest limitation is that egocentric video, flagged as a promising scaling direction, hasn't yet been explored at the scale its availability would permit. The evaluation protocol—no per-task fine-tuning, held-out scenes and object instances—is the right one for measuring what pretraining actually contributes, and the paper applies it consistently across all four data scales.