EXIMO: VLM Guided Exploration of VLA Policies

The core insight in Eximo is deceptively simple: a pretrained VLA already knows how to pick up a banana; it just doesn't know that a monkey would want one. A VLM does. So rather than collecting thousands of new teleoperation hours or running sparse-reward RL from scratch on compositional tasks, Eximo uses the VLM as a closed-loop orchestrator that breaks goals into sub-instructions the VLA can already execute.

The three-stage pipeline — explore, imitate, optimize — is where the engineering choices get interesting. During exploration, the VLM watches the robot's camera feed, reasons inside a think block, and issues natural-language sub-goals to the VLA in real time. Only successful episodes get stored. That filter matters: the imitation stage trains the VLA on clean, high-quality trajectories, and crucially it trains the VLA to respond to the original task goal, not the VLM's decomposed sub-goals. The VLM's scaffolding is compiled away. At evaluation time, no VLM is needed — latency gone, API calls gone, and performance actually improves over keeping the orchestrator live.

The final stage adds a residual off-policy RL controller on top of the finetuned VLA. Because the SFT stage already gets the policy to non-trivial success rates, the RL exploration problem is tractable rather than needle-in-a-haystack. The paper evaluates across twenty-two manipulation tasks on the Aloha platform in simulation, covering chained placements, reasoning tasks where the target object must be inferred semantically, and spatial tasks requiring left/right discrimination. The base VLA used throughout is the 3B-parameter GROD model.

The ablations are the most useful part. VLM orchestration alone raises success rates substantially over the unorchestrated base, particularly on long-horizon tasks like PlateBowlOnRack. SFT on the filtered orchestrated data then beats the orchestrated agent at evaluation time — the distillation result. And GROD + SFT + RL beats the base GROD + RL even when the latter is given extra environment steps to compensate for the data Eximo spent during exploration. The advantage isn't just a head start; the finetuned model converges higher.

One honest failure mode is documented: trying to distill the VLM's guidance into the residual policy rather than the base VLA doesn't work. The offline residual policy learns fine, but when online RL takes over, the distribution shift between VLM-orchestrated tuples and on-policy rollouts kills the benefit. The paper tries mixing VLM-guided and non-guided rollouts during RL training and finds the no-VLM baseline still wins at evaluation. The residual policy can't leverage the orchestrated tuples because it's learning to correct the base VLA's actions, not the VLM's.

The remaining assumption worth flagging: a ground-truth success detector is required throughout. The paper acknowledges this and names VLM-based success detection as future work. In simulation that's fine; in the real world it's the gap between a compelling result and a deployable system. The vision of a fully autonomous loop — VLM as orchestrator, reward model, and environment resetter — is sketched but not demonstrated.