A discrete action vocabulary, not a bigger model, is what closes the VLM-to-robot gap

The central bet in Show-Harness is that the bottleneck between VLMs and robots is not model capability but interface design. Rather than fine-tuning a VLM to regress continuous motor signals, the system exposes a small vocabulary of discrete semantic action units — move end-effector forward, back, left, right, up, down, rotate clockwise or counterclockwise about a chosen axis, grasp, release, done — and lets an embodiment-specific interpreter translate each unit deterministically into physical motion. The VLM never sees joint angles or motor torques; it reasons over symbols it already understands.

This separation does real work. Reducing the interpreter's translation step from 2 cm to 1 cm improves zero-shot success on stacking and peg insertion from 60% to 82% without touching the model or retraining anything. A comparable VLA baseline reaches only 18% at that precision level with the same demonstrations, climbing to 62% only after additional fine-grained training. The interface absorbs the precision change; the VLA has to relearn it.

Two deployment modes share the same interface. Frontier models like Gemini run zero-shot through API calls, with no fine-tuning required. Smaller open-source models — the paper's default is a 2B-parameter model — can be adapted with a few GPU-hours of LoRA fine-tuning, updating only a small fraction of parameters. Fine-tuning the 2B model takes under two hours on a single H200. The 2B scale hits a reasonable balance: 1B models oscillate near targets, while larger backbones mainly help on precision-critical tasks like stacking.

Generalization results hold across held-out objects, lighting and background shifts, and sim-to-real transfer. The fine-tuned policy trained entirely on simulated demonstrations succeeds on real hardware; the VLA baselines trained on the same simulated data fail. Cross-embodiment transfer between a 7-DoF Franka and a bimanual AgileX requires only swapping the interpreter, not retraining the model.

GUMI, the GUI-based data collection interface built on the same action vocabulary, is a practical contribution that often gets buried in ablations but deserves attention. Because every semantic unit maps to a keyboard shortcut, humans collect demonstrations without teleoperation hardware, frontier agents can operate the same interface autonomously, and the resulting data trains both semantic-action and continuous-control policies from a single rollout. Remote collection without physical colocation with the robot becomes straightforward.

The ablation on action-space representation is clarifying. Arbitrary symbols paired with explicit written conventions about their physical effects nearly match the default semantic names. Arbitrary symbols with no conventions succeed in only 1 out of 20 episodes, with fewer than a quarter of inferred mappings correct. The grounding comes from the conventions, not from the names themselves — a useful reminder that what looks like "natural language understanding" in robot control is often just well-specified context.

The system is currently limited to parallel-jaw grippers on single- and dual-arm platforms. Dexterous hands and humanoids remain untested, and the perception side lacks tactile or force feedback.