DriveZero shows human driving logs are a ceiling, not a floor, for autonomous planners

The central bet in DriveZero is that human driving logs are a ceiling, not a floor. Most end-to-end systems treat recorded trajectories as the sole source of behavioral supervision, which means every quirk, hesitation, and gap in the logs becomes a constraint on what the policy can learn. DriveZero breaks that constraint by splitting the problem: train a privileged action model through closed-loop reinforcement learning, train a visual backbone by distilling frozen foundation models, then unify them via distillation into a camera-only planner that never touches a human trajectory.

The RL teacher, DriveRL, is a 5.7-million-parameter policy trained from scratch with PPO across up to 196,608 parallel worlds built from nuPlan logs. It outputs Beta distributions over longitudinal jerk and steering-angle rate, with background traffic handled by a mix of log replay, IDM, and optionally self-play. On nuPlan's community splits it scores a mean of 93.01 across six closed-loop evaluations, exceeding the Log-Replay expert on every one. A value-guided test-time search - sampling alternative first actions and scoring them with the trained critic over a short rollout - pushes that to 93.57 without any retraining.

The visual backbone, DriveVFM, takes a different route entirely. Rather than annotating detection, segmentation, or depth labels, it distills four frozen foundation models - DINOv3, SigLIP2, SAM, and Depth Anything V2 - into a single ViT using their own features as supervision targets. PHI Standardization normalizes the heterogeneous feature distributions so no single teacher dominates gradient flow. The ablation is clear: adding SAM to the DINOv3 + SigLIP2 base lifts PDMS by 0.41 points; adding Depth Anything V2 adds another 0.31.

Distillation into the student planner, DriveZero, uses winner-takes-all trajectory regression against teacher rollouts rather than logged human futures. Crucially, because DriveRL is goal-conditioned, the same logged scene can be queried with augmented navigation intents, generating diverse yet internally consistent supervision that recorded data structurally cannot provide. That augmentation is what tips the balance: RL distillation alone scores 93.61 PDMS on NAVSIMv1 navtest, slightly below the 93.92 from human trajectories, but adding goal augmentation reaches 94.41 - clearing the human-trajectory baseline by nearly half a point.

Scaled with out-of-distribution simulation data, DriveZero-Scale reaches 95.3 PDMS on navtest, surpassing the human driver benchmark of 94.8, 57.1 EPDMS on the harder NAVSIMv2 navhard split, and 46.6 HD-Score zero-shot on HUGSIM - an 8.1-point improvement over the previous best on that benchmark. The system has also been deployed on a small real-vehicle fleet, with DriveRL controlling vehicles in dense urban traffic using perception outputs rather than ground-truth state.

The architecture is genuinely modular in a useful sense: perception and action are optimized in the regimes where each learns best, and the interface between them is just trajectory supervision at training time.