Embodied-Navigator: Point, Think, Memorize, and Align for Efficient Navigation

The central insight in TAMP-Nav is deceptively simple: stop asking a vision-language model to reason in 3D. VLMs are trained on 2D image-text pairs, and forcing them to regress 3D coordinates or output atomic directional commands creates a geometric mismatch that produces spatial hallucinations and wastes training data. Instead, the agent acts as a pixel pointer—it clicks a 2D coordinate on its egocentric view, and a deterministic SLAM controller handles the projection into 3D space and the physical movement. That single design choice reduces average interaction steps to roughly 9 per trajectory, compared to around 30 for comparable open-source systems.

The memory architecture addresses a genuine tension. Keeping full visual history causes attention dilution; uniform sampling loses critical observations on long paths. TAMP-Nav's solution is heterogeneous: at topological decision nodes, it stores a full visual frame plus the chain-of-thought reasoning that was triggered there. Between nodes, it discards visual features entirely and retains only a lightweight Space-Time Indicator token encoding position, orientation, and temporal index via rotary position embeddings. On long-horizon trajectories exceeding 50 atomic forward actions, this yields a 49.8% success rate against 30.9% for StreamVLN and 41.9% for DualVLN.

The reasoning trigger itself is learned, not scheduled. After reinforcement learning, the proportion of CoT steps allocated to straight corridors drops from 38% in the supervised baseline to 11%. The agent concentrates thinking at crossroads, doorways, and near targets—matching a dense-CoT upper bound of 66.8% SR with only a 26.3% reasoning ratio, at 66.2% SR on R2R-CE Val-Unseen.

The RL training uses a two-level GRPO formulation. At the trajectory level, rewards cover task success, path efficiency via SPL, and a reasoning-density penalty that discourages over-thinking. At the step level, five local rewards cover target approach, collision avoidance, stop correctness, reasoning utility, and output format. Annealed guided sampling weights early rollout selection by local reward quality, then decays toward uniform exploration. Training on trajectory-level rewards alone converges at a 0.59 success reward; adding step-level advantages lifts it to 0.64; the full framework with guided sampling reaches 0.68.

The whole system trains on 90k trajectories—substantially fewer than the 763k used by DualVLN—and runs a complete task in 16.58 seconds on a single A800 GPU versus 37–41 seconds for competing systems. Real-world deployment on a Unitree Go2 quadruped, with no robot-specific fine-tuning, achieves 60.0% SR across 100 trials spanning indoor and outdoor environments, against 49.0% and 53.0% for the two baselines.

Limitations are stated plainly. The STI token encodes only planar position and orientation, so multi-floor buildings remain a weak point. The GRPO reward thresholds are tuned for R2R-CE and RxR-CE distributions and may need retuning elsewhere. Online RL on physical hardware is not supported.