Mid-run steering beats post-hoc reflection for long-horizon agent self-improvement

Post-hoc reflection is the dominant pattern in agent self-improvement: finish a run, critique the trace, update the harness, try again. PILOT argues this is the wrong loop. By the time the reflection runs, the trajectory that revealed the problem is already over. The knowledge arrives too late to help the run that earned it.

The fix is architectural. PILOT splits execution and oversight into a worker and a supervisor that stay connected throughout the run via a live two-way channel. The worker handles tool calls, environment feedback, and implementation details in its own context. The supervisor watches for drift, stalled strategies, and implementation errors from a separate context focused on the goal and recent events. When something goes wrong, the supervisor can steer the worker mid-run or abort it entirely - not after the fact.

The paper's two case studies make this concrete. In one, a worker spent over twenty minutes tuning variants of a CoreWars strategy that kept producing ties. The supervisor interrupted, told it to abandon the synthetic opponents and find a published warrior instead. The worker pivoted, adapted a known warrior from a public archive, and passed every threshold. In the other, a worker was accumulating bias errors in a distributed tensor-parallelism implementation - adding the bias once per rank before the all-reduce instead of once after. The supervisor caught it, explained the correct placement, and the worker's fix passed all thirteen verifier tests across three world sizes.

Those are qualitative wins. The quantitative picture is more interesting. In a one-shot setting across two frozen backbones and three benchmarks, PILOT ranks first in five of six backbone-benchmark combinations. On Terminal-Bench 2.0 it outperforms the strongest single-agent baseline by up to 9.8 percentage points. In a self-improvement setting where the harness accumulates skills across iterations, the best observed pass rate rises 14.6 points with GLM-5.1 and 12.4 points with Kimi-K2.6. The skill library grows by 21 and 31 skills respectively. Mean output tokens per task fall by 42.9% and 47.4% - the agent stops re-deriving procedures it already knows.

The live-steering gains concentrate where you'd expect: hard tasks, where long execution chains give errors more room to compound. On easy tasks, no successful run required supervisor intervention at all.

The honest limitation is cost. Iterating over the full benchmark many times is expensive, so the evaluation covers only three benchmarks and two open-weight models. The supervisor and worker also share the same backbone throughout, leaving the question of heterogeneous pairings - a stronger supervisor model overseeing a cheaper worker - entirely unexplored. That's a real gap for anyone thinking about production deployment.