Scaffold design is now a learnable capability, not just expert intuition

Agent capability has always been treated as a property of model weights. JIT-Agent challenges that assumption directly: the operational scaffold wrapping a model—how it manages memory, forms plans, invokes tools, and advances control—can matter as much as the model itself, and that scaffold can now be generated on demand by a trained meta-agent.

The core mechanism is a four-module protocol decomposing any harness into memory, planning, action, and capability-orchestration components. JIT-Agent, a 27B model trained on Qwen3.6-27B, receives a task specification and a small retrieved context of prior harnesses, then emits an executable harness tailored to that specific task. Training proceeds in three stages: supervised imitation of teacher-generated harnesses, repair learning from failed executions converted into bounded fix trajectories, and an online evolutionary stage called Evo-GDPO that rewards proposals surpassing the current harness archive on reward, latency, and cost simultaneously.

The results are concrete. Equipping DeepSeek-V4-Flash with a JIT-generated harness pushes it past GPT-5.6 on DeepSearchQA and OdysseyBench. GLM-5.2, already a strong baseline, gains further across multiple benchmarks. Across all 24 backbone-benchmark comparisons against a standard ReAct harness, the JIT harness wins every time. Crucially, these gains come cheaper: in every one of the six controlled backbone-harness comparisons, JIT-Agent uses fewer tokens and costs less per case than any fixed alternative. On DeepSeek-V4-Flash xBench-DS, token consumption drops from 527K (NanoBot, the cheapest fixed harness) to 212K while performance rises from 78.0 to 82.0.

The qualitative analysis is where the paper earns its argument. The same generator produces a DAG-based artifact pipeline for a contact-processing task and a recursive delegating subagent architecture for a multi-hop identity question. These are not prompt variations—they differ in state representation, control topology, and exposed capabilities. The shared protocol constrains interfaces, not behavior, which is precisely what makes the approach transferable across model families.

Two honest caveats: the benchmark suite is entirely self-reported against models and harnesses whose versioning is fluid, and the 27B generator itself carries non-trivial inference cost that the per-case API cost figures do not fully account for. Streaming evolution finishes above the static variant on all three reported benchmarks, though the cumulative accuracy advantage is modest.

Harness design has been manual, task-specific, and bottlenecked on expert intuition. Training a model to do it just-in-time, with repair and evolution built into the training objective, is a tractable and well-executed step toward making scaffold quality a scalable dimension of agent systems.