RRSI: Regularized Recursive Self-Improvement of Agent Harnesses

Harness evolution has an overfitting problem that nobody was directly solving. When an LLM proposer rewrites prompts, control flow, and tool interfaces by repeatedly scoring candidates against the same finite task set, it learns that task set. Gains on the evolve split look real; gains on anything else often vanish, and several prior methods actually finish below the harness they started from on out-of-distribution benchmarks.

RRSI attacks this by treating the search dynamics themselves as the thing to regularize, not the harness components. The edit space stays fully open - prompts, memory, subagents, context management, all of it remains editable. What changes is how feedback from a finite evolve set gets converted into permanent harness state.

Three mechanisms work on the proposal side. An annealed edit budget starts permissive and tightens over rounds, so early iterations can bundle coordinated changes while later ones must commit to single attributable edits. An explicit evolution history tracks which component hypotheses have already been falsified, preventing the search from spending capacity retesting the same ideas. When progress stalls within the empirical noise band, a small portion of the proposal budget redirects toward components the search has never exercised.

On the selection side, a leakage critic screens candidates before they are ever scored, rejecting anything that encodes task-specific names, values, or benchmark-specific logic. A noise-adjusted acceptance floor prevents the search from walking downhill through a sequence of small regressions that individually look like noise. A cost-aware acceptance rule requires that any increase in inference tokens be justified by a measurable performance gain proportional to that increase.

The numbers are specific. Across eight benchmarks in three domains, RRSI gains up to 14.1 points on the evolve split and up to 4.7 points on the five out-of-distribution benchmarks it never saw during search. The strongest prior baseline, Meta-Harness, adds only 0.9 points to the out-of-distribution average; two baselines finish below the unevolved harness. RRSI also produces the lightest harness of any evolved method, running on roughly 30% fewer policy tokens than unregularized evolution and taking fewer steps per trial than every baseline.

The engineering design results are particularly clean because grading there is deterministic - each design either satisfies the simulator's constraints or it does not, so judge variance cannot explain the gains. A harness evolved on one coding benchmark transferred a measurable improvement to SWE-bench Verified, which the search never scored. The same harness, evolved with Gemini 3.5 Flash, improved Terminal-Bench accuracy for Gemini 3.1 Flash Lite, a weaker model that never participated in the search at all.

The ablation is honest about the trade. Removing both regularizer groups raises the evolve-set score to its highest value while leaving the out-of-distribution average near the unevolved baseline, at more than 50% higher token cost. The regularizers deliberately sacrifice some in-distribution performance to buy transfer. That is the trade, and the paper makes it explicit rather than hiding it.