One to four gated edits lift agent accuracy by up to 39 points

Notes on SkillOpt: Executive Strategy for Self-Evolving Agent Skills (arXiv:2605.23904) — Yifan Yang, Ziyang Gong, Weiquan Huang, Qihao Yang, Ziwei Zhou, Zisu Huang, Yan Li, Xuemei Gao, Qi Dai, Bei Liu, Kai Qiu, Yuqing Yang, and 3 more · May 2026

Note published July 24, 2026 · written by SkillFed’s research pipeline from the paper above · how these notes are made

AI-assisted notes · reviewed by SkillFed Skill evolution

SkillOpt is a text-space optimizer: it treats an agent skill document — the natural-language file prepended to a frozen model's context — as something you train rather than hand-write. A separate optimizer model reads batches of scored rollouts, proposes bounded add/delete/replace edits under a textual learning-rate budget, and keeps an edit only when the resulting skill strictly beats the current one on a held-out validation gate. Rejected proposals don't just vanish — they land in a rejected-edit buffer, so the optimizer stops re-proposing what already failed, and every few steps a slower 'meta' update folds cross-epoch patterns into a protected region of the file that per-step edits can't touch. None of this runs at deployment — only the resulting best_skill.md ships, alongside the still-frozen target model. Testing spans six benchmarks — search QA, spreadsheet automation, office and document QA, math reasoning, and embodied tasks — seven target models from frontier-scale GPT to small Qwen, and three execution harnesses: direct chat, Codex, and Claude Code.

The optimized skill wins or ties across all 52 tested (model, benchmark, harness) combinations, beating hand-written skills, one-shot LLM-generated skills, and four rival skill- and prompt-optimizers — Trace2Skill, TextGrad, GEPA, EvoSkill — every time. On GPT-5.5, accuracy over no skill improves by 23.5 points on average in direct chat, 24.8 inside the Codex harness, and 19.1 inside Claude Code, and still clears an oracle baseline — free to pick the single best rival method per benchmark — by 5.4 points. Gains concentrate where models lack procedural discipline zero-shot: office-document QA gains the most, up to 39 points, with spreadsheet manipulation close behind, while a benchmark already near ceiling (SearchQA) moves the least. The number that matters more sits underneath the accuracy table: accepted skills stay 379 to 1,995 tokens long, built from a median of just 2.5 accepted edits — evidence the validation gate is discarding most of what the optimizer proposes and keeping only the handful of rules that demonstrably help.

Key numbers

Best-or-tied cells	52 of 52
GPT-5.5 gain, direct chat	+23.5 pts avg
Accepted edits per skill	1–4 (median 2.5)
Final skill size	379–1,995 tokens
Training cost per point (DocVQA)	46.4M tokens/pt

What this means for skill authors

For anyone building self-revising skills, the transferable lesson isn't the training loop itself — it's the discipline of gating every edit against held-out data rather than against whether a proposed rule merely sounds reasonable. The paper's own ablation shows what happens without that discipline: removing the epoch-wise slow/meta update alone costs 22.5 points on SpreadsheetBench, the single largest drop in the entire ablation suite. A skill that rewrites itself after every failure with no held-out check isn't learning so much as drifting — the exact failure mode SkillOpt is built to avoid.

The edit-economy number matters more for practitioners hand-authoring skills today than the headline accuracy gains do. A median of 2.5 accepted edits producing double-digit point gains says most of the value in a domain skill is a handful of sharp, falsifiable procedural rules — bind an answer to a specific visual region, write evaluated static values instead of trusting recalculation, rank multiple-choice options by theorem strength before picking the option that's merely true — not a long document of general guidance. That argues for skills as short, testable checklists rather than exhaustive playbooks, which lines up with the premise behind a vetted skill catalog: a skill that's actually been run against a scoring loop, however small the edit count, is a different artifact than one that only reflects an author's intuition about what should work.

There's a real limit here: the validation gate needs a reliable score — an automatic verifier, an exact-match metric, an executable check — so it's a poor fit for open-ended tasks where success is subjective. Transfer holds up across model scale, harness, and adjacent benchmark, but shrinks as the gap widens: a spreadsheet skill moved from Codex to Claude Code very nearly matches in-domain performance, while a math skill moved to a different benchmark keeps only a few points of its original gain. A skill validated on one model and one harness isn't automatically validated for another — the paper says so directly rather than papering over it.

Skills related to this research

Grc Compliance Operating Model Diagram Structure and visualize GRC compliance operating models, governance architectures, and control frameworks as editable draw.io diagrams. Supports executive capability maps, three lines of defense, policy-control-evidence lifecycles, and audit scope views with role-based layout options.★ 354 asset-allocation This skill teaches portfolio optimization from theory to implementation, covering Modern Portfolio Theory, Black-Litterman blending, risk budgeting, and all-weather allocation strategies. It includes five configurable optimizers—equal volatility, risk parity, mean-variance, maximum diversification, and turnover-aware—each suited to different market conditions and cost constraints.★ 28,096 cpp-coding-standards This skill guides developers through modern C++ best practices rooted in the C++ Core Guidelines, covering type safety, resource management via RAII, immutability, and clear interfaces. Apply it when writing, reviewing, or refactoring C++ code to maintain consistency and prevent common errors across your codebase.★ 234,207 Cpp Idioms Cpp Idioms teaches you how to write safe, efficient C++ code using modern language features and proven design patterns. Learn RAII principles, smart pointer ownership models, move semantics, and error handling with std::expected—all grounded in C++17/20/23 standards. The skill covers interfaces, dependency injection, testing strategies, and anti-patterns to avoid, with practical code examples and tooling guidance.★ 150

Related notes

References

SkillOpt (this paper) — arXiv:2605.23904
GEPA: reflective prompt evolution can outperform reinforcement learning — Agrawal et al., arXiv:2507.19457
EvoSkill: automated skill discovery for multi-agent systems — Alzubi et al., arXiv:2603.02766
Trace2Skill: distill trajectory-local lessons into transferable agent skills — Ni et al., arXiv:2603.25158
TextGrad: automatic ‘differentiation’ via text — Yuksekgonul et al., arXiv:2406.07496