0 of 9 add-skill edits improved pass rate; cost fell up to 31.7% vs. static bundles

Notes on SkillMOO: Multi-Objective Optimization of Agent Skills for Software Engineering (arXiv:2604.09297) — Jingzhi Gong, Ruizhen Gu, Zhiwei Fei, Yazhuo Cao, Lukas Twist, Alina Geiger, Shuo Han, Dominik Sobania, Federica Sarro, Jie Zhang · April 2026

Note published July 24, 2026 · written by SkillFed’s research pipeline from the paper above · how these notes are made

AI-assisted notes · reviewed by SkillFed Skill evolution

SkillMOO treats a coding agent's agent skill bundle as something to search over, not a fixed asset written once. An LLM optimizer agent proposes edits — pruning a skill, substituting one, reordering the bundle, rewriting content — while a separate solver agent runs the edited bundle against a task. NSGA-II ranks the resulting candidates each generation on two objectives at once, pass rate and inference cost, keeping only the Pareto-efficient ones. Both agents run on GLM-5, evolving a population of 4 over 3 generations. The pipeline goes up against all 16 SkillsBench software-engineering tasks — build repair, code migration, data engineering, performance debugging, ML reproduction among them — each one repeated across 10 independent runs.

SkillMOO takes the top pass-rate rank on 11 of the 12 tasks that had any non-zero pass rate at all, while cutting inference cost by as much as 31.7% against hand-built static bundles and lifting pass rate by up to 21 percentage points. On fix-build-agentops the swing is stark: pass rate rises from 0.16 to 0.37 while cost falls 31.7% in the same run. Running the search itself is cheap — $1.76 to $13.73 per task — and a breakdown of 38 recorded edits shows why the wins concentrate where they do: removing a peripheral or redundant skill improved pass rate in most attempts, replacing one improved it in 5 of 7 tries, but adding a skill to a bundle improved pass rate in 0 of 9 attempts.

Key numbers

Tasks where SkillMOO topped pass-rate rank (of 12 non-zero-pass tasks)	11
Peak cost reduction vs. static bundle	31.7%
Peak pass-rate gain	+21 pp
Add-skill edits that improved pass rate	0 of 9
Search overhead per task	$1.76–$13.73

What this means for skill authors

The 0-for-9 record on additions is the finding worth sitting with. It isn't that extra skills are neutral padding — they cost tokens on every invocation, and here they bought back nothing often enough that the search never once rewarded adding one. For anyone authoring or maintaining a skill bundle by hand, that flips the default question from "what else might help" to "what can I cut without losing pass rate," since pruning and substitution are what actually moved the needle across all 38 edits studied.

It's also evidence that pass-rate-only evaluation hides a real cost. A bundle can look like a strict improvement on a leaderboard that only tracks success while quietly costing up to 31.7% more in tokens for a gain a smaller bundle would have matched. Making cost a first-class search objective, rather than a number reported after the fact, is what surfaces edits like this — and the search itself is cheap enough here ($1.76–$13.73 a task) to be worth running whenever a bundle gets reused across many invocations.

For a catalog of skills built and shared by many authors, the natural extension is treating cost-per-success as part of what gets checked before a skill is trusted, not an afterthought bolted onto a pass-rate number. That's the same instinct behind vetting a skill on more than one axis before it earns a place in a shared catalog — a skill that nudges success up while dragging cost along for every downstream user is a worse citizen than a single success metric would show.

Skills related to this research

Okf Okf lets you build, organize, and validate Open Knowledge Format (OKF) bundles—plain markdown files with YAML frontmatter stored in a portable directory structure. Initialize new bundles, author concept documents, generate indexes, and render interactive graph viewers. The format is vendor-neutral and lock-in free, consumable by any tool that reads markdown.★ 8 wiki-export Wiki Export transforms your Obsidian vault's wikilink graph into multiple structured formats—JSON, GraphML, Cypher scripts, and interactive HTML—ready for use in Gephi, Neo4j, and custom tools. Optionally filter by project or visibility tags, and assign community IDs for graph analysis and visualization.★ 3,032 wiki-import wiki-import brings a knowledge graph into your vault from two sources: a graph.json file for page stubs with relationships, or an OKF bundle for complete pages with full bodies. Choose how to handle existing pages—merge, skip, or overwrite—and the skill reconstructs frontmatter, wikilinks, and vault metadata automatically.★ 3,032 Math Modeling Solver Math Modeling Solver guides you through competition problem-solving in five structured stages: decomposing the problem, searching literature for precedent, matching models to subproblems, deriving algorithms and generating code, and bridging to paper writing. It provides decision matrices for 95+ scenarios, algorithm cookbooks, complete worked examples, and Python/MATLAB templates to help you move from problem statement to solution.★ 154

Related notes

References

Gong, J., Gu, R., Fei, Z., Cao, Y., Twist, L., Geiger, A., Han, S., Sobania, D., Sarro, F., Zhang, J. — "SkillMOO: Multi-Objective Optimization of Agent Skills for Software Engineering" (arXiv:2604.09297, 2026)
Deb, K., Pratap, A., Agarwal, S., Meyarivan, T. — "A Fast and Elitist Multi-Objective Genetic Algorithm: NSGA-II" (IEEE Transactions on Evolutionary Computation, 2002)