Delete one repair action, skill-library success falls from 79.5% to 13.2%

Notes on SkillOps: Managing LLM Agent Skill Libraries as Self-Maintaining Software Ecosystems (arXiv:2605.13716) — Hongjian Pu, Xinyuan Song, Liang Zhao · May 2026

Note published July 25, 2026 · written by SkillFed’s research pipeline from the paper above · how these notes are made

LLM agents that keep a library of reusable skills eventually run into a problem invisible at the single-skill level: skills get added, patched, reused in new contexts, and wired to shifting dependencies until the library itself degrades. Call it skill technical debt — library-level defects (near-duplicate skills, stale interfaces, missing validators, broken dependency chains) that don't break any one skill locally but quietly corrode future retrieval, composition, and execution. SkillOps is the maintenance layer built to catch it, and it's method-agnostic: every skill gets formalized as a typed Skill Contract — preconditions, operation, output artifacts, validators, known failure modes — and the whole library sits inside a Hierarchical Skill Ecosystem Graph with typed edges for dependency, compatibility, redundancy, and alternative relations. A five-dimension health scan (utility, redundancy, compatibility, failure-risk, validation-gap) drives six typed repair actions — merge, repair, retire, add_validator, add_adapter, instantiate — turning a raw library into a maintained one that any existing retrieval or planning agent can use unchanged.

On ALFWorld, testing against synthetically degraded skill libraries ranging from 200 to 2,000 skills (built from a 229-skill base with 0–90% degradation density), SkillOps running standalone hits 79.5% task success — 8.8 percentage points above the strongest baseline, an LLM-based skill planner, with zero extra task-time LLM calls. Used purely as a plug-in that cleans a library before handing it to an existing agent, it still lifts retrieval-heavy baselines by up to 2.90 points. The lead holds as libraries grow and decay further: SkillOps stays in the 79.5–83.8% band while retrieval baselines fall 35.9–42.7 points over the same scale range. The ablations tell the sharper story. Pull the add_adapter repair action — the one that inserts type-conversion shims between mismatched skill interfaces — and success collapses to 13.2%; pull add_validator and it drops to 38.0%. Removing the entire library-time maintenance loop, by contrast, costs just 7.6 points (79.5% to 71.9%). A full maintenance pass over a 2,000-skill library runs about 10.8K tokens, roughly $0.0026.