Self-repairing skills add 8.4 points — raw context alone scores worse than no skills at all

Notes on SkillFlow:Benchmarking Lifelong Skill Discovery and Evolution for Autonomous Agents (arXiv:2604.17308) — Ziao Zhang, K. Shi, Shiting Huang, Avery Nie, Yu Zeng, Yiming Zhao, Zhenlong Fang, Qisheng Su, Haibo Qiu, Wei Yang, Qingnan Ren, Shu Zou, and 4 more · April 2026

Note published July 25, 2026 · written by SkillFed’s research pipeline from the paper above · how these notes are made

SkillFlow introduces 166 tasks across 20 workflow families spanning five domains — finance and economics, operations and supply chain, healthcare and life sciences, governance and strategy, and data and document intelligence. Every family runs on a shared Domain-Agnostic Execution Flow (DAEF), an abstract workflow skeleton that holds a task's operational structure constant while individual tasks within the family escalate in difficulty. Agents operate under an Agentic Lifelong Learning protocol: they start with an empty skill library, work through a family's 8-9 tasks in sequence, and after each one they receive the execution trace plus a verifier-written rubric describing what was missing or wrong. From that feedback they write a skill patch — adding, revising, or deleting entries in the library — before the next task begins. The domain never changes under them; only the skill library they carry forward does.

Results diverge sharply by model. Claude Opus 4.6 raises task success from 62.65% to 71.08% over a family — an 8.43-point gain — while invoking its own skills on 45.78% of tasks. An ablation that gives Opus 4.6 the same trajectories and rubrics but no persistent library to write into reaches only 51.04%, below even its own zero-skill starting point, so the gain isn't just extra context sitting in the prompt. Other models fare worse: Claude Sonnet 4.5 and MiniMax M2.5 gain 6+ points, GPT 5.4 and Claude Opus 4.5 gain more modestly, and Claude Sonnet 4.6 shows no net change at all. Several regress — Qwen3-Coder-480B and MiniMax M2.7 each lose 0.6 points, and GPT 5.3 Codex falls 6.02 points, from 52.41% to 46.39%. The sharpest mismatch is Kimi K2.5, which uses its own skills on 66.87% of tasks for a net gain of only 0.60 points. The gap looks like repair, not authorship: most models write some skill after failing a task, but weaker ones enshrine a flawed one in the library and keep reusing it, turning a single mistake into a repeated pattern across later tasks.