Self-tested skill edits beat teacher-curated skill banks by up to 9.3 points

Notes on SkillMaster: Toward Autonomous Skill Mastery in LLM Agents (arXiv:2605.08693) — Min Yang, J. Piao, Xuanye Xia, Xiaochong Lan, Jiajun Chen, Yongshun Gong, Yong Li · May 2026

Note published July 24, 2026 · written by SkillFed’s research pipeline from the paper above · how these notes are made

SkillMaster gives an LLM agent the right to edit its own skill bank instead of treating skills as content pushed in by an external teacher. After every training episode the agent enters a review phase and must call exactly one of three tools — propose a new skill, revise an existing one, or leave the bank alone — grounded in the trajectory that just played out. Whether an edit is worth keeping is decided by a counterfactual utility reward: the system reruns four related probe tasks under the old skill bank and the candidate-edited version, and only rewards edits that raise success rates or cut wasted steps across those probes. A modified GRPO variant, DualAdv-GRPO, keeps this skill-editing signal from destabilizing the task-execution signal by normalizing advantages for action tokens and skill-editing tokens separately before merging them into one policy update.

On ALFWorld the trained agent reaches 98.7% overall success, 8.8 points above the strongest prior baseline — SkillRL's teacher-curated skill library at 89.9% — with four of six task families solved perfectly. On WebShop success climbs from 72.7% to 82.0%. An ablation isolates where the gain actually comes from: bolting the same post-episode review step onto plain GRPO without the utility reward moves success only from 77.6% to 78.4%, so reflection by itself explains almost none of the improvement. A separate test strips the starting skill bank down to nothing, and the trained agent still reaches 89.4% versus 74.8% for the teacher-driven baseline run under the same conditions — most of the payoff sits in the learned editing behavior, not in inherited skill content.