One shared reward signal pushes skill-augmented agents to 97.5% success, beating every tested baseline — skill-augmented or skill-free — by up to 20 points

Notes on Skill1: Unified Evolution of Skill-Augmented Agents via Reinforcement Learning (arXiv:2605.06130) — Yaorui Shi, Yuxin Chen, Zhengxi Lu, Yuchun Miao, Shugui Liu, Qi Gu, Xunliang Cai, Xiangjun Wang, An Zhang · May 2026

Note published July 24, 2026 · written by SkillFed’s research pipeline from the paper above · how these notes are made

Skill1 trains one policy, via reinforcement learning, to run all three stages of a skill library lifecycle end to end: it writes a text query and re-ranks candidates to pick a skill, solves the task conditioned on that pick, then distills a new skill out of its own trajectory. The departure from prior work sits in credit assignment. Prior systems wire each stage to its own reward source — a heuristic match score for selection, a self-assessed quality score for distillation, task outcome for utilization — training three objectives that never talk to each other. Skill1 derives every signal from one binary task-outcome value instead, split into a low-frequency trend (an exponential moving average of each skill's success rate that supervises re-ranking through an NDCG-style ranking reward) and a high-frequency variation (current outcome minus the best retrieved skill's trend, which only pays out when a distilled skill beats the library's existing ceiling). One GRPO update per rollout batch trains query generation, re-ranking, and distillation together.

On ALFWorld, Skill1 reaches 97.5% average success, ahead of the strongest prior skill-augmented RL baseline (RetroAgent, 94.9%) by 2.6 points and the strongest skill-free RL baseline (GiGPO, 90.8%) by 6.7 points; on WebShop it scores 89.7 (82.9% success), again best among tested methods. The ablations are the more interesting result: pulling the skill library out entirely costs 16.6 points (down to 80.9%), dropping only the selection signal costs 5.7, dropping only distillation costs 5.1, and zeroing both auxiliary objective weights at once costs 7.3 — worse than either removal alone, the paper's evidence that the three capabilities are coupled, not merely additive. Training curves show selection precision converging first (0.95 by step 20), with utilization and distillation catching up afterward (0.8 by step 60), and the growing library adds real overhead: 1.3–1.7x the per-step wall-clock time of skill-free GRPO.