Skill abstraction beats raw-trajectory memory by up to 25 points

Notes on SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning (arXiv:2602.08234) — Peng Xia, Jianwen Chen, Han Wang, Jiaqi Liu, Kaide Zeng, Yu Wang, Siwei Han, Yiyang Zhou, Xujiang Zhao, Haifeng Chen, Zeyu Zheng, Cihang Xie, and 1 more · February 2026

Note published July 25, 2026 · written by SkillFed’s research pipeline from the paper above · how these notes are made

SkillRL splits an LLM agent's raw rollouts into two streams and distills each differently: successful trajectories become demonstrations, failed ones get compressed by a teacher model (OpenAI o3) into short failure lessons — the point of failure, the flawed reasoning, and the general principle to avoid repeating it. Both feed a two-tier SkillBank: general skills that apply across an entire environment, and task-specific skills retrieved by embedding similarity to the task at hand. Before reinforcement learning starts, a cold-start SFT phase teaches the Qwen2.5-7B-Instruct backbone to actually retrieve and apply skills — handing skills to an untrained model turns out to buy almost nothing on its own. During GRPO training the library keeps growing through recursive skill evolution: after each validation epoch, failures in weak task categories get fed back to the teacher model, which proposes new or refined skills, so the SkillBank and the policy improve in tandem instead of the library staying frozen.

On ALFWorld and WebShop, SkillRL reaches 89.9% and 72.7% success — 12.3 points above a same-backbone GRPO baseline on ALFWorld, and enough to beat GPT-4o and Gemini-2.5-Pro on that benchmark by 41.9 and 29.6 points respectively, despite running a 7B open model against much larger closed ones. It also leads memory-augmented RL baselines (Mem0+GRPO, SimpleMem+GRPO, EvolveR, MemRL) by wide margins, and averages 47.1% across seven search-augmented QA benchmarks against Search-R1's 38.5% and EvolveR's 43.1%. The ablations matter more than the leaderboard numbers for anyone building agent memory: dropping the general/task-specific hierarchy costs 13.1 points on ALFWorld and 11.3 on WebShop, and swapping the distilled skill library back for raw trajectory storage costs up to 25 points — the paper's clearest evidence that abstraction, not just more RL, drives the gain. Skill distillation also compresses context 10–20x versus raw trajectories, keeping average prompt length under 1,300 tokens against roughly 1,450 for a raw-memory baseline — about 10% leaner while scoring far higher.