A trained 8B model beats Gemini-2.5-Pro at curating agent skills: 61.2% vs 50.7%

Notes on SkillOS: Learning Skill Curation for Self-Evolving Agents (arXiv:2605.06614) — Siru Ouyang, Jun Yan, Yanfei Chen, Rujun Han, Zifeng Wang, Bhavana Dalvi, Rui Meng, Chun-Liang Li, Yizhu Jiao, Kaiwen Zha, Maohao Shen, Vishy Tirumalashetty, and 4 more · May 2026

Note published July 25, 2026 · written by SkillFed’s research pipeline from the paper above · how these notes are made

AI-assisted notes · reviewed by SkillFed Skill evolution

SkillOS splits the self-evolving agent into two roles: a frozen executor that solves tasks and retrieves skills via BM25 retrieval, and a separately trained curator that edits an external SkillRepo of skills stored as Markdown files (the same SKILL.md format Anthropic uses). The curator issues insert/update/delete function calls after every task, but training doesn't grade each edit in isolation — training instances are groups of related tasks, so a skill written after task 1 gets its verdict from whether it actually helps solve tasks 2 through N in that same group. A composite reward (future task success, function-call validity, an LLM-judged content-quality score, and a compression term penalizing verbatim trajectory copying) trains the curator with GRPO while the executor stays frozen.

Across ALFWorld, WebShop, and three reasoning benchmarks (AIME24, AIME25, GPQA-Diamond), the trained curator beat a memory-free baseline and heuristic procedural memory systems (ReasoningBank, MemP) on success rate and interaction efficiency at once, and it transferred to executors (Qwen3-32B, Gemini-2.5-Pro) it never saw in training. The sharper result: the 8B trained curator outperformed using Gemini-2.5-Pro directly as the curator on the same executor. Watch the curator over the course of training and a pattern emerges: inserts dominate early, updates take over later, deletes stay rare. The skills themselves change shape too, accumulating failure-handling and conditional-branch sections as the repository shifts from narrow task-specific entries toward reusable meta-skills like verification and fallback planning.

Key numbers

ALFWorld success rate vs. best baseline (Qwen3-8B exec.)	61.2% vs 55.7%
Interaction steps vs. best baseline (ALFWorld)	−6.0%
WebShop success rate vs. best baseline	16.5% vs 12.0%
8B trained curator vs. Gemini-2.5-Pro-as-curator (avg. SR, same executor)	61.2% vs 50.7%
Success rate without grouped-task training (ablation)	57.3%, down from 61.2%

What this means for skill authors

The finding worth sitting with is the ablation, not the leaderboard: swapping grouped task streams for random task order drops ALFWorld success from 61.2% to 57.3% — the single largest degradation in the paper, bigger than removing either auxiliary reward term. That says the hard part of skill curation isn't writing a plausible-sounding skill file; it's grading a curation decision by whether it actually helps a later, related task, which requires the kind of temporal structure most memory-writing pipelines never bother to construct.

For anyone building or consuming self-updating skill libraries, the more uncomfortable result is that letting a strong, untrained model freely curate (the Gemini-2.5-Pro-as-curator variant) underperforms a small model trained against downstream outcomes. Model capability and curation judgment are different skills — a capable model can write skills that score well on a content-quality judge without being what the executor actually needs. That's a direct argument for evaluating a skill's real usefulness against task outcomes rather than trusting any single model's self-assessment of quality, which is the spirit behind vetting catalog entries against actual behavior rather than an author's or a lone LLM's say-so.

The frozen executor is the biggest asterisk: nothing here shows the approach still holds once the acting model is being updated too, not just its skill library. Add that skills are single flat Markdown files rather than the folder-plus-scripts structure real skill authors use, and that retrieval runs on plain BM25 — the authors flag all three as open limitations. The cross-domain transfer result is real but uneven: a curator trained on reasoning tasks carried over well to agentic tasks, while curators trained on ALFWorld or WebShop stayed more environment-specific. Treat "the curator generalizes" as a within-related-domains claim, not a universal one.

Skills related to this research

wiki-retrieve wiki-retrieve upgrades vault search from page-level to chunk-level granularity, combining contextual prefixes, BM25 sparse retrieval, and dense reranking to reduce retrieval failures. Setup is opt-in with built-in privacy gates: run fully on-machine with synthetic prefixes, or opt into Anthropic API or claude CLI tiers for better context generation.★ 10,007 seedance-recipes Seedance-recipes provides production-ready recipe patterns for video content across genres: product, lifestyle, drama, music video, landscape, commercial, animation, and more. Each recipe preserves core creative constraints while inviting customization of subject, camera, lighting, and sound. Use recipes as proven starting shapes, not rigid templates.★ 5,445 fine-tuning-with-trl This skill teaches post-training techniques for aligning language models to human preferences. It covers supervised fine-tuning, direct preference optimization (DPO), and online reinforcement learning methods like RLOO and GRPO, with complete workflows and practical examples.★ 221,503 fine-tuning-with-trl Fine-tuning with TRL provides post-training methods to align language models with human preferences through multiple approaches. Train models on instruction data with SFT, optimize for preference alignment via DPO without a separate reward model, or run full RLHF pipelines combining supervised fine-tuning, reward modeling, and PPO optimization. GRPO offers memory-efficient online reinforcement learning for resource-constrained setups.★ 11,165

Related notes

References

Ouyang, S. et al. "SkillOS: Learning Skill Curation for Self-Evolving Agents." arXiv:2605.06614 (2026).
Shridhar, M. et al. "ALFWorld: Aligning Text and Embodied Environments for Interactive Learning." ICLR (2021).
Yao, S. et al. "WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents." NeurIPS (2022).
Rein, D. et al. "GPQA: A Graduate-Level Google-Proof Q&A Benchmark." (2023).
Anthropic. "Agent Skills" documentation — the SKILL.md (YAML frontmatter + Markdown) format SkillOS adopts (2025).