MUSE-Autoskill's Self-Written Skills Beat Human-Authored Ones, 85.24% vs. 81.17%

Notes on MUSE-Autoskill: Self-Evolving Agents via Skill Creation, Memory, Management, and Evaluation (arXiv:2605.27366) — Huawei Lin, Peng Li, Jie Song, Fuxin Jiang, Tieying Zhang · May 2026

Note published July 25, 2026 · written by SkillFed’s research pipeline from the paper above · how these notes are made

AI-assisted notes · reviewed by SkillFed Skill evolution

MUSE-Autoskill wraps an LLM agent in a five-stage skill lifecycle — creation, memory, management, evaluation, refinement — instead of treating each skill as a one-off script. New skills are built through a create tool that emits a SKILL.md interface file plus optional scripts, resources, and unit tests; code-backed skills are graded on their own tests, while procedural ones go through sandboxed execution and trajectory checks. Every skill also carries a skill-level memory file where the agent appends lessons from past use, alongside short-term task memory and persistent cross-session notes; long conversations stay within budget through a DAG-based context compression scheme that summarizes or merges old turns rather than discarding them outright. A management stage periodically merges overlapping skills and prunes ones that keep failing. That setup gets evaluated across three agent runtimes — Hermes, Codex, and Claude Code — on SkillsBench (75 tasks across four domains, 5 runs each) and SkillLearnBench (100 verified instances over 20 skill-dependent tasks).

MUSE-Autoskill wins every skill condition on both benchmarks. On SkillsBench, with no skills it scores 46.95% against 37.24% (Hermes), 44.80% (Codex), and 42.43% (Claude Code); with human-authored skills it reaches 59.67% against 48.02%, 57.58%, and 56.15%; with its own self-created skills it hits 53.42%, ahead of Codex's 47.52% and Claude Code's 44.27%. The sharper result sits inside the subset where self-created skills actually exist: on the 47 of 75 tasks where MUSE managed to produce a usable skill, those self-authored skills score 85.24% versus 81.17% for the human-written skills covering the same tasks — the agent's own tooling beats what people wrote, on the ground it manages to cover. Skills also survive a change of runtime — what the paper calls cross-agent skill transfer: skills MUSE creates but hands to Hermes reach 51.90% accuracy, edging out Hermes's own score with human-authored skills (48.02%) and out-transferring skills authored by Codex or Claude Code.

Key numbers

Self-created vs. human-authored skills (covered subset)	85.24% vs. 81.17%
Self-created skill coverage on SkillsBench	47 of 75 tasks produced a usable skill
Cross-agent transfer accuracy (MUSE skills → Hermes)	51.90%, vs. 48.02% for Hermes with human skills
One-time skill-creation overhead	364K tokens / 156.3s per task
SkillsBench, self-created skills (full 75-task set)	MUSE 53.42% vs. Codex 47.52%, Claude Code 44.27%

What this means for skill authors

The coverage caveat matters as much as the headline number. MUSE produces a usable skill for 47 of 75 SkillsBench tasks — good enough to beat human authors where it lands, silent on the other 28. Each skill is generated from a single successful trajectory and re-tested on that same task — a generous setup. The paper's own hvac-control example undercuts it: a skill tuned to one run's simulator noise later produced gains outside the verifier's stability margin on a fresh run. A skill that passed its one recorded trial isn't yet a skill proven to generalize.

The transfer number is the one worth sitting with. A skill authored under MUSE's conventions, handed to a different runtime with no fine-tuning, still beats that runtime's own human-written skills. That suggests the interface — a documented SKILL.md contract plus tests — is doing more of the work than the identity of whichever agent wrote it. It's a point in favor of judging a skill by its interface and test coverage rather than by who authored it, which is the same instinct behind vetting skills before they enter a shared catalog rather than trusting provenance alone.

It's also an argument for skills as living records rather than static drops: MUSE's per-skill memory file and its merge/prune step exist because a skill's first version usually isn't its best one. A catalog that only checks a skill once, at submission, would miss exactly the kind of regression the hvac-control case describes — the qualitative case for scanning skills on update, not just on arrival.

Skills related to this research

audio-router audio-router intelligently directs audio processing requests to the right specialized skill, whether for playback control, sound analysis, or reactive audio handling. Built for Claude, it streamlines how audio workflows are orchestrated across multiple focused capabilities.★ 8 seedance-recipes Seedance-recipes provides production-ready recipe patterns for video content across genres: product, lifestyle, drama, music video, landscape, commercial, animation, and more. Each recipe preserves core creative constraints while inviting customization of subject, camera, lighting, and sound. Use recipes as proven starting shapes, not rigid templates.★ 5,445 ugc-ads ugc-ads builds short-form product videos optimized for social platforms, combining jump-cut pacing with native lip-synced dialogue delivered as a first-person talking head. The skill auto-detects product category from your URL and structures the narrative across five beats—setup, name reveal, product reveal, twist, and punchline—then renders in your choice of vertical formats.★ 37 10 Music Video This skill transforms your music into precise visual prompts for Seedance 2.0 on Higgsfield, a platform that synchronizes generated video to your uploaded audio track. Learn to reference specific beats, describe genre-matched visuals, and apply proven hooks—from particle explosions on drops to color shifts on harmonic changes—that lock your visuals to every moment of your song.★ 712

Related notes

References

MUSE-Autoskill: Self-Evolving Agents via Skill Creation, Memory, Management, and Evaluation (Lin et al., 2026)
Voyager: An Open-Ended Embodied Agent with Large Language Models (Wang et al., 2023)
Reflexion: Language Agents with Verbal Reinforcement Learning (Shinn et al., 2023)
MemGPT: Towards LLMs as Operating Systems (Packer et al., 2023)
Generative Agents: Interactive Simulacra of Human Behavior (Park et al., 2023)