Role-labeled skill groups: reward 36%→49%, runtime cut 42%

Notes on Group of Skills: Group-Structured Skill Retrieval for Agent Skill Libraries (arXiv:2605.06978) — Kun Zeng, Yunchen Huo, Siyu Zhang, Zihan Ye, Yuecheng Zhuo, Haoyu Liu, Yuquan Lu, Junhao Wen, Xiaoying Tang · May 2026

Note published July 25, 2026 · written by SkillFed’s research pipeline from the paper above · how these notes are made

AI-assisted notes · reviewed by SkillFed Skill retrieval

Retrieval for agent skill libraries has been treated as a search problem: find the k most relevant skills and hand them to the agent. GoSkills treats it as a presentation problem instead. It builds anchor-centered skill groups offline from a typed skill graph — clusters of at most three skills carrying explicit roles like prerequisite, formatter, or checker. At query time it picks an anchor group, pulls in supporting groups through a separate group graph, caps the result to a bounded set of atomic skill payloads, and renders everything into a fixed four-field execution contract: Start, Support, Check, Avoid. Nothing about the downstream agent, the skill payload format, or the execution environment has to change — GoSkills sits in front of an existing library rather than replacing it.

Tested across six LLM backbones on SkillsBench and ALFWorld against four baselines — no skills, full-library exposure, flat vector top-k retrieval, and Graph of Skills (a dependency-aware structural retrieval baseline) — GoSkills improves over the flat baselines throughout and, per the paper's own framing, often (though not universally) improves on Graph of Skills as well. For GPT-5.4 on SkillsBench, reward climbs from 36.4% under the strongest baseline to 48.9%, while agent-only runtime drops from 603.7s to 352.9s. On ALFWorld the same backbone goes from 93.6% to 95.3% reward with runtime nearly halved (65.0s to 38.2s). A separate retrieval-gate check hit a 1.00 must-hit rate on annotated visible requirements across all 40 test cases, in both prompting modes tested. The ablations show each stage earning its place: drop anchor selection and the rate falls to 0.82, drop group expansion and it falls to 0.78, drop coverage backfill and it bottoms out at 0.73.

Key numbers

SkillsBench reward, GPT-5.4 (best baseline → GoSkills)	36.4% → 48.9%
Agent runtime, SkillsBench GPT-5.4 (best baseline → GoSkills)	603.7s → 352.9s
Visible-requirement must-hit rate, full method	1.00 (40/40 tasks)
Must-hit rate with anchor selection removed	0.82
Skill budget exposed per query	top-4 skills, 3-skill groups

What this means for skill authors

The paper's real target isn't retrieval accuracy — it's the gap between "relevant" and "usable." A flat top-k list can be perfectly on-topic and still leave an agent guessing which skill to run first and what depends on what downstream. That's a packaging problem, not a search problem, and it's the same gap that argues for skills carrying structured role and dependency metadata rather than a description field optimized purely for embedding similarity. A vetted catalog that only ranks by relevance is solving half the problem GoSkills identifies.

The ablation numbers are the more durable finding here, independent of whether GoSkills itself gets adopted. Anchor selection, group expansion, and coverage backfill each protect a different failure mode, and pulling any one out costs roughly 7 to 8 points of reward plus 18 to 27 points of requirement coverage — not a graceful degradation. That's an argument against treating "add a dependency graph" as a single feature you either have or don't: a structural retrieval layer is only as good as its weakest stage, and this paper is unusually explicit about measuring each one separately.

Read the runtime gains with the paper's own caveat in mind: the comparison is on agent-only runtime, not on the cost of building and maintaining the group graph, which happens once per library and never enters the reported clock. That's a fair framing for a fixed catalog queried repeatedly, but it means the headline speedup is amortized, not total-cost-of-ownership.

Skills related to this research

wiki-retrieve wiki-retrieve upgrades vault search from page-level to chunk-level granularity, combining contextual prefixes, BM25 sparse retrieval, and dense reranking to reduce retrieval failures. Setup is opt-in with built-in privacy gates: run fully on-machine with synthetic prefixes, or opt into Anthropic API or claude CLI tiers for better context generation.★ 10,007 redis This skill walks you through deploying and operating Redis across single-instance, Sentinel, and cluster modes. Learn core commands for strings, hashes, lists, and sorted sets; configure persistence with RDB snapshots and AOF; implement caching, rate limiting, pub/sub, and distributed locking patterns; and run Redis in Docker with monitoring.★ 44 Figma Generate Library Build production-ready design systems in Figma by orchestrating multi-phase workflows that extract tokens from code, create variable foundations, construct components with proper variant sets, and document design principles. Works alongside the figma-use skill to enforce quality patterns from real-world systems.★ 1,925 adr-backfill adr-backfill lets you record decisions that shipped long ago but were never written up as ADRs—typically surfaced by doc audits or code archaeology. It enforces stricter honesty than forward drafting by requiring a visible backfill marker so future readers know the record was reconstructed, not contemporaneous. The skill demands two independent evidence sources and refuses to draft when confidence in the reasoning is too low.★ 49

Related notes

References

Zeng, K., Huo, Y., Zhang, S., Ye, Z., Zhuo, Y., Liu, H., Lu, Y., Wen, J., & Tang, X. (2026). Group of Skills: Group-Structured Skill Retrieval for Agent Skill Libraries. arXiv:2605.06978.
Li et al. (2026a). Graph of Skills: Dependency-Aware Structural Retrieval for Massive Agent Skills. arXiv:2604.05333.
Li et al. (2026c). SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks. arXiv:2602.12670.
Shridhar, M. et al. (2021). ALFWorld: Aligning Text and Embodied Environments for Interactive Learning. International Conference on Learning Representations (ICLR).