A skill graph beats context-stuffing: +25.55% reward, -56.72% tokens
Notes on Graph of Skills: Dependency-Aware Structural Retrieval for Massive Agent Skills (arXiv:2604.05333) — Dawei Liu, Zongxia Li, Hongyang Du, Xiyang Wu, Shihang Gui, Yongbei Kuang, Lichao Sun · April 2026
Note published · written by SkillFed’s research pipeline from the paper above · how these notes are made
AI-assisted notes · reviewed by SkillFed Skill retrievalGraph-of-Skills (GoS) builds a typed skill graph offline over a local skill library, linking skills through four edge types: dependency, workflow, semantic, and alternative. Dependency edges come free — induced deterministically from I/O schema overlap between skills, no LLM involved — while the other three relation types are filled in by a sparse, top-k LLM validation pass rather than an exhaustive pairwise check. At query time, GoS blends semantic and lexical retrieval scores to seed the graph, then runs a reverse-aware Personalized PageRank diffusion so upstream prerequisites — a parser here, a format converter there — accumulate relevance even when they share no vocabulary with the query. A final rerank-and-hydrate step packs the top-scoring skills into a token-budgeted bundle.
GoS is benchmarked against two baselines — full-library loading and flat top-k vector retrieval — on SkillsBench (11 technical domains) and ALFWorld (140 embodied household episodes), across three model families: Claude Sonnet 4.5, MiniMax M2.7, and GPT-5.2 Codex. It wins on reward in all six model-benchmark combinations. The peak result — 1,000-skill SkillsBench, GPT-5.2 Codex — puts GoS at 34.4% average reward against 27.4% for full loading and 21.5% for vector retrieval, while total tokens drop 56.72% versus full loading. Vector retrieval is often the weakest of the three baselines: on SkillsBench it trails full-library loading by 5–7 reward points across every model tested, because it fetches the topically obvious skill and skips the unglamorous prerequisite sitting right next to it. A library-size sweep from 500 to 2,000 skills shows full loading's token bill nearly tripling — 1.93M to 5.84M — while GoS holds close to flat, and its reward margin over both baselines widens as the library grows.
Key numbers
| Peak reward gain vs. full-library loading | +25.55% |
| Token reduction vs. full-library loading | -56.72% |
| Vanilla token growth, 500→2,000 skills | 1.93M→5.84M (~3x) |
| Vector-only reward vs. full loading (MiniMax M2.7) | -6.8 pts (17.2→10.4) |
| Reward lost removing lexical retrieval + rerank | -7.7 pts |
Skills related to this research
Related notes
- A typed skill graph gives up just 3.5 retrieval points across a 10x library scale-up →
- Retrieving the right skill lifts coding Pass@1 78%, reaching 84% of oracle-level performance →
- Role-labeled skill groups: reward 36%→49%, runtime cut 42% →
- A fine-tuned 8B retriever hits 83 NDCG@10 — a 12B off-the-shelf model manages 55 →
- Forced to search a 34k-skill library instead of being handed the right one, agents' pass rate falls from 55.4% to 38.4% -- a hair above the 35.4% no-skill floor →
- Task-decomposition reranking beats the best baseline 78.7 vs 73.1 on ALFWorld-unseen, using just 1.3 skills per task →
- Malicious-Skill Detection Hits 0.920 F1 at $0.006 per Skill →
- A skill improves every time anyone uses it, not just its author →
- Three rounds of automated repair add up to 12pp of consistency — and beat a mature production system by 13.76pp →
References
- Liu, Li, Du, Wu, Gui, Kuang, Sun. Graph-of-Skills: Dependency-Aware Structural Retrieval for Massive Agent Skills. arXiv:2604.05333, 2026.
- Li et al. SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks. arXiv:2602.12670, 2026.
- Haveliwala. Topic-Sensitive PageRank. WWW 2002.
- Shridhar et al. ALFWorld: Aligning Text and Embodied Environments for Interactive Learning. 2020.
- Karpukhin et al. Dense Passage Retrieval for Open-Domain Question Answering. EMNLP 2020.