200k+ skills wired into one graph, +40% reward across three backbones

Notes on SkillNet: Create, Evaluate, and Connect AI Skills (arXiv:2603.04448) — Yuanying Liang, R. Zhong, Haoming Xu, Chen Jiang, Yi Zhong, Runnan Fang, Jiasheng Gu, Shumin Deng, Yu-Zhe Yao, Mengru Wang, Shuofei Qiao, Xin Xu, and 37 more · February 2026

Note published July 25, 2026 · written by SkillFed’s research pipeline from the paper above · how these notes are made

AI-assisted notes · reviewed by SkillFed Skill retrieval Bridge: retrieval × benchmarks

A nineteen-institution team spanning Zhejiang University, Alibaba, Tencent, Ant Group, and UCLA built SkillNet, infrastructure that creates, grades, and connects agent skills at scale instead of just hosting them. Skills get pulled from execution trajectories, GitHub repositories, PDFs and slide decks, and raw natural-language prompts through an LLM-driven pipeline, then run through deduplication (directory-structure and MD5 comparison) plus rule-based and model-based filtering before landing in one of ten functional categories. The distinguishing piece is a three-layer skill ontology: a taxonomy of categories and tags, a relation graph linking individual skills via similar_to, compose_with, belong_to, and depend_on edges, and a package library for modular deployment. Skills here aren't vetted one at a time and left to sit — they're wired into a queryable network.

Every admitted skill is graded on five axes — Safety, Completeness, Executability, Maintainability, Cost-awareness — by an LLM evaluator (GPT-5o-mini) on a Good/Average/Poor scale, with Executability getting a second check: code-bearing skills actually run in sandboxes. Against a 200-skill sample independently rated by three PhD-level annotators, the automated scores stayed under a mean absolute error of 0.03 on every dimension, with quadratic weighted kappa essentially at 1.000. Then came the real test: across ALFWorld, WebShop, and ScienceWorld, three backbone models — DeepSeek V3.2, Gemini 2.5 Pro, o4 Mini — equipped with SkillNet-derived skill collections beat both a ReAct baseline and an ExpeL-style experience-retrieval baseline, on seen and unseen task splits alike, with zero overlap between the skill-source trajectories and the test tasks.

Key numbers

Average reward gain over ReAct	+40%
Reduction in interaction steps	-30%
Total candidate skills aggregated	200,000+
Curated high-quality skills retained	150,000+
Evaluator MAE vs. human raters (200-skill sample)	<0.03

What this means for skill authors

The evaluation framework is the part worth studying closely if you author or vet skills: five named axes (safety, completeness, executability, maintainability, cost-awareness), each graded on a coarse three-level scale, checked against human annotators until the automated grader's error landed under 0.03. That's a template for what "vetted" can mean beyond a star count or a download tally — it names the specific failure modes (hazardous operations, missing prerequisites, hallucinated tool calls, broken backward compatibility, runaway API cost) instead of leaving quality as a vibe.

The relation graph is the more novel bet: skills aren't scored in isolation, they're linked by similarity, composition, hierarchy, and dependency, and an agent is expected to select and chain them at runtime rather than load one static bundle. That matches the direction SkillFed's own catalog work is headed — dual-tool scanning plus a vetted catalog is the completeness and executability half of this story; the relational layer is the part most catalogs, including ours, haven't built yet.

One caution for authors: separate benchmarking work cited alongside SkillNet shows curated, externally-authored skills help agents while self-generated skills from the same models often don't. SkillNet doesn't test that split directly, but if the pattern holds broadly, it argues for human- or pipeline-vetted skill catalogs over agents freelancing their own skills — the bet a federation model is already making.

Skills related to this research

wiki-export Wiki Export transforms your Obsidian vault's wikilink graph into multiple structured formats—JSON, GraphML, Cypher scripts, and interactive HTML—ready for use in Gephi, Neo4j, and custom tools. Optionally filter by project or visibility tags, and assign community IDs for graph analysis and visualization.★ 3,032 fine-tuning-with-trl This skill teaches post-training techniques for aligning language models to human preferences. It covers supervised fine-tuning, direct preference optimization (DPO), and online reinforcement learning methods like RLOO and GRPO, with complete workflows and practical examples.★ 221,503 fine-tuning-with-trl Fine-tuning with TRL provides post-training methods to align language models with human preferences through multiple approaches. Train models on instruction data with SFT, optimize for preference alignment via DPO without a separate reward model, or run full RLHF pipelines combining supervised fine-tuning, reward modeling, and PPO optimization. GRPO offers memory-efficient online reinforcement learning for resource-constrained setups.★ 11,165 Unsloth Training Skill This skill teaches efficient LLM fine-tuning via Unsloth, covering GRPO (reinforcement learning with reward functions) and SFT (supervised training). Learn FP8 training for 60% VRAM reduction, automatic packing for 2–5x speedup, vision model tuning, mobile deployment via QAT, Docker containerization, and export to GGUF and other formats.★ 26

Related notes

References

SkillNet: Create, Evaluate, and Connect AI Skills (arXiv:2603.04448)
Reflexion: language agents with verbal reinforcement learning (Shinn et al., NeurIPS 2023)
ExpeL: LLM Agents Are Experiential Learners (Zhao et al., AAAI 2024)
ALFWorld / WebShop / ScienceWorld benchmarks