SkillWiki Converts Structured Sources to Governed Skills at 96% — Legacy Skill Files Lag 32 Points Behind

Notes on SkillWiki: A Living Knowledge Infrastructure for Agent Skills (arXiv:2606.16523) — Dingcheng Huang, Yu Ding, Bingshuo Liu, Qingbin Liu, Xi Chen, Jiang Bian, Hongliang Sun, Zhiying Tu, Dianhui Chu, Xiaoyan Yu, Dianbo Sui · June 2026

Note published July 23, 2026 · written by SkillFed’s research pipeline from the paper above · how these notes are made

AI-assisted notes · reviewed by SkillFed Skill evolution Bridge: evolution × benchmarks

SkillWiki treats an agent skill as a governed, versioned asset, not a stored prompt or memory snippet — the same move Wikipedia made for articles and Git made for code. The system ingests five kinds of raw material (trajectories, documents, API specifications, scripts, and historical skill files), then runs them through a knowledge-grounded construction pipeline that extracts reusable actions and workflows while keeping every skill linked back to its originating evidence via a Skill Provenance Graph. Each skill carries a lifecycle state — one of eight, from Raw Experience and Candidate through Verified, Released, Degraded, Deprecated, and Archived — and sits in a three-tier taxonomy of atomic, functional, and strategic skills. Nothing changes directly: edits pass through a Git-style governance workflow of candidate diffs, review, and release, run largely by autonomous self-management agents that flag breaking changes, with humans holding override authority at every step.

To test the production side, the authors built a 125-artifact benchmark — 25 each of trajectories, documents, API specs, scripts, and historical skills, pulled from sources including SkillsBench, WebArena, and Anthropic's published skills — then ran it through the pipeline, with DeepSeek-V4-Flash handling knowledge parsing and candidate generation. 99 of 125 artifacts (79%) came out the other side as governed skills. The yield split sharply by source type: trajectories and API specifications converted at 96% (24/25 each), scripts at 72% (18/25), documents at 68% (17/25), historical skill files lowest at 64% (16/25). A separate case study traced one API-documentation-derived skill through all eight lifecycle states end to end — production, verification, release, execution, repair, versioning, deprecation, archival — showing the governance loop closes in practice, not just on paper.

Key numbers

Overall conversion rate	99/125 (79%)
Trajectory / API-spec conversion	96% (24/25 each)
Script conversion	72% (18/25)
Document conversion	68% (17/25)
Historical-skill conversion (weakest)	64% (16/25)

What this means for skill authors

The framing here is the useful part. Skill-acquisition research has mostly asked how to build one more skill, not how to keep ten thousand of them straight. Once skills are long-lived assets that other skills depend on, get invoked in production, and eventually go stale, the missing piece isn't a better extraction algorithm — it's the infrastructure code already gets from version control: lineage, diffs, review gates, a rollback path. Treating skill health as ongoing telemetry (success rate, failure patterns, reflection memories from failed runs) rather than a one-time eval score is the right instinct. It manages a skill like a production service, not a static file.

The conversion numbers are the more actionable data point for anyone building or vetting a skill pipeline. Sources with inherent structure — trajectories, API specs — convert almost automatically at 96%. Sources that require inferring intent from prose or legacy artifacts lose roughly a third to manual triage: documents at 68%, old skill files at 64%. That's not a flaw unique to SkillWiki; it's a general ceiling on how much of "turn any knowledge into a working skill" can be automated versus reviewed. Anyone promising near-100% automatic skill generation from arbitrary documentation should be read against that gap.

For a federated skill catalog, the relevant idea isn't the demo UI, it's the schema: a skill worth trusting needs traceable provenance and a lifecycle state, not just a passing test run. That's the same judgment a vetted catalog has to make before recommending a skill to an agent — where it came from, whether it's been flagged as degraded, whether its version trails current dependencies. SkillWiki doesn't solve trust or provenance at the marketplace level, but it's a clean illustration of what the metadata layer under any skill-vetting system eventually needs to look like.

Skills related to this research

ad-account-auditor This skill systematically reviews paid advertising accounts to validate return-on-ad-spend calculations and ensure measurement reliability. Use it before increasing budget allocation to catch tracking gaps, attribution errors, or configuration issues that could distort performance data.★ 2,469 building-ci-pipelines Master continuous integration and deployment by building robust pipelines across multiple platforms. This skill guides you through setting up automated workflows that handle testing, building, and deploying your applications with minimal manual intervention.★ 390 Wiki Lint Wiki Lint performs read-only audits of wiki spaces, surfacing contradictions, orphaned pages, stale claims, concept gaps, broken links, and provenance issues. It returns a severity-grouped triage list with evidence and suggested fixes, leaving all decisions to the maintainer.★ 9 Package Publishing This skill handles the end-to-end automation of package distribution and versioning tasks. Integrate it into your CI/CD pipeline to eliminate repetitive publishing steps, manage releases systematically, and keep your artifact repositories in sync across environments.★ 13

Related notes

References

Huang et al., "SkillWiki: A Living Knowledge Infrastructure for Agent Skills," arXiv:2606.16523 (2026)
Dabbish, Stuart, Tsay & Herbsleb, "Social Coding in GitHub: Transparency and Collaboration in an Open Software Repository," CSCW 2012
Giles, "Internet Encyclopaedias Go Head to Head," Nature 438(7070) (2005)
Li et al., "SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks," arXiv:2602.12670 (2026)
Zhou et al., "WebArena: A Realistic Web Environment for Building Autonomous Agents," arXiv:2307.13854 (2024)