Code review, testing, and security auditing claim 35% of task assignments; requirements analysis gets 2%
Notes on Inside the Skill Market: From Software Engineering Activities to Reusable Agent Skills (arXiv:2607.09065) — Jialun Cao, Xinru Yan, Songqiang Chen, Yaojie Lu, Zhongxin Liu, S. Cheung · July 2026
Note published · written by SkillFed’s research pipeline from the paper above · how these notes are made
AI-assisted notes · reviewed by SkillFed Agentic benchmarksThe first large-scale empirical study of software-engineering skills starts with a brutal filtering funnel: 775,790 skills pulled from four public marketplaces — ClawHub, SkillHub, SkillNet, and SkillsMP — shrink to 11,497 after deduplication, rule-based and LLM-based relevance filtering, and dead-link removal. Every surviving skill gets mapped onto an eight-stage SE lifecycle taxonomy running from requirements through release, then broken down further by activity, measured for length and internal structure (plain instructions versus scripts versus full agent workflows), version history, and — wherever a marketplace exposes one — its own safety or quality score.
The catalog that results is lopsided. Implementation, testing, and code review alone account for 65.4% of all skills; requirement analysis sits at 2.2% and release work at 3.2% — the code-adjacent, easy-to-demo stages crowd out the harder-to-package ones. Zoom into individual activities and the concentration sharpens: code review, test automation, and security auditing alone claim 35.5% of all task assignments, against just 176 skills for data engineering and 270 for LLM agent development. Most of what ships isn't executable at all — 63.8% are pure instruction-based skills, natural-language guidance with no code attached, and only 13.6% carry a script, library, or application payload. Maintenance is thin, too: on ClawHub, 969 skills have exactly one version ever published. Where safety scanning exists, it surfaces real problems — on ClawHub, 19.6% of skills are flagged suspicious by both VirusTotal and SkillSpector at once, and credential-handling practices rate acceptable only a third of the time.
Key numbers
| Unique SE skills in final corpus (4 marketplaces) | 11,497 |
| Skills that are instruction-only, no executable code | 63.8% |
| Task assignments claimed by top 3 activities (review, test, security) | 35.5% |
| Requirement-analysis skills (rarest lifecycle stage) | 2.2% |
| ClawHub skills flagged suspicious by both scanners | 19.6% |
Skills related to this research
Related notes
- 40,285 Skills Later, Supply Still Doesn't Match Demand →
- Nearly 1 in 5 Skill Forks Add Security-Sensitive Instructions →
- 39 of 49 Real-World SWE Skills Move the Pass Rate by Exactly Zero →
- Flat retrieval breaks once a skill library hits the tens-to-hundreds range →
- 0 of 9 add-skill edits improved pass rate; cost fell up to 31.7% vs. static bundles →
- 23% of Agent Skills Already Bundle Executable Code, Not Just Prompts →
- Self-Authored Skills Crashed One Model's Accuracy From 78% to Under 1% →
- Comparing a skill's claims to its code lifts misalignment detection from 0.45 to 0.89 Macro-F1 →
References
- Inside the Skill Market: From Software Engineering Activities to Reusable Agent Skills (Cao et al., 2026)
- Voyager: An Open-Ended Embodied Agent with Large Language Models (Wang et al., 2023)
- MetaGPT: Meta Programming for a Multi-Agent Collaborative Framework (Hong et al., 2024)
- SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering (Yang et al., 2024)
- Software Reuse (Krueger, 1992)