Curated Skills Lift Success Rates 16.2 Points — Self-Generated Ones Cost You 1.3

Notes on SoK: Agentic Skills - Beyond Tool Use in LLM Agents (arXiv:2602.20867) — Yanna Jiang, Delong Li, Hai Deng, Baihe Ma, Xu Wang, Qin Wang, Guangsheng Yu · February 2026

Note published July 25, 2026 · written by SkillFed’s research pipeline from the paper above · how these notes are made

AI-assisted notes · reviewed by SkillFed Skill security Bridge: security × benchmarks

An agentic skill, formalized here as a four-part contract S = (C, π, T, R), needs all four pieces to count: an applicability condition that decides when it fires, an executable policy, a termination condition, and a callable interface. That's what separates a skill from a raw tool call (no reuse contract), a one-off plan (no persistence), or stored memory (no execution policy) — three things builders already reach for and easily mistake for skills. The definition anchors a lifecycle — discovery, practice, distillation, storage, composition, evaluation, update — plus two taxonomies for what's already shipping. Seven system-level design patterns cover how skills get packaged and run, from metadata-first progressive disclosure and executable code skills through workflow-enforcing gates, self-evolving skill libraries, hybrid natural-language-plus-code macros, meta-skills, and marketplace-distributed plugins. A second, orthogonal grid crosses representation (natural language, code, policy, hybrid) against operating scope (web, OS, software engineering, robotics).

The security case study is concrete. OpenClaw's community registry, ClawHub, grew from zero to over 10,700 published skills within weeks, while the underlying project passed 200,000 GitHub stars faster than any software repository in history. Into that marketplace, the ClawHavoc campaign slipped nearly 1,200 malicious skills before anyone caught them — exfiltrating API keys, crypto wallets, and browser credentials, because OpenClaw skills run with the agent's full system permissions. The evaluation evidence comes from a separate benchmark, SkillsBench: 86 tasks across 11 domains, 7,308 trajectories, three conditions — no skills, curated skills, self-generated skills. Curated skills lifted the average pass rate 16.2 percentage points, from 24.3% to 40.6%; skills an agent generated for itself made things 1.3 points worse. The spread by domain was wide: healthcare gained 51.9 points and manufacturing 41.9, while software engineering managed only 4.5 and mathematics 6.0. A smaller model running curated skills beat a larger model running none.

Key numbers

Curated-skill success-rate lift	+16.2 pp avg (24.3% → 40.6%)
Self-generated skill effect	-1.3 pp vs. no-skill baseline
Malicious skills in ClawHavoc breach	~1,200 skills
ClawHub registry growth	0 → 10,700+ skills in weeks
Domain variance (curated-skill gain)	+51.9 pp (healthcare) vs. +4.5 pp (software eng.)

What this means for skill authors

Put those two findings side by side and the shape of the problem is obvious: a registry that accepts anything gets exactly what an open pipeline invites — thousands of skills published with nothing checking the gap between who wrote a skill and the agent about to run it with full system permissions. The paper's own trust-tiered execution ladder — metadata-only, instruction-access, supervised execution, autonomous execution — reads like an admission that most marketplaces default to the risky end, because gating slows the growth number everyone wants to cite.

The SkillsBench spread matters more than its headline average for anyone actually authoring skills. Healthcare and manufacturing have narrow, well-specified procedures, so a curated skill's applicability condition holds cleanly outside the one trace it came from; software engineering and math are broad enough that a generic skill barely beats writing the plan fresh each time. Self-generated skills fail for a related reason — an agent optimizing for the task in front of it has no incentive to get the applicability condition or termination criterion right for a task it'll face later, so the four-part contract quietly degrades into a memorized shortcut. One caveat is real: this comes from a single benchmark, not yet independently replicated, so the domain splits are evidence in progress, not settled fact.

The same logic applies to pairing marketplace distribution with a curation gate before a skill reaches a user: dual-tool scanning is built for the ClawHavoc failure mode on one side and the narrow, unverified applicability condition behind self-generated skills on the other. Neither claim needs a SkillFed number to land — the paper supplies both halves of the argument on its own.

Skills related to this research

Figma Generate Library Build production-ready design systems in Figma by orchestrating multi-phase workflows that extract tokens from code, create variable foundations, construct components with proper variant sets, and document design principles. Works alongside the figma-use skill to enforce quality patterns from real-world systems.★ 1,925 documentation-lookup Documentation-lookup pulls live reference material for popular frameworks and libraries when you need setup guidance, API details, or code examples. It resolves library identifiers and retrieves version-specific documentation to ensure your answers stay current and accurate.★ 5,778 codexer Codexer is a Python research assistant that leverages Context7 MCP servers to investigate libraries, fetch documentation, and evaluate dependencies for production use. It enforces strict coding standards including PEP 8 compliance, mandatory type hints, and modular architecture, with automated quality gates via black, flake8, and mypy. Use it to conduct thorough package research, compare solutions, and implement clean, reliable Python code.★ 6 excalidraw-diagram-generator Turn natural language into Excalidraw diagrams without drawing manually. Supports flowcharts, mind maps, architecture diagrams, sequence diagrams, ER diagrams, and data flow visualizations. The skill analyzes your description, identifies key elements and relationships, then generates a ready-to-open .excalidraw file.★ 6

Related notes

References

Jiang et al., "SoK: Agentic Skills - Beyond Tool Use in LLM Agents" (arXiv:2602.20867, 2026)