skillfed

Nearly 1 in 5 Skill Forks Add Security-Sensitive Instructions

Notes on An Empirical Study of Downstream Adaptation for Agent Skills (arXiv:2607.03238) — Xinjian Wu, Jingzhi Gong, Gunel Jahangirova, Zhenpeng Chen, Jie M. Zhang · July 2026

Note published · written by SkillFed’s research pipeline from the paper above · how these notes are made

AI-assisted notes · reviewed by SkillFed Skill security Bridge: security × benchmarks

Researchers screened GitHub for agent skill repositories with at least 20,000 stars and 2,000 forks and landed on six, including Anthropic's own anthropics/skills, obra/superpowers, and affaan-m/everything-claude-code. Those six repos alone had 67,264 public forks. Filtering out inactive branches, upstream-sync noise, and edits that never touched an actual skill package brought that down to 1,220 branch comparisons -- each one a developer's real, deliberate change to an existing skill. An LLM agent read every diff and built a taxonomy from the patch content itself rather than from commit messages; two human auditors checked a random sample and hit κ=0.83 agreement, solid enough to trust the result: 1,126 validated adaptation instances, sorted into 46 patterns across 13 families.

The headline finding is a reuse paradox: skills ship for drop-in reuse, yet the most common fork-level edit is rewriting the skill's own metadata/description field -- the routing signal an agent uses to decide the skill is relevant in the first place. Adding procedure steps, decision rules, and hard constraints follows close behind. SKILL.md gets touched in 79.8% of every adaptation instance, against 23.3% for bundled scripts -- the markdown instruction file, not the code underneath it, is the real control plane developers are fighting with. These edits rarely land alone: procedure, decision, and policy changes co-occur far more than chance would predict, with lift as high as 3.53. And 18.6% of adaptations introduce content matching security-sensitive patterns, access to sensitive files topping the list. 73.1% of those matches sit inside SKILL.md or bundled docs rather than executable scripts -- invisible to any review process that only scans code.

Key numbers

Skill-adaptation instances analyzed1,126
Adaptation patterns / families46 / 13
Adaptations touching SKILL.md79.8%
Adaptations adding security-sensitive content18.6%
Of those, hidden in prose rather than scripts73.1%

Skills related to this research

Lesson Study Cycle Designer Lesson Study Cycle Designer structures the Japanese jugyō kenkyū process for teacher teams investigating student learning challenges. It generates a research theme, detailed cycle plan with meeting agendas, a research lesson designed for collaborative observation, and protocols focused on case students. The skill grounds itself in evidence from Stigler & Hiebert, Lewis, Perry & Murata, Dudley, and Takahashi & McDougal on how teachers develop shared professional knowledge through structured cycles of planning, teaching, observing, and analysing.★ 472 markdown-mermaid-writing This skill establishes markdown with embedded Mermaid diagrams as the source format for scientific documents, reports, and technical writing. It includes 24 diagram type references, 9 document templates, and comprehensive style guides to enforce consistent, version-control-friendly documentation. Text-based diagrams render natively on GitHub, GitLab, and Notion without build steps, remain editable, and stay smaller than prose descriptions.★ 31,940 localize-campaign Transform your campaign for global reach by localizing all assets—emails, ads, social posts, landing pages, and scripts—across multiple target markets simultaneously. The skill handles translation routing, cultural adaptation using established frameworks, compliance modifications per region, and SEO localization, then packages deployment-ready assets with quality scores for each market. It goes beyond word-for-word translation to ensure creative content resonates culturally and all regulatory requirements are met.★ 641 3dgs-visualizer This skill provides an interactive explorer for the 3D Gaussian Splatting research landscape, letting you generate publication-quality comparison charts across hundreds of papers and methodologies. Organize your analysis by category, filter by technique, and create visuals that communicate complex research relationships clearly. Perfect for researchers synthesizing 3DGS literature or presenting findings to peers.★ 129

Related notes

References

  1. Wu, X., Gong, J., Jahangirova, G., Chen, Z., & Zhang, J. M. (2026). An Empirical Study of Downstream Adaptation for Agent Skills. arXiv:2607.03238.
  2. Anthropic and the Agent Skills community. (2025). Agent Skills: A standardized way to give AI agents new capabilities and expertise.
  3. Liu, Y., Wang, W., Feng, R., Zhang, Y., Xu, G., Deng, G., Li, Y., & Zhang, L. (2026). Agent Skills in the Wild: An Empirical Study of Security Vulnerabilities at Scale. arXiv.
  4. Zhou, Y., Shu, W., Su, Y., Du, W., Fang, Y., & Lin, X. (2026). A Comprehensive Survey on Agent Skills: Taxonomy, Techniques, and Applications. arXiv:2605.07358.