SKILL.md files average 10.5 quality smells, and age doesn't cure them

Notes on From Anatomy to Smells: An Empirical Study of SKILL.md in Agent Skills (arXiv:2607.01456) — David Hong, Aaron Imani, Iftekhar Ahmed · July 2026

Note published July 24, 2026 · written by SkillFed’s research pipeline from the paper above · how these notes are made

An Agent Skill's domain knowledge lives inside SKILL.md — a mandatory file with metadata up top and then a markdown body authors can write however they want. No required sections, no schema for the body. This study is the first systematic look at what actually ends up in that free space. The authors pulled 238 real skills from the skills.sh marketplace, working down from 133,149 listed packages by filtering on weekly downloads and repository diversity, then hand-coded the bodies into a taxonomy of 13 higher-level and 44 lower-level semantic components — task, introduction, references, principles, and so on. To judge quality, they ran a multivocal literature review across 29 online sources — docs, vendor guides, practitioner posts — and distilled 26 concrete authoring best practices. Violating each one gets a name: a skill smell. A hybrid detector, static rules for some smells and an LLM classifier for the rest, then scanned the corpus and tracked 142 skills with commit history (1,199 commits total) to see whether smells clear up as a skill matures.

Only 7 of the 13 semantic components turn out to have any explicit guidance anywhere in the 29 reviewed sources — the rest of what goes into a skill file is authored on instinct. Against that thin guidance, 237 of the 238 files, over 99%, carry at least one skill smell, averaging 10.5 per file. The single most common one, Rationalization Loophole — a file that never tells the agent it isn't allowed to talk itself out of a required step — shows up in 94% of the corpus. Eleven of the 26 smells clear the 50% mark across the sample. The commit-history analysis is the part that stings: across the 142 skills tracked over 1,199 commits, smells introduced early show no measurable tendency to get fixed later. Popularity and iteration don't self-correct quality.