Skill security 42 papers

SkCC treats SKILL.md as source code, not the final artifact. A syntax parser lowers the raw markdown into a typed abstract syntax tree; an IR builder then normalizes that tree into SkIR, a strongly-typed intermediate representation capturing a skill's procedures, permissions, input/output schemas, constraints, and MCP dependencies independent of any target framework. Before anything gets emitted, a static Security Optimizer runs four passes over the IR — structural validation, permission auditing, pattern-based injection of missing safety constraints like timeouts, loop bounds, and destructive-operation guards, and risk-tier classification. From there, four emitters render the same hardened IR differently per framework: Claude gets XML-tagged sections, Gemini switches to YAML once nesting passes a depth threshold, Kimi keeps full markdown, Codex gets XML-tagged markdown to dodge JSON's token overhead. Parsing and static analysis run once per skill no matter how many targets follow, which is why adaptation cost moves from O(m×n) — m skills times n frameworks, each pair hand-rewritten — to O(m+n).

Every agent skill is two artifacts stapled together: a structured half that declares callable actions, and a prose half that tells the agent when to fire them — prose that gets reinterpreted fresh, probabilistically, on every invocation. Static analyzers parse the structured half and ignore the prose; LLM-based auditors read the prose but can't reproducibly prove a tainted input reaches a dangerous sink. Semia lifts each skill into the Skill Description Language (SDL) — a Datalog fact base capturing triggers, data flow, authorization gates, and documented claims. It generates that fact base through Constraint-Guided Representation Synthesis (CGRS), a propose-validate-verbalize loop that keeps refining a candidate until it's both structurally sound and faithful to the source text. Once the facts exist, checks for indirect prompt injection , secret leakage, or a confused deputy become plain Datalog reachability queries.

Agent Skills bundle a SKILL.md manifest with scripts, reference documents, and repository context, so auditing one before it loads means reviewing several files as a unit rather than screening a single prompt. Sorting a package into benign, suspicious, or malicious takes four stages: pull role-weighted evidence out of every file, route only the uncertain cases to a semantic verifier, arbitrate between competing attack-chain readings — a hidden override, a disguised data transfer, a remote bootstrap dependency — then check the verdict against the package's semantics-preserving rewrites , escalating to "malicious" only once every rewrite agrees. That pipeline is SkillGuard-Robust, tested on SkillGuardBench plus two public-ecosystem extensions, across five evaluation views spanning 254 to 404 packages each.

HarmfulSkillBench starts as a sweep, not a benchmark: every one of the 98,440 skills published on ClawHub and Skills.Rest gets scored against a purpose-built harmful skill taxonomy -- cyber attacks, fraud and scams, privacy violation, sexual content generation, and more -- split into a Tier 1 prohibited-use band and a lighter Tier 2 high-risk band. The scorer is LLM-driven, tuned against 500 hand-labeled samples to an F1 of 0.82. Whatever clears the threshold feeds the benchmark itself: 200 harmful skills across 20 categories, drawn from both registries plus original entries, each pushed through four evaluation conditions that isolate one variable at a time -- a bare harmful request with no skill present, the same request delivered through a pre-installed skill, and skill-delivered variants where the harmful intent is stated outright or left implicit in the skill's own instructions.

A regex scanner can read a skill's code but not the English prose in its SKILL.md — exactly where prompt injection and social-engineering instructions like to hide. Formal static analyzers read code more rigorously and still can't parse prose. SkillSieve's answer isn't a smarter single scanner; it's three escalating layers, each one filtering for the next. Layer 1 is a zero-cost static analysis pass — regex, AST parsing, metadata heuristics — that clears roughly 86% of skills in under 40ms apiece. What's left goes to Layer 2, an LLM that splits judgment into four parallel sub-checks (intent alignment, permission justification, covert-behavior detection, cross-file consistency) rather than one monolithic prompt. Skills that still look risky hit Layer 3: a multi-model jury — GLM-5.1, Qwen3-235B, DeepSeek-V3.1 — that votes independently and only debates when the votes split. The full pipeline ran end to end on a single $440 ARM single-board computer against 49,592 real skills pulled from OpenClaw's ClawHub registry (a 2026-04-04 snapshot spanning 16,797 authors), plus adversarial samples built around five evasion techniques.

Agent Skills — the filesystem-based packaging format that lets agents load domain-specific instructions and bundled scripts on demand — has never had a systematic security audit until now. This one splits a skill's life into four phases: Creation, Distribution, Deployment, Execution. Each opens a different door: authors get unrestricted control over SKILL.md and its scripts, marketplaces distribute without review, and the access a user actually grants at install time runs broader than what they think they're agreeing to. From that lifecycle view comes a threat taxonomy of seven categories and seventeen concrete attack scenarios, grouped into three layers: delivery and trust establishment, runtime attack, and persistent/lateral impact.

Coding agents install third-party agent skills from open marketplaces with no mandatory security review, and because those skills run as operational directives with system-level privileges — file writes, shell commands, network calls — a single bad one can compromise the host outright. Document-Driven Implicit Payload Execution ( DDIPE ) is a supply-chain attack that skips the obvious move of writing malicious instructions into a skill's prompt text, and instead embeds the payload inside the skill's code examples and configuration templates — the parts of a skill an agent reuses verbatim while doing normal work, so the attack fires without ever being explicitly asked for. An LLM-driven generation pipeline scaled this from 81 hand-built seed skills to 1,070 adversarial variants spanning 15 MITRE ATT&CK categories, roughly a 13x expansion, then ran them against four production coding-agent frameworks — Claude Code, OpenHands, Codex, and Gemini CLI — across five underlying models.

Researchers built the first large-scale audit of credential leakage across a live agent-skill marketplace. Starting from 170,226 artifacts on SkillsMP, they drew a stratified random sample of 17,022 skills and ran each through a three-stage pipeline: static secret extraction using regex and AST parsing , dynamic sandbox testing seeded with mock credentials, and a cross-reference step that checks whether a skill's stated natural-language purpose matches what its code actually does at runtime. That last step is the paper's structural bet: credential handling in an agent skill isn't just a code-review problem, because the thing an LLM trusts (the description) and the thing that executes (the code) can diverge, and neither view alone catches the gap.

SkillClone doesn't read a SKILL.md file as one flat document. It splits each into three channels — YAML frontmatter, natural-language instructions, and embedded code — and embeds each one separately with TF-IDF plus latent semantic analysis. The pairwise channel similarities then feed a logistic regression, alongside a whole-document TF-IDF score and a document-shape feature. The real trick is a set of quadratic cross-channel interaction terms. Take a pair with near-identical YAML, moderately similar instructions, and zero code overlap — because the code got refactored into separate sub-files. That pair still scores as a strong clone detection hit instead of getting diluted by the empty channel. A four-branch decision tree then sorts each detected pair into a classical clone type, with the sharpest focus on Type-4 semantic clones : same functional intent, different words, different code. Validation runs on SkillClone-Bench, a 300-pair benchmark built without manual labeling — half the positive pairs from verified GitHub fork lineages, half from seven controlled mutation operators, and a negative set deliberately stacked with same-category pairs that share vocabulary without being clones.

SkillProbe is a three-stage, multi-agent auditing pipeline built for agent skill marketplaces like ClawHub. First, a Gatekeeper stage filters for compliance violations and known-malicious patterns. Then an Alignment Detector checks semantic-behavioral alignment — whether a skill's declared behavior actually matches what its code does — sorting each skill into a four-way match / over-declaration / under-declaration / mixed classification. The final stage, a Flow Simulator, models combinatorial risk : individually benign skills chaining into dangerous behavior once an agent invokes several together. Brute-forcing every possible combination would mean an O(2^N) search, so the simulator instead matches against a fixed rule set, keeping cost linear in catalog size — O(N × number of rules). The whole pipeline is itself packaged as skills that drive specialized auditing agents, a design the authors call " Skills-for-Skills ." The team ran it across 2,500 real skills pulled from ClawHub, then separately re-audited the top 20 most-downloaded skills with 8 different LLM series to see how verdicts shift depending on which model is doing the judging.

Researchers crawled 238,180 unique agent skills from ClawHub, Skills.sh, SkillDirectory, and GitHub — the largest cross-platform census of the skill ecosystem so far. Every skill passed through three lenses: each marketplace's own scanner, an independent Cisco skill scanner, and the authors' own LLM-based classifier. For skills flagged high-risk by both an external scanner and their model, the team went a step further and checked the skill against the GitHub repository it actually ships from — README, code, commit history — rather than judging the SKILL.md file in isolation.

The pipeline mines open-source agentic repositories in three stages: structural analysis of a repo's layout, dense retrieval to find reusable procedural capabilities buried in the code, and translation of what it finds into the standardized SKILL.md format . Run against two Manim-based educational video generators — TheoremExplainAgent's planner-coder pair and Code2Video's three-agent planner-coder-critic setup — it produces two working skills: a theorem-walkthrough animator and a visual-layout critic. Neither ships without clearing a four-gate check first: static code analysis, semantic prompt injection screening, sandboxed behavioral testing, and permission validation against an allowed-tools manifest.

Clawdrain is a trojanized OpenClaw skill that hides a multi-turn "Segmented Verification Protocol" (SVP) inside its SKILL.md instructions. Before it hands back a simple BBC-headline lookup, the skill demands a growing, comma-separated "calibration sequence" — and a companion script grades every submission PROGRESS, REPAIR, or TERMINAL. REPAIR forces a retry of the same step, so the exchange stretches into extra tool calls that get replayed back into context on every subsequent turn.

SkillFortify bills itself as the first formal-methods framework for agent skill supply chains: six components, each carrying a proof instead of a heuristic. Start with the DY-Skill attacker model , a Dolev-Yao adaptation spanning the skill lifecycle -- install, load, configure, execute, persist -- proven maximal, meaning no stronger symbolic adversary needs modeling. A static analyzer built on abstract interpretation checks skill code against a capability lattice : four authority levels (none, read, write, admin) crossed with eight resource types -- filesystem, network, environment, shell, skill-invoke, clipboard, browser, database. Its soundness proof means a clean report isn't a best guess; it's a bound on what the code can actually do at runtime. Capability-based sandboxing, formalized as an object-capability model , comes with its own proof that it blocks 'authority amplification' -- a child skill inheriting permissions its parent never had. A dependency graph resolves through SAT solving into a deterministic lockfile, and a trust score algebra propagates trust multiplicatively down dependency chains, decaying it for skills that go unmaintained, with a proven monotonicity property. All six run against SkillFortifyBench, a 540-skill benchmark -- 270 malicious, 270 benign -- assembled from real attack campaigns.

An agentic skill, formalized here as a four-part contract S = (C, π, T, R), needs all four pieces to count: an applicability condition that decides when it fires, an executable policy, a termination condition, and a callable interface. That's what separates a skill from a raw tool call (no reuse contract), a one-off plan (no persistence), or stored memory (no execution policy) — three things builders already reach for and easily mistake for skills. The definition anchors a lifecycle — discovery, practice, distillation, storage, composition, evaluation, update — plus two taxonomies for what's already shipping. Seven system-level design patterns cover how skills get packaged and run, from metadata-first progressive disclosure and executable code skills through workflow-enforcing gates, self-evolving skill libraries , hybrid natural-language-plus-code macros, meta-skills, and marketplace-distributed plugins. A second, orthogonal grid crosses representation (natural language, code, policy, hybrid) against operating scope (web, OS, software engineering, robotics).

Researchers built SkillInject, a benchmark of 202 injection-task pairs that measures skill-based prompt injection — adversarial instructions planted inside agent skills , the third-party extensions that hand an LLM agent new tools, knowledge, and code paths. The attacks run the full spectrum, from blatantly malicious injected commands to subtle instructions folded into otherwise legitimate skill content, built to blend in rather than stand out. Each test pairs an injection with a genuine task and scores the agent on two axes at once: whether it dodges the harmful instruction, and whether it still finishes the legitimate one.

SkillJect is an automated framework for manufacturing poisoned agent skills, and it works two channels at once instead of leaning on a single hand-crafted trick. The artifact channel buries the actual malicious payload inside an auxiliary helper script bundled alongside the skill. The instruction channel rewrites SKILL.md around it, using a front-loaded inducement strategy : right at the top of the file, the helper script gets framed as a mandatory setup step, complete with a plausible file path and an executable example command, rather than as an optional or suspicious add-on. Tuning that framing is a closed-loop multi-agent process — a Victim Agent runs the skill on a real task, an Evaluate Agent reads the execution trace to judge whether the payload fired, was ignored, refused, or half-triggered, and an Attack Agent rewrites the instruction text, never the payload itself, for another pass.

A skill loads in three strict stages, and that staging is the whole point. A SKILL.md file's YAML frontmatter — name and description, a few dozen tokens — sits in the system prompt from startup. The full instruction body loads only once a task actually triggers the skill, and bundled scripts or reference docs load only when those instructions call for them. This progressive disclosure pattern is why an agent can hold a library of thousands of skills without paying a context-window tax for the ones it never uses — and it's the real difference between a skill and an ordinary tool call. A tool executes and returns a result; a skill rewrites what the agent knows and is allowed to do before it responds at all. The survey frames skills and MCP as two halves of one stack, skills carrying procedural know-how and MCP carrying connectivity to external systems, then works through how skills actually get built. Some are hand-authored SKILL.md files. Others grow inside a reusable library through reinforcement learning during training rollouts, or get discovered on the fly by agents exploring software nobody wrote instructions for.

Agent Skills — the Markdown files that tell an LLM agent which tools exist and how to use them — get rendered to HTML for human review. The model itself usually reads the raw Markdown instead, verbatim. Researchers at Shandong University built an attack around that gap: a hidden-comment prompt injection , a single HTML comment carrying high-priority instructions, appended to an otherwise clean, benign Skill. HTML comments disappear in rendered output — a human reviewer sees nothing unusual, but the model still gets the hidden text as context. They tested it with a plain developer request, 'format my code,' against two open models, DeepSeek-V3.2 and GLM-4.5-Air, comparing the clean Skill, the poisoned Skill, and the poisoned Skill paired with a short defensive system prompt.

Researchers ran 98,380 agent skills pulled from two community registries, skills.rest and skillsmp.com, through a four-stage detection funnel: static pattern matching against 14 rules tied to a six-phase kill chain , dynamic behavioral verification inside sandboxed containers, then manual dual-rater review (Cohen's κ=0.89) and vulnerability labeling. Static matching alone flagged 4,287 candidates. The full pipeline confirmed 157 as malicious — 632 distinct vulnerabilities spread across 13 attack techniques — with the dynamic-verification stage alone hitting 99.6% precision, a 90x jump over pattern matching by itself.