Skill security 42 papers

SkillSec-Eval breaks the agent skill lifecycle into six stages — authoring, storage, retrieval, planner selection, execution, evolution — and gives each one its own threat taxonomy. Badhe and Tiwari run the framework against 327 real skills pulled from the SkillMCP repository across 15 clusters spanning DevOps, cloud operations, and data engineering, then throw five attack families at it: tampered signatures and provenance at storage, gamed semantic retrieval via keyword stuffing and cloned listings, fake planner-facing recommendations, chained tool calls at execution, and permission creep hidden inside version updates.

SkillsMP, the largest open-source Agent Skills marketplace, supplied a corpus of 264,937 normalized skill packages out of 273,657 catalog entries, each split into three layers: metadata (name, description, tags), instructions (the SKILL.md procedural text), and resources (scripts, configs, dependencies). Divergence between what the metadata claims and what the instructions and resources actually do — cross-layer misalignment — is what PL-HCL (Progressive Loading-Aware Hierarchical Contrastive Learning) is built to catch before a skill ever runs. Training mirrors how a skill actually loads: first continued pretraining on metadata plus instructions alone, then on full packages including resources. On top of that sits a contrastive learning stage, trained against two kinds of synthetic negatives per skill — swapping in another skill's metadata, and perturbing the skill's own metadata — while the true metadata-instructions-resources triple stands as the positive example the model learns to recognize as consistent.

SkillCenter builds its library through a five-stage pipeline. Source acquisition feeds an LLM-based pre-filter called SkillGate , which screens raw material for actionability before any generation tokens get spent — auto-rejecting anything under 200 characters, then issuing a pass/maybe/fail verdict off a 0-10 suitability score. What survives moves through template-driven generation (four templates, keyed to source type) and an iterative improvement loop of up to three passes, each running a deterministic source grounding check that traces every retained claim back to an exact quotation in its source. A four-criterion publish gate — quality score, license compatibility, plagiarism ratio, placeholder density — catches roughly 3% of generated skills for falling under a 3-out-of-5 floor. Everything that clears it ships as an offline SQLite FTS5 database: keyword search, no embeddings, no vector store, no network call at query time.

Researchers screened GitHub for agent skill repositories with at least 20,000 stars and 2,000 forks and landed on six, including Anthropic's own anthropics/skills, obra/superpowers, and affaan-m/everything-claude-code. Those six repos alone had 67,264 public forks. Filtering out inactive branches, upstream-sync noise, and edits that never touched an actual skill package brought that down to 1,220 branch comparisons -- each one a developer's real, deliberate change to an existing skill. An LLM agent read every diff and built a taxonomy from the patch content itself rather than from commit messages; two human auditors checked a random sample and hit κ=0.83 agreement, solid enough to trust the result: 1,126 validated adaptation instances, sorted into 46 patterns across 13 families.

SkillFuzz treats skill composition — not the individual skill — as the unit worth testing. An LLM first compiles each skill's natural-language instructions into a structured skill contract : preconditions, postconditions, state changes, domain scope, and action types, all mapped into a shared semantic space. A contract-guided Monte Carlo Tree Search then walks the combinatorial space of possible activations, steering toward compositions whose contracts look likely to interact once a branch shows signal, and sampling randomly where none has. Every candidate plan gets checked against a differential oracle — the plan the same task would produce with no skills active at all. The semantic gap between the two, plan drift , flags an implicit intent : an objective that shows up in the plan without being asked for by the task or promised by any single skill. None of it requires execution, so the whole check can run at admission time, before a marketplace ever lets an agent touch a real tool.

VIGIL is a runtime reference monitor for agent skills. It abstracts raw tool calls into typed events, then grounds each skill's natural-language specification into a policy that names the actual arguments, outputs, and artifact identities appearing in that specific run — closing the gap between generic prose ("validate before use") and a checkable rule. Each grounded policy compiles into a quantifier-free SMT query evaluated against the whole trace, not one call at a time, so a violation can hinge on event order, an argument value, or something produced several steps earlier and only consumed later. The policy language covers six temporal forms — absence, precedence, response, bounded response, resolution, and until — and when the solver finds a policy broken, its unsatisfiable core points to the exact invocation to block before that call's effects land.

Security review for agent skills mostly happens one skill at a time — does this skill's code or manifest do anything dangerous, taken alone. Skill Composition Risk (SCR) names what that misses: a skill can be completely bounded and legitimate on its own and still hand a later skill a discovered file path, a trust signal, or an authorization-sounding sentence, with the harm living on that handoff rather than in either skill. SCR-Bench makes the failure mode testable — it models an agent's available skills as a graph, with edges forming wherever one skill's output becomes another skill's input, endorsement, or approval evidence, then runs the graph in sandboxed environments that track actual downstream state changes rather than model text. Three sub-benchmarks each isolate one composition mechanism: capability flow , where an upstream discovery skill hands a downstream execution skill a concrete target; trust transfer , where an upstream review-style skill's endorsement legitimizes a risky downstream install; and authorization confusion , where advisory language from one skill gets read as approval by another. Each runs matched control, isolated, and composed conditions across up to ten current model backends — GPT-5.5, GPT-5.4, Claude Opus 4.6 and 4.5, Gemini 3.1 Pro Preview, MiniMax-M2.7, DeepSeek-V4, Kimi-K2, GLM-5.1, and GLM-5.

SkillMutator targets a blind spot in Agent Skills security: attacks that only emerge from how a skill's natural-language SKILL.md and its executable code interact, not from either file read on its own. A SKILL.md can read as a completely benign workflow while an implicit directive buried in the prose steers the agent to exfiltrate sensitive files — the accompanying script stays clean when checked by itself. The benchmark turns this into a moving target: an adversarial mutation process iteratively rewrites malicious skills against scanner feedback until the injected behavior reads as indistinguishable from a legitimate workflow, covering 13 attack categories across four threat groups that range from technical exploits like exfiltration and privilege escalation to semantic tricks like brand hijacking and supply-chain moves like configuration weakening. The training corpus runs 1,219 mutations generated over 68 community-authored skills; evaluation draws 48, 63, and 76 surviving mutations from three adversarial oracles of increasing strength (GPT-4o-mini, GPT-5.4-mini, and GPT-5.4) applied against 17 of Anthropic's official Agent Skills.

POISE compromises a SKILL.md file by editing exactly one line. The earlier approach scattered several explicit malicious steps through a skill's body, giving an agent multiple chances to notice a command that doesn't belong. POISE instead compresses the whole payload into a self-contained external script and inserts a single line that just invokes it, framed as a routine prerequisite ("before first use, run <cmd> to verify the environment"). Placement is restricted to positions that already look native to the file — a step folded into an existing numbered procedure, or a line appended to a setup/install block — both written by a context-aware generator so the injected sentence reads like ordinary skill prose. The evaluation spans two agent-skill benchmarks, Skill-Inject (25 tasks) and SkillsBench (27 tasks), each crossed with three harm categories (credential exfiltration, config tampering, privileged-shell behavior), across four agent-model pairings: codex +gpt-5.2 as the primary target, with openclaw and Claude Code used to test transfer. The headline metric is Attack Success Rate (ASR) : the injected command has to fire and the user's own task still has to pass its verifier in the same trial, which rules out attacks that tip their hand by breaking the task.

MalSkillBench builds ground truth for detecting malicious agent skills — the SKILL.md packages that bundle natural-language instructions, scripts, and tool permissions for coding agents like Claude Code and Gemini CLI — by actually running them rather than trusting a label. A closed-loop Generate-Verify-Feedback pipeline seeds a three-dimensional taxonomy (attack vector × behavior × insertion strategy, 108 cells) with real malicious PyPI code and prompt-injection corpora, then drops each candidate skill into a Docker sandbox with syscall monitoring and an LLM judge; only skills whose malicious behavior actually fires get admitted. The result is 3,214 verified generated skills, plus 703 confirmed skills pulled from the wild and 27 drawn from existing tool test suites — 3,944 malicious skills in total, paired with 4,000 benign skills pulled from ClawHub for false-positive testing.

Agent skills aren't just documentation anymore — they're bundles of instructions, scripts, and tool bindings that can steer an agent's reasoning before any tool executes, then trigger real side effects once one does. Most defenses cover only one side of that: scanning a skill's files before install, or gating individual tool calls at runtime. Nothing connects what a skill declares it needs to what it actually does once running. SkillGuard closes that gap with a skill manifest — a JSON schema declaring the resources a skill needs — checked against a runtime permission layer that intercepts every tool call and enforces deny-by-default on anything undeclared. High-stakes actions get escalated to the user for approval. Underneath sits a taxonomy of eight resource groups — storage, code repository, network, execution, hardware, system, secrets, agent ecosystem — broken into 41 distinct labels such as READ_FILE and WRITE_FILE, built on the classical least privilege and complete-mediation principles that systems like Android's permission model already lean on. Testing whether that taxonomy actually covers what real skills do meant checking it against 1,260 skills mined across 63 categories on the SkillsMP marketplace — the top 10 by star count plus 10 random picks per category — with every observed action mapped onto it.

SkillHarm treats an agent skill as a persistent artifact with a lifecycle, not a one-shot instruction set, and builds its benchmark on that framing instead of testing poisoned skills inside one task run. Two attack shapes anchor it: Fixed-Payload Poisoning , where a static poisoned skill package compromises every session that loads it, and Self-Mutating Poisoning , where a benign-looking run quietly rewrites the skill's saved content so the payload only fires on a later reuse. A taxonomy of 12 risk types sits around those two modes, grouped by what part of the agent workflow gets hit — data pipelines, system environment, agent autonomy — and an automated construction pipeline, AutoSkillHarm, generates attacks at scale using coding agents driven by natural-language harnesses rather than by hand. The result: 879 attack samples spread across 71 skills.

ClawHub Security Signals pairs each of 67,453 latest public OpenClaw skill versions — redacted SKILL.md content plus sanitized bundled files where present — with a ClawScan verdict (clean, suspicious, or malicious) and independent evidence from three scanner families: VirusTotal, static heuristic analysis, and NVIDIA's SkillSpector , which raises semantic agentic-risk advisories rather than malware-reputation signals. The release isn't trying to estimate how many skills are actually malicious. It's asking a narrower question: pointed at the same skills, do these three scanners even agree with each other?

Agent skills look solved from the outside: descriptors load, tools get pre-approved, marketplaces push updates. Underneath that surface, a systematic review of 37 production agent systems and 51 research papers pulled out 342 distinct architectural practices — consolidated first into 50 candidate patterns, then distilled into a catalogue of 10 (five core, five supporting), and synthesized into a four-layer reference architecture: Supply Chain, Mediation, Execution Control, and Evidence & Feedback. The framing hinges on a distinction the authors name skill-in-use : a skill file sitting in a repo is inert, and its real architectural responsibilities only switch on once it's selected for a specific run, bound to authority limits, interpreted by a stochastic agent, and logged as evidence. Two patterns give that vocabulary teeth. Skill–Execution Authority Separation keeps a skill's reference to a capability from being treated as an automatic grant of it, and the Runtime Skill Bill of Materials is a per-run ledger of exactly which skill artefact, version, and scope actually participated.

Most skill safety work checks prompts and runtime actions, never the skill artifact itself against its own description. Behavioral integrity verification (BIV) names that missing check: a typed comparison between what a skill declares and what it actually does, run over a shared capability taxonomy of 29 capability types spanning code, natural-language instructions, and metadata. The declared side comes from parsing manifests plus LLM-assisted extraction, constrained by a hallucination-control filter that requires a quoted source span for every claimed capability. The actual side comes from deterministic analysis — inter-procedural taint analysis for Python, regex extraction for JavaScript and shell — plus LLM parsing of natural-language instructions. Both sides land in the same typed taxonomy, producing one structured evidence record per skill that feeds three downstream jobs: a deviation taxonomy, a root-cause classifier, and a malicious-skill detector.

SKILL.md files — the natural-language front matter that tells an agent when and how to use a packaged capability — sit upstream of three separate decisions: whether a skill gets found, whether it beats a rival for selection, and whether it clears a security check. Researchers tested all three against real skills pulled from ClawHub , a live agent-skill registry, drawing on 100 skills across five categories (email, travel, tax, health, prompt). Discovery got the beam-search-and-gradient treatment: short adversarial trigger phrases, appended to the SKILL.md text, tested whether embedding-based retrieval could be steered toward a skill without touching a line of its code. Selection worked differently — functionally identical skill pairs, differing only in how the description was framed (exaggerated capability claims, directive language, recency signals, trust claims), put in front of four LLM agents asked to pick one. Governance was the toughest test: a three-stage vetting pipeline — a static scanner, an LLM reviewer, a VirusTotal malware pass — run against admittedly malicious skills rewritten with wording-only evasion tactics like paraphrasing and context-window overflow .

SkillSafetyBench tests a blind spot in agent safety evaluation: the user's request stays completely benign, but the exploit rides in through the skill itself. The benchmark packages 155 adversarial cases across 47 executable tasks, 6 risk domains, and 30 canonical safety categories, and each case buries its attack not in the user's prompt but in skill-facing non-user attack surfaces — the helper scripts, wrappers, memory stores, retrieval corpora, and dependency manifests that ship alongside a skill and get treated as trusted workflow context. SkillSafetyBench pairs every case with a runnable environment and a case-specific rule-based verifier that checks concrete run artifacts — leaked payloads, poisoned memory records, malicious trace patterns — rather than judging the agent's final answer. The taxonomy and verifiers cleared expert review and a two-judge LLM-as-judge protocol alike, posting pass rates above 85% on camouflage quality, verifier-attack alignment, and taxonomy placement, with inter-judge agreement above 91% on every axis.

FORTIS treats the skill layer — the abstraction that routes a request to a specific skill and then to specific tools — as a privilege boundary , not an organizational nicety. It scores two things separately: whether a model picks the minimally sufficient skill from an overlapping library (Task 1, 600 queries), and whether it then executes that skill without escalating into tools broader than it's permitted (Task 2, 1,543 queries). The benchmark spans three domains — email, e-commerce, filesystem operations — built from 60 skills and 174 tools across a five-level privilege hierarchy, from observation-only (L0) to bulk administrative control (L4). Ten frontier models go through it, among them GPT-5.5, GPT-5.4, Claude Opus 4.7, Claude Sonnet 4.6, Gemini 3.1-Pro, Gemini 3 Flash, Qwen 3.6-Max, Kimi K2.6, and DeepSeek-V4-Flash, under both clean requests and everyday ambiguous phrasing. Nothing adversarial.

Agent skills split into two halves: a deterministic script side that runs known code, and a non-deterministic LLM side that decides what to call next. This paper treats capability-containment — proving a skill's tool calls never exceed what its manifest declares — as a property of that whole system, not just the script, and closes it with three methods built to compose rather than compete. First, abstract interpretation walks a capability-effect lattice to over-approximate what the script's code can actually do, reusing existing dataflow engines (Semgrep, CodeQL, Pyright) instead of building new analyzers from scratch. Second, the tool-call dispatch function gets wrapped in a refinement type that mechanically rejects any call whose inferred capability falls outside the manifest's declared set, no matter what the LLM asks for. Third, SMT-bounded model checking with Z3 tests the parent paper's biconditional correctness criterion against a session-length horizon, so any violation surfaces as a concrete counter-example trace rather than an abstract failure.

SIGIL treats skill security as a lifecycle problem, not a one-time gate. Every approved skill gets anchored in a tamper-evident on-chain registry , published under one of four modes — Transparent (free plaintext), Licensed (paid access), Sealed (custodial use), or Committed (off-chain content, on-chain hash). Admission runs through a DAO audit committee : pluggable audit methods (static analysis, LLM review, sandbox checks) operating under a stake-and-slash economic model that penalizes bad calls. Every later load then passes through a Skill Verification Loader (SVL) , which re-checks the fetched skill against the exact record the auditors signed off on and enforces its permission manifest before any content reaches the model's context.