97.6% of Injection and Poisoning Caught, Only 90.2% When Skills Interact

Notes on Sealing the Audit-Runtime Gap for LLM Skills (arXiv:2605.05274) — Tingda Shen, Yebo Feng, Konglin Zhu, Xiaojun Jia, Yang Liu, Lin Zhang · May 2026

Note published July 23, 2026 · written by SkillFed’s research pipeline from the paper above · how these notes are made

AI-assisted notes · reviewed by SkillFed Skill security

SIGIL treats skill security as a lifecycle problem, not a one-time gate. Every approved skill gets anchored in a tamper-evident on-chain registry, published under one of four modes — Transparent (free plaintext), Licensed (paid access), Sealed (custodial use), or Committed (off-chain content, on-chain hash). Admission runs through a DAO audit committee: pluggable audit methods (static analysis, LLM review, sandbox checks) operating under a stake-and-slash economic model that penalizes bad calls. Every later load then passes through a Skill Verification Loader (SVL), which re-checks the fetched skill against the exact record the auditors signed off on and enforces its permission manifest before any content reaches the model's context.

Tested against 1,023 real skills pulled from ClawHub and other public registries across six attack categories, SIGIL holds up best exactly where its on-chain binding is aimed. Explicit injection and implicit poisoning land at 97.6% accuracy, local tampering at 100%; rug-pull attacks — the closest analog to a supply-chain compromise — come in at 95.1%. The floor is cross-skill interaction attacks at 90.2%, where individually clean skills combine into an unsafe outcome: the hardest category by design, since no single skill's record looks wrong on its own. The DAO ensemble beats any single audit method run alone and holds its accuracy even with 20–40% of auditors colluding. Verification itself is cheap: under 86 ms of batched checking at load time, and auditing's own token cost stays under 3% of a typical monthly LLM subscription's quota.

Key numbers

Real-world skills tested	1,023 skills, 6 attack types
Injection & poisoning detection	97.6% accuracy
Cross-skill interaction detection (weakest category)	90.2% accuracy
Batched load-time verification	under 86 ms
Auditing's own token cost	under 3% of a $20/mo quota

What this means for skill authors

The real contribution here isn't the blockchain — it's naming the gap plainly. An audit report is a claim about a piece of content at one moment; a runtime fetch is a separate event with no built-in obligation to match it. Most of today's skill supply chain defenses pick one side of that gap: signing covers distribution but not content drift after approval, and policy engines at runtime have no record of what was actually vetted. Forcing every load through a verifier that checks the fetched artifact against the same record the auditors signed off on is exactly the fix a rug-pull needs — and a one-time review can't provide it.

The 90.2% floor on cross-skill interaction attacks is the number worth sitting with, not the headline 97.6%. That's the attack class where the danger lives in how two individually safe skills interact rather than in either skill's own file — precisely the blind spot single-skill vetting, dual-tool or otherwise, is built to miss. SIGIL's answer, intersecting permission manifests across concurrently loaded skills, narrows that gap; it doesn't close it, and the paper's own numbers say so.

The transferable lesson for any vetted catalog isn't "run a DAO" — it's that an audit record divorced from the runtime artifact is a liability no matter how good the audit was. A scan that runs once at intake and can't be re-checked when a skill is actually loaded leaves exactly the gap this paper names. Treating "the skill you vetted" and "the skill you're running" as a claim that has to be verified, not assumed, is the design habit worth carrying into any skill-vetting pipeline, SkillFed's included.

Skills related to this research

Committee Committee orchestrates parallel analysis across three AI models on unified market datasets, each delivering independent investment perspectives before cross-validation and consensus extraction. The skill structures data preparation, round-one opinions, optional verification rounds, and final recommendation synthesis with confidence weighting and risk flagging.★ 99 ios-debugger-agent ios-debugger-agent automates the process of building, launching, and running your current iOS project on an active simulator instance. This skill eliminates repetitive setup steps, letting you focus on development rather than tooling. Ideal for developers seeking faster iteration cycles during iOS app testing and debugging.★ 3,861 architecture-decision This skill guides you through authoring Architecture Decision Records (ADRs) that capture major technical choices with full context. It validates decisions against existing architectural stances, engine compatibility, and GDD requirements, then generates a structured record with status, dependencies, and consequences tracked. Use it to retrofit missing sections into existing ADRs or create new ones from scratch.★ 23,424 create-control-manifest This skill transforms accepted architecture decision records into a single, layer-organized reference guide that tells programmers exactly what to do and what to avoid. It pulls rules from ADRs, technical preferences, and engine docs, then surfaces them by Foundation, Core, Feature, and Presentation layers—answering "what do I do?" faster than reading the reasoning behind each decision.★ 23,424

Related notes

References

Shen, Feng, Zhu, Jia, Liu, Zhang. "Sealing the Audit-Runtime Gap for LLM Skills." arXiv:2605.05274 (2026).
Jamshidi et al. "Securing the Model Context Protocol: Defending LLMs against tool poisoning and adversarial attacks." (2025).
Beurer-Kellner et al. "Snyk Finds Prompt Injection in 36%, 1467 Malicious Payloads in a ToxicSkills Study of Agent Skills Supply Chain Compromise." (2026).
Holzbauer et al. "Malicious Or Not: Adding Repository Context to Agent Skill Classification." arXiv:2603.16572 (2026).