97.6% of Injection and Poisoning Caught, Only 90.2% When Skills Interact
Notes on Sealing the Audit-Runtime Gap for LLM Skills (arXiv:2605.05274) — Tingda Shen, Yebo Feng, Konglin Zhu, Xiaojun Jia, Yang Liu, Lin Zhang · May 2026
Note published · written by SkillFed’s research pipeline from the paper above · how these notes are made
AI-assisted notes · reviewed by SkillFed Skill securitySIGIL treats skill security as a lifecycle problem, not a one-time gate. Every approved skill gets anchored in a tamper-evident on-chain registry, published under one of four modes — Transparent (free plaintext), Licensed (paid access), Sealed (custodial use), or Committed (off-chain content, on-chain hash). Admission runs through a DAO audit committee: pluggable audit methods (static analysis, LLM review, sandbox checks) operating under a stake-and-slash economic model that penalizes bad calls. Every later load then passes through a Skill Verification Loader (SVL), which re-checks the fetched skill against the exact record the auditors signed off on and enforces its permission manifest before any content reaches the model's context.
Tested against 1,023 real skills pulled from ClawHub and other public registries across six attack categories, SIGIL holds up best exactly where its on-chain binding is aimed. Explicit injection and implicit poisoning land at 97.6% accuracy, local tampering at 100%; rug-pull attacks — the closest analog to a supply-chain compromise — come in at 95.1%. The floor is cross-skill interaction attacks at 90.2%, where individually clean skills combine into an unsafe outcome: the hardest category by design, since no single skill's record looks wrong on its own. The DAO ensemble beats any single audit method run alone and holds its accuracy even with 20–40% of auditors colluding. Verification itself is cheap: under 86 ms of batched checking at load time, and auditing's own token cost stays under 3% of a typical monthly LLM subscription's quota.
Key numbers
| Real-world skills tested | 1,023 skills, 6 attack types |
| Injection & poisoning detection | 97.6% accuracy |
| Cross-skill interaction detection (weakest category) | 90.2% accuracy |
| Batched load-time verification | under 86 ms |
| Auditing's own token cost | under 3% of a $20/mo quota |
Skills related to this research
Related notes
- 84% of Malicious Agent-Skill Exploits Hide in the Docs, Not the Code →
- A skill's own examples smuggle malicious code past agent defenses up to 33.5% of the time →
- Frontier Agents Cave to Poisoned Skill Files Up to 80% of the Time →
- Debug Logging Alone Causes 73.5% of Agent-Skill Credential Leaks →
- More Than Half of 13,728 Real-World Skills Hide a Critical Risk in Plain Prose →
- Best Skill Scanner Catches 17% of Cross-Modal Attacks — a Distilled 7B Model Hits 88% →
- A fine-tuned 8B retriever hits 83 NDCG@10 — a 12B off-the-shelf model manages 55 →
- One shared reward signal pushes skill-augmented agents to 97.5% success, beating every tested baseline — skill-augmented or skill-free — by up to 20 points →
- A trained 8B model beats Gemini-2.5-Pro at curating agent skills: 61.2% vs 50.7% →
References
- Shen, Feng, Zhu, Jia, Liu, Zhang. "Sealing the Audit-Runtime Gap for LLM Skills." arXiv:2605.05274 (2026).
- Jamshidi et al. "Securing the Model Context Protocol: Defending LLMs against tool poisoning and adversarial attacks." (2025).
- Beurer-Kellner et al. "Snyk Finds Prompt Injection in 36%, 1467 Malicious Payloads in a ToxicSkills Study of Agent Skills Supply Chain Compromise." (2026).
- Holzbauer et al. "Malicious Or Not: Adding Repository Context to Agent Skill Classification." arXiv:2603.16572 (2026).