Zero of 8 production agent systems verify skill-in-use

Notes on Harnessing Agent Skills: Architectural Patterns and a Reference Architecture for Skill-Mediated LLM Agents (arXiv:2606.20631) — Boming Xia, Liming Zhu, Zhenchang Xing, Qinghua Lu, Dino Sejdinovic, Xiwei Xu · May 2026

Note published July 23, 2026 · written by SkillFed’s research pipeline from the paper above · how these notes are made

Agent skills look solved from the outside: descriptors load, tools get pre-approved, marketplaces push updates. Underneath that surface, a systematic review of 37 production agent systems and 51 research papers pulled out 342 distinct architectural practices — consolidated first into 50 candidate patterns, then distilled into a catalogue of 10 (five core, five supporting), and synthesized into a four-layer reference architecture: Supply Chain, Mediation, Execution Control, and Evidence & Feedback. The framing hinges on a distinction the authors name skill-in-use: a skill file sitting in a repo is inert, and its real architectural responsibilities only switch on once it's selected for a specific run, bound to authority limits, interpreted by a stochastic agent, and logged as evidence. Two patterns give that vocabulary teeth. Skill–Execution Authority Separation keeps a skill's reference to a capability from being treated as an automatic grant of it, and the Runtime Skill Bill of Materials is a per-run ledger of exactly which skill artefact, version, and scope actually participated.

Cross-checking the reference architecture against eight production systems — Claude Code, GitHub Copilot, Microsoft Agent Framework, OpenAI Codex, OpenClaw, Hermes Agent, Aeon, and Snowflake Cortex Agents — shows exactly where the industry's engineering effort has landed, and where it hasn't. Every one of the eight fully realizes skill representation and storage; six of eight fully realize activation and context loading. The parts of the pipeline that get a skill into a run are mature and largely convergent across vendors. Evidence & Feedback tells the opposite story: across its three responsibility blocks and eight systems (24 cells total), only three come back fully realized, concentrated in just two systems (Aeon and Snowflake). Independent verification that a skill's guidance was actually followed — as opposed to merely invoked — never comes back fully realized anywhere; one system gets partial credit, the rest show no foregrounded evidence of it at all. The cross-instantiation catches something else too: vendor vocabulary conflates distribution with admission control. Install and update get described as if they gate capability, but the two diverge in half the systems studied, and capability authorization diverges from runtime action gating in three of eight.