Compiling a skill for its model drops regressions from 15% to 4.5%

Notes on SkVM: Revisiting Language VM for Skills across Heterogenous LLMs and Harnesses (arXiv:2604.03088) — Le Chen, Erhu Feng, Yubin Xia, Haibo Chen · April 2026

Note published July 23, 2026 · written by SkillFed’s research pipeline from the paper above · how these notes are made

AI-assisted notes · reviewed by SkillFed Frontier & other

Scale first: two public catalogs hold 118,000 agent skills between them — 28,990 on clawhub.ai, 89,280 on skills.sh. Running that catalog against eight LLMs and three harnesses (BareAgent, OpenCode, OpenClaw) on SkillsBench-style tasks turns up a blunt pattern: a skill isn't actually a portable unit. It's raw context that one model's harness happens to parse correctly and another's doesn't. SkVM's answer borrows from compiler theory — treat the skill as source code, the LLM as a heterogeneous processor, and decompose what the skill demands into primitive capabilities: 26 of them, spanning code generation, reasoning, tool execution, and procedure-following, each gated by a proficiency level. A one-time profiling pass scores how well a given (model, harness) pair supports each primitive, and that profile drives capability-based compilation, environment binding, and concurrency extraction before the skill ever runs — plus JIT code solidification and adaptive recompilation while it does.

The failure rates are the headline. An unmodified skill actively regresses performance on 15% of tasks, and across the eight models tested, 87% of tasks return zero benefit for at least one of them — a skill tuned to shine on your own model can sit inert, or do active harm, somewhere else in the fleet. Missing environment dependencies are their own failure mode: completion rates fall to 33-67% for weaker models, and even strong models burn 2-4x more tokens working around gaps a setup script could have closed. Compiling with SkVM pulls all of this down hard. Task completion improves by an average of 15.3%. Regressions fall from 15% of tasks to 4.5%. Token consumption drops by up to 40% on capable models paired with richer harnesses, and cross-harness variance shrinks from a 13-point spread to at most 5. The runtime payoff lands hardest on repeated, parameterizable work: code solidification cuts latency 19-50x on tasks like PDF extraction (low tens of seconds down to a few hundred milliseconds), and extracting parallelism from a skill's own steps adds up to a 3.2x speedup on top.

Key numbers

Public skills audited across two catalogs	118,000
Tasks where an unmodified skill regresses	15%
Tasks with zero benefit for at least one of 8 models	87%
Token reduction after compilation (capable models)	up to 40%
Latency cut via code solidification	19–50x

What this means for skill authors

The 87%-of-tasks number is the one worth sitting with. It means testing a skill against a single model tells you almost nothing about how it behaves on the seven you didn't test — the skill format carries no guarantee of behavior, only of syntax. That's a portability failure most skill authors can't currently see, because nobody ships a way to check for it before publishing.

It also reframes what a skill catalog owes its users. Correctness testing against one reference setup is necessary, not sufficient — the real risk surface is the (skill, model, harness) triple, and a skill can pass every functional check on the author's own stack while quietly doing nothing, or worse, on someone else's. SkillFed's dual-tool scanning targets catalog trust rather than execution-time compilation, but the diagnosis generalizes past SkVM's specific runtime: a passing scan on one setup is not a portability guarantee for every setup the skill will actually run on.

Profiling costs minutes and cents per (model, harness) pair, cached and reused after that first pass — cheap enough that catalogs, not just individual runtimes, could eventually publish a capability profile alongside a skill, instead of leaving every downstream agent to rediscover the same mismatches on its own.

Skills related to this research

ios-storyboard This skill equips AI agents to work directly with iOS storyboard files, enabling updates and restructuring of interface layouts within healthcare applications. It integrates with the Agent Skills open format to provide domain-specific capabilities for managing UI components and screen configurations in clinic-focused iOS development.★ 182 swift-ui-architect swift-ui-architect equips AI agents with the expertise to design and implement robust SwiftUI applications using proven architectural patterns. The skill enforces separation of concerns through MVVM-C layering, Swift Package Manager organization, and coordinator-based navigation, enabling agents to generate production-ready iOS codebases that scale with team complexity.★ 182 compose-slot-api-pattern This skill shows how to build flexible, reusable Compose components by delegating content to callers through `@Composable` lambda slots instead of accumulating primitive parameters. You'll learn to structure components around layout responsibility, use scope receivers to expose layout features, and co-locate sensible defaults in a `Defaults` object—the pattern Material 3's `ListItem` exemplifies.★ 1,565 mobile-android-design Learn to craft responsive Android interfaces using Jetpack Compose and Material Design 3 (Material You). This skill covers adaptive layouts for phones, tablets, and foldables, dynamic color theming, and accessibility best practices.★ 38,308

Related notes

References

SkVM: Revisiting Language VM for Skills across Heterogenous LLMs and Harnesses (Chen, Feng, Xia & Chen, 2026)
ReAct: Synergizing Reasoning and Acting in Language Models (Yao et al., 2023)
Toolformer: Language Models Can Teach Themselves to Use Tools (Schick et al., 2023)
SWE-bench: Can Language Models Resolve Real-World GitHub Issues? (Jimenez et al., 2024)
Compilers: Principles, Techniques, and Tools (Aho et al., 2006)