Typed contracts + call templates: 82 vs. 47 ALFWorld wins, −23% tokens per game

Notes on Skill-as-Pseudocode: Refactoring Skill Libraries to Pseudocode for LLM Agents (arXiv:2605.27955) — Xinze Li, Yuhang Zang, Yixin Cao, Aixin Sun · May 2026

Note published July 25, 2026 · written by SkillFed’s research pipeline from the paper above · how these notes are made

AI-assisted notes · reviewed by SkillFed Frontier & other

Skill-as-Pseudocode (SaP) rewrites markdown skill libraries into typed pseudocode, so agents stop re-deriving schemas and call syntax from prose on every retrieval. The pipeline clusters similar procedural passages across a library and extracts a typed contract for each cluster. Before promotion, each contract has to clear a four-check deterministic verifier — coverage, binding, replacement, risk. Verified contracts get inlined into a rewritten skill file alongside restored concrete action templates, so one retrieval now hands the agent both a typed signature for what a skill does and syntax it can invoke directly.

On the 134-game ALFWorld unseen split with gpt-4o-mini, pooled across three seeds, SaP wins 82 of 402 paired games against 47 for the Graph-of-Skills baseline (pooled McNemar test, p = 8.2×10⁻⁵) — and does it while cutting input tokens 22.8% and LLM calls 14.5% per game. The mechanism shows up directly in action quality: the agent's next environment action after a retrieval succeeds 29.3% of the time under SaP versus 20.5% under GoS, widening to a 1.57x gap on explicit skill-lookup events. That gain depends on one design choice: an ablation that promotes contracts to standalone top-level retrieval results, instead of nesting them under their parent skill, drops reward by roughly 27%.

Key numbers

ALFWorld paired wins (SaP vs. GoS)	82/402 vs. 47/402
Significance (pooled McNemar)	p = 8.2×10⁻⁵
Input tokens per game	−22.8% ± 6.4%
LLM calls per game	−14.5% ± 4.1%
Reward drop if retrieval hierarchy is flattened	−27% (22.4%→16.4%, seed 42)

What this means for skill authors

The failure mode here is familiar to anyone who has watched an agent misfire against a skill doc: it reads the prose, extracts a plausible-looking call, gets rejected by the environment, and re-reads the same prose without learning anything new. SaP's fix isn't better prose — it's refusing to let 'what a skill does' and 'how to call it' live in the same paragraph, and enforcing that split with a deterministic filter instead of trusting an LLM's judgment about what to keep. For anyone authoring skills meant for automated retrieval, that's the lesson: a typed I/O contract and a copy-pasteable invocation template are two separate artifacts, not two sentences in the same section.

The economics favor doing this at scale. Refactoring 500 skills (5,709 procedural units) end to end cost about $1.3 and 30 minutes — cheap next to what a single mis-invoked action costs an agent in retries. Token and call savings are per-episode, so they compound with usage instead of staying fixed. The catch is the ablation: how the refactored contracts get exposed to retrieval matters almost as much as the refactoring itself. Flatten the hierarchy and reward drops by roughly 27% (22.4% → 16.4%).

This generalizes past markdown-only libraries. A catalog that vets skills for correctness of content but still ships them as free-form prose is leaving a structural failure mode unaddressed — dual-tool scanning can confirm a skill is safe and does what it claims without confirming an agent can actually call it correctly on the first try. Packaging, not just content, is a quality axis worth checking for.

Skills related to this research

Figma Pull design files, nodes, and rendered images from Figma via REST API to automate design workflows. Extract design tokens, colors, spacing, and read file comments—all read-only access using your OAuth token. Ideal for design-to-code pipelines and design system automation.★ 13 Seo Images Seo Images helps you maximize the discoverability of your visual assets through intelligent optimization techniques. Improve how search engines index and rank your images by refining metadata, alt text, and technical attributes. Unlock better visibility for your image-based content across search results.★ 506 Figma Generate Library Build production-ready design systems in Figma by orchestrating multi-phase workflows that extract tokens from code, create variable foundations, construct components with proper variant sets, and document design principles. Works alongside the figma-use skill to enforce quality patterns from real-world systems.★ 1,925 documentation-lookup Documentation-lookup pulls live reference material for popular frameworks and libraries when you need setup guidance, API details, or code examples. It resolves library identifiers and retrieves version-specific documentation to ensure your answers stay current and accurate.★ 5,778

Related notes

References

Li, Zang, Cao & Sun, "Skill-as-Pseudocode: Refactoring Skill Libraries to Pseudocode for LLM Agents," arXiv:2605.27955 (2026).
"Graph-of-Skills: Dependency-Aware Structural Retrieval for Massive Agent Skills," arXiv:2604.05333 (2026).
Shridhar et al., "ALFWorld: Aligning Text and Embodied Environments for Interactive Learning," ICLR (2021).