35 iterations, 6 of 6 runs: programmatic skill graphs triple Voyager's hit rate on Minecraft's diamond-tool climb

Notes on Evolving Programmatic Skill Networks (arXiv:2601.03509) — Haochen Shi, Xingdi Yuan, Bang Liu · January 2026

Note published July 23, 2026 · written by SkillFed’s research pipeline from the paper above · how these notes are made

AI-assisted notes · reviewed by SkillFed Skill evolution

Evolving Programmatic Skill Networks builds agent skills as executable, typed programs — not prompts or embeddings — arranged in a directed graph that keeps growing and rewriting itself as the agent acts. The framework, called the Programmatic Skill Network (PSN), grows through two linked mechanisms. Day to day, Reflect traces a failed run backward through the skill graph to pin down which specific subskill broke — a symbolic stand-in for backpropagation — and a maturity-aware update gate throttles how often a skill gets rewritten based on its track record, so reliable skills stop churning while shaky ones stay open to revision. Periodically, a refactoring pass collapses redundant or overlapping skills into one of five canonical patterns — parametric coverage, behavioral coverage, sibling specialization, common-subskill extraction, and duplication — with every rewrite gated behind a rollback check against recent tasks. Both mechanisms are evaluated on the MineDojo Minecraft tech-tree benchmark and on Crafter.

On MineDojo's diamond-tool climb, PSN closes all six runs, averaging 35 iterations; Voyager, the standard open-ended-skill baseline, closes only one of three runs and needs 102. The gap survives a weaker backbone: swapping GPT-5-mini for the open mixture-of-experts model Qwen3-Coder-Next still reaches the goal, just slower (49 vs. 35 iterations), with a shallower skill graph — a reuse ratio (the share of skills called by more than one other skill) of 0.15 against 0.40 for the stronger model — and deeper repair chains (5.0 vs. 2.7 skills touched per multi-skill fix). Retries compound quickly: first-attempt success of 67% reaches 100% within three tries, and on a fixed compositional-task suite, turning refactoring on lifts PSN to 84.62% success against 68.75% for a Voyager variant. Across both benchmarks, PSN keeps its earlier skills intact as it learns new ones, while Voyager shows marked backward interference.

Key numbers

Diamond-tool runs solved	6 of 6, avg. 35 iterations
Voyager baseline	1 of 3 runs, 102 iterations
Reuse ratio, strong vs. weak LLM	0.40 vs. 0.15
Compositional-task success, PSN vs. Voyager-R	84.62% vs. 68.75%
First-attempt success → within 3 tries	67% → 100%

What this means for skill authors

For skill authors, the number worth sitting with isn't the six-run Minecraft demo — it's the reuse ratio. PSN turns "is this skill library actually composing well" into a graph metric, the fraction of skills reused by more than one caller, that moves with model quality (0.40 vs. 0.15) instead of staying a vibe. A catalog that tracked reuse ratio and repair-chain depth per skill could flag libraries quietly rotting from unrepaired dependencies before a user hits the failure, rather than after.

The gating rule is worth borrowing outside PSN's own execution loop: let a skill's edit-worthiness depend on its own track record, not on whatever else nearby just broke. Any review process — including a vetted catalog's revision workflow — benefits from that discipline; a skill that already works shouldn't get destabilized just because a sibling skill got rewritten.

The catch is representation. Fault localization and canonical refactoring both lean on skills being typed programs with explicit pre/postconditions — a stronger structural contract than the markdown or prompt bundles most agent skills ship as today. The backward-interference gap here — PSN keeps its old skills while Voyager forgets them — reads less like a planning win and more like evidence that the format a skill is stored in, not just the agent's reasoning, decides whether reuse compounds or erodes over time.

Skills related to this research

agent-builder Agent Builder guides you through creating AI agents using the Claude Agent SDK and headless mode—no raw API keys needed. Build custom tools with MCP servers, control execution with hooks, and deploy agents in Python or via CLI commands. Covers SDK patterns, security practices, and production workflows.★ 274 homelab-network-readiness A planning and validation framework for homelab network upgrades involving VLANs, local DNS resolvers, firewall rules, and remote access. Work through required inventory, trust zones, and staged migration steps to avoid locking yourself out of critical infrastructure. Includes safety rules, anti-patterns, and a review checklist to confirm your changes won't break management access or recovery paths.★ 234,207 Cometchat Android V6 Kotlin Theming Apply and customize CometChat themes in Kotlin Views using the CometChatTheme singleton, XML attributes, or programmatic color and font overrides. Supports dark mode detection and per-component style classes for complete visual control.★ 61 Fix Schema Fix Schema resolves session log validation failures by updating Zod schemas to handle new data formats without breaking existing JSONL files. It guides you through locating the relevant schema, updating type definitions, adding tests, and creating a pull request with all changes verified.★ 1,262

Related notes

References

Shi, H., Yuan, X., & Liu, B. (2026). Evolving Programmatic Skill Networks. arXiv:2601.03509.
Wang, G., Xie, Y., Jiang, Y., Mandlekar, A., Xiao, C., Zhu, Y., Fan, L., & Anandkumar, A. (2023). Voyager: An Open-Ended Embodied Agent with Large Language Models.
Fan, L., Wang, G., Jiang, Y., Mandlekar, A., Yang, Y., Zhu, H., Tang, A., Huang, D.-A., Zhu, Y., & Anandkumar, A. (2022). MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge.
Hafner, D. (2021). Benchmarking the Spectrum of Agent Capabilities (Crafter).