skillfed

Remove the curriculum and Voyager's item discovery collapses 93%

Notes on Voyager: An Open-Ended Embodied Agent with Large Language Models (arXiv:2305.16291) — Guanzhi Wang, Yuqi Xie, Yunfan Jiang, A. Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi (Jim) Fan, Anima Anandkumar · May 2023 · Trans. Mach. Learn. Res.

Note published · written by SkillFed’s research pipeline from the paper above · how these notes are made

AI-assisted notes · reviewed by SkillFed Skill evolution

Voyager pairs GPT-4 with three components instead of one clever prompt. An automatic curriculum proposes the next Minecraft objective by reading the agent's live state against its history of completed and failed tasks. A skill library stores the results as executable JavaScript functions, indexed by embeddings of their own descriptions and pulled back out via nearest-neighbor lookup. Before anything is committed to that library, an iterative prompting mechanism folds in game feedback and interpreter errors, and a dedicated self-verification critic judges whether the skill actually worked. None of this involves fine-tuning: GPT-4 (gpt-4-0314) is queried as a black box for curriculum, code generation, and verification; GPT-3.5 handles cheaper auxiliary text; embeddings come from text-embedding-ada-002.

Across 160 prompting iterations, Voyager discovered 63 unique items — roughly 3.3x what the ReAct, Reflexion, and AutoGPT baselines managed — while covering 2.3x more map distance and clearing tech-tree tiers 6.4x to 15.3x faster by iteration count (15.3x at the wooden tier alone). It was the only method to unlock the diamond tier at all, and even then in just one of three runs. The ablations show where that gap comes from: pull the automatic curriculum and discovered items collapse by 93%; pull self-verification instead and they collapse by 73%, the largest single feedback-channel effect the authors measured; swap GPT-4 for GPT-3.5 in code generation and unique items drop by a further 5.7x. Dropped into a freshly reset world with an empty inventory, Voyager reused its accumulated skill library to solve four tasks it had never explicitly targeted; every baseline solved none of them within 50 iterations.

Key numbers

Unique items discovered63 in 160 iterations (3.3x baselines)
Wooden-tier tech tree unlock15.3x faster than baselines
Map distance traversed2.3x farther than baselines
Item discovery w/o curriculum-93%
Item discovery w/o self-verification-73%

Skills related to this research

create-x-content Create X Content drafts 2–5 post variants from a brief, each with a distinct framing (how-to, problem-first, hype, mechanism-breakdown, etc.) that align with your personal voice guide. The skill self-checks each variant against banned phrases and format rules before saving, ensuring posts sound authentically like you.★ 1,062 Video Prompting Guide Learn structured approaches to writing prompts that generate high-quality videos across leading AI models. This guide covers shot composition, camera techniques, lighting vocabulary, and style keywords, plus model-specific tips and real-world examples for product demos, documentaries, and more.★ 660 Education Tutor Education Tutor provides one-on-one academic support across subjects and grade levels, breaking down difficult concepts through multiple teaching methods tailored to how you learn best. It creates structured study schedules, generates practice problems, and offers test preparation guidance to build real understanding rather than surface memorization.★ 7 seo-site-health-audit This skill analyzes your website's technical health through a business-impact lens rather than generic crawler scores. It surfaces SEO issues ranked by their actual effect on search visibility and user experience, helping you focus optimization efforts where they'll drive the most meaningful results.★ 500
Odyssey : Empowering Minecraft Agents with Open-World Skills Recent studies have delved into constructing generalist agents for open-world environments like Minecraft. Despite the encouraging results, existing efforts… Liu et al. · similarity 0.89 Skill Reinforcement Learning and Planning for Open-World Long-Horizon Tasks We study building multi-task agents in open-world environments. Without human demonstrations, learning to accomplish long-horizon tasks in a large open-world… Yuan et al. · similarity 0.87 MindAgent: Emergent Gaming Interaction Large Language Models (LLMs) have the capacity of performing complex scheduling in a multi-agent system and can coordinate these agents into completing… Gong et al. · similarity 0.86 Multi-task curriculum learning in a complex, visual, hard-exploration domain: Minecraft An important challenge in reinforcement learning is training agents that can solve a wide variety of tasks. If tasks depend on each other (e.g. needing to… Kanitscheider et al. · similarity 0.86

Related notes

References

  1. Wang et al., "Voyager: An Open-Ended Embodied Agent with Large Language Models" (arXiv:2305.16291, 2023)
  2. Yao et al., "ReAct: Synergizing Reasoning and Acting in Language Models" (arXiv:2210.03629, 2022)
  3. Shinn et al., "Reflexion: an autonomous agent with dynamic memory and self-reflection" (arXiv:2303.11366, 2023)
  4. Fan et al., "MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge" (arXiv:2206.08853, 2022)