skillfed

Letting a robot invent its own practice tasks adds 20.6 points of task success

Notes on Playful Agentic Robot Learning (arXiv:2606.19419) — Junyi Zhang, Jiaxin Ge, H. Yoo, Letian Fu, Zihan Yang, Yao Liu, R. Saravanan, Shaofeng Yin, Justin Yu, Dantong Niu, Zirui Wang, Roei Herzig, and 8 more · June 2026

Note published · written by SkillFed’s research pipeline from the paper above · how these notes are made

AI-assisted notes · reviewed by SkillFed Skill evolution

RATs (Robotics Agent Teams) gives a robot a practice stage before it's ever handed a task — a multi-agent Code-as-Policy system built around three coordinating roles. A task proposer invents exploratory goals, an execution team writes and runs candidate robot code with step-level verification and retry-on-failure, and a memory team distills what worked into a persistent skill library. Task selection isn't random: a "Goldilocks" score multiplies novelty (how rarely a given object-skill pairing has been attempted) by learnability (how close the agent's current success rate on that skill sits to 50%), pushing practice toward the edge of what the agent can already half-do. It's intrinsic motivation from developmental psychology, rebuilt as a computable scoring function instead of a vague heuristic.

After 50 play iterations per environment, the resulting skill libraries lifted held-out task success from 23.2% to 43.8% on LIBERO-PRO (+20.6 points) and from 21.0% to 38.0% on MolmoSpaces (+17.0 points) — both well clear of the strongest vision-language-action baseline tested, π0.5, which topped out at 12.8%. The skills traveled, too: plugging the LIBERO-PRO library into an otherwise unmodified baseline agent raised RoboSuite success from 40.3% to 49.1%, and the same frozen library pushed real-robot pick-and-place from 30.0% to 38.8%, with no real-world play and no model finetuning — a form of sim-to-real transfer achieved purely by retrieving code into context. An ablation shows where the gain actually comes from: swap curiosity-driven task selection for randomly sampled tasks under the identical 50-iteration budget and the number barely moves, 23.2% to 24.7%, while curiosity-driven selection alone reaches 32.3%. What the agent chooses to practice is doing the work, not how much it practices.

Key numbers

LIBERO-PRO task success (play-learned skills)23.2% → 43.8% (+20.6 pp)
MolmoSpaces task success21.0% → 38.0% (+17.0 pp)
Cross-sim transfer to RoboSuite40.3% → 49.1% (+8.9 pp)
Real-robot transfer, no finetuning30.0% → 38.8% (+8.8 pp)
Random play vs. curiosity-driven play24.7% vs. 32.3%

Skills related to this research

robot-framework-skill This skill enables you to create Robot Framework test cases with keyword-driven automation for both web interfaces and APIs. It leverages structured syntax to build maintainable, reusable test scenarios that integrate seamlessly into your testing workflows.★ 339 srdf srdf empowers you to build and refine Semantic Robot Description Format files that power MoveIt2 motion planning workflows. Whether you're defining robot kinematics, collision geometry, or planning groups, this skill streamlines SRDF authoring with text-driven CAD generation and real-time validation.★ 10,959 urdf Create and modify URDF (Unified Robot Description Format) files to define robot kinematics, joint structures, and link hierarchies. This skill helps roboticists and engineers build frame-correct kinematic models for simulation, visualization, and control system integration. Streamline your robot description workflow with intelligent URDF generation and editing capabilities.★ 10,959 Education Tutor Education Tutor provides one-on-one academic support across subjects and grade levels, breaking down difficult concepts through multiple teaching methods tailored to how you learn best. It creates structured study schedules, generates practice problems, and offers test preparation guidance to build real understanding rather than surface memorization.★ 7
Learning Latent Plans from Play We propose learning from teleoperated play data (LfP) as a way to scale up multi-task robotic skill learning. Learning from play (LfP) offers three main… Lynch et al. · similarity 0.86 ELSIM: End-to-end learning of reusable skills through intrinsic motivation Taking inspiration from developmental learning, we present a novel reinforcement learning architecture which hierarchically learns and represents… Aubret et al. · similarity 0.85 Agentic Skill Discovery Language-conditioned robotic skills make it possible to apply the high-level reasoning of Large Language Models (LLMs) to low-level robotic control. A… Zhao et al. · similarity 0.84 Offline Multi-agent Continual Cooperation via Skill Partition and Reuse Extracting skills from multi-agent offline dataset improves learning efficiency via sharing task-invariant coordination skills among tasks. In settings where… Xiao et al. · similarity 0.83

Related notes

References

  1. J. Zhang, J. Ge, H. Yoo, L. Fu, et al. Playful Agentic Robot Learning. arXiv:2606.19419, 2026.
  2. M. Fu, J. Yu, K. El-Refai, et al. CaP-X: A framework for benchmarking and improving coding agents for robot manipulation. arXiv:2603.22435, 2026.
  3. G. Wang, Y. Xie, Y. Jiang, et al. Voyager: An open-ended embodied agent with large language models. Transactions on Machine Learning Research (TMLR), 2024.
  4. Y. Zhu, J. Wong, A. Mandlekar, et al. robosuite: A modular simulation framework and benchmark for robot learning. arXiv:2009.12293, 2020.
  5. Y. Kim, W. Pumacay, O. Rayyan, et al. MolmoSpaces: A large-scale open ecosystem for robot navigation and manipulation. arXiv:2602.11337, 2026.