Pasting a guide into the prompt costs points; compiling it into a skill adds up to 25

Notes on MMG2Skill: Can Agents Distill In-the-Wild Guides into Self-Evolving Skills? (arXiv:2606.01993) — Xinyu Che, Junqi Xiong, Yunfei Ge, Xinping Lei, Shihao Li, Hang Yan, Hanyu Li, Yuanxing Zhang, Zhiqi Bai, Jinhua Hao, Ming Sun, Jiaheng Liu · June 2026

Note published July 25, 2026 · written by SkillFed’s research pipeline from the paper above · how these notes are made

MMG2Skill formalizes guide-to-skill learning: turning human-written how-tos scraped off the web — app tutorials, wiki recipes, strategy guides — into procedures a vision-language model (VLM) agent can actually execute. MMG2Skill-Bench backs the idea with 130 success-inferable tasks spread across three domains: OSWorld-style desktop GUI control (40 tasks), Minecraft crafting and building (30 tasks), and the card games Doudizhu and Mahjong via RLCard (60 tasks), each one paired with a real multimodal guide pulled from the wild. The framework itself runs as a closed loop with four stages — compile the guide into an editable SKILL.md (procedures, when they apply, what success looks like, how to recover from failure), run a fixed VLM agent conditioned on that skill file, read the resulting trajectories for root-cause diagnoses, and rewrite the skill accordingly — all without ever touching a benchmark score.

Across six current VLM backbones, compiling and revising the skill lifts macro-average task success by 12.8 to 25.3 percentage points over the vanilla agent — the smallest gain lands on the strongest baseline model, the largest on the weakest. The guide alone doesn't buy that: pasting its raw text straight into the prompt is flat on GUI tasks and actively costs up to 1.67 points on the Game domain, because a human-oriented guide's steps don't line up with what the agent actually sees on screen. Structured extraction and trajectory-level root-cause feedback split the credit differently by domain — revision supplies essentially the entire gain on Minecraft and card play, while on GUI tasks most of the lift comes from the initial structured extraction, with revision adding a smaller top-up. Repeated revision passes are non-monotonic — some attempts make skills worse — so an analyzer-based early stopping rule that watches its own diagnosis confidence, with no access to ground truth, cuts 25 to 53 percent of revision attempts without giving up the accuracy gains.