A three-tier skill hierarchy — not a bigger skill library — drives a 79.6% jump on long-horizon GUI tasks

Notes on Mirage-1: Augmenting and Updating GUI Agent with Hierarchical Multimodal Skills (arXiv:2506.10387) — Yuquan Xie, Zaijing Li, Rui Shao, Gongwei Chen, Kaiwen Zhou, Yinchuan Li, D. Jiang, Liqiang Nie · June 2025

Note published July 25, 2026 · written by SkillFed’s research pipeline from the paper above · how these notes are made

AI-assisted notes · reviewed by SkillFed Skill evolution

MLLM-based GUI agents that plan step-by-step from screenshots run out of usable knowledge once a task stretches across many steps and several apps. The fix: a Hierarchical Multimodal Skills (HMS) module that abstracts recorded trajectories into three tiers — execution skills (a literal recorded action sequence for one concrete goal), core skills (a parameterized function distilled from several similar execution skills), and meta skills (a further roll-up of core skills into a broad, reusable capability). Paired with HMS is Skill-Augmented Monte Carlo Tree Search (SA-MCTS), which uses the offline-learned hierarchy to prune the action space an online tree search has to explore, then writes newly discovered successful trajectories back into HMS so the library keeps growing after deployment instead of freezing at training time.

Seeded from 1,000 trajectories each on mobile (AITW) and web (Multimodal-Mind2Web) data, HMS started with roughly 1,600 mobile skills (93 meta / 463 core / 1,063 execution) and 1,800 web skills (125 / 643 / 1,049) before any online exploration ever ran. Tested with the OS-Atlas grounding model, Mirage-1 beat its baseline agent by 32.3% on AndroidWorld, 19.1% on MobileMiniWob++, and 15.8% on Mind2Web-Live — and by 79.6% on the paper's own 30-task, 12-app long-horizon benchmark, AndroidLH, built specifically to force multi-app task chains. The ablations pin down which tier is doing the work: pulling out meta skills costs 19.5 points, more than double the 8.7 points lost by pulling out execution skills, and skipping offline skill augmentation entirely costs 22.3 points.

Key numbers

Gain on AndroidLH long-horizon benchmark (vs. OS-Atlas baseline)	+79.6%
Gain on AndroidWorld (vs. OS-Atlas baseline)	+32.3%
Gain on Mind2Web-Live (vs. OS-Atlas baseline)	+15.8%
Performance lost when meta skills are removed	-19.5%
Performance lost when offline skill augmentation is removed	-22.3%

What this means for skill authors

The ablation numbers carry the real design lesson, and it isn't specific to GUI agents: a skill library earns its keep by being layered, not just large. Meta skills — the most abstract, most reusable tier — cost more to lose than the raw execution traces sitting underneath them. Anyone packaging skills for an agent to reuse should read that as an argument against flat macro dumps: a skill worth keeping over time is one distilled into something callable with parameters, not the verbatim recording of a single run.

The harder implication is that this skill library never stops moving. SA-MCTS writes every successful online trajectory back into HMS, retrieved later by skill retrieval over embeddings rather than exact match, so the skill set an agent draws on next week isn't the one it shipped with. That's a genuinely different problem from vetting a static skill file: a one-time check can clear a skill at submission, but it says nothing about what that skill's own memory looks like after a few hundred successful runs have folded themselves back in. Self-updating skill libraries need drift-triggered re-verification, not just approval at the door — the gap a vetted, continuously-scanned catalog is built to keep in view rather than assume away.

Where the gains concentrate tells its own story: the smallest lift shows up on single-focus benchmarks, the largest on the suite built to force composition across a dozen apps. Skills that only have to work in isolation are the easy case; skills meant to be chained toward one goal are where a system's real behavior shows up. Correctness of one skill checked alone is necessary but not sufficient once agents start composing several of them.

Skills related to this research

Montecarlo This skill executes Monte Carlo simulations across 10,000 market scenarios to forecast income probabilities, margin safety metrics, and portfolio performance over a 28-month horizon. It auto-detects current portfolio values and supports a 4-layer structure (Growth, Income, Hedge, and GOOGL positions) with monthly deployment tracking.★ 315 risk-analyzer Risk Analyzer quantifies portfolio risk through Value at Risk calculations—parametric, historical, and Monte Carlo methods—alongside conditional VaR, stress testing, and drawdown analysis. It identifies key risk concentrations and factor exposures to guide mitigation decisions.★ 406 risk-analysis Risk-analysis provides quantitative methods for measuring portfolio risk through Value at Risk, Conditional Value at Risk, maximum drawdown analysis, and Monte Carlo simulation. It includes historical and parametric VaR calculation, tail-risk analysis via extreme value theory, and stress-testing frameworks using both historical scenarios and hypothetical shock models.★ 28,096 forward-risk Quantify potential portfolio losses across normal and extreme market conditions using parametric VaR, Monte Carlo simulation, Expected Shortfall, and scenario analysis. Decompose risk by position and factor to identify which holdings drive portfolio volatility. Stress-test portfolios against historical crises and custom market shocks.★ 159

Related notes

References

Mirage-1: Augmenting and Updating GUI Agent with Hierarchical Multimodal Skills
AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agents
Mind2Web: Towards a Generalist Agent for the Web
OS-Atlas: A Foundation Action Model for Generalist GUI Agents