A 3.9M-parameter skill sequencer closes 80% of the gap to hand-picked "gold" skill sets

Notes on Generative Skill Composition for LLM Agents (arXiv:2606.32025) — Xinyu Zhao, Zhen Tan, Vaishnav Tadiparthi, Nakul Agarwal, Kwonjoon Lee, Ehsan Moradi Pari, Hossein Nourkhiz Mahjoub, Tianlong Chen · June 2026

Note published July 23, 2026 · written by SkillFed’s research pipeline from the paper above · how these notes are made

LLM agents built on skill libraries hit a bottleneck once the library grows: choosing what to load stops being a lookup problem and becomes a joint decision over subset, count, and order — three choices that retrieval and full-library approaches never handle together. SkillComposer treats this as structured skill composition. A frozen Qwen3-Embedding-0.6B encoder turns the task and skill metadata into a 256-dimensional vector; a compact autoregressive decoder (3 layers, 256 hidden dimensions, 4 attention heads) reads that vector and emits skill identifiers one at a time until a STOP token. Subset, count, and order all fall out of that single decoding pass. At inference the decoder doesn't work alone: its logits are fused with a TF-IDF relevance score and a separately trained set-membership head, what the paper calls retrieval-augmented decoding. Training data is 9,872 task-skill records built around a real, 196-skill human-curated library — 65 real SkillBench tasks plus synthetic single- and multi-skill compositions grounded in a 196-node skill dependency graph.

On composition quality, the 3.9M-parameter SkillComposer beats a fully fine-tuned 600M-parameter model, TF-IDF/BM25/dense-embedding retrieval, and an LLM-judge (Gemini-2.5-flash) on Set F1. It also degrades far more gracefully when tested on real held-out tasks after training only on synthetic data — an 11-point Set F1 drop versus a 27.5-point drop for the fine-tuned model. The gap carries downstream: on SkillsBench, loading SkillComposer's predicted plan into two production coding agents lifts pass rate over a no-skill baseline by 23.1 points on GPT-5.2-Codex and 18.2 points on Gemini-3-Pro-Preview, beats top-3 retrieval, and closes roughly 80% of the gap to a hand-curated gold-skill ceiling — all while spending fewer prompt tokens than either the full-library baseline or top-3 retrieval. Ablations confirm both the retrieval prior and the set-membership fusion are load-bearing (drop either and Set F1 falls by several points); case studies expose a recurring failure mode, too: on tasks needing two or three chained skills, the model sometimes stops one skill short, a bias the authors trace to the training corpus leaning toward shorter compositions.