Splitting skill-writing into create, merge, and improve lifts a 27B agent up to 4.5 points -- merge drives generalization, improve drives specification, and neither alone gets you there

Notes on SkillComposer: Learning to Evolve Agent Skills for Specification and Generalization (arXiv:2606.06079) — Qi Zhang, Zhaopeng Feng, Xiao Shi, Xiaomeng Hu, Chu Liu, Pengjun Xie, Xiaobin Wang, Jieping Ye, Bryan Hooi, Haobo Wang, Junbo Zhao · June 2026

Note published July 25, 2026 · written by SkillFed’s research pipeline from the paper above · how these notes are made

Agent skills — reusable natural-language instructions that steer a model's reasoning without touching its weights — usually get built one of two ways: hand-written by a person, or extracted whole from a single successful trajectory. SkillComposer's case against both is that skill quality has two separate axes, generalization and specification, and no one-shot extraction step can serve both. So it breaks skill-writing into three trainable edits. Create turns a raw task trajectory into a skill. Merge folds two overlapping skills into one broader skill. Improve refines an existing skill against a fresh trajectory generated under its own guidance. All three train the same way, via rejection sampling: an edit only becomes training data if it lifts the executor's pass@1 by at least 0.4 over the unedited baseline. The resulting 4B-parameter composer runs three ways. Offline, create and merge build a static skill library ahead of time. Online, create and improve refine one skill per task with no library at all. Hybrid retrieves from the offline library, then specializes it per task.

On τ²-Bench (agent tasks), LiveCodeBench v6 (code), and AppWorld — a benchmark held out of training entirely — the trained composer beats a no-skill baseline and beats MemP, a prior automatic procedural-memory method. MemP actually pushes LiveCodeBench score below the no-skill baseline: unfiltered skill extraction can make an agent worse, not better. Hybrid mode wins τ²-Bench; online wins on code. The advantage survives a harder test: cross-domain generalization. Skills trained only on a Retail customer-service domain still lift performance on unseen Airline and Telecom domains, and skills composed by the 4B model still help a 27B executor that never saw the training data. Ablating the three operations shows why: merge alone accounts for nearly all of the offline library's gain, and bolting improve onto that same library actually costs code accuracy. Merge and improve pull in different, complementary directions — neither subsumes the other.