A skill grown from a 35B model's traces adds 57.65 points to a 122B agent on table QA it's never seen

Notes on Trace2Skill: Distill Trajectory-Local Lessons into Transferable Agent Skills (arXiv:2603.25158) — Jingwei Ni, Yihao Liu, Xinpeng Liu, Yutao Sun, Mengyu Zhou, Pengyu Cheng, Dexin Wang, Xiaoxi Jiang, Guanjun Jiang · March 2026

Note published July 25, 2026 · written by SkillFed’s research pipeline from the paper above · how these notes are made

Trace2Skill turns a pile of raw agent execution traces into one consolidated skill file, not a growing memory store, not an order-dependent chain of edits. A frozen agent runs a batch of tasks first and produces labeled trajectories, successes and failures both. Failures go to an interactive error analyst that inspects the actual output artifacts, checks them against ground truth, and validates a candidate fix before writing a patch; successes go to a lighter single-pass analyst that pulls out reusable behavior patterns. A hierarchical merge step then folds every patch in the pool together at once — deduplicating, resolving conflicts, keeping only what recurs across independently analyzed trajectories — into one updated skill document. The same model generates the trajectories, analyzes them, and merges the result, so there's no separate teacher model, no fine-tuning, no test-time retrieval. The pipeline runs in two modes: deepening an existing human-written skill, and building one from scratch out of a weak LLM-drafted seed, across spreadsheet editing, math reasoning, document QA, and office-file tasks.

The baseline number sets up the whole case: Anthropic's official xlsx skill lifts a 122B spreadsheet agent's verified pass rate from 27.67% to 48.33%, but the identical skill drags a 35B agent below its own no-skill floor — 9.67% versus 19.00%. A hand-written skill that helps one model size can actively hurt another. Skills grown by Trace2Skill don't carry that fragility: one evolved purely from 35B-model trajectories improves a 122B agent by up to 57.65 percentage points on the out-of-distribution WikiTableQuestions benchmark, and the gains cross model families too — Gemma- and GPT-family agents both improve on Qwen-authored skills and on skills grown from their own traces. Parallel consolidation also beats order-dependent sequential editing, on speed (about 3 minutes versus roughly 60 for fully sequential updates) and on downstream quality: one consolidated skill outperforms a ReasoningBank-style retrieval memory built from the identical trajectory pool, 69.83% versus 56.00% verified pass rate for the 122B agent. The recurring lessons collapse into a handful of standard operating procedures — four SoPs account for between 16.4% and 55.1% of all patches in one run, covering things like re-verifying formula writes and safe row-deletion order, instead of staying scattered as one-off trajectory-specific tips.