Text-based skill edits beat trained baselines by 6.7 points

Notes on SkillGrad: Optimizing Agent Skills Like Gradient Descent (arXiv:2605.27760) — Hanyu Wang, Yifan Lan, Bochuan Cao, Lu Lin, Jinghui Chen · May 2026

Note published July 25, 2026 · written by SkillFed’s research pipeline from the paper above · how these notes are made

AI-assisted notes · reviewed by SkillFed Skill evolution

SkillGrad treats an agent's skill package — its metadata, always-loaded skill body, and conditional resource files — as a parameter to update, not a document to hand-edit. Each task run against the skill produces a trajectory; a diagnoser model reads the failures alongside contrastive successes and writes a text-based gradient — a plain-language statement of what needs to change and why. A momentum agent tracks these diagnoses across many runs and keeps only the patterns that recur, so a single bad trajectory can't swing the skill on its own. A separate patcher model executes the update as layer-aware edits: broadly useful fixes go into the always-loaded body, narrower ones into resource files that load only when needed.

On SpreadsheetBench Verified and an out-of-distribution WikiTableQuestions split, across GPT-5.4 and GPT-4.1 backbones, the loop beat two training-based skill-evolution baselines — EvoSkill and Trace2Skill — by 6.7 points on average. The ablations rank the components: pulling momentum out costs more accuracy than pulling out the contrastive half of diagnosis, so the stabilizing memory is doing more of the work than the raw feedback signal feeding it. There's an uncomfortable number buried in the same table — a skill GPT-5.4 writes for itself, with no structured feedback loop, scored worse than giving it no skill at all, until SkillGrad's update loop fixed it.

Key numbers

Avg. accuracy gain vs. strongest training-based baseline	+6.7 pp
Accuracy loss with momentum agent removed	−6.67 pp (72.50%→65.83%)
Accuracy loss with contrastive diagnosis removed	−4.17 pp (72.50%→68.33%)
Self-generated skill vs. no-skill baseline, pre-optimization	55.83% vs. 62.50%
Held-out tasks that triggered the conditional resource layer	71.9%

What this means for skill authors

The buried result matters more than the headline number. A skill an LLM writes for itself — no eval loop, no diagnosis, just "write me a skill for this" — underperformed the no-skill baseline by about seven points before any optimization touched it. That's not noise. It's evidence that an unvetted skill is a liability by default, not a neutral convenience — the same premise a dual-tool scan runs on: treat every third-party or self-authored skill as unproven until something checks it against a task suite, rather than trusting it because it parses.

The ablation ordering is the more durable lesson for anyone maintaining a skill over time. Momentum — deciding a pattern is real because it recurs across a batch, not because it showed up once — mattered more than the diagnosis step feeding it. That argues against rewriting a skill file after every single failure, and for batching evidence before committing a change. It also validates splitting a skill into an always-loaded core and conditional resources: the core plateaued at roughly 150 lines while the resource layer kept absorbing detail, firing on nearly three-quarters of held-out tasks.

There's a practical ceiling too: accuracy plateaued around the tenth batch update and was already drifting down by the thirteenth. Gradient-style skill editing needs a stopping rule, not an assumption that more iterations always help — the same discipline that keeps a skill from getting edited into a worse version of itself.

Skills related to this research

Crypto Backtest Crypto Backtest transforms natural language trading ideas into validated strategies with professional backtesting and detailed reports. It combines multiple indicators—RSI, moving averages, Bollinger Bands, and more—to build robust entry and exit conditions for spot trading only. Users describe their strategy, confirm the generated plan, and receive comprehensive backtest results.★ 18 audio-router audio-router intelligently directs audio processing requests to the right specialized skill, whether for playback control, sound analysis, or reactive audio handling. Built for Claude, it streamlines how audio workflows are orchestrated across multiple focused capabilities.★ 8 seedance-recipes Seedance-recipes provides production-ready recipe patterns for video content across genres: product, lifestyle, drama, music video, landscape, commercial, animation, and more. Each recipe preserves core creative constraints while inviting customization of subject, camera, lighting, and sound. Use recipes as proven starting shapes, not rigid templates.★ 5,445 multi-factor Multi-factor ranks stocks by computing and standardizing multiple factors—momentum, reversal, volatility, and volume—then combines them into a composite score to select top performers for equal-weight portfolios. Built-in support for value metrics like PE and ROE on supported markets. The newer ZooSignalEngine integrates 450+ pre-built alphas from the registry for flexible long-only, short-only, or long-short strategies.★ 28,096

Related notes

References

Wang, Lan, Cao, Lin & Chen. "SkillGrad: Optimizing Agent Skills Like Gradient Descent." arXiv:2605.27760 (2026).
Alzubi et al. EvoSkill — failure-driven skill evolution with validation-based selection, cited as a comparison baseline (2026).
Ni et al. Trace2Skill — trajectory-to-skill distillation over frozen execution trajectories, cited as a comparison baseline (2026).