Curated Skills Add 16.6 Points — Self-Authored Ones Cost You 11.5

Notes on SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks (arXiv:2602.12670) — Xiangyi Li, Wenbo Chen, Yimin Liu, Shenghan Zheng, Xiaokun Chen, Yifeng He, Yubo Li, B. You, Haotian Shen, Jiankai Sun, Shuyi Wang, Qunhong Zeng, and 30 more · February 2026

Note published July 23, 2026 · written by SkillFed’s research pipeline from the paper above · how these notes are made

AI-assisted notes · reviewed by SkillFed Agentic benchmarks

SkillsBench runs Agent Skills — structured procedural packages loaded into an LLM agent's context at inference time — through a controlled, paired test instead of a single-condition demo. The benchmark assembles 87 tasks across 8 domains (software engineering, natural science, cybersecurity, finance, and more), each shipped with a curated Skill and a deterministic verifier so pass/fail isn't a judgment call. The task pool itself passed through selection: 400 candidate submissions from 142 contributors, an automated gate (structural integrity, oracle execution, AI-text detection, leakage checks), then human review of at least 30 minutes per task, for a 22% acceptance rate. Every task runs three trials under matched no-Skills and curated-Skills conditions, across 18 model-harness combinations spanning OpenHands, Claude Code, Gemini CLI, and Codex CLI paired with models like GPT-5.5, Gemini 3.1 Pro, GLM 5.1, and MiniMax M2.7.

Average pass rate across all 18 configurations climbs from 33.9% without Skills to 50.5% with them — a 16.6 percentage-point gain, or a 25.5% normalized gain against the no-Skills ceiling. The gain is uneven: individual configurations range from +4.1 to +25.7 points, and domains split just as widely, from +28.8 points in natural science down to +11.6 in software engineering. Bundle size matters more than raw coverage — a single focused Skill adds +18.0 points, two-to-three add +19.0, and four-or-more collapse to +10.1, roughly half the benefit. A smaller model carrying Skills (MiniMax M2.7 at 34.9%) outperforms a larger model running bare (GLM 5.1 in OpenHands without Skills, at 32.7%). The sharpest result: Skills a model writes for itself don't just underperform curated ones, they push pass rates 8.1 to 11.5 points below the no-Skill baseline on the same configurations where curated Skills add 18.2 to 24.8 points.

Key numbers

Curated Skills avg pass-rate gain	+16.6 pp (33.9% → 50.5%)
Normalized gain	25.5%
4+ skill bundles gain (vs +19.0 pp for 2-3)	+10.1 pp
Self-authored Skills vs no-Skill baseline	-8.1 to -11.5 pp
Tasks with negative Skill delta	13 of 87

What this means for skill authors

For skill authors, the bundle-size result is close to a design rule: pack no more than three modules into a Skill before the marginal instruction starts subtracting value instead of adding it. The failure analysis backs this up — of the 13 tasks out of 87 where Skills made things worse, the recurring causes were heavyweight pipelines that displaced a model's already-strong default strategy, not missing capability. Comprehensiveness reads as diligence to a human reviewer; to the agent consuming it at inference time, it's often just noise competing with a plan it already had working.

The more consequential number is what happens when a model writes its own Skill: 8.1 to 11.5 points below doing nothing at all, on the exact configurations where a curated version of the same Skill adds 18.2 to 24.8 points. That gap isn't a marginal edge case — it's the difference between a Skill helping and a Skill actively sabotaging the run, and it makes the strongest case yet for treating a skill catalog as a vetting problem rather than a hosting problem. Unreviewed, self-published instructions aren't neutral filler; they're a measurable regression risk, which is the qualitative argument for scanning and curating skills before they ever reach an agent's context.

There's a portability wrinkle too: the same model shows different gains depending on which harness surrounds it (Gemini 3.1 Pro reads 60.8% natively but drops to 52.8% inside OpenHands), so a Skill validated in one agent runtime doesn't automatically carry its measured benefit into another. Authors and evaluators should treat harness as a variable in any efficacy claim, not a footnote to it.

Skills related to this research

Rtk Rtk helps you set up and troubleshoot Rust Token Killer, a CLI tool that rewrites shell commands into token-efficient versions for AI agents. Install via Homebrew, script, or Cargo, then initialize RTK for your specific agent—Claude Code, Codex, Gemini CLI, Cursor, Copilot, Windsurf, Cline, or OpenCode. Use direct `rtk` commands like `rtk git status` or `rtk read` when automatic hooks aren't available.★ 38 team-frontend-debug team-frontend-debug orchestrates a multi-role team for frontend quality assurance, routing feature lists to a testing pipeline or bug reports to a debugging pipeline. Both flows leverage Chrome DevTools MCP for browser inspection, DOM analysis, console monitoring, and performance tracing. The coordinator role parses your input, spawns specialized workers (tester, reproducer, analyzer, fixer, verifier), and manages progress across phases.★ 2,142 ai-llm-inference This skill delivers operational patterns for tuning LLM inference performance, cost, and reliability in production. It covers latency budgeting, continuous batching, KV-cache optimization, speculative decoding, quantization strategies, and parallelism choices—with decision trees and checklists to guide infrastructure and serving stack tuning.★ 69 paper-writing Paper-writing chains five specialized sub-skills into a single end-to-end workflow: outline planning, figure generation, LaTeX authoring, PDF compilation, and iterative review. Feed it a research narrative or existing plan, specify your target venue (ICLR, NeurIPS, ICML, CVPR, ACL, AAAI, ACM, IEEE), and optionally reference a style guide—the skill handles the rest, producing a polished paper directory with source and compiled output.★ 13,939

Related notes

References

Li, Liu, Chen, et al., "SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks" (arXiv:2602.12670, 2026)