One decay slope predicts both routing collapse and execution rescue — fixing it takes routing accuracy from 71.3% to 91.7%

Notes on The Scaling Laws of Skills in LLM Agent Systems (arXiv:2605.16508) — Qiguang Chen, Q. Yu, Yuhang Gu, Zhuoye Huang, Hanjing Li, Hongyu Liu, Siming Liu, Jinhao Liu, Dengyun Peng, Jiangying Wang, Zheng Yan, Fanqing Meng, and 3 more · May 2026

Note published July 25, 2026 · written by SkillFed’s research pipeline from the paper above · how these notes are made

AI-assisted notes · reviewed by SkillFed Skill retrieval Bridge: retrieval × security

Researchers stress-tested skill routing across 15 frontier LLMs and 1,141 real-world skills pulled from public Agent Skills repos, Claude Code, MCP servers, and community GitHub, organized into 14 software-automation domains. Using 4,075 tasks swept across library sizes from 10 to 500 skills and pipelines of 1 to 10 steps, they logged more than 3 million routing and execution decisions, then fit two separate laws: a routing law for single-step skill selection and an execution law for what happens once a chosen skill actually runs.

Single-step routing accuracy decays logarithmically as the library grows, holding with R² above 0.97 across every model tested. The failure mode shifts as libraries scale: near-duplicate skills compete locally first (worst in the 0.55–0.75 similarity band), then task language stops anchoring clearly and routing drifts across skill families, and eventually black-hole skills — overly broad, vaguely-described skills that absorb disproportionate routing mass — take over, but only once weak anchoring and vague phrasing coincide. On the execution side, correct upstream output rescues difficult downstream decisions by roughly 4×, and the same slope that describes routing collapse also predicts that rescue effect across models without being refit (ρ=0.74, p<0.001). Editing the library with that structure in mind — auditing near-duplicates, tightening description boundaries, narrowing or removing overly broad skills, anchoring prompts — pushes held-out routing accuracy from 71.3% to 91.7%, cuts in-library hijack from 22.4% to 4.1%, and carries through to real execution: pass rate rises from 49.3% to 61.6% on ClawBench and 28.4% to 34.5% on ClawMark.

Key numbers

Held-out routing accuracy after law-guided library edit	71.3% → 91.7%
In-library hijack rate after fix	22.4% → 4.1%
Downstream transfer (ClawBench / ClawMark pass rate)	49.3%→61.6% / 28.4%→34.5%
Decay slope: name-only vs. constraint+example descriptions	b=0.32 vs. b=0.08
Correct-state execution rescue on hard downstream decisions	~4×

What this means for skill authors

For anyone writing a SKILL.md, the description-quality number is the actionable one: a name-only description decays four times faster than one that spells out constraints and examples (b=0.32 versus b=0.08). That's not a style preference — it's a measured multiplier on how fast your skill's routing accuracy erodes as the library around it grows. The risk concentrates in a specific place, too: the 0.55–0.75 similarity band, where a thin description is up against near-duplicate cousins the model can barely tell apart. Write a vague skill into that zone and you're choosing to lose the routing coin flip as the catalog scales.

The black-hole skill finding is a structural argument, not just an authoring tip. An overly broad skill can sit harmlessly in a library of 20 and become a routing sink at 500, because capture only triggers once vague queries meet weak-anchored neighbors — exactly the kind of interaction a raw, unaudited scrape of 1,141 GitHub skills won't surface until it's already expensive to fix. That's the case for editing a library on the way in, not after the fact: a vetted, dual-tool-scanned catalog is positioned to audit description boundaries and near-duplicate overlap before scale turns a merely-broad skill into a parasitic one.

The ~4× rescue effect from correct upstream state cuts both ways for anyone chaining skills into a pipeline. It rewards getting the first call right, but a botched first step doesn't fail gracefully: whether the resulting bad state drags a downstream skill down or gets absorbed depends on the capability gap between the two — below roughly a 0.25 gap, weak ties amplify the error; above it, strong ties can still recover. Multi-step agents should treat the first invocation in a chain as higher-stakes than a flat per-step error rate implies.

Skills related to this research

design-better Design Better applies timeless UX/UI craft heuristics—drawn from Nielsen Norman, Laws of UX, WCAG, and modern AI-product patterns—to separate polished interfaces from merely functional code. It pairs with your design system tokens for visual decisions while owning hierarchy, interaction, accessibility, and motion craft. Use it when building or refactoring UI and want design best practices baked into code generation and review.★ 168 wiki-lint Wiki Lint performs comprehensive health checks on your knowledge base, identifying orphaned pages, broken internal links, contradictory information, and stale content that needs updating. It applies a 30-day confidence decay rule to flag concepts that haven't been reviewed, detects missing cross-course connections, and ensures course overviews and glossary entries are complete. After scanning, it prompts you to choose which issues to fix, then automatically updates all related documentation.★ 195 content-writer Create search-engine-ready content that ranks. This skill handles keyword research integration, semantic structure, and on-page optimization signals so your content performs from day one. Perfect for marketers building content strategies at scale.★ 2,469 flash-sale-scaling Flash Sale Scaling equips you with infrastructure patterns to survive extreme traffic events—50–100× normal load arriving in seconds. The skill covers platform-specific strategies for Shopify, WooCommerce, and BigCommerce, plus custom implementations using Redis atomic inventory, queue-based order intake, and graceful degradation under load.★ 40

Related notes

References

Chen, Q., Yu, Q., Gu, Y., et al. (2026). The Scaling Laws of Skills in LLM Agent Systems. arXiv:2605.16508.
Kaplan, J., et al. (2020). Scaling Laws for Neural Language Models. arXiv:2001.08361.
Yao, S., et al. (2023). ReAct: Synergizing Reasoning and Acting in Language Models. ICLR.
Qin, Y., et al. (2023). ToolLLM: Facilitating Large Language Models to Master 16000+ Real-World APIs. arXiv:2307.16789.
Patil, S. G., et al. (2023). Gorilla: Large Language Model Connected with Massive APIs. arXiv:2305.15334.