Four-Way Attribution Gates Every Update to a Million-Scale Skill Corpus

Notes on SkillsVote: Lifecycle Governance of Agent Skills from Collection, Recommendation to Evolution (arXiv:2605.18401) — Hongyi Liu, Haoyan Yang, Tao Jiang, Bo Tang, Feiyu Xiong, Zhiyu Li · May 2026

Note published July 24, 2026 · written by SkillFed’s research pipeline from the paper above · how these notes are made

AI-assisted notes · reviewed by SkillFed Skill evolution

SkillsVote treats the full lifecycle of an agent skill — collection, recommendation, attribution, and evolution — as one governance problem instead of four separate ones. The starting point is a million-scale open-source skill corpus, profiled for what environment each skill assumes, how well-formed it is, and whether its claims can actually be checked; only skills that clear the verifiability bar get synthetic tasks built against them. At use time, an agent skips flat similarity lookup and instead runs agentic library search over structured skill folders, surfacing the instructional context a task needs before execution starts.

After a run finishes, the trajectory is decomposed into skill-linked subtasks, and the outcome gets attributed across four sources — the skill's own guidance, the agent's own exploration, the environment, and the raw result signal — rather than credited wholesale to "it worked." Only outcomes traceable to genuine, reusable skill-guided success pass through this evidence-gated update filter into the library; everything else gets dropped, not folded in as noise. Tested on Terminal-Bench 2.0 and SWE-Bench Pro, that discipline pays off two ways: skills keep improving online as an agent works through a live task stream, and skills built offline — from historical trajectories or curated open-source libraries — transfer as frozen resources, no retraining needed.

Key numbers

raw open skill ecosystems, baseline problem	Redundant, uneven, environment-sensitive artifacts
open-source skill corpus profiled for governance	Million-scale
benchmarks used to evaluate gains	2 (Terminal-Bench 2.0, SWE-Bench Pro)
outcome-attribution categories per trajectory	4 (skill, agent exploration, environment, result)
complementary improvement pathways	2 (online evolution, offline frozen-library transfer)

What this means for skill authors

The interesting design decision isn't "let skills evolve" — most skill-learning work claims that. It's the refusal to treat a completed task as proof a skill helped. A run can succeed because the environment was forgiving, or because the agent explored its way past a bad skill rather than because of it, and folding either case back into the library corrupts the exact resource the next agent will lean on. Splitting credit four ways before writing anything back is a much higher bar than "did the eval pass," and it's a check that has to sit outside the model that produced the skill in the first place.

For anyone maintaining a shared skill catalog, the profiling stage — triaging environment requirements, quality, and verifiability before a skill is even eligible for the corpus — echoes the instinct behind vetting a skill before it's trusted at scale: check what it claims against what it actually needs, on the way in, not after the damage is done. SkillsVote runs that check at corpus-construction time; a catalog built around dual-tool scanning runs a version of the same check at admission time. Different stage of the pipeline, same posture.

For skill users, the online/offline split is the practical takeaway. A frozen library built from curated, vetted skills is a safer default to depend on than one that's silently rewriting itself from every task-stream discovery that merely looked successful. Evidence-gating is an argument for keeping that write-back conservative — admit the rare, well-attributed win, not every trajectory that happened to end in a passing eval.

Skills related to this research

performance-attribution Break down why your portfolio outperformed or underperformed its benchmark by isolating contributions from sector positioning, individual stock picks, and market timing. Performance Attribution uses Brinson decomposition, factor regression, and timing models to reveal the true drivers of excess return across multiple time periods.★ 28,096 performance-attribution Decompose active returns into the sources that drove outperformance or underperformance—allocation decisions, security selection, and their combined effects. Supports equity attribution via Brinson-Fachler, factor-based decomposition, fixed-income yield and curve analysis, and currency impact for international portfolios.★ 159 ad-account-auditor This skill systematically reviews paid advertising accounts to validate return-on-ad-spend calculations and ensure measurement reliability. Use it before increasing budget allocation to catch tracking gaps, attribution errors, or configuration issues that could distort performance data.★ 2,469 doc-validator doc-validator enforces consistency rules across your entire SDD documentation corpus—catching broken links, orphaned artifacts, cumulative-tag gaps, duplicate IDs, and traceability gaps across all 8 layers. It runs declarative checks against your corpus structure and can optionally repair safely-fixable issues like missing reciprocal links or stale metadata. Use it before releases or after batch generation to ensure your documentation chain stays machine-verifiable and traceable.★ 16

Related notes

References

Hongyi Liu, Haoyan Yang, Tao Jiang, Bo Tang, Feiyu Xiong, Yuyu Luo, Zhiyu Li, "SkillsVote: Lifecycle Governance of Agent Skills from Collection, Recommendation to Evolution," arXiv:2605.18401 (May 2026).
Terminal-Bench 2.0 (agentic terminal-use benchmark, referenced as one of the two evaluation suites).
SWE-Bench Pro (extended, professional-grade SWE-Bench evaluation suite, referenced as the second evaluation suite).