71% of Mined Scientific Skills Don't Exist in Any Skill Marketplace

Notes on SKILLFOUNDRY: Building Self-Evolving Agent Skill Libraries from Heterogeneous Scientific Resources (arXiv:2604.03964) — Shuaike Shen, Wenduo Cheng, Mingqian Ma, A. Turcan, M. J. Zhang, Jian Ma · April 2026

Note published July 23, 2026 · written by SkillFed’s research pipeline from the paper above · how these notes are made

AI-assisted notes · reviewed by SkillFed Skill evolution

SkillFoundry turns scientific papers, repos, notebooks, and docs into a maintained skill library, not a one-time scrape. It maps a target domain as a domain knowledge tree, then mines the branches with the most resources and the weakest verified coverage. Each artifact gets reduced to an operational contract — scope, inputs/outputs, execution steps, environment assumptions, provenance, and tests — and compiled into an executable skill package. A closed-loop validation pass then expands, repairs, merges, or prunes entries as new results come in, so the library keeps evolving instead of going stale.

The mined library holds up under scrutiny: of 286 skills pulled this way across 27 domains, 71.1% do not exist in the SkillHub or SkillSMP marketplaces, meaning most of what a working science agent needs still isn't packaged anywhere. Plugging the mined skills into a coding agent improved paper-level accuracy on five of the six MoSciBench datasets, lifting the overall average across all six from 43.85% to 53.05%. Built on demand for two genomics tasks, mined skills pushed cell-type-annotation coverage from 81.1% to 99.2% and cut the scDRS workflow's error (RMSE) from 0.11 to 0.02 — though the full seven-criteria qualitative rubric was met clean in only one of three runs.

Key numbers

Skills mined into the library	286 (27 domains)
Mined skills absent from SkillHub/SkillSMP	71.1%
MoSciBench avg paper-accuracy (across all 6 datasets)	43.85% → 53.05%
Cell-type annotation coverage (Codex)	81.1% → 99.2%
scDRS: all 7 qualitative criteria met clean	1 of 3 runs

What this means for skill authors

The headline number isn't the accuracy lift, it's the 71.1% novelty rate. Two marketplaces for agent skills already exist — SkillHub, SkillSMP — and the bulk of what a working science agent needs isn't listed in either. Call it an unmapped demand surface, not a stale catalog. That's the strongest outside argument yet for continuous, automated ingestion and validation over periodic manual curation — the coverage bet SkillFed's dual-tool scanning and vetted catalog are already making.

The genomics numbers are the more honest test, because they aren't benchmark leaderboard entries — they're a single external agent (Biomni) picking up mined skills with no retraining and cutting its scDRS error roughly five-fold, from RMSE 0.11 to 0.02. That's the deployment case most teams are actually in: bolting skills onto an agent someone else built, not training one from scratch. The caveat sits right next to it: the full seven-criteria rubric only passed clean in one of three runs. A mined and validated skill package is proven to execute correctly; reproducing the exact right answer run after run is a separate, harder bar — and one worth testing for before anyone ships this class of skill.

For skill authors, the coding-agent result argues for a division of labor rather than a winner. Mined skills raised cell-type-annotation coverage from 81.1% to 99.2% and closed most of the gap to a hand-built domain specialist, but not all of it. Automated mining is a fast way to raise the floor across many domains at once, not a substitute for an author who knows one domain's edge cases well enough to beat a purpose-built agent outright.

Skills related to this research

Figma Generate Library Build production-ready design systems in Figma by orchestrating multi-phase workflows that extract tokens from code, create variable foundations, construct components with proper variant sets, and document design principles. Works alongside the figma-use skill to enforce quality patterns from real-world systems.★ 1,925 exa-search Exa-search combines keyword and semantic web retrieval for research-grade lookups on scientific and technical topics. Route queries to web search for topic research or URL extraction for fetching specific pages and academic PDFs. Use category filters and domain allowlists to prioritize peer-reviewed journals, preprints, and institutional sources.★ 31,940 parallel-web Parallel Web unifies web intelligence workflows through multiple capabilities: search for current information and research topics, extract content from specific URLs, enrich datasets with web-sourced fields, discover entities by criteria, run exhaustive multi-source reports, and set up recurring web monitoring. Route requests to the right capability based on your need.★ 31,940 documentation-lookup Documentation-lookup pulls live reference material for popular frameworks and libraries when you need setup guidance, API details, or code examples. It resolves library identifiers and retrieves version-specific documentation to ensure your answers stay current and accurate.★ 5,778

Related notes

References

Shen, Cheng, Ma, Turcan, Zhang, and Ma, "SkillFoundry: Building Self-Evolving Agent Skill Libraries from Heterogeneous Scientific Resources," arXiv:2604.03964 (2026).
Huang et al., "Biomni: A General-Purpose Biomedical AI Agent," bioRxiv (2025).
Wang et al., "SpatialAgent: An Autonomous AI Agent for Spatial Biology," bioRxiv (2025).
Zhang et al., "Polygenic Enrichment Distinguishes Disease Associations of Individual Cells," Nature Genetics 54(10):1572–1580 (2022).
Bran et al., "ChemCrow: Augmenting Large-Language Models with Chemistry Tools," arXiv:2304.05376 (2023).