29 mined skills beat prompt-search code optimizers by up to 12.5 points — without running the code

Notes on EffiSkill: Agent Skill Based Automated Code Efficiency Optimization (arXiv:2603.27850) — Zimu Wang, Yuling Shi, Mengfan Li, Zijun Liu, Jie M. Zhang, Chengcheng Wan, Xiaodong Gu · March 2026

Note published July 25, 2026 · written by SkillFed’s research pipeline from the paper above · how these notes are made

AI-assisted notes · reviewed by SkillFed Skill retrieval

EffiSkill treats recurring slow-to-fast code rewrites as reusable agent skills instead of one-off prompts. Stage I mines Operator Skills (concrete transformation mechanics) and Meta Skills (diagnosis, retrieval, and composition logic) from 900 Python and 900 C++ slow/fast program pairs, compiling a portable skill library. Stage II applies that library to unseen programs through an execution-free pipeline: diagnose the bottleneck, retrieve matching skills, compose a plan, generate candidates — no compiling or running the program during inference.

On EffiBench-X (623 tasks across six languages, evaluated here on the Python and C++ subsets), EffiSkill raises optimization success over the strongest baseline by 3.7 to 12.5 percentage points, tested with GPT-5-mini and Qwen3-Coder-30B-A3B-Instruct. The mined library holds 29 skills, but usage entropy shows only about 10 to 13 are effectively active — a concentrated core, not a sprawling catalog. Swap the retrieved skills for random ones and the effect is stark: Qwen3-Coder's eight-candidate success rate drops from 36.6% to 13.8%. The gain comes from matching the right mechanism to the right bottleneck, not from having any code-transform hint in context.

Key numbers

Success-rate lift vs. strongest baseline	+3.7 to +12.5 pts
Mining corpus (slow/fast pairs)	900 Python + 900 C++
Skill library size	29 skills (~10–13 effectively active)
OPT@8 collapse, Qwen3-Coder, random vs. retrieved skills	36.6% → 13.8%

What this means for skill authors

The core wager is that a "skill" here is narrower and more mechanical than a workflow doc — a slow-to-fast transformation pattern, mined at the operator level, matched to a bug without ever compiling or running the candidate fix. For skill authors, that's a useful discipline: the highest-leverage skill isn't the biggest one, it's the one whose firing condition (does this pattern actually apply here) is diagnosable from static inspection alone, before any sandbox gets involved.

The ablation is the real lesson for anyone building a skill catalog rather than a single skill. Swapping a retrieved skill for a random one from the same library didn't just blunt the gain — it nearly cut Qwen3-Coder's success rate in half. Skill retrieval quality, not skill quantity, is what the number is actually measuring: dumping every trick into context and hoping the model picks correctly loses badly to a well-matched single skill. That's the argument for curation over volume, and it's why a catalog entry should be treated as a match to verify against the task, not a keyword to autocomplete.

One thing the ablation buries: only 10 to 13 of the 29 mined skills carry most of the usage. A compact, high-frequency core does the real work; the long tail exists but rarely fires. That's the shape we'd expect from any well-vetted skill catalog — curation means separating that core from the noise, not maximizing how many entries the catalog holds.

Skills related to this research

wiki-retrieve wiki-retrieve upgrades vault search from page-level to chunk-level granularity, combining contextual prefixes, BM25 sparse retrieval, and dense reranking to reduce retrieval failures. Setup is opt-in with built-in privacy gates: run fully on-machine with synthetic prefixes, or opt into Anthropic API or claude CLI tiers for better context generation.★ 10,007 audio-router audio-router intelligently directs audio processing requests to the right specialized skill, whether for playback control, sound analysis, or reactive audio handling. Built for Claude, it streamlines how audio workflows are orchestrated across multiple focused capabilities.★ 8 seedance-recipes Seedance-recipes provides production-ready recipe patterns for video content across genres: product, lifestyle, drama, music video, landscape, commercial, animation, and more. Each recipe preserves core creative constraints while inviting customization of subject, camera, lighting, and sound. Use recipes as proven starting shapes, not rigid templates.★ 5,445 Materialize Terraform Self Managed This skill provides Terraform modules for deploying Materialize across AWS (EKS), Azure (AKS), and GCP (GKE), handling cloud infrastructure, Kubernetes setup, and instance configuration. Use it to build networking, databases, storage, and the Materialize operator stack, or to troubleshoot and upgrade existing deployments.★ 3

Related notes

References

Wang et al., EffiSkill: Agent Skill Based Automated Code Efficiency Optimization, arXiv:2603.27850 (2026)
Qing et al., EffiBench-X: A Multi-Language Benchmark for Measuring Efficiency of LLM-Generated Code, arXiv:2505.13004 (2025)
Gao et al., Search-Based LLMs for Code Optimization, arXiv:2408.12159 (2024)
Yao et al., ReAct: Synergizing Reasoning and Acting in Language Models, arXiv:2210.03629 (2022)