Skill retrieval 30 papers

Agent Skills packs — the folder-based standard for handing an agent on-demand expertise — expose only a short description until a task matches it, then load an indexed body, then the specific passages: that's progressive disclosure . The first controlled test of the idea pits raw-document navigation against flat and hierarchical skill-pack designs, plus a classical hybrid retriever, across three agent harnesses (Codex, Pi, Claude-Code) and three model families. The benchmark itself is new: LoongDoc turns the ∞Bench long-document set into a live environment where agents navigate a sandboxed file system instead of a fixed context window , testing single books and libraries of 5, 10, and 20 books stitched together.

SkillReranker treats skill selection as a graph-matching problem, not a similarity search. It decomposes a task into an ordered sequence of subtasks and intermediate sub-states, then parses every candidate skill the same way: a precondition state — what has to be true before it fires — and a completion state — what's true once it runs. Those states become nodes in a directed acyclic execution graph , with each skill wired in as an edge from whichever node matches its precondition to whichever downstream node matches its completion. That structure is a direct answer to what the paper calls the granularity gap : task requirements are specific, skill descriptions are written generic for reuse, and flat semantic matching keeps mistaking topical overlap for functional fit. Once the graph exists, SkillReranker walks it node by node, applying a split condition — comparing the strength of skills that complete before a node, start after it, or merely pass across it — to carve the task into intervals. Inside each interval, a cross-encoder scores the surviving candidates by summing the log-probability of two relevance checks, one against the full task instruction and one against just that interval's subtask text (equivalent to their geometric mean), and keeps the top scorer.

LLM agents built on skill libraries hit a bottleneck once the library grows: choosing what to load stops being a lookup problem and becomes a joint decision over subset, count, and order — three choices that retrieval and full-library approaches never handle together. SkillComposer treats this as structured skill composition . A frozen Qwen3-Embedding-0.6B encoder turns the task and skill metadata into a 256-dimensional vector; a compact autoregressive decoder (3 layers, 256 hidden dimensions, 4 attention heads) reads that vector and emits skill identifiers one at a time until a STOP token. Subset, count, and order all fall out of that single decoding pass. At inference the decoder doesn't work alone: its logits are fused with a TF-IDF relevance score and a separately trained set-membership head, what the paper calls retrieval-augmented decoding . Training data is 9,872 task-skill records built around a real, 196-skill human-curated library — 65 real SkillBench tasks plus synthetic single- and multi-skill compositions grounded in a 196-node skill dependency graph.

Existing graph-based multi-agent design treats agents as closed-set entities : fix a roster of agents, roles, or groups first, then optimize who talks to whom. SIGMA drops that assumption. Given a task and a library of reusable skills, its skill-incidence graph predicts a skill-agent incidence matrix that assigns skills onto task-conditioned agent slots, builds each agent's embedding from the skills it was handed, and only then decodes a communication topology over the agents it just built. At runtime, skill-specific mailboxes route messages straight to the capability responsible for them — the incidence structure isn't just a planning artifact, it does the actual routing during execution.

Compositional skill routing formalizes what happens when a query needs more than one skill: decompose it into atomic sub-tasks, retrieve a skill for each, then compose the results into an executable plan. SkillWeaver, the framework built to do this, has three parts — an LLM decomposer, a bi-encoder retriever indexed with FAISS , and a dependency-aware DAG planner that orders and parallelizes the resulting steps. To measure it, the authors built CompSkillBench: 300 compositional queries (150 easy/2-skill, 100 medium/3-skill, 50 hard/4-to-5-skill) run against 2,209 real skills pulled from the public Model Context Protocol (MCP) server ecosystem and sorted into 24 functional categories.

daVinci-kernel splits CUDA/Triton kernel generation across three roles under one shared LLM backbone. A Selection Agent retrieves candidate optimization techniques through BM25 pre-filtering plus LLM reranking, and a Policy Agent writes the kernel across multiple turns, conditioned on whichever techniques got selected. The third role, a Summary Agent, turns a rollout's winning trick into a new reusable entry for the shared skill library — but only once execution-based verification confirms the speedup reproduces at least 1.2x when the technique is re-run on its original task, not merely praised by a judge model. All three roles are cold-started with SFT on diversity-filtered demonstrations, then jointly optimized end-to-end with multi-turn REINFORCE and per-agent advantage estimation, so the policy, the retriever, and the summarizer improve together rather than in separate stages.

A retriever can nail the capability family — the right general kind of skill for the job — and still hand back the wrong specific one. SkillResolve-Bench 1.0 names this failure same-capability execution-risk retrieval : each query pairs a helpful skill against a query-specific risky sibling — same capability, but wired to a stale resource, an unmet precondition, or the wrong procedure. The benchmark draws 661 helpful/risky pairs (630 from SRA-Bench, 31 from SkillsBench) and tests them against a 7,982-candidate pool that includes 6,660 public SkillRet skills, split query-disjoint at 446/68/147. A new metric, harmful sibling rate (HSR@K), tracks whether the risky sibling shows up in the top-K results at all.

SkillDAG models a large skill library as a typed directed graph instead of a flat embedding index. Five edge types — depends_on, specializes, composes_with, similar_to, conflicts_with — encode relationships cosine similarity can't see: prerequisite chains, narrower-vs-general variants, synergistic pairings, redundant duplicates, and incompatible combinations. Construction starts cold: each skill gets embedded twice, once for what it does and once for what it requires (a HyDE-style query expansion), and an LLM classifier assigns edge types between candidate pairs. At inference time the agent calls a single search() function that returns three parallel signals — top-K semantic matches, typed-edge neighbors from graph traversal, one-hop conflict warnings — and decides for itself how to weigh them, rather than have a fixed formula fuse them upstream. A propose-then-commit protocol lets the agent register new edges it discovers mid-episode, subject to acyclicity and non-contradiction checks, so the graph accumulates structure across runs instead of freezing at construction.

LLM agents that compose multiple skills to solve a task depend on a retriever picking the right combination. That's a different problem from picking the right individual skill. A retriever can score every skill against a query correctly and still return two skills that each look relevant but cannot actually work together for that query — query-conditioned skill compatibility isn't recoverable from per-pair relevance scores. The fix here starts from a byproduct most LLM data-synthesis pipelines throw away: when an LLM decides whether a candidate skill belongs in a query's retrieval set, its rejections already record which skills should not be retrieved together for that query. Reject-as-Resource (R3) keeps those rejections and turns them into training signal instead of discarding them. Built on that idea, R3-Skill is a bilingual Chinese-English benchmark: 10,246 skills (deduplicated from 95,212 raw candidates) across 8 thematic domains, 41,592 accepted queries, and 32,828 of the LLM's own rejected pairings, tagged across an eight-class taxonomy of rejection reasons.

SelSkill treats every point where an agent could call an agent skill as a binary decision — invoke or skip — instead of assuming relevance implies use. It flags candidate decision points with predictive entropy (how uncertain the model is about its next token), then forks the trajectory at that exact prefix into two rollouts: one calls the skill, one doesn't. Both run to completion and get labeled by outcome and step-efficiency — success beats failure, and between two successes the shorter one wins. Those local invoke/skip pairs get combined with whole-trajectory, episode-level preferences (successful runs vs. failed ones) and trained jointly with Direct Preference Optimization (DPO). The loss is masked to just the few assistant turns right after each branch, so the gradient targets the invocation decision itself, not the whole trajectory.

Skill injection is usually treated as a retrieval problem: rank candidates, keep the top K, paste their descriptions in unchanged. That static treatment turns out to be the bottleneck, not the skills themselves. On one tau2-bench airline task with an 82-skill library, per-skill benefit over a no-skill baseline ranges from -0.20 to +0.40: an insurance-policy skill that shares surface keywords with the request actively hurts, while four unrelated policy-reference skills each help substantially. Feed the whole library in at once and pass rate collapses to 24.8% on average across benchmarks — worse than injecting nothing at all (40.2%).

Web agents that click through visual interfaces usually get more capable by spending more at inference: sampling multiple rollouts, adding a verifier pass, or running an offline discovery phase before the real task starts. Trace VisualWebArena trajectories and the waste has a shape: agents loop on repeat actions that burn steps without changing the page state, tool-discovery phases quietly cost roughly 100 steps per tool once a library passes 50 tools, and most baseline pipelines reuse under 11% of their prompt through caching. PANDO answers with a single-rollout design — no sampling, no pre-evaluation discovery budget — built around a structured Skill Library of pattern-triggered rules and parameterized routines, a reflector that checks whether a step actually moved the task forward, confidence-based demotion for skills that keep failing, hierarchical routing that reserves expensive reasoning for novel planning, and a prompt layout built to keep prompt cache utilization high.

A skill library doesn't have to get big before it starts working against you. Two Databricks researchers pin down exactly when: past the handful of skills a task actually needs, agents get measurably worse. They define an oracle skill set for each task — the skill or skills that, tested alone, lift pass rate by at least 4 percentage points over no skill at all — then track the pass-rate gap between an agent given only that oracle set and the same agent buried in the full library. A probability identity over which skills the agent actually invokes splits that gap cleanly into two effects: skill shadowing , where the agent picks the wrong skill more often as the library expands, and context overhead , where a longer list of skill descriptions hurts execution even when selection is right. They bound both effects mathematically and test the split on SkillsBench , running Claude Haiku 4.5 and Sonnet 4.6 agents against libraries of 52, 102, and 202 skills across 38 (task, model) pairs and 2,545 trajectories.

Researchers stress-tested skill routing across 15 frontier LLMs and 1,141 real-world skills pulled from public Agent Skills repos, Claude Code, MCP servers, and community GitHub, organized into 14 software-automation domains. Using 4,075 tasks swept across library sizes from 10 to 500 skills and pipelines of 1 to 10 steps, they logged more than 3 million routing and execution decisions, then fit two separate laws: a routing law for single-step skill selection and an execution law for what happens once a chosen skill actually runs.

SkillGraph reframes an agent's skill library as a directed graph instead of a flat, similarity-ranked list. Each node is a skill distilled by a teacher model (OpenAI o3) from the agent's own successful and failed trajectories; each typed edge encodes one of three relations — prerequisite, enhancement, or co-occurrence — carrying a weight that strengthens or decays with experience. Given a new task, the system doesn't just pull the nearest skills by embedding distance: it seeds from task-relevant nodes, walks backward through prerequisite edges (depth 2) to recover what has to happen first, walks forward through a width-3 beam to find what builds on it, then topologically sorts the result into an ordered subgraph capped at eight skills — what the paper calls graph-aware retrieval . That ordered plan conditions the policy, and the graph keeps evolving during GRPO reinforcement learning: new skills get inserted from failures and near-duplicates get merged, while skills that succeed inconsistently get split apart and chronic failures get deprecated.

Agent skills now have a formal definition: a skill is the tuple S = (M, R, C) — a root instruction document M, a set of auxiliary resources R (references, templates, scripts), and applicability conditions C that govern when the skill gets retrieved and used. The point of formalizing it is closing the procedural gap — giving an agent a tool through an API or the Model Context Protocol tells it what it can do, not when to do it, how to sequence it with other capabilities, or how to judge whether the result succeeded. Literature tracked from April 2023 to April 2026 sorts into four lifecycle stages — representation, skill acquisition , skill retrieval and selection, and skill evolution — spanning eight application domains, from software engineering and web/GUI automation to robotics, finance, healthcare, and social simulation.

SkillSelect-Serve treats a reusable agent skill as a Skill Service — a structured record of what it does, its input/output shape, which tools it needs, how many tokens it costs to load, and a risk label — rather than a passage to rank by text similarity. A five-stage pipeline builds these profiles from raw skill documents, then converts each incoming task into a structured requirement object without naming a candidate skill. Standard BM25/dense/cross-encoder retrieval pulls a high-recall candidate pool, and a task-conditioned suitability estimator scores each one. The final stage, constrained projection , is a greedy skip-and-continue packer: it walks the ranked list and admits a skill into the bundle only if the whole package still fits the token budget, stays under an aggregated risk ceiling, and draws only on tools the deployment actually has. Infeasible candidates get skipped, not stalled on — their budget rolls forward to the next-best option.

Retrieval for agent skill libraries has been treated as a search problem: find the k most relevant skills and hand them to the agent. GoSkills treats it as a presentation problem instead. It builds anchor-centered skill groups offline from a typed skill graph — clusters of at most three skills carrying explicit roles like prerequisite, formatter, or checker. At query time it picks an anchor group, pulls in supporting groups through a separate group graph , caps the result to a bounded set of atomic skill payloads, and renders everything into a fixed four-field execution contract : Start, Support, Check, Avoid. Nothing about the downstream agent, the skill payload format, or the execution environment has to change — GoSkills sits in front of an existing library rather than replacing it.

LLM agent skills today live almost entirely as prose — SKILL.md -style files that bundle a skill's invocation rules, execution phases, and side effects into a single block of natural-language text. The Scheduling-Structural-Logical ( SSL ) representation pulls that block apart into three typed layers: a scheduling record for when and how a skill gets invoked, a structural graph of execution scenes (prepare, acquire, act, verify, and so on), and a logical graph of atomic actions with explicit resource boundaries. The three-way split borrows directly from Schank and Abelson's cognitive-linguistics work — Memory Organization Packets, Script Theory, Conceptual Dependency . An LLM normalizer (DeepSeek-V3.2) converts raw SKILL.md text into SSL's JSON schema; a 100-skill human audit found 83% of the normalized fields faithfully grounded in the source document.

LEGO breaks the digital front-end chip-design flow — spec generation, RTL coding, testbench generation, simulation, debug — into six fixed steps and represents every reusable capability within those steps as a circuit skill : a seven-field unit (name, function, constraints, entry command, IO spec, schema, completion criteria) that any code agent can invoke like a subroutine. An automated three-stage pipeline builds the library from there — summarize capabilities, map to a workflow step, standardize the fields — mining more than 100 papers and 11 open-source EDA-agent projects (VerilogCoder, MAGE, Spec2RTL-Agent, AutoBench, the hierarchy-verilog line of work, RTLFixer, and others) down to 42 skills across 24 functional groups. A companion retrieval layer, Agent Skill RAG , skips embedding models entirely: it loads only short skill descriptions up front and pulls the full entry — symptom, root cause, fix — once a match is found, keeping lookup at sub-millisecond speed while letting new fixes get appended as plain text.