Turning a corpus into a skill tree cuts RAG hallucination from 50% to 4.5%

Notes on Don't Retrieve, Navigate: Distilling Enterprise Knowledge into Navigable Agent Skills for QA and RAG (arXiv:2604.14572) — Yiqun Sun, Peng Wei, Lawrence B. Hsieh · April 2026

Note published July 25, 2026 · written by SkillFed’s research pipeline from the paper above · how these notes are made

AI-assisted notes · reviewed by SkillFed Skill retrieval

Corpus2Skill turns a document corpus into something an agent walks instead of queries. An offline compile pass embeds every document, clusters them hierarchically with K-Means, and has an LLM write a summary at each level, producing a tree of SKILL.md and INDEX.md files plus a cross-branch entity index — the same filesystem format and progressive disclosure mechanism used for procedural agent skills, repurposed here to describe what a corpus contains rather than how to do a task. At serve time the agent is preloaded with only skill names and one-line descriptions, then drills from a top-level summary through progressively finer INDEX.md files down to a specific document, backing out of branches that turn out thin. Five baselines take the comparison on WixQA, an enterprise customer-support benchmark of 6,221 support articles and 200 expert-written queries with gold answers: BM25, dense embedding retrieval, hybrid retrieval, RAPTOR-style hierarchical retrieval, and an agentic RAG agent with search tools.

Navigation wins on every quality metric on WixQA. Token F1 lands at 0.456, a 21% relative gain over agentic RAG (0.378) and 25% over dense retrieval (0.364), with factuality 6 points above the next-best method. It grounds better too: hallucination rate falls to 4.5%, against 50% for the same agent searching instead of navigating — though it costs more, at $0.153 per query versus $0.082 for agentic RAG and $0.007–$0.012 for the single-shot retrievers. Swapping the serving model from Sonnet 4.6 to Haiku 4.5 cuts that to $0.093 (−39%) while keeping 92% of the F1 score, which suggests the compiled tree does most of the work, not the model walking it. A follow-up sweep across ten RAGBench subsets complicates the picture: navigation wins on five single-domain sets (by 0.01–0.14 F1), ties on two heterogeneous ones, and loses on three — an open-domain factoid pool, a homogeneous-tabular corpus, and a long-extractive-document set — by 0.04–0.17 F1, because their top-level cluster summaries collapse into generic, indistinguishable labels.

Key numbers

Token F1 vs. agentic RAG	0.456 vs. 0.378 (+21% relative)
Hallucination rate vs. agentic RAG	4.5% vs. 50%
Cost per query	$0.153 (1.9x agentic RAG, 13-22x flat retrieval)
RAGBench 10-subset split	5 wins, 2 ties, 3 losses (0.04-0.17 F1 swing)
Misrouting failures on WixQA	19 of 45 flagged failures traced to wrong top-level branch

What this means for skill authors

The reframe worth sitting with: a hierarchical skill directory here isn't a knowledge base bolted onto an agent skill — it is the skill. The same SKILL.md/INDEX.md scaffolding that ships procedural instructions ships a navigable index instead, and none of it needs a vector database at serve time. For anyone authoring skills meant to carry institutional or product knowledge rather than a workflow, that's a real alternative to embedding a retrieval call inside the skill: compile the corpus once, ship the directory, let the agent read its way down.

The generalization study is the part worth acting on — it draws a hard boundary, not a caveat. Navigation earns its cost premium only where a corpus has real topical structure to recover. The paper's own cluster-width ablation reproduces the same collapse in miniature once summaries get too coarse to route on, and the failure analysis on WixQA itself finds 19 of 45 flagged failures traced to the agent choosing the wrong top-level branch. Flat, homogeneous, or open-domain corpora are still better served by plain retrieval, on this evidence. That's a design decision to make before packaging a corpus as a navigable skill, not after deploying it.

It's also a good test for what a vetted skill catalog should check before listing a corpus-as-skill package: not just that the skill runs, but what shape of corpus it was actually validated against. A misrouted top-level branch fails silently — the agent still answers, just from the wrong evidence — which is exactly the grounding failure that scanning for functional correctness alone won't catch.

Skills related to this research

Ml Engineering Learn to build and operate production machine learning systems with guidance on model deployment, infrastructure setup, and monitoring. Covers MLOps workflows, LLM integration patterns, and best practices for scaling models in real-world environments.★ 6 java Java teaches Spring Boot 3.x development with enterprise architecture patterns, SOLID principles, and modern Java 17+ features. It covers layered code organization, RESTful APIs, proper exception handling, and testing strategies for production applications.★ 202 paper2slides paper2slides transforms research papers into polished presentation decks by orchestrating PDF parsing, outline generation, and slide rendering. The skill guides you through editorial decisions—length, visual quality checks, and design preferences—while handling the technical conversion steps automatically.★ 304 dev-browser dev-browser enables AI agents and developers to automate browser interactions through sandboxed JavaScript execution. Scripts run safely in an isolated QuickJS WASM environment without access to the host system, while maintaining persistent page state across multiple interactions. Navigate websites, extract data, and control web applications programmatically with reliable, repeatable automation.★ 6,481

Related notes

References

Sun, Wei & Hsieh (2026). Don't Retrieve, Navigate: Distilling Enterprise Knowledge into Navigable Agent Skills for QA and RAG. arXiv:2604.14572.
Sarthi et al. (2024). RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval. ICLR.
Lewis et al. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. NeurIPS 33.
Cohen et al. (2025). WixQA: A Multi-Dataset Benchmark for Enterprise Retrieval-Augmented Generation. arXiv:2505.08643.
Friel, Belyi & Sanyal (2024). RAGBench: Explainable Benchmark for Retrieval-Augmented Generation Systems. arXiv:2407.11005.