Compiling skills into LoRA weights beats prompting them in-context by up to 21 points, on 64% fewer prefill tokens

Notes on LatentSkill: From In-Context Textual Skills to In-Weight Latent Skills for LLM Agents (arXiv:2606.06087) — Ao Yu, Chenyue Zhou, Tianyi Xu, Zihan Guo, Rong Shan, Zhihui Fu, Jun Wang, Weiwen Liu, Yong Yu, Weinan Zhang, Jianghao Lin · June 2026

Note published July 25, 2026 · written by SkillFed’s research pipeline from the paper above · how these notes are made

AI-assisted notes · reviewed by SkillFed Skill evolution

LatentSkill replaces the standard approach of stuffing a skill's text into the prompt at every decision step with a hypernetwork that compiles the skill document directly into weights. Given the skill's text, the hypernetwork runs one forward pass and emits a skill-specific LoRA adapter — a low-rank update mounted onto a frozen Qwen3-8B backbone — so the model conditions on task history alone, with no per-step skill tokens sitting in context. It's pretrained on 171,000 deduplicated GitHub skill documents (roughly 300 million tokens), then fine-tuned with trajectory-supervised learning against teacher agent runs on ALFWorld and a multi-dataset Search-QA suite. Every result is measured against two baselines: a vanilla agent with no skill access, and an in-context skill agent that inserts the identical skill text into the prompt at each step.

On ALFWorld, LatentSkill reaches 74.3% success on seen tasks and 69.4% on unseen ones — beating the in-context baseline's 52.9% and 56.0% by 21.4 and 13.4 points — while cutting prefill tokens by 64.1% (0.44k vs. 1.21k). On Search-QA, exact match climbs to 35.6% against the baseline's 32.6%, a 3.0-point gain running on 72.2% fewer skill tokens per step. The generated adapters aren't just smaller, they're structured. LoRAs for related skills cluster tightly in weight space by domain. Turn up the adapter's injection coefficient and behavior scales smoothly, then collapses once it passes an optimum. And when a skill's textual decomposition matches the granularity of its weight update, skills combine through parameter-space arithmetic — pushing one compositional task from 61.5% to 84.6% success. Under adversarial prompt manipulation the gap widens further: a prompt injection hijack attack collapses the in-context baseline to 8.57% success while LatentSkill holds at 38.6%.

Key numbers

ALFWorld success, LatentSkill vs. in-context skill (seen/unseen)	74.3%/69.4% vs. 52.9%/56.0% (+21.4/+13.4 pts)
ALFWorld prefill-token reduction	64.1% fewer tokens (0.44k vs. 1.21k)
Search-QA exact match, LatentSkill vs. in-context skill	35.6% vs. 32.6% (+3.0 pts), 72.2% fewer tokens
Aligned component-merged composition vs. single skill (seen split)	84.6% vs. 61.5% success
Success under prompt-injection hijack attack, in-context vs. LatentSkill	8.57% vs. 38.6%

What this means for skill authors

The headline is a rare double win: LatentSkill is cheaper to run and better at the task, not a nicely-priced tradeoff between the two. That combination — accuracy up, prefill cost down by roughly two-thirds — reframes what a skill costs to use after the first read. Instead of a block of prose an agent re-reads at every step, a skill becomes a one-time weight patch that runs free afterward. For any architecture that pulls the same skill across dozens of steps in a long-horizon task, those savings compound directly into latency and spend, without the usual accuracy tax.

The composability result is the one to read carefully before assuming this generalizes to any skill library. Naively merging two skill adapters — whether directly or by concatenating their text — actually underperforms the single skill alone on the unseen split; composition only helps once a skill's textual structure lines up with the granularity of its weight update. That's a real constraint on how a skill would need to be authored for this pipeline — monolithic prose blocks likely compose worse than skills factored into clean, separable components. It also marks this as platform infrastructure rather than a solo author's tool: the hypernetwork needs pretraining on a large skill corpus (171K documents here) before it compiles a novel skill well.

The prompt-injection numbers argue for weight-space skills as a mitigation, not a cure. A hijack attack collapses the in-context baseline from 52.9% to 8.57%, because the skill text sitting in the prompt is exactly the surface an attacker can manipulate. LatentSkill degrades too, just to a materially higher floor, because its instructions never appear as text a prompt can override. That's the same instinct behind scanning what a skill actually does against what it claims to do before it reaches a catalog — no single representation, plaintext or weights, is immune on its own, so the check has to happen at more than one layer.

Skills related to this research

seedance-recipes Seedance-recipes provides production-ready recipe patterns for video content across genres: product, lifestyle, drama, music video, landscape, commercial, animation, and more. Each recipe preserves core creative constraints while inviting customization of subject, camera, lighting, and sound. Use recipes as proven starting shapes, not rigid templates.★ 5,445 fal-text-to-image fal-text-to-image provides access to multiple text-to-image generation models including FLUX variants, SDXL, and specialized options like Recraft for design assets. Configure parameters like guidance scale, inference steps, image size presets, and batch generation to control output quality and speed.★ 49 project-maintainer This skill performs a complete health check on projects following the Open Source Framework pipeline. It verifies phase gates, syncs task tracking across multiple sources, identifies stale content and broken links, and surfaces exactly what needs attention next. Use it for maintenance checks, progress updates, or whenever you need clarity on project status.★ 351 statistical-testing Statistical Testing provides researchers with advanced methods for hypothesis testing, Bayesian inference, survival analysis, time series modeling, and meta-analysis. The skill covers multiple comparison corrections, effect size calculations, and bootstrap/permutation approaches with APA-compliant reporting standards.★ 869

Related notes

References

Yu, A., Zhou, C., Xu, T., Guo, Z., Shan, R., Fu, Z., Wang, J., Liu, W., Yu, Y., Zhang, W., & Lin, J. (2026). LatentSkill: From In-Context Textual Skills to In-Weight Latent Skills for LLM Agents. arXiv:2606.06087.
Hu, E. J., et al. (2021). LoRA: Low-Rank Adaptation of Large Language Models. arXiv:2106.09685.
Charakorn, R., Cetin, E., Tang, Y., & Lange, R. T. (2025). Text-to-LoRA: Instant Transformer Adaption. arXiv:2506.06105.
Shridhar, M., Yuan, X., Côté, M., et al. (2021). ALFWorld: Aligning Text and Embodied Environments for Interactive Learning. arXiv:2010.03768.
Yao, S., Zhao, J., Yu, D., et al. (2023). ReAct: Synergizing Reasoning and Acting in Language Models. arXiv:2210.03629.