92.5% Skill Reuse From 816 Tokens

Notes on Skill-Pro: Learning Reusable Skills from Experience via Non-Parametric PPO for LLM Agents (arXiv:2602.01869) — Qirui Mi, Zhijian Ma, Mengyue Yang, Haoxuan Li, Yisen Wang, Haifeng Zhang, Jun Wang · February 2026

Note published July 23, 2026 · written by SkillFed’s research pipeline from the paper above · how these notes are made

AI-assisted notes · reviewed by SkillFed Skill evolution

Agents built on chain-of-thought reasoning treat every episode as new — even when the underlying subtask has been solved dozens of times, the reasoning chain gets thrown away the moment the episode ends. Skill-Pro converts finished trajectories into explicit, executable procedures instead of discarding them, formalized as a Skill-MDP: each stored skill bundles an activation condition (when to fire), an execution procedure (the ordered action sequence), and a termination condition (when to hand control back to the base policy). Improving that library without touching model weights is handled by Non-Parametric PPO. Hindsight attribution over a trajectory produces semantic gradients — natural-language descriptions of how a skill should change — and a PPO Gate reruns PPO's clipped-surrogate trust-region check against historical trajectories to decide whether the edited candidate is actually accepted into the library. Score-based maintenance, which scores each skill by its average advantage whenever it's active, prunes it back out the moment it stops paying its way.

Tested in-domain, across tasks, and across agents — reusing skills learned on one base model across three held-out backbones (Gemma-3-4B, Qwen3-32B, and LLaMA-3.3-70B-Instruct) — Skill-Pro reused stored skills 92.5%, 82.5%, and 90.0% of the time. The entire skill library totaled 816 tokens; baselines that store raw trajectories, distilled insights, notes, or workflow graphs ran 40,510 to 391,706 tokens. The ablations are the more telling numbers: pull out the PPO Gate and reuse collapses 76% to 0.222, with the running skill score turning negative — bad skills accumulate once nothing rejects them. Swap score-based pruning for plain FIFO eviction and it gets worse, an 85.8% collapse to 0.131, because the library fills with whatever was written most recently rather than whatever actually works.

Key numbers

In-domain skill reuse rate	92.5%
Cross-task skill reuse rate	82.5%
Total skill-library token footprint	816 tokens (vs. 40,510–391,706 for baselines)
Reuse rate with PPO Gate removed	0.222 (–76% vs. full method)
Reuse rate with FIFO pruning instead of scoring	0.131 (–85.8%)

What this means for skill authors

The ablations are effectively a controlled demonstration of what happens to any skill store without a verification gate: candidates that look plausible in isolation get admitted, low performers never get evicted, and the library's average quality degrades until reuse itself becomes unreliable. That's the exact failure mode a vetted catalog exists to prevent — SkillFed's dual-tool scanning does qualitatively the same job as Skill-Pro's PPO Gate, checking a candidate against evidence before it enters general circulation rather than trusting that it looked fine when it was written.

For anyone budgeting context window against a growing skill collection, the token numbers make the practical case for procedures over transcripts: a procedure-shaped skill costs roughly one to three orders of magnitude less to store than the trajectory it was extracted from, and unlike a raw log it doesn't need re-reading and re-interpreting at use time. The catch is where the cost moves, not whether it disappears — Skill-Pro pays for that compression at write time, rerunning the PPO Gate against historical trajectories every time a skill is proposed or revised. An agent that authors many skills and reuses few gets a worse trade than one that writes rarely and pulls from a shared library constantly, which is the same incentive that should shape a team's choice between authoring a new skill and reusing an existing one from a vetted catalog.

Skills related to this research

fine-tuning-with-trl Fine-tuning with TRL provides post-training methods to align language models with human preferences through multiple approaches. Train models on instruction data with SFT, optimize for preference alignment via DPO without a separate reward model, or run full RLHF pipelines combining supervised fine-tuning, reward modeling, and PPO optimization. GRPO offers memory-efficient online reinforcement learning for resource-constrained setups.★ 11,165 Bencium Impact Designer Bencium Impact Designer creates distinctive, production-grade frontend interfaces that reject generic AI patterns. It guides you through design thinking—understanding purpose, tone, and constraints—then commits boldly to a cohesive aesthetic direction before generating working code. The skill emphasizes intentional creative choices, from brutalist minimalism to retro-futuristic maximalism, ensuring every interface is visually memorable and functionally polished.★ 359 Bencium Innovative Ux Designer Bencium Innovative UX Designer crafts polished frontend interfaces that break free from cookie-cutter patterns. It guides you through intentional design choices—from tone and aesthetic direction to typography and color—then generates real, accessible code that feels thoughtfully made rather than machine-generated.★ 359 fine-tuning-with-trl This skill teaches you to apply reinforcement learning techniques for aligning language models with human preferences. It covers supervised fine-tuning for instruction following, direct preference optimization for preference alignment, PPO and GRPO for reward-based training, and reward model development—all integrated with HuggingFace Transformers.★ 2,795

Related notes

References

Mi et al., 2026 — Skill-Pro: Learning Reusable Skills from Experience via Non-Parametric PPO for LLM Agents (arXiv:2602.01869)
Schulman et al., 2017 — Proximal Policy Optimization Algorithms (the trust-region mechanism Skill-Pro adapts non-parametrically)
Wang et al., 2024 — Agent Workflow Memory (AWM), one of the paper's baseline memory methods
Zhao et al., 2024 — ExpeL: LLM Agents Are Experiential Learners, a baseline insight-distillation method
Xu et al., 2025 — A-Mem: Agentic Memory for LLM Agents, a baseline note-based memory method