Skill selection, not RL, drives the gain — 8.3 of 10.4 points

Notes on OpenClaw-Skill: Collective Skill Tree Search for Agentic Large Language Models (arXiv:2606.16774) — Tianyi Lin, Chuan-da Sun, Jingyi Zhang, Changxu Wei, Huanjin Yao, Shunyu Liu, Xikun Zhang, Liu Liu, Jiaxing Huang · June 2026

Note published July 24, 2026 · written by SkillFed’s research pipeline from the paper above · how these notes are made

AI-assisted notes · reviewed by SkillFed Skill evolution

OpenClaw-Skill builds a searchable tree of skills instead of a flat library. Its construction method, Collective Skill Tree Search (CSTS), breaks a long-horizon agentic task into an ordered chain of subtasks. For each one, several heterogeneous LLMs attempt it independently, and each resulting trajectory gets distilled into a candidate skill node. Every candidate is scored twice. A collective quality score averages judgments across multiple judge models; a transferability score hands the skill to models that never produced it and checks how well those outsiders perform using it as guidance. Only the top-scoring node per subtask survives into the tree, and the surviving path becomes supervised training data. Collective Skill Reinforcement Learning (CSRL) then layers a GRPO-style policy update on top: rollouts conditioned on different candidate skills for the same subtask are normalized against each other as one group, so the policy learns to prefer whichever skill actually worked, not whichever one it happened to sample.

Across four Qwen backbones (Qwen3-4B/8B and Qwen3.5-4B/9B), OpenClaw-Skill lifts overall QwenClawBench scores by 4.3 to 10.4 points. The flagship Qwen3.5-9B variant moves from 34.5 to 44.9. Gains concentrate in categories tied to verification and error recovery: the 9B model's CS category jumps from 30.2 to 78.4, and its SVM category from 33.2 to 70.9. On PinchBench's 123-task suite, the same model raises best-run success from 61.1% to 68.2% and average success from 47.1% to 53.6%. An ablation shows where the improvement actually comes from: skill construction alone (CSN-Gen plus CSN-Assess) accounts for 8.3 of the 10.4-point total gain on Qwen3.5-9B, while CSRL contributes the remaining 2.1. Most of the benefit is from generating and filtering better skills; reinforcement learning only refines at the margin.

Key numbers

QwenClawBench overall, Qwen3.5-9B backbone	34.5 → 44.9 (+10.4 pts)
Single-category jump (CS, 9B)	30.2 → 78.4
PinchBench 123-task best success (9B)	61.1% → 68.2%
Gain split: skill construction vs. RL (9B)	CSTS +8.3 pts, CSRL +2.1 pts of +10.4 total
Still trails closed frontier model on same benchmark	44.9 vs. 59.5 (Claude Opus 4.6)

What this means for skill authors

For anyone authoring or curating agent skills, the transferability score is the part worth sitting with. It formalizes a check most skill-authoring workflows skip entirely: does this skill actually help a model that didn't write it, or does it just encode one model's idiosyncratic reasoning style dressed up as a general procedure? Running every candidate through multiple judges, then handing it to executor models that never produced it, before it earns a place in the tree — that's a cheap, automatable version of the cross-model validation any federated skill catalog needs at scale. A skill that only works for its author model is a liability the moment it's shared.

The honest caveat sits in the paper's own comparison table: even OpenClaw-Skill 9B at 44.9 overall lands well below closed frontier models evaluated on the same benchmark — Claude Opus 4.6 scores 59.5, Qwen3.6-Plus scores 57.4. This is a fine-tuning and skill-curation technique layered onto a small open backbone, not a way to leapfrog frontier agents outright. Read the reported gains as narrowing a capability gap, not closing one.

The more durable lesson is in the ablation, not the leaderboard position: most of the improvement came from generating and filtering better skills, not from the reinforcement-learning stage bolted on afterward. If you're building or vetting skills for a catalog rather than training a model, that's the argument — invest in generation diversity and independent, cross-model scoring before reaching for RL fine-tuning at all.

Skills related to this research

unsloth-fine-tuning Unsloth accelerates LLM fine-tuning on consumer and datacenter GPUs through optimized LoRA and QLoRA training, cutting both speed and memory requirements dramatically. It handles supervised fine-tuning, reinforcement learning with GRPO, vision model adaptation, and TTS training across 300+ model architectures, with direct export to GGUF for deployment on Ollama and llama.cpp.★ 2,896 php-best-practices PHP Best Practices audits your code against 51 rules covering type systems, modern PHP 8.x features, PSR standards, SOLID principles, error handling, performance, and security. It detects your project's PHP version and applies only relevant guidance, from union types and match expressions in 8.0 through property hooks and the pipe operator in 8.4+.★ 58 fine-tuning-with-trl This skill teaches post-training techniques for aligning language models to human preferences. It covers supervised fine-tuning, direct preference optimization (DPO), and online reinforcement learning methods like RLOO and GRPO, with complete workflows and practical examples.★ 221,503 fine-tuning-with-trl Fine-tuning with TRL provides post-training methods to align language models with human preferences through multiple approaches. Train models on instruction data with SFT, optimize for preference alignment via DPO without a separate reward model, or run full RLHF pipelines combining supervised fine-tuning, reward modeling, and PPO optimization. GRPO offers memory-efficient online reinforcement learning for resource-constrained setups.★ 11,165

Related notes

References

OpenClaw-Skill: Collective Skill Tree Search for Agentic Large Language Models (Lin et al., 2026)
ReAct: Synergizing Reasoning and Acting in Language Models (Yao et al., 2023)
Mastering the Game of Go without Human Knowledge (Silver et al., 2017)
Tree Search for Language Model Agents (Koh et al., 2025)
Equipping agents for the real world with agent skills (Anthropic, 2025)