skillfed

Verification-gated skills add up to 12 points on KernelBench — pull retrieval at inference and most of it vanishes

Notes on daVinci-kernel: Co-Evolving Skill Selection, Summarization, and Utilization via RL for GPU Kernel Optimization (arXiv:2606.16497) — Dayuan Fu, Mohan Jiang, Tong Wang, Dian Yang, Jiarui Hu, Liming Liu, Jinlong Hou, Pengfei Li · June 2026

Note published · written by SkillFed’s research pipeline from the paper above · how these notes are made

AI-assisted notes · reviewed by SkillFed Skill retrieval

daVinci-kernel splits CUDA/Triton kernel generation across three roles under one shared LLM backbone. A Selection Agent retrieves candidate optimization techniques through BM25 pre-filtering plus LLM reranking, and a Policy Agent writes the kernel across multiple turns, conditioned on whichever techniques got selected. The third role, a Summary Agent, turns a rollout's winning trick into a new reusable entry for the shared skill library — but only once execution-based verification confirms the speedup reproduces at least 1.2x when the technique is re-run on its original task, not merely praised by a judge model. All three roles are cold-started with SFT on diversity-filtered demonstrations, then jointly optimized end-to-end with multi-turn REINFORCE and per-agent advantage estimation, so the policy, the retriever, and the summarizer improve together rather than in separate stages.

On KernelBench's loosest threshold (correctness plus any measured speedup), the 14B model beats the strongest prior RL-trained kernel model, Dr. Kernel-14B, by 6.8, 12.1, and 10.1 points on Level 1, 2, and 3 (37.2% vs 30.4%, 70.6% vs 58.5%, 32.2% vs 22.1%). Under the stricter 1.2x-speedup bar, the Level 2 and Level 3 gaps hold (+9.0pp and +4.5pp) while Level 1 narrows to roughly a point. An ablation that keeps the RL-trained weights but withholds skill injection at inference time erases most of that advantage: Level 2 falls from 27.1% to 21.2% and Level 3 from 7.3% to 2.5% — the retrievable skills, not just the fine-tuning that produced them, are doing real work at solve time.

Key numbers

KernelBench L1 solve rate (any speedup) vs prior best37.2% vs 30.4%
KernelBench L2 solve rate (any speedup) vs prior best70.6% vs 58.5%
KernelBench L3 solve rate (any speedup) vs prior best32.2% vs 22.1%
L2 solve rate without skill injection at inference27.1% → 21.2%
L3 solve rate without skill injection at inference7.3% → 2.5%

Skills related to this research

wiki-retrieve wiki-retrieve upgrades vault search from page-level to chunk-level granularity, combining contextual prefixes, BM25 sparse retrieval, and dense reranking to reduce retrieval failures. Setup is opt-in with built-in privacy gates: run fully on-machine with synthetic prefixes, or opt into Anthropic API or claude CLI tiers for better context generation.★ 10,007 Context7 Context7 fetches the latest library documentation to give you accurate, up-to-date API information and code examples. It eliminates hallucinations from outdated training data by retrieving version-specific details when you ask about library usage, request examples, or explicitly invoke it.★ 3 ai-bot-log-audit Track and interpret how AI agents interact with your infrastructure by examining server logs for bot activity patterns. Gain actionable insights into crawl frequency, request timing, and resource usage to refine your content strategy and improve bot-to-human traffic balance. Essential for teams managing high-volume AI agent access.★ 140 Skill LLM Wiki Bootstrap helps you build and operate a personal knowledge base structured around immutable raw sources, a curated wiki layer, and a SCHEMA.md operating contract. Set up preferences, ingest sources, query your wiki, or configure full-text search—the skill handles scaffolding, content compilation, and consistency checks.★ 98
CUDA Agent: Large-Scale Agentic RL for High-Performance CUDA Kernel Generation GPU kernel optimization is fundamental to modern deep learning but remains a highly specialized task requiring deep hardware expertise. Despite strong… Dai et al. · similarity 0.87

Related notes

References

  1. Dayuan Fu, Mohan Jiang, Tongyu Wang, Dian Yang, Jiarui Hu, Liming Liu, Jinlong Hou, Pengfei Liu, "daVinci-kernel: Co-Evolving Skill Selection, Summarization, and Utilization via RL for GPU Kernel Optimization," arXiv:2606.16497 (2026).
  2. Wei Liu et al., "Dr. Kernel: Reinforcement Learning Done Right for Triton Kernel Generation," arXiv:2602.05885 (2026).
  3. Anne Ouyang et al., "KernelBench: Can LLMs Write Efficient GPU Kernels?," arXiv:2502.10517 (2025).
  4. Guanzhi Wang et al., "Voyager: An Open-Ended Embodied Agent with Large Language Models," arXiv:2305.16291 (2023).
  5. Peng Xia et al., "SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning," arXiv:2602.08234 (2026).