Self-graded exploration closes a 32-point reasoning gap — no labels needed

Notes on Unsupervised Skill Discovery for Agentic Data Analysis (arXiv:2606.06416) — Zhisong Qiu, Kang Song, Shengwei Tang, Shuofei Qiao, Lei Liang, Huajun Chen, Shumin Deng · June 2026

Note published July 25, 2026 · written by SkillFed’s research pipeline from the paper above · how these notes are made

DataCOPE builds data-analysis skills without ever seeing a labeled example. Instead of grading trajectories against ground-truth answers, it manufactures its own quality signal out of the agent's exploration: for open-ended report tasks, an Adaptive Checklist Verifier writes a task-specific checklist, scores each report by how much of the checklist it verifiably covers, and rewrites the checklist itself whenever the agent starts gaming it; for fixed-answer reasoning tasks, an Answer Agreement Verifier clusters trajectories by their final answer and uses self-consistency — the relative size of a trajectory's answer cluster — as a secondary confidence signal. A Data-Analytic Agent samples the trajectories, the verifier sorts them into contrastive high- and low-quality groups, and a Skill Manager rewrites a Markdown skill file from that contrast, looping through generation, verification, and distillation with no human ever touching the exploration set.

Averaged over four matched base models — Claude, GPT, DeepSeek, and Qwen variants — DataCOPE lifts report-style accuracy on Deep Data Research from 47.4% to 57.1%, and reasoning accuracy on DABStep from 29.1% to 61.4%, with the reasoning gain concentrated on the benchmark's hard split. Both beat Anthropic's own Skill Creator tool run through Claude Code over the same exploration data, which manages only 51.3% and 51.7% respectively — the gap traces to the unsupervised verifier signal, not merely to having an agent write skills. The two verifier halves aren't interchangeable: on DABStep, keeping only self-consistency and dropping answer clustering scores worse than using no verifier signal at all (47.9% vs. 53.9%), because trajectories can converge confidently on the same wrong answer. The discovered skills also compress behavior, not just improve it — one configuration cut average token use by 73.4% (241k to 64k tokens) while accuracy rose from 44% to 64% under a fixed 15-turn budget. The one place unsupervised still trails supervised: with every exploration trajectory hand-labeled, the comparison baseline reaches 72.2% on DABStep, about nine points past what DataCOPE gets for free.