No labels, no verifier: self-play skill-writing lifts GPT-4.1 from 11.1% to 16.5% on context learning

Notes on From Context to Skills: Can Language Models Learn from Context Skillfully? (arXiv:2604.27660) — Shuzheng Si, Haozhe Zhao, Yu Lei, Qingyi Wang, Dingwei Chen, Zhitong Wang, Zhenhailong Wang, Kangyang Luo, Zheng Wang, Gang Chen, Fanchao Qi, Minjia Zhang, and 1 more · April 2026

Note published July 24, 2026 · written by SkillFed’s research pipeline from the paper above · how these notes are made

Ctx2Skill builds a skill file straight from a raw, unlabeled context using a five-role self-play loop — no human-written skills, no external verifier. Each round, a Challenger agent writes probing tasks and pass/fail rubrics from the context and its own skill set, a Reasoner agent tries to answer them guided by its current skill set, and a neutral Judge scores every rubric pass or fail. Failed cases go to a Reasoner-side Proposer–Generator pair that diagnoses the missing knowledge and rewrites the Reasoner's skill file; cases the Reasoner solves too easily go to a matching Challenger-side pair that tightens the next round's tasks. Left unchecked, a loop like this drifts toward increasingly narrow, extreme tasks — the authors call it adversarial collapse — so a Cross-Time Replay mechanism keeps two small probe sets, the hardest failure and easiest success seen each round, and picks whichever historical skill file scores best on both, rather than defaulting to the final round.

Evaluated on CL-bench — 500 contexts, 1,899 tasks, and 31,607 rubrics spanning domain-knowledge reasoning, rule systems, procedures, and empirical discovery — Ctx2Skill's skill files raise solve rates across three closed backbones with no weight updates at all: GPT-4.1 climbs from 11.1% to 16.5%, GPT-5.1 from 21.1% to 25.8%, GPT-5.2 from 18.2% to 21.4%, beating both single-pass prompted skill-writing and AutoSkill4Doc (a windowed-context adaptation of an existing execution-feedback skill method) on every category. The ablations show where the gains actually come from. Removing the Challenger's own skill evolution costs the most — GPT-4.1 drops to 13.8% — and removing Cross-Time Replay is next-worst, since skills pulled from a single fixed iteration degrade monotonically as self-play keeps running. Skills transfer asymmetrically, too: GPT-5.1-authored skills lift GPT-4.1 almost as much as GPT-4.1's own self-generated skills do, but GPT-4.1-authored skills give GPT-5.1 only about half of its own self-generated gain.