1,858 reusable skills mined from 22,511 real chats — zero weight updates

Notes on AutoSkill: Experience-Driven Lifelong Learning via Skill Self-Evolution (arXiv:2603.01145) — Yutao Yang, Junsong Li, Qianjun Pan, Bihao Zhan, Yuxuan Cai, Linge Du, Jie Zhou, Kai Chen, Qin Chen, Xin Li, Bo Zhang, Liang He · March 2026

Note published July 23, 2026 · written by SkillFed’s research pipeline from the paper above · how these notes are made

AutoSkill is a training-free plug-in layer that turns repeated user requests into versioned skill artifacts instead of treating them as disposable chat history. Two loops run in tandem. A serving loop rewrites the incoming query, ranks stored skills with a hybrid BM25+dense retrieval score, and injects only the ones that clear a similarity threshold. A background evolution loop extracts a candidate skill from the user's own queries — never the model's replies — and hands it to an LLM judge that compares it against its single nearest neighbor and decides add, merge, or discard. A merge doesn't concatenate text: it bumps the existing skill's version number and folds in only the new, non-conflicting constraints, so a skill like a writing-style preference accumulates as one evolving artifact rather than a pile of near-duplicate prompts.

Running the extraction pipeline across four WildChat-1M subsets — English and Chinese, GPT-3.5 and GPT-4, each filtered to conversations with more than 8 turns — turns up 1,858 extracted skills from 22,511 conversations. The resulting SkillBank skews toward programming and technical writing: Programming & Software Dev. is the largest category at 482 skills, roughly a quarter of the corpus, and "python" is the single most common tag. It also catches softer behavioral preferences — roleplay personas, translation conventions. Case studies confirm the versioning mechanism works as designed: one English text-rewriting skill reached version 0.1.34 after 34 rounds of incremental refinement from recurring feedback, while a rarely-revisited Chinese counseling-persona skill sat untouched at 0.1.0.