Self-generated web skills score below no skill at all — contract-based repair triples success to 28.1%

Notes on ContractSkill: Repairable Contract-Based Skills for Multimodal Web Agents (arXiv:2603.20340) — Zijian Lu, Yiping Zuo, Yupeng Nie, Xinhui He, Weibei Fan, Lianyong Qi, S. Jin · March 2026

Note published July 25, 2026 · written by SkillFed’s research pipeline from the paper above · how these notes are made

ContractSkill converts a draft web-agent skill into a contract artifact — a structured record of the goal, preconditions, an ordered list of steps (each carrying a selector, action, optional arguments, and post-assertions), postconditions, recovery rules, and termination checks. A deterministic verifier walks that structure against the live page and returns one of five typed error codes — element not found, wrong page state, an unmet postcondition, a step-budget timeout, or an invalid input — pinpointing exactly which step broke and why. That code drives fault localization. Five narrow patch operators then swap a brittle selector, insert a missing precondition or postcondition, add a recovery fallback, or correct an argument — touching only the broken step instead of regenerating the whole procedure.

The setup: VisualWebArena (100 tasks) and MiniWoB (20 tasks, 10 instances each — 600 episodes), run through two backbones, GLM-4.6V and Qwen3.5-Plus. The failure mode that motivated the work shows up directly in the baselines — a self-generated skill scores below using no skill at all. On VisualWebArena, GLM-4.6V drops from 12.5% success with no skill to 9.4% with a self-written one; Qwen3.5-Plus drops from 28.1% to 10.9%. ContractSkill reverses that, reaching 28.1% and 37.5% — roughly triple the self-generated numbers, and past the no-skill baseline in both cases. A MiniWoB ablation isolates why: swapping the verify-and-patch loop for a plain LLM rewrite of the failed skill recovers almost none of the gain (60.5–62.0%, at or below the self-generated baseline), and letting repair skip failure localization gives up another 11–12.5 points. Repaired artifacts also outlive the model that wrote them: with the originating model removed from the loop, a contract patched by one backbone and handed to the other still succeeds 80.4% of the time on VisualWebArena, against 32.6% for a skill the second model wrote for itself on the same tasks.