SciVis agent skills swing task scores +5% to +66% — but bigger gains don't guarantee more tasks finish

Notes on SciVisAgentSkills: Design and Evaluation of Agent Skills for Scientific Data Analysis and Visualization (arXiv:2606.05525) — Kuangshi Ai, Haichao Miao, Kaiyuan Tang, Shusen Liu, Chaoli Wang · June 2026

Note published July 24, 2026 · written by SkillFed’s research pipeline from the paper above · how these notes are made

SciVisAgentSkills packages tool-specific procedural knowledge for four scientific visualization platforms — ParaView, napari, VMD, and TTK — into self-contained, version-pinned agent skills. Each one pins software versions, distills official docs into working usage patterns, and borrows code straight from existing domain agents (ParaView-MCP, BioImage-Agent, GMX-VMD-MCP, TopoPilot). Failure-mode fixes get folded in too — headless rendering, viewport capture, the stuff that trips agents up in practice. The team ran these skills across two coding-agent harnesses — Claude Code on Sonnet-4.5, Codex on GPT-5.2 — against SciVisAgentBench: 108 expert-designed multi-step tasks across five suites — ParaView, molecular visualization, bioimage, topology, and object identification. Three trials per configuration, scored by a multimodal LLM judge (Claude Opus 4.6) plus deterministic completion checks and, for ParaView, image-similarity metrics.

Skills raised the overall score in every agent-suite combination tested, but by wildly uneven margins. The two biggest jumps landed on tasks where tool coverage in model training runs thinnest: topology visualization took Claude Code's score from 45.2 to 73.6 (+63%), object identification from 41.5 to 69.1 (+66%). The smallest gain sat on VMD molecular visualization for Claude Code — just 61.5 to 64.3 (+5%) — suggesting the base model already handles a mature, well-documented tool well enough that a skill barely moves the needle. Token efficiency split by harness, not by skill size: Claude Code showed a consistent drop in output tokens across suites (ParaView output fell from 425K to 101K, a 76% drop), while its input tokens told a more mixed story — down 64% on topology tasks but essentially flat on ParaView. Codex went the other way in places — bioimage input tokens rose 68%. Image-quality metrics for ParaView (PSNR, SSIM, LPIPS) barely moved with skills added. One result breaks the clean story: Codex's object-identification score rose, yet its completion rate fell from 92.6% to 80.3% — better answers on the runs that finished, more runs failing outright.