Debug Logging Alone Causes 73.5% of Agent-Skill Credential Leaks

Notes on How Your Credentials Are Leaked by LLM Agent Skills: An Empirical Study (arXiv:2604.03070) — Zhihao Chen, Ying Zhang, Yi Liu, Gelei Deng, Yuekang Li, Yanjun Zhang, Jianting Ning, L. Zhang, Lei Ma, Zhiqiang Li · April 2026

Note published July 25, 2026 · written by SkillFed’s research pipeline from the paper above · how these notes are made

Researchers built the first large-scale audit of credential leakage across a live agent-skill marketplace. Starting from 170,226 artifacts on SkillsMP, they drew a stratified random sample of 17,022 skills and ran each through a three-stage pipeline: static secret extraction using regex and AST parsing, dynamic sandbox testing seeded with mock credentials, and a cross-reference step that checks whether a skill's stated natural-language purpose matches what its code actually does at runtime. That last step is the paper's structural bet: credential handling in an agent skill isn't just a code-review problem, because the thing an LLM trusts (the description) and the thing that executes (the code) can diverge, and neither view alone catches the gap.

The audit surfaced 520 affected skills carrying 1,708 distinct security issues, sorted into a ten-pattern taxonomy split between unintentional exposure and deliberate malice. One mundane habit accounts for most of it: debug logging — print statements and console.log calls — makes up 73.5% of all vulnerability issues, because agent frameworks pipe stdout straight into the model's context window. A credential that would sit harmlessly in a terminal on a human's machine becomes something the LLM itself reads and can repeat. Catching these cases required reading code and description together in 76.3% of instances. Credential exposure here is fundamentally cross-modal — a scanner limited to one channel misses most of what's there. Once found, the leaks weren't theoretical: 89.6% of leaked credentials were immediately exploitable, and 92.5% of those needed no elevated privileges beyond running the skill as intended. Disclosure worked reasonably well upstream — 91.6% of hardcoded-secret cases got fixed, and every skill judged deliberately malicious was pulled — but the fork-based distribution model undercuts even a clean fix: secrets removed from 107 upstream repositories kept circulating across 50-plus independent forks that never pulled the patch.