Prompt injection defense was a credit assignment problem all along

The core problem with defending LLMs against prompt injection has never been a lack of effort—it's been a measurement problem disguised as a training problem. When you assign a single reward signal to an entire response, you cannot distinguish between the part of that response that correctly answered the user's question and the part that helpfully appended the attacker's injected sentence. SecOPD fixes this by operating at the token level.

The mechanism is elegant in its simplicity. During training, the model generates a response from an attacked input—one where malicious instructions have been embedded in the untrusted data channel. A frozen copy of the same base model then scores each generated token, but using the clean version of that same input, with the injection stripped out. Tokens the clean model would have generated anyway receive favorable feedback; tokens that only appear because the injection nudged the model there get suppressed. No external judge, no task-specific reward model—just the base model's own clean-context preferences as a teacher signal.

The results against adaptive attacks are striking. The prior best defense, Meta-SecAlign, uses DPO with sequence-level preference labels and achieves a 94.0% attack success rate against PISmith, a reinforcement-learning-trained adaptive attacker that optimizes specifically against the defended model. SecOPD on the same Qwen3.6-27B base drops that to 9.0%. That is not a marginal improvement—it is roughly an order-of-magnitude reduction against the strongest available attack.

What makes this more credible than typical security benchmark results is the out-of-distribution transfer. SecOPD was trained on text-completion tasks, yet on AgentDojo—where success means the model actually calls a malicious tool, not just produces suspicious text—it achieves 4.7% ASR versus 5.5% for Meta-SecAlign. The separation between trusted instructions and untrusted data appears to generalize beyond the training domain.

The utility story is also worth examining carefully. A GRPO baseline that uses sequence-level on-policy training does push adaptive ASR down to around 47%, but its average utility across seven benchmarks drops noticeably, with Minerva-Math performance falling substantially. SecOPD's average utility stays close to the undefended model. A manual audit of Minerva-Math failures found that most errors were formatting or answer-extraction issues rather than genuine reasoning degradation—a useful distinction that the aggregate numbers alone would obscure.

The paper is honest about what this does not cover. It assumes the system can reliably signal which input parts are trusted—agents that must infer trust from context are out of scope. It does not address jailbreaks or direct prompt injections from malicious users. And the authors explicitly note that a 9.0% ASR is a milestone, not a solution; future adaptive attacks trained against SecOPD-defended models may close the gap.

The deeper implication is methodological. The failure of DPO and GRPO here is not a failure of those algorithms generally—it is a failure of applying sequence-level credit assignment to a problem that is inherently token-local. Once you frame prompt injection defense as a credit assignment problem, the fix becomes obvious. That framing is the contribution.