Pith. sign in

REVIEW 6 cited by

InjecGuard: Benchmarking and Mitigating Over-defense in Prompt Injection Guardrail Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2410.22770 v3 pith:S2EXWZWR submitted 2024-10-30 cs.CL cs.AIcs.CR

Hao Li , Xiaogeng Liu This is my paper
classification cs.CLcs.AIcs.CR
keywords promptmodelsover-defenseinjecguardinjectionattacksguardnotinject
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Prompt injection attacks pose a critical threat to large language models (LLMs), enabling goal hijacking and data leakage. Prompt guard models, though effective in defense, suffer from over-defense -- falsely flagging benign inputs as malicious due to trigger word bias. To address this issue, we introduce NotInject, an evaluation dataset that systematically measures over-defense across various prompt guard models. NotInject contains 339 benign samples enriched with trigger words common in prompt injection attacks, enabling fine-grained evaluation. Our results show that state-of-the-art models suffer from over-defense issues, with accuracy dropping close to random guessing levels (60%). To mitigate this, we propose InjecGuard, a novel prompt guard model that incorporates a new training strategy, Mitigating Over-defense for Free (MOF), which significantly reduces the bias on trigger words. InjecGuard demonstrates state-of-the-art performance on diverse benchmarks including NotInject, surpassing the existing best model by 30.8%, offering a robust and open-source solution for detecting prompt injection attacks. The code and datasets are released at https://github.com/leolee99/InjecGuard.

Discussion (0). Sign in to comment.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Transferable Direct Prompt Injection via Activation-Guided MCMC Sampling

    cs.AI 2025-09 conditional novelty 7.0 of 10

    An activation-guided energy model plus MCMC sampling creates transferable direct prompt injection attacks in a black-box setting, reaching 49.6% average attack success across five LLMs.

  2. From Neural Intent to Cryptographic Authorization: Securing AI-Driven Enterprise Workflows

    cs.CR 2026-07 conditional novelty 6.0 of 10

    NCS gates every privileged AI-agent tool call behind an offline-signed, hash-chained instruction stream, cutting injected-parameter attacks to zero by construction.

  3. AgentWatcher: A Rule-based Prompt Injection Monitor

    cs.CR 2026-04 conditional novelty 6.0 of 10

    An attention-attribution plus rule-reasoning detector reduces prompt-injection success to near zero on agent and long-context benchmarks with small utility loss.

  4. Detection Method for Prompt Injection by Integrating Pre-trained Model and Heuristic Feature Engineering

    cs.CL 2025-06 conditional novelty 5.0 of 10

    A dual-channel detector combining DeBERTa and heuristic rules is claimed to beat existing prompt injection detectors on three benchmarks and to reduce attack success on GLM-4, Llama 3, Qwen 2.5, and GPT-4o.

  5. JavelinGuard: Low-Cost Transformer Architectures for LLM Security

    cs.LG 2025-06 reject novelty 4.0 of 10

    A study of five small transformer classifier architectures for LLM jailbreak and prompt injection detection claims low-latency accuracy comparable to large models, led by the multi-task Raudra design.

  6. Multi-Stage Prompt Inference Attacks on Enterprise LLM Systems

    cs.CR 2025-07 reject novelty 3.0 of 10

    Multi-stage prompt inference attacks against enterprise LLMs are formalized and defenses are proposed, but the preprint gives no reproducible evidence for its central claims.

Pith tools