Pith. sign in

REVIEW 9 cited by

JailGuard: A Universal Detection Framework for LLM Prompt-based Attacks

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2312.10766 v4 pith:6PR5XLRB submitted 2023-12-17 cs.CR

Xiaoyu Zhang , Cen Zhang , Tianlin Li , Yihao Huang , Xiaojun Jia , Ming Hu , Jie Zhang , Yang Liu
show 2 more authors
Shiqing Ma Chao Shen
This is my paper · ORCID
classification cs.CR
keywords attacksdetectionjailguardimageinputsprompt-basedsystemstext
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

The systems and software powered by Large Language Models (LLMs) and Multi-Modal LLMs (MLLMs) have played a critical role in numerous scenarios. However, current LLM systems are vulnerable to prompt-based attacks, with jailbreaking attacks enabling the LLM system to generate harmful content, while hijacking attacks manipulate the LLM system to perform attacker-desired tasks, underscoring the necessity for detection tools. Unfortunately, existing detecting approaches are usually tailored to specific attacks, resulting in poor generalization in detecting various attacks across different modalities. To address it, we propose JailGuard, a universal detection framework deployed on top of LLM systems for prompt-based attacks across text and image modalities. JailGuard operates on the principle that attacks are inherently less robust than benign ones. Specifically, JailGuard mutates untrusted inputs to generate variants and leverages the discrepancy of the variants' responses on the target model to distinguish attack samples from benign samples. We implement 18 mutators for text and image inputs and design a mutator combination policy to further improve detection generalization. The evaluation on the dataset containing 15 known attack types suggests that JailGuard achieves the best detection accuracy of 86.14%/82.90% on text and image inputs, outperforming state-of-the-art methods by 11.81%-25.73% and 12.20%-21.40%.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 9 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Eyes-on-Me: Scalable RAG Poisoning through Transferable Attention-Steering Attractors

    cs.LG 2025-10 conditional novelty 7.0 of 10

    Eyes-on-Me makes RAG data poisoning reusable: a transferable attention-steering attractor is optimized once, then combined with different attack payloads at near-zero cost.

  2. Safe responses matter: Output-aware safety guardrail mitigate over-refusal in MLLMs

    cs.LG 2026-06 conditional novelty 6.0 of 10

    A lightweight multi-instance contrastive classifier on MLLM hidden states predicts output harmfulness, matching input-side safety while sharply lowering over-refusal.

  3. SoK: Systematizing LLM Prompt Security: Taxonomies, Datasets, and Unified Evaluation of Attacks and Defenses

    cs.CR 2025-10 conditional novelty 6.0 of 10

    A systemization of LLM jailbreak security that adds linked taxonomies, an evaluation platform, and JailbreakDB, while its main attack–defense comparison results remain deferred.

  4. Paper Summary Attack: Jailbreaking LLMs through LLM Safety Papers

    cs.CL 2025-07 conditional novelty 6.0 of 10

    Summaries of LLM safety papers, paired with a completion-style payload containing a harmful query, jailbreak aligned LLMs at high reported success rates and expose a defense paper versus attack paper bias.

  5. AMIA: Automatic Masking and Joint Intention Analysis Makes LVLMs Robust Jailbreak Defenders

    cs.CV 2025-05 conditional novelty 6.0 of 10

    AMIA is an inference-only jailbreak defense that masks text-irrelevant image patches and prompts single-pass intention analysis, lifting average defense success on LVLMs from 52.4% to 81.7%.

  6. Test-Time Immunization: A Universal Defense Framework Against Jailbreaks for (Multimodal) Large Language Models

    cs.CR 2025-05 conditional novelty 6.0 of 10

    TIM detects jailbreaks at test time using a gist-token classifier and adaptively fine-tunes the model with LoRA on detected harmful prompts, cutting attack success rates to near zero on tested benchmarks.

  7. Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generation

    cs.CL 2025-08 reject novelty 5.0 of 10

    Sparse autoencoder activation perturbation (SFPF) applied on top of existing jailbreak prompts raises attack success rate on Qwen3-32B, but with no defense evaluation and weak reproducibility.

  8. A Multimodal Automatic Redteaming Evaluation based on Atomic Jailbreak Strategy Decoupling and Combination

    cs.CR 2026-08 conditional novelty 4.0 of 10

    A new jailbreak framework, HACA, combines atomic text and image attack strategies selected by a cross-modal planner and generates attacks with LLMs and text-to-image models, reaching 95.48% average attack success acro...

  9. A Survey of Safety on Large Vision-Language Models: Attacks, Defenses and Evaluations

    cs.CR 2025-02 conditional novelty 4.0 of 10

    A survey of LVLM safety that adds a lifecycle taxonomy and new benchmark results showing Janus-Pro-7B has weaker safety than several open-source LVLMs.

Pith tools