Pith. sign in

REVIEW 5 cited by

AttnGCG: Enhancing Jailbreaking Attacks on LLMs with Attention Manipulation

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2410.09040 v1 pith:C2Z7526C submitted 2024-10-11 cs.CL

Zijun Wang , Haoqin Tu , Jieru Mei , Bingchen Zhao , Yisen Wang , Cihang Xie This is my paper · ORCID
classification cs.CL
keywords attacksattentionjailbreakingllmsmodelsattngcgattackeffective
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

This paper studies the vulnerabilities of transformer-based Large Language Models (LLMs) to jailbreaking attacks, focusing specifically on the optimization-based Greedy Coordinate Gradient (GCG) strategy. We first observe a positive correlation between the effectiveness of attacks and the internal behaviors of the models. For instance, attacks tend to be less effective when models pay more attention to system prompts designed to ensure LLM safety alignment. Building on this discovery, we introduce an enhanced method that manipulates models' attention scores to facilitate LLM jailbreaking, which we term AttnGCG. Empirically, AttnGCG shows consistent improvements in attack efficacy across diverse LLMs, achieving an average increase of ~7% in the Llama-2 series and ~10% in the Gemma series. Our strategy also demonstrates robust attack transferability against both unseen harmful goals and black-box LLMs like GPT-3.5 and GPT-4. Moreover, we note our attention-score visualization is more interpretable, allowing us to gain better insights into how our targeted attention manipulation facilitates more effective jailbreaking. We release the code at https://github.com/UCSC-VLAA/AttnGCG-attack.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Eyes-on-Me: Scalable RAG Poisoning through Transferable Attention-Steering Attractors

    cs.LG 2025-10 conditional novelty 7.0 of 10

    Eyes-on-Me makes RAG data poisoning reusable: a transferable attention-steering attractor is optimized once, then combined with different attack payloads at near-zero cost.

  2. On Surjectivity of Neural Networks: Can you elicit any behavior from your model?

    cs.LG 2025-08 conditional novelty 7.0 of 10

    Pre-LayerNorm transformers and linear attention are almost always surjective, so any target output has an input that produces it in the continuous embedding space.

  3. D-Fusion: Direct Preference Optimization for Aligning Diffusion Models with Visually Consistent Samples

    cs.CV 2025-05 conditional novelty 7.0 of 10

    Mask-guided self-attention fusion creates well-aligned target images that stay visually close to poorly-aligned base images, with full denoising trajectories, and DPO on these pairs improves alignment.

  4. Benign-to-Toxic Jailbreaking: Inducing Harmful Responses from Harmless Prompts

    cs.CV 2025-05 conditional novelty 7.0 of 10

    Adversarial images optimized to map harmless text prefixes to toxic tokens jailbreak vision-language models more effectively than continuing toxic text.

  5. AgentSentinel: An End-to-End and Real-Time Security Defense Framework for Computer-Use Agents

    cs.CR 2025-09 conditional novelty 6.0 of 10

    AgentSentinel combines system-level tracing with LLM-based auditing to block 79.6% of attacks in the authors' 60-scenario computer-use agent benchmark.

Pith tools