Pith. sign in

REVIEW 5 major objections 5 minor 50 references

DETAM: Defending LLMs Against Jailbreak Attacks via Targeted Attention Modification

T0 review · 5 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that jailbreak attacks on aligned LLMs can be blocked at inference time by reallocating attention inside a few sensitive attention heads, without fine-tuning or input rewriting.

desk verdict The stress-test finding is correct: the mask update in Eq. 8 is a no-op under the paper's own convention, so DETAM's reported results are unexplained by the stated mechanism. read the letter →

arxiv 2504.13562 v1 pith:HAQCIVTZ submitted 2025-04-18 cs.CL

Yu Li , Han Jiang , Zhihua Wei This is my paper · ORCID
classification cs.CL
keywords jailbreakdefenseattentionmodificationinference-timesafetyalignmentheadsattacksuccessrateover-refusalLLM
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper aims to show that jailbreak defenses can be built without fine-tuning or input rewriting: by identifying the attention heads whose allocation of attention distinguishes successful from unsuccessful defenses, and then amplifying attention to the user's core intention tokens during generation, an aligned model can block attacks while preserving normal behavior. Across five jailbreak methods (GCG, DeepInception, DSN, RS, and ReNeLLM) on LLaMA-2-7b-chat, the reported average attack success rate falls to 1.2% by keyword matching and 0.8% by an LLM judge, with 4.0% and 2.2% on Vicuna-13b-v1.5, beating the tested baselines. The method also generalizes to unseen in-the-wild jailbreak prompts, transfers sensitive heads across same-family models, and keeps false refusals low on XSTest and ORBench. A sympathetic reader would care because the approach points to an interpretable, training-free lever on safety: attention itself, rather than input sanitization or fine-tuning.

What carries the argument

The load-bearing object is the set of 'sensitive attention heads', defined as heads whose average relative attention difference $S_{i,j} = (A_{i,j}(P) - A_{i,j}(T))/A_{i,j}(P)$ between query tokens $P$ and attack tokens $T$ shifts most between successful and unsuccessful defenses. Heads with sensitivity score $\Delta\bar{S}_{i,j}$ below $-\alpha$ are selected. During generation, DETAM modifies the causal mask $M$ so that entries at the positions of the localized core-intention tokens are multiplied by $\beta > 1$; after softmax, attention to intention tokens rises and attention to attack tokens falls. The correction is applied only in the selected heads and only for the first few generated tokens, which keeps utility impact low. The intent localization step is done with a prompt (with few-shot examples for LLaMA-2-7b-chat), and the paper notes an SLM or a single-pass variant as alternatives.

What would settle it

Collect the jailbreak prompts on which DETAM reports low ASR, run the same intent-localization prompt, and compare the predicted core-intention span with the tokens that actually carry the harmful request. If the localizer frequently misses the harmful request or includes the attack template, then the defense's success cannot be attributed to amplifying the true intention; alternatively, compare DETAM with oracle intention spans against DETAM with predicted spans: if ASR is unchanged, the localization mechanism is not load-bearing.

Watch

Extended reading notes

Core claim

The central discovery is that defense success is visible in the attention distribution before the model decides to refuse or comply. On certain heads, when a defense succeeds the model allocates relatively more attention to the user's actual query tokens and less to the jailbreak template tokens; when it fails, the pattern reverses. DETAM formalizes this as a sensitivity score per head and, at inference, scales up the mask entries at the localized core-intention positions in the selected heads, so the softmax reallocates weight toward the intention and away from the attack text. The paper reports that this reduces average attack success to near zero across five attacks on LLaMA-2-7b-chat and Vicuna-13b-v1.5 while maintaining utility, and that the identified heads transfer across models in the same family without retraining. It also reports that the same head-level intervention partially mitigates backdoor triggers, suggesting injected backdoors also operate by shifting attention.

Load-bearing premise

The whole procedure depends on the intent localizer correctly identifying the user's core intention tokens; if it labels attack phrasing as intention, the amplification strengthens the attack, and the paper reports no accuracy evaluation of the localization step on the jailbreak inputs it defends.

Editorial extensions

If this is right

  • If DETAM works as reported, jailbreak defense can be applied to any decoder-only transformer at inference time with no training, so deployments can update defenses without retraining.
  • Sensitive heads identified on one model of a family can be reused on sibling models, making the one-time identification cost amortizable across model versions.
  • Because the intervention is localized to a few heads and early decoding steps, it offers a path to safety that does not degrade instruction-following as much as input rewriting or decoding-side baselines.
  • The same mechanism partially blocks backdoor triggers, indicating that backdoor and jailbreak attacks share an attention-level effect that one head-level correction can counter.
  • DETAM's generalization to WildJailbreak suggests the head-level signal is not an artifact of the five benchmark attack families, but a common property of how attack text diverts attention.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: If the prompt-based intent localizer is the weak link, then attacks that make the harmful request span many tokens without a single 'core intention' span could evade DETAM, because the amplification would have no clean target; this is a testable failure mode not analyzed in the paper.
  • Inference: The reported head transferability across same-family models suggests these heads encode a general safety-relevant attention pattern; one could probe this by checking whether the same heads are implicated in refusal behavior on benign sensitive queries.
  • Inference: DETAM's success on backdoor triggers hints that a unified attention-level defense might cover several manipulation classes; a direct test would be whether the same selected heads mitigate prompt-injection attacks that instruct the model to override system priorities.
  • Inference: The single-pass variant (having the model state its intent before answering) could make the method nearly cost-free in practice, but it trades off against the risk that the stated intent itself becomes a target for manipulation; that tradeoff is worth measuring.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. DETAM proposes a fine-tuning-free, inference-time jailbreak defense. The authors first identify attention heads that behave differently under successful versus unsuccessful defenses by comparing attention allocated to query tokens versus attack tokens, and then, at generation time, modify the attention of those heads to emphasize the user's core intention as found by a prompt-based intent-localization step. The method is evaluated on JBB-Behaviors with five jailbreak attacks, on WildJailbreak, on Llama-3-family models, on utility/over-refusal datasets, and on backdoor attacks. The paper reports very low attack success rates (e.g., average ASRk/ASRa of 1.2%/0.8% on LLaMA-2-7b-chat and 4.0%/2.2% on Vicuna-13b-v1.5) and claims consistent superiority over baselines.

Significance. If the mechanism were correctly specified and the results reproducible, DETAM would be a useful contribution: it avoids fine-tuning and input modification, offers an interpretable head-level analysis, reports transfer of sensitive heads within a model family, evaluates over-refusal with XSTest and ORBench, and includes a backdoor-defense extension. The paper also gives credit for evaluating on WildJailbreak and recent Llama-3 models, which are partly independent of the tuning data. However, the central mechanism as written in Eq. (8) is a no-op under the standard causal-mask convention, and key experimental details are missing. The empirical claims therefore cannot be assessed as submitted.

major comments (5)
  1. [Section 3.3, Eq. (8)] As written, the mask modification is a no-op. Under the standard causal mask convention used in Eq. (1), M has entries 0 for allowed positions and -inf for masked positions. Multiplying an entire row by beta leaves every entry unchanged (0*beta=0 and -inf*beta=-inf), so the modified attention in Eq. (9) is identical to vanilla attention. The described mechanism therefore cannot explain the ASR reductions in Tables 1-3. Please clarify whether the intended operation is a column-wise addition of beta to mask entries for key positions l in IP, or an explicit post-softmax rescaling of attention to intention tokens, and reproduce the experiments with the corrected rule. Releasing code would help resolve this ambiguity.
  2. [Sections 3.2, 4.1, and 4.4] The sensitive-head selection and the hyperparameters alpha and beta are fitted on the same JBB benchmark used for the headline evaluation. The manuscript does not report N, the number of samples in D_success and D_unsuccess, the attack methods used to construct those sets, or the judgment criterion for success/failure. Consequently the JBB results include a potentially fitted component, and only the WildJailbreak and Llama-3 transfer results are independent. Please report N and the exact construction of the selection sets, and evaluate with a held-out split or cross-validation to separate genuine generalization from benchmark tuning.
  3. [Section 3.3 and Appendix E.2] The intent-localization step is load-bearing: if localization mislabels attack tokens as intention tokens, the attention modification amplifies the adversarial wording instead of suppressing it. No accuracy evaluation of the localization step is reported on jailbreak inputs, and the prompt template is only validated indirectly through end-to-end ASR. Please provide localization accuracy or error statistics on the jailbreak evaluation sets, and include an ablation with an oracle localizer versus the actual prompt-based localizer to show how much of the defense depends on localization quality.
  4. [Tables 1-3] No variance or multiple-seed results are reported, although several differences between DETAM and the second-best method are small (e.g., LLaMA-2 average ASRk 1.2% versus 1.8% for SafeDecoding in Table 1; WildJailbreak ASRk 10% versus 12% for SafeDecoding in Table 3). Please report means and standard deviations over at least three seeds, or bootstrap confidence intervals over the 100/200 prompts, to support the claim that DETAM consistently outperforms baselines.
  5. [Section 5.1 and Figure 4(b)] There is a direct contradiction in the head-selection criterion. Section 3.2 selects heads with Delta S_bar_i,j < -alpha, but the caption of Figure 4(b) says red regions (Delta S_bar_i,j > alpha) represent the selected heads. This makes the selection rule and the visualization ambiguous. Please align the text, equation, and figure, and state clearly which sign convention is correct.
minor comments (5)
  1. [Abstract and Section 1] The phrase 'we are the first to analysis' should be 'we are the first to analyze'.
  2. [Section 2 and Appendix A] The reference '(Yu et al., 2023; ?)' contains a dangling '?', and PAIR is described in Appendix A but does not appear in the main evaluation results.
  3. [Figure 3 and Appendix F.1] The method name is inconsistently typeset as 'D ETAM' in the Figure 3 caption and 'DeTAM' in Appendix F.1; please unify to DETAM.
  4. [Section 5.1, Figure 4(c)] The control-group comparison in Figure 4(c) is described only qualitatively; please specify the statistical test used and report the resulting p-value or effect size.
  5. [Appendix E.3] The statement that prompt-based intent identification doubles inference time should be stated in the main experimental discussion, not only in the appendix, since it is a practical cost of the defense.

Circularity Check

2 steps flagged · score 6.0 of 10

Claimed attention redistribution is self-identical under the paper's own mask definition, and headline JBB ASRs are partly fitted; out-of-sample transfer results keep the circularity from being total.

  1. self definitional [Section 3.3, Eqs. (8)-(9)]
    "Formally, the original Mask matrix Mi,j is adjusted as follows: M′(k,l) = ( β·M(k,l), if xk∈IP; M(k,l), otherwise ) (8) ... the modified attention calculation process can be expressed as: Ai,j = softmax(Qi,jKT_i,j/√dk + M′_i,j) (9)"

    In Eq. (1), M is an additive causal mask; under the standard convention its entries are 0 for visible positions and -inf for masked positions. Both values are fixed points of multiplication by any positive β, so β·M(k,l) = M(k,l) for every entry, making M′ = M. Eq. (9) is therefore identical to Eq. (1) by construction. The paper's claim that 'the positions of the core intention tokens are amplified by β' and that this yields a 'relative reduction in the attention weights assigned to the attack tokens' cannot follow from this update; the stated mechanism reduces to no modification, so the reported ASR reductions are not derived from the equations given.

  2. fitted input called prediction [Section 3.2 (Eqs. 6-7), Section 4.4 (Fig. 3), Section 4.2 (Table 1)]
    "We construct two subsets, Dsuccess and Dunsuccess, by selecting N samples from successful and unsuccessful defenses... Attention heads with scores below a predefined threshold α (∆¯Si,j < −α) are selected for further investigation. ... Therefore, we select α = 0.1 and β = 5.0 as the default parameter for LLaMA2."

    The sensitive heads are located by comparing successful and unsuccessful defenses on jailbreak data, and α and β are chosen in Fig. 3 by minimizing ASR on the JBB/LLaMA-2 setup; Table 1 then reports the resulting JBB ASRs (LLaMA: ASRk=1.2%, ASRa=0.8%) as the headline result. The headline numbers therefore receive their selection signal from the same benchmark they are claimed to predict. This is partial rather than total: WildJailbreak (Table 3) and transfer to Llama-3 (Table 5) use already-fixed settings and are genuinely out-of-sample.

full rationale

The paper's central derivation contains two load-bearing self-referential elements. First, the only stated implementation of 'attention redistribution' is Eq. (8), and under the additive causal mask convention used in Eq. (1), M' = M by construction; the mechanism is self-identical, so the claimed causal story does not connect to the empirical ASR reductions. Second, the sensitive-head selection and hyperparameters are tuned on the same JBB attacks/behaviors that Table 1 evaluates, making the headline per-attack ASRs partly in-sample fits. These issues are partially offset by genuine out-of-sample evidence: WildJailbreak is unseen real-world data evaluated with the fixed settings, and Table 5 transfers LLaMA-2 sensitive heads to Meta-Llama-3 without retuning. The limitations section (Sec. 7) acknowledges the attack scope is limited, which is a coverage caveat rather than circularity. Because the no-op is a mechanical identity rather than a fitted tautology, and because independent evaluations exist, the score is 6 rather than 8-10. This is a moderate-to-severe circularity/self-consistency finding, not a claim that all results are fabricated.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The method rests on three empirical assumptions: sensitive heads can be located by attention-difference analysis, boosting attention to localized intention tokens restores refusal, and the localization is accurate. Hyperparameters α, β and the unstated N for head selection are fitted on the evaluation benchmark.

free parameters (3)
  • alpha (head selection threshold) = 0.1 (LLaMA-2-7b-chat), 0.03 (Vicuna-13b-v1.5)
    Threshold for selecting sensitive attention heads; chosen per model based on ablation on the JBB evaluation benchmark (Section 4.4).
  • beta (mask scaling factor) = 5.0
    Scaling factor for attention mask boost; selected via sweep shown in Figure 3(b) on the evaluation data.
  • N (sample count for sensitivity estimation) = not reported
    Number of successful/unsuccessful defense samples used to compute ΔS̄; never specified, affecting reproducibility.
assumptions (4)
  • domain assumption Attention heads sensitive to jailbreak attacks can be identified by comparing attention scores between successful and unsuccessful defenses.
    Section 3.2 assumes the difference ΔS̄ localizes safety-relevant heads; no independent verification.
  • domain assumption Boosting attention to localized core-intention tokens suppresses attack-token influence and restores refusal.
    Section 3.3 builds the defense on this mechanism; the paper tests it only through end-to-end ASR.
  • domain assumption The prompt-based intent localization correctly identifies the user's core intention tokens in jailbreak inputs.
    Appendix E.2 provides the prompt; no accuracy evaluation is reported for this step.
  • domain assumption Sensitive heads identified on one model transfer to other models of the same family and architecture.
    Claimed in Section 5.1 with one transfer experiment (LLaMA-2 heads to Meta-Llama-3); limited evidence.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DETAM: Defending LLMs Against Jailbreak Attacks via Targeted Attention Modification." pith.science (2026). https://pith.science/paper/HAQCIVTZ

@misc{pith2026250413562,
  author       = {Pith},
  title        = {Pith review of: DETAM: Defending LLMs Against Jailbreak Attacks via Targeted Attention Modification},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HAQCIVTZ}},
  note         = {Machine review of arXiv:2504.13562}
}
read the original abstract

With the widespread adoption of Large Language Models (LLMs), jailbreak attacks have become an increasingly pressing safety concern. While safety-aligned LLMs can effectively defend against normal harmful queries, they remain vulnerable to such attacks. Existing defense methods primarily rely on fine-tuning or input modification, which often suffer from limited generalization and reduced utility. To address this, we introduce DETAM, a finetuning-free defense approach that improves the defensive capabilities against jailbreak attacks of LLMs via targeted attention modification. Specifically, we analyze the differences in attention scores between successful and unsuccessful defenses to identify the attention heads sensitive to jailbreak attacks. During inference, we reallocate attention to emphasize the user's core intention, minimizing interference from attack tokens. Our experimental results demonstrate that DETAM outperforms various baselines in jailbreak defense and exhibits robust generalization across different attacks and models, maintaining its effectiveness even on in-the-wild jailbreak data. Furthermore, in evaluating the model's utility, we incorporated over-defense datasets, which further validate the superior performance of our approach. The code will be released immediately upon acceptance.

Figures

Figures reproduced from arXiv: 2504.13562 by the authors.

Figure 1
Figure 1. An Illustration of LLM Vulnerabilities to [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. An illustration of DETAM. (Left) Identifying the attention heads sensitive to jailbreak attacks by analyzing attention distribution differences between successful and unsuccessful defenses. (Right) Dynamically reallocating attention within the identified heads during inference to prioritize the user’s intention. types of jailbreak attacks, and offers a low-cost and highly efficient solution. 3 Methods DETAM is intro… view at source ↗
Figure 3
Figure 3. Ablation study for DETAM on LLaMA2. fewer valid inputs being falsely refused. The results of our evaluation, presented in [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Impact of Attention Head Selection on DETAM. (a) Performance comparison of randomly selected vs. sensitive attention heads. (b) Visualization of sensitive attention head selection. Red regions represent attention heads with ∆S¯ i,j > α, while blue regions represent att…
Figure 5
Figure 5. Figure 5: Attention distribution at the last token position [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Attention distribution at the fifth head in all [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

50 extracted references · 12 canonical work pages

  1. [1]

    Bang An, Sicheng Zhu, Ruiyi Zhang, Michael-Andrei Panaitescu-Liess, Yuancheng Xu, and Furong Huang. 2024. Automatic pseudo-harmful prompt generation for evaluating false refusals in large language models. arXiv preprint arXiv:2409.00598

    arXiv 2024

  2. [2]

    Maksym Andriushchenko, Francesco Croce, and Nicolas Flammarion. 2024. Jailbreaking leading safety-aligned llms with simple adaptive attacks. arXiv preprint arXiv:2404.02151

    arXiv 2024

  3. [3]

    Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862

    arXiv 2022

  4. [4]

    Bender, Timnit Gebru, Angelina McMillan - Major, and Shmargaret Shmitchell

    Emily M. Bender, Timnit Gebru, Angelina McMillan - Major, and Shmargaret Shmitchell. 2021. https://doi.org/10.1145/3442188.3445922 On the dangers of stochastic parrots: Can language models be too big? In FAccT '21: 2021 ACM Conference on Fairness, Accountability, and Transparency, Virtual Event / Toronto, Canada, March 3-10, 2021 , pages 610--623. ACM

    arXiv 2021

  5. [5]

    Bochuan Cao, Yu Cao, Lu Lin, and Jinghui Chen. 2024. https://api.semanticscholar.org/CorpusID:262827619 Defending against alignment-breaking attacks via robustly aligned llm . In Annual Meeting of the Association for Computational Linguistics

    work page 2024

  6. [6]

    Ting-Yun Chang, Jesse Thomason, and Robin Jia. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.574 When parts are greater than sums: Individual LLM components can outperform full models . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 10280--10299, Miami, Florida, USA. Association for Computational Linguistics

    doi:10.18653/v1/2024.emnlp-main.574 2024

  7. [7]

    Patrick Chao, Edoardo Debenedetti, Alexander Robey, Maksym Andriushchenko, Francesco Croce, Vikash Sehwag, Edgar Dobriban, Nicolas Flammarion, George J Pappas, Florian Tramer, et al. 2024. Jailbreakbench: An open robustness benchmark for jailbreaking large language models. arXiv preprint arXiv:2404.01318

    arXiv 2024

  8. [8]

    Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J Pappas, and Eric Wong. 2023. Jailbreaking black box large language models in twenty queries. arXiv preprint arXiv:2310.08419

    arXiv 2023

Show all 50 references
  1. [9]

    Zhuowei Chen, Qiannan Zhang, and Shichao Pei. 2025. https://doi.org/10.48550/arXiv.2502.10438 Injecting universal jailbreak backdoors into llms in minutes

    2025 doi arXiv

  2. [10]

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al. 2023. Vicuna: An open-source chatbot impressing gpt-4 with 90\ See https://vicuna. lmsys. org (accessed 14 April 2023), 2(3):6

    2023

  3. [11]

    Justin Cui, Wei-Lin Chiang, Ion Stoica, and Cho-Jui Hsieh. 2024. Or-bench: An over-refusal benchmark for large language models. arXiv preprint arXiv:2405.20947

    2024 arXiv

  4. [12]

    Josef Dai, Xuehai Pan, Ruiyang Sun, Jiaming Ji, Xinbo Xu, Mickel Liu, Yizhou Wang, and Yaodong Yang. 2023. Safe rlhf: Safe reinforcement learning from human feedback. International Conference on Learning Representations

    2023

  5. [13]

    Peng Ding, Jun Kuang, Dan Ma, Xuezhi Cao, Yunsen Xian, Jiajun Chen, and Shujian Huang. 2024. https://doi.org/10.18653/v1/2024.naacl-long.118 A wolf in sheep`s clothing: Generalized nested jailbreak prompts can fool large language models easily . In Proceedings of the 2024 Conf...

    2024 doi

  6. [14]

    Pappas, Hamed Hassani, Yang Zhang, Eric Wong, and Shiyu Chang

    Jiabao Ji, Bairu Hou, Alexander Robey, George J. Pappas, Hamed Hassani, Yang Zhang, Eric Wong, and Shiyu Chang. 2024. https://api.semanticscholar.org/CorpusID:267938320 Defending large language models against jailbreak attacks via semantic smoothing . ArXiv, abs/2402.16192

    2024 arXiv

  7. [15]

    Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. 2024 a . Mixtral of experts. arXiv preprint arXiv:2401.04088

    2024 arXiv

  8. [16]

    Liwei Jiang, Kavel Rao, Seungju Han, Allyson Ettinger, Faeze Brahman, Sachin Kumar, Niloofar Mireshghallah, Ximing Lu, Maarten Sap, Yejin Choi, et al. 2024 b . Wildteaming at scale: From in-the-wild jailbreaks to (adversarially) safer language models. arXiv preprint arXiv:2406.18510

    2024 arXiv

  9. [17]

    Shuyang Jiang, Yusheng Liao, Ya Zhang, Yanfeng Wang, and Yu Wang. 2024 c . https://proceedings.neurips.cc/paper_files/paper/2024/file/be0a8ecf8b2743a4117557c5eca0fb79-Paper-Conference.pdf Taia: Large language models are out-of-distribution data learners . In Advances in Neural...

    2024

  10. [18]

    Zhuoran Jin, Pengfei Cao, Hongbang Yuan, Yubo Chen, Jiexin Xu, Huaijun Li, Xiaojian Jiang, Kang Liu, and Jun Zhao. 2024. https://doi.org/10.18653/v1/2024.findings-acl.70 Cutting off the head ends the conflict: A mechanism for interpreting and mitigating knowledge conflicts in ...

    2024 doi

  11. [19]

    Xuan Li, Zhanke Zhou, Jianing Zhu, Jiangchao Yao, Tongliang Liu, and Bo Han. 2023. Deepinception: Hypnotize large language model to be jailbreaker. arXiv preprint arXiv:2311.03191

    2023 arXiv

  12. [20]

    Yanzhou Li, Tianlin Li, Kangjie Chen, Jian Zhang, Shangqing Liu, Wenhan Wang, Tianwei Zhang, and Yang Liu. 2024. https://openreview.net/forum?id=duZANm2ABX Badedit: Backdooring large language models by model editing . In The Twelfth International Conference on Learning Represe...

    2024

  13. [21]

    Fan Liu, Zhao Xu, and Hao Liu. 2024 a . Adversarial tuning: Defending against jailbreak attacks for llms. arXiv preprint arXiv:2406.06622

    2024 arXiv

  14. [22]

    Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. 2023. Autodan: Generating stealthy jailbreak prompts on aligned large language models. arXiv preprint arXiv:2310.04451

    2023 arXiv

  15. [23]

    Zichuan Liu, Zefan Wang, Linjie Xu, Jinyu Wang, Lei Song, Tianchun Wang, Chunlin Chen, Wei Cheng, and Jiang Bian. 2024 b . Protecting your llms with information bottleneck. In Neural Information Processing Systems

    2024

  16. [24]

    Yichuan Mo, Yuji Wang, Zeming Wei, and Yisen Wang. 2024. Fight back against jailbreaking via prompt adversarial tuning. In The Thirty-eighth Annual Conference on Neural Information Processing Systems

    2024

  17. [25]

    OpenAI. 2023. https://arxiv.org/abs/2303.08774 GPT-4 technical report . CoRR, abs/2303.08774

    2023 arXiv

  18. [26]

    Javier Rando and Florian Tram \` e r. 2024. https://openreview.net/forum?id=GxCGsxiAaK Universal jailbreak backdoors from poisoned human feedback . In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024 . OpenReview.net

    2024

  19. [27]

    Mengjie Ren, Boxi Cao, Hongyu Lin, Cao Liu, Xianpei Han, Ke Zeng, Wan Guanglu, Xunliang Cai, and Le Sun. 2024. https://doi.org/10.18653/v1/2024.acl-long.330 Learning or self-aligning? rethinking instruction fine-tuning . In Proceedings of the 62nd Annual Meeting of the Associa...

    2024 doi

  20. [28]

    Alexander Robey, Eric Wong, Hamed Hassani, and George J. Pappas. 2023. https://arxiv.org/abs/2310.03684 Smoothllm: Defending large language models against jailbreaking attacks . Preprint, arXiv:2310.03684

    2023 arXiv

  21. [29]

    Paul R \"o ttger, Hannah Kirk, Bertie Vidgen, Giuseppe Attanasio, Federico Bianchi, and Dirk Hovy. 2024. https://doi.org/10.18653/v1/2024.naacl-long.301 XST est: A test suite for identifying exaggerated safety behaviours in large language models . In Proceedings of the 2024 Co...

    2024 doi

  22. [30]

    do anything now

    Xinyue Shen, Zeyuan Johnson Chen, Michael Backes, Yun Shen, and Yang Zhang. 2023. https://api.semanticscholar.org/CorpusID:260704242 "do anything now": Characterizing and evaluating in-the-wild jailbreak prompts on large language models . In Conference on Computer and Communic...

    2023

  23. [31]

    Chenyu Shi, Xiao Wang, Qiming Ge, Songyang Gao, Xianjun Yang, Tao Gui, Qi Zhang, Xuanjing Huang, Xun Zhao, and Dahua Lin. 2024. https://doi.org/10.18653/v1/2024.acl-long.253 Navigating the O ver K ill in large language models . In Proceedings of the 62nd Annual Meeting of the ...

    2024 doi

  24. [32]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton - Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu,...

    2023 arXiv

  25. [33]

    Yihan Wang, Zhouxing Shi, Andrew Bai, and Cho-Jui Hsieh. 2024. https://doi.org/10.18653/v1/2024.findings-acl.948 Defending LLM s against jailbreaking attacks via backtranslation . In Findings of the Association for Computational Linguistics: ACL 2024, pages 16031--16046, Bangk...

    2024 doi

  26. [34]

    Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. 2024 a . Jailbroken: How does llm safety training fail? Advances in Neural Information Processing Systems, 36

    2024

  27. [35]

    Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. 2024 b . Jailbroken: How does llm safety training fail? Advances in Neural Information Processing Systems, 36

    2024

  28. [36]

    Zeming Wei, Yifei Wang, Ang Li, Yichuan Mo, and Yisen Wang. 2023. Jailbreak and guard aligned language models with only few in-context demonstrations. arXiv preprint arXiv:2310.06387

    2023 arXiv

  29. [37]

    Yueqi Xie, Jingwei Yi, Jiawei Shao, Justin Curl, Lingjuan Lyu, Qifeng Chen, Xing Xie, and Fangzhao Wu. 2023. Defending chatgpt against jailbreak attack via self-reminders. Nature Machine Intelligence, 5(12):1486--1496

    2023

  30. [38]

    Zhangchen Xu, Fengqing Jiang, Luyao Niu, Jinyuan Jia, Bill Yuchen Lin, and Radha Poovendran. 2024. https://api.semanticscholar.org/CorpusID:267658033 Safedecoding: Defending against jailbreak attacks via safety-aware decoding . Annual Meeting of the Association for Computation...

    2024

  31. [39]

    Yifan Yao, Jinhao Duan, Kaidi Xu, Yuanfang Cai, Eric Sun, and Yue Zhang. 2023. https://arxiv.org/abs/2312.02003 A survey on large language model (LLM) security and privacy: The good, the bad, and the ugly . CoRR, abs/2312.02003

    2023 arXiv

  32. [40]

    Jiahao Yu, Xingwei Lin, Zheng Yu, and Xinyu Xing. 2023. Gptfuzzer: Red teaming large language models with auto-generated jailbreak prompts. arXiv preprint arXiv:2309.10253

    2023 arXiv

  33. [41]

    Jiahao Yu, Haozheng Luo, Jerry Yao-Chieh Hu, Wenbo Guo, Han Liu, and Xinyu Xing. 2024. https://api.semanticscholar.org/CorpusID:270199867 Enhancing jailbreak attack against large language models through silent tokens . ArXiv, abs/2405.20653

    2024 arXiv

  34. [42]

    Yifan Zeng, Yiran Wu, Xiao Zhang, Huazheng Wang, and Qingyun Wu. 2024. Autodefense: Multi-agent llm defense against jailbreak attacks. arXiv preprint arXiv:2403.04783

    2024 arXiv

  35. [43]

    Zhexin Zhang, Junxiao Yang, Pei Ke, Shiyao Cui, Chujie Zheng, Hongning Wang, and Minlie Huang. 2024 a . Safe unlearning: A surprisingly effective and generalizable solution to defend against jailbreak attacks. arXiv preprint arXiv:2407.02855

    2024 arXiv

  36. [44]

    Zhexin Zhang, Junxiao Yang, Pei Ke, and Minlie Huang. 2024 b . https://api.semanticscholar.org/CorpusID:265212812 Defending large language models against jailbreaking attacks through goal prioritization . In Annual Meeting of the Association for Computational Linguistics

    2024

  37. [45]

    Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. 2023. Instruction-following evaluation for large language models. arXiv preprint arXiv:2311.07911

    2023 arXiv

  38. [46]

    Weikang Zhou, Xiao Wang, Limao Xiong, Han Xia, Yingshuang Gu, Mingxu Chai, Fukang Zhu, Caishuang Huang, Shihan Dou, Zhiheng Xi, et al. 2024 a . Easyjailbreak: A unified framework for jailbreaking large language models. arXiv preprint arXiv:2403.12171

    2024 arXiv

  39. [47]

    Yukai Zhou, Zhijie Huang, Feiyang Lu, Zhan Qin, and Wenjie Wang. 2024 b . Don't say no: Jailbreaking llm by suppressing refusal. arXiv preprint arXiv:2404.16369

    2024 arXiv

  40. [48]

    Andy Zou, Zifan Wang, J Zico Kolter, and Matt Fredrikson. 2023. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043

    2023 arXiv

  41. [49]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  42. [50]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.