Pith. sign in

REVIEW 4 major objections 5 minor 50 references

Exploring Jailbreak Attacks on LLMs through Intent Concealment and Diversion

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that a parameter-free, black-box attack can jailbreak instruction-aligned LLMs in a single target-model query by splitting harmful requests into hierarchical reasoning fragments, padding them with semantically related…

desk verdict A plausible new SOTA jailbreak recipe and a useful two-scenario benchmark, but the single-query efficiency claim hides auxiliary LLM calls and the paper ships no code, data, or seeds. read the letter →

arxiv 2505.14316 v1 pith:QX5ZKWTT submitted 2025-05-20 cs.CR cs.AI

Tiehan Cui , Yanxu Mao , Peipei Liu , Congying Liu , Datao You This is my paper · ORCID
classification cs.CRcs.AI
keywords jailbreakattacklargelanguagemodelsafetyintentconcealmenthierarchicalsplitsemanticexpansionsingle-queryblack-boxBiSceneEval
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that the safety guards of instruction-tuned LLMs share a common failure mode: they break when a harmful request is decomposed into nested reasoning subtasks and scattered with semantically related filler words. The proposed attack, ICE, performs this decomposition without any training or parameter access, so it works as a black-box, single-query jailbreak and transfers across models. The paper reports more than 70% average attack success rate on six mainstream LLMs and a 15.16x speedup over the ReNeLLM baseline on the AdvBench benchmark. It also introduces BiSceneEval, a dataset that separates harmful inquiries from toxic text-generation targets, arguing that safety evaluation must test both blocking harmful inputs and detecting harmful outputs. If the claims hold, current alignment methods are vulnerable to structured reasoning prompts, and defenses need to combine static safety training with real-time semantic decomposition.

What carries the argument

The load-bearing mechanism is the four-step ICE pipeline. Hierarchical Split builds a dependency graph of the original prompt, selects non-root verbs, randomly prunes some dependency relations, and assigns increasing level numbers to word groups so that the sentence is partitioned into nested ranges. Semantic Expansion collects a sentiment label, a synonym of a verb, a representative noun phrase from a noun's definition, and two composition/hazard words plus a toxicity word for the most toxic term; the paper obtains these with DistilBERT, WordNet, and an LLM. Reasoning Mask is the central obfuscation device: contiguous word groups at each hierarchy level are replaced by random uppercase placeholder letters, each placeholder is explained in a 'M is (...)' sentence, and these sentences are concatenated in random order together with the semantic-expansion words. Environmental Construction then frames the masked sentence as a benign reconstruction task. The argument is that across instruction-aligned models, the reconstruction demand overrides the safety filter, because the model's multi-step reasoning capacity is overloaded and the harmful intent is never present as a single recognizable instruction.

What would settle it

Count every API call in the ICE pipeline—target-model query, the LLM call that generates toxic-word descriptions, the sentiment-classifier call, and the WordNet lookups—on the AdvBench sample used for Table 3, and compare that total cost with ReNeLLM's measured 132.03 seconds and 20 queries. Also run ICE on a reasoning-enhanced model with explicit multi-step validation and measure attack success; if the total cost exceeds ReNeLLM's, or the attack success collapses on reasoning-enhanced models, the paper's central efficiency and transferability claims fail.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central claim is that LLMs' safety alignment breaks under cognitive overload: when an instruction is split into small fragments, each is replaced by a placeholder letter and an explanation sentence of the form 'M is (...)', and the whole thing is framed as a fictional reasoning problem, the model treats the content as a neutral task, reconstructs the original harmful request, and then answers it in detail. ICE realizes this with four steps: Hierarchical Split uses a dependency parse and part-of-speech tags to break the prompt into multi-level fragments; Semantic Expansion adds a sentiment word, WordNet-derived synonyms and noun definitions, plus toxicity-related description words produced by an LLM; Reasoning Mask replaces contiguous fragments with randomly assigned uppercase placeholders and shuffles the explanation sentences; Environmental Construction wraps the masked sentence in an instruction to restore the sentence and solve a problem. On AdvBench, the paper reports keyword-based attack success of 99.2% for GPT-3.5, 99.8% for GPT-4, 96.9% for Claude-1, and 88.9% for Llama2, with single-query execution and 8.71 seconds per sample, and it reports over 70% average attack success across six instruction-aligned LLMs. The second contribution, BiSceneEval, contains 319 harmful inquiries and 433 toxic-response texts with baseline attack success under 1%, and its text-generation scenario measures whether a model can be made to reproduce toxic content verbatim, not merely to answer a harmful question.

Load-bearing premise

The load-bearing premise is that 'single-query' means one call to the target model, because ICE's prompt construction itself calls a commercial LLM, a sentiment classifier, and WordNet, and those auxiliary calls are not counted in the query number or time cost; if a defender counts any API call, the single-query and speedup claims do not hold.

Editorial extensions

If this is right

  • Safety filters that block on obvious harmful phrases will not catch ICE, because no contiguous harmful instruction appears in the final prompt; detection has to reconstruct the request from placeholders.
  • Because the attack needs a single target-model query, defenses that assume iterative query patterns or that rate-limit repeated attempts will not stop it.
  • The high transferability the paper reports across GPT, Claude, and Llama families implies the vulnerability is shared by current instruction-tuned models rather than an artifact of one safety implementation.
  • Evaluation datasets should include text-generation restoration (e.g., Restore-ASR) alongside question-answering, since a model can refuse questions yet still output toxic text when asked to restore a sentence.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper's reported numbers, the 'single-query' claim counts only target-model calls: prompt construction uses an LLM call, a sentiment classifier, and WordNet lookups, so an end-to-end cost audit would likely raise the time per sample above the reported 8.71 seconds.
  • A testable extension is to run ICE on a reasoning-enhanced model with explicit multi-step validation; if attack success collapses there, the cognitive-overload mechanism is confirmed, while sustained success would mean the attack is generic obfuscation.
  • A defensive corollary the paper does not implement is that a model which resolves each placeholder to its fragment and checks the reconstructed sentence for toxicity before answering should block ICE; this can be evaluated directly.
  • The same reconstruction-based principle could be ported to multi-turn or multimodal settings, where the reconstruction instruction is less conspicuous; the paper does not test these settings.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The manuscript proposes ICE, a black-box jailbreak framework that splits a malicious query into hierarchical reasoning fragments, expands the fragments with semantically related words (including toxicity-related descriptors generated by GPT-4o), and then conceals the fragments behind a 'reasoning mask' so that a single target-model query induces harmful output. The paper also introduces BiSceneEval, a two-scenario dataset with Harmful Inquiries and Toxic Responses, and reports experiments on AdvBench and BiSceneEval across several LLMs, claiming state-of-the-art ASR and a 15.16x time speedup over ReNeLLM.

Significance. If the ASR and efficiency claims are reproducible, ICE is a practically relevant contribution: it is a non-parametric, transferable attack that needs only one target-model call, and BiSceneEval addresses a real evaluation gap by separating pre-inference prevention from post-inference detection. The paper is also honest in its limitations section about open questions for reasoning-enhanced models. However, the current evidence does not yet fully support the headline claims because the query/time accounting is underspecified, no variance estimates are provided despite randomized algorithm steps, and the same LLM (GPT-4o) is used as both a prompt-generation component and the GPT-ASR judge.

major comments (4)
  1. [§4.1.1, Table 3, §3.4] The 'single-query' and 15.16x speedup claims are not supported as stated because 'Query Times' and TCPS are never defined, and the per-sample GPT-4o call in Eq. (8), together with the DistilBERT and WordNet lookups, is not counted in the reported query budget. Please define whether the query count includes all LLM API calls or only target-model calls, state explicitly what TCPS includes, and recompute the ICE and ReNeLLM comparisons under a symmetric cost model.
  2. [§1 contribution bullet, Tables 3–5] The claim of 'more than 70% average ASR' on '6 mainstream LLMs' cannot be verified from the reported experiments: Table 3 reports results for only 5 models, Tables 4–5 report results for 7 models, and no seeds, repeated runs, or confidence intervals are given even though Sections 3.3–3.5 contain multiple random choices (random subset selection, random breakpoint pairing, random placeholder ordering). Please specify the exact model set used for the aggregate claim and provide variance/error bars for the main ASR results.
  3. [§3.1] The framework is described as 'parameter-free,' but the method relies on several thresholds and design choices, including the NCD cutoff of 0.6, the toxicity cutoff of 0.9, the Levenshtein threshold of 0.95, the relation set Rpreserved, and the random selection distributions in Sections 3.3–3.4. If 'parameter-free' is intended to mean 'without learned parameters,' this should be stated explicitly; otherwise the term overstates the method's generality.
  4. [§4.1.1 and §3.4] GPT-4o is used both to construct part of the prompt in Eq. (8) and to judge attack success in the GPT-ASR metric. This evaluator–constructor overlap is a potential confound for the reported GPT-ASR values, and the paper does not discuss it. Please address this overlap explicitly and, ideally, validate a subset of AdvBench responses with human raters or an independent judge.
minor comments (5)
  1. [Figure 2 and Appendix C] Figure 2 and Appendix C contain garbled font sequences (e.g., '/uni00000015/...') that appear to be rendering artifacts; these should be replaced with readable text, and the author affiliation contains a typo ('Henan Univeristy').
  2. [Table 3] In the 'Query Times' column, the entry '100P' for AutoDAN is unclear, and the units for the other entries are not specified; please define what is being counted (e.g., target-model calls, total API calls, or iterations).
  3. [Tables 4–5] The phrase 'Model-specific ASR' is not defined; please clarify whether it refers to per-model ASR, per-category ASR, or both.
  4. [§4.2.1] The statement that 'identical inputs across different models' were used is ambiguous because ICE's random components mean the generated prompt depends on random draws; please clarify whether one fixed prompt per sample was generated and then reused across models, or whether a separate prompt was generated per model.
  5. [§4.1.1] For Restore-ASR, please specify whether the 0.95 threshold applies to Levenshtein similarity or Levenshtein distance, and describe how partial restorations are scored.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the jailbreak attack is not derived from its own evaluation, though a minor GPT-4o oracle overlap and one non-load-bearing self-citation warrant a low score.

full rationale

The paper's central claims—ICE achieves high ASR with a single target-model query and generalizes across LLMs—are supported by direct experiments on the external AdvBench dataset and on the newly constructed BiSceneEval dataset, rather than by fitting a parameter to the result. The attack pipeline is a parameter-free prompt-construction procedure: hierarchical split, semantic expansion, reasoning masking, and environmental construction are all defined as syntactic/semantic transformations of the input prompt, with no equation that assumes the target outcome. The evaluation metrics (KW-ASR, GPT-ASR, Hybrid-ASR, Human-ASR, Restore-ASR) are applied after the attack, not used as inputs to construct the attack. The most notable concern is an evaluation-oracle overlap: GPT-4o is used in Section 3.4 (Eq. 8) to generate toxic-word descriptions that become part of the prompt, and GPT-4o is also used as the GPT-ASR judge in Section 4.1.1. This is a methodological/overlap caveat rather than circularity, because the GPT-ASR judge does not define the attack success condition in a way that forces a positive outcome; a jailbreak still depends on the target LLM's actual response. Similarly, the 'single-query' and TCPS claims exclude auxiliary GPT-4o, DistilBERT, and WordNet calls in prompt construction, which is an efficiency-accounting issue, not a circular-derivation issue. The only self-citation is Mao et al. 2025 in Section 2.2, used to justify NCD-based deduplication; this is a preprocessing step and is not load-bearing for the main attack or evaluation results, and the underlying NCD method is independently attributable to Jiang et al. 2023. Overall, no circular reduction is present under the definitions in the reviewer instructions, so the appropriate score is low rather than a finding of construction-level circularity.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

ICE introduces no new physical or ontological entities. Its central results rest on hand-set thresholds, on external language tools, and on assumptions about LLM cognitive overload and the reliability of automated harm evaluation.

free parameters (5)
  • NCD deduplication threshold = 0.6
    Section 2.2 chooses NCD below 0.6 to define near-duplicates; this directly changes dataset size and composition.
  • Toxicity filtering cutoff = 0.9
    Section 2.3 drops texts below 0.9 toxicity returned by the OpenAI moderation endpoint; this affects BiSceneEval difficulty.
  • Restore-ASR Levenshtein threshold = 0.95
    Section 4.1.1 sets the text-similarity threshold that defines success in the text generation scenario; small changes would move the reported ASR.
  • Preserved dependency relations Rpreserved = neg, fixed, compound, amod, advmod, nmod
    Equation (3) in Section 3.3 fixes which syntactic relations survive splitting; this hand-picked set is central to the hierarchical decomposition.
  • Random split and selection distribution = uniform over verbs, breakpoint pairs, and word choices; no seed
    Algorithm 1 and Section 3.4 introduce stochasticity; the paper does not report the distribution's effect on ASR or provide seeds for replication.
assumptions (4)
  • domain assumption Instruction-aligned LLMs share a cognitive overload vulnerability: nested reasoning decompositions reduce safety-filter effectiveness while preserving enough semantics for reconstruction.
    Stated as motivation in Section 3.1; the entire ICE method depends on this hypothesis, but no independent ablation isolates it.
  • domain assumption GPT-4o, DistilBERT, and WordNet provide sufficiently accurate semantic expansion outputs, including sentiment words, related words, definition phrases, and toxic-word descriptions.
    Section 3.4, Equations (7) through (9), relies on these external tools without reporting their accuracy or variance.
  • domain assumption Keyword dictionaries, GPT-4o-as-judge, and Levenshtein-based Restore-ASR are valid measures of harmful content and attack success.
    Section 4.1.1 defines these metrics; the Limitations section itself concedes that they may overestimate stealth and impact.
  • domain assumption The OpenAI moderation endpoint (omni-moderation-latest) correctly classifies and filters harmful texts for BiSceneEval.
    Section 2.3 uses it as the automated classifier; no human agreement statistics are reported for this step.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Exploring Jailbreak Attacks on LLMs through Intent Concealment and Diversion." pith.science (2026). https://pith.science/paper/QX5ZKWTT

@misc{pith2026250514316,
  author       = {Pith},
  title        = {Pith review of: Exploring Jailbreak Attacks on LLMs through Intent Concealment and Diversion},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QX5ZKWTT}},
  note         = {Machine review of arXiv:2505.14316}
}
read the original abstract

Although large language models (LLMs) have achieved remarkable advancements, their security remains a pressing concern. One major threat is jailbreak attacks, where adversarial prompts bypass model safeguards to generate harmful or objectionable content. Researchers study jailbreak attacks to understand security and robustness of LLMs. However, existing jailbreak attack methods face two main challenges: (1) an excessive number of iterative queries, and (2) poor generalization across models. In addition, recent jailbreak evaluation datasets focus primarily on question-answering scenarios, lacking attention to text generation tasks that require accurate regeneration of toxic content. To tackle these challenges, we propose two contributions: (1) ICE, a novel black-box jailbreak method that employs Intent Concealment and divErsion to effectively circumvent security constraints. ICE achieves high attack success rates (ASR) with a single query, significantly improving efficiency and transferability across different models. (2) BiSceneEval, a comprehensive dataset designed for assessing LLM robustness in question-answering and text-generation tasks. Experimental results demonstrate that ICE outperforms existing jailbreak techniques, revealing critical vulnerabilities in current defense mechanisms. Our findings underscore the necessity of a hybrid security strategy that integrates predefined security mechanisms with real-time semantic decomposition to enhance the security of LLMs.

Figures

Figures reproduced from arXiv: 2505.14316 by the authors.

Figure 1
Figure 1. The construction process of BiSceneEval. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. The proportion of each category to the entire [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Overview of the ICE attack process. attack performance. ICE can be abstracted into four steps: 1) Hierarchical Split. This method or￾ganizes input prompts into hierarchical fragments based on part-of-speech and syntactic dependen￾cies. 2) Semantic Expansion. This step inte￾grates enhanced terms representing additional se￾mantic information related to the prompt. 3) Rea￾soning Mask. Hierarchical fragments and en￾hanc… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: The TCPS and query times of different jail [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Data volume reduction across three steps, [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: Examples of Three Jailbreaking Methods, with Attack Methods Highlighted in Orange Boxes and Their [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

50 extracted references · 21 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    OpenAI. 2023b. 2023. Gpt-4 technical report. https://cdn. openai.com/papers/gpt-4.pdf

    work page 2023

  4. [4]

    Anthropic. 2024. Introducing claude. https://www.anthropic.com/news/introducing-claude

    work page 2024

  5. [5]

    Rishabh Bhardwaj and Soujanya Poria. 2023. Red-teaming large language models using chain of utterances for safety-alignment. arXiv preprint arXiv:2308.09662

    arXiv 2023

  6. [6]

    Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. In Proceedings of the 34th International Conference on Neural Information Processing Systems, pages 1877--1901

    2020

  7. [7]

    Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J Pappas, and Eric Wong. 2023. Jailbreaking black box large language models in twenty queries. arXiv preprint arXiv:2310.08419

    arXiv 2023

  8. [8]

    Boyi Deng, Wenjie Wang, Fuli Feng, Yang Deng, Qifan Wang, and Xiangnan He. 2023. Attack prompt generation for red teaming and defending large language models. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 2176--2189

    2023

Show all 50 references
  1. [9]

    Peng Ding, Jun Kuang, Dan Ma, Xuezhi Cao, Yunsen Xian, Jiajun Chen, and Shujian Huang. 2024. A wolf in sheep’s clothing: Generalized nested jailbreak prompts can fool large language models easily. In Proceedings of the 2024 Conference of the North American Chapter of the Assoc...

    2024

  2. [10]

    Gilad Gressel, Rahul Pankajakshan, and Yisroel Mirsky. 2024. Are you human? an adversarial benchmark to expose llms. arXiv preprint arXiv:2410.09569

    2024 arXiv

  3. [11]

    Xingang Guo, Fangxu Yu, Huan Zhang, Lianhui Qin, and Bin Hu. 2024. Cold-attack: Jailbreaking llms with stealthiness and controllability. In Forty-first International Conference on Machine Learning

    2024

  4. [12]

    Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi, Maarten Sap, Dipankar Ray, and Ece Kamar. 2022. Toxigen: A large-scale machine-generated dataset for implicit and adversarial hate speech detection. In Proceedings of the 60th Annual Meeting of the Association for Computational...

    2022

  5. [13]

    Yangsibo Huang, Samyak Gupta, Mengzhou Xia, Kai Li, and Danqi Chen. 2023. Catastrophic jailbreak of open-source llms via exploiting generation. In The Twelfth International Conference on Learning Representations

    2023

  6. [14]

    Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, et al. 2024. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409.12186

    2024 arXiv

  7. [15]

    low-resource

    Zhiying Jiang, Matthew Yang, Mikhail Tsirlin, Raphael Tang, Yiqin Dai, and Jimmy Lin. 2023. “low-resource” text classification: A parameter-free classification method with compressors. In Findings of the Association for Computational Linguistics: ACL 2023, pages 6810--6828

    2023

  8. [16]

    Daniel Kang, Xuechen Li, Ion Stoica, Carlos Guestrin, Matei Zaharia, and Tatsunori Hashimoto. 2024. Exploiting programmatic behavior of llms: Dual-use through standard security attacks. In The Second Workshop on New Frontiers in Adversarial Machine Learning

    2024

  9. [17]

    Yifan Li, Hangyu Guo, Kun Zhou, Wayne Xin Zhao, and Ji-Rong Wen. 2024. Images are achilles' heel of alignment: Exploiting visual vulnerabilities for jailbreaking multimodal large language models. arXiv preprint arXiv:2403.09792

    2024 arXiv

  10. [18]

    Tong Liu, Yingjie Zhang, Zhe Zhao, Yinpeng Dong, Guozhu Meng, and Kai Chen. 2024. Making them ask and answer: Jailbreaking large language models in few queries via disguise and reconstruction. In 33rd USENIX Security Symposium (USENIX Security 24), pages 4711--4728

    2024

  11. [19]

    Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. 2023. Autodan: Generating stealthy jailbreak prompts on aligned large language models. arXiv preprint arXiv:2310.04451

    2023 arXiv

  12. [20]

    Binny Mathew, Punyajoy Saha, Seid Muhie Yimam, Chris Biemann, Pawan Goyal, and Animesh Mukherjee. 2021. Hatexplain: A benchmark dataset for explainable hate speech detection. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 14867--14875

    2021

  13. [21]

    OpenAI. 2023 a . https://platform.openai.com/docs/guides/moderation Openai moderation guide . Accessed: 2024-12-27

    2023

  14. [22]

    OpenAI. 2023 b . https://openai.com/index/upgrading-the-moderation-api-with-our-new-multimodal-moderation-model/ Upgrading the moderation api with our new multimodal moderation model . Accessed: 2024-12-27

    2023

  15. [23]

    Xiangyu Qi, Kaixuan Huang, Ashwinee Panda, Peter Henderson, Mengdi Wang, and Prateek Mittal. 2024. Visual adversarial examples jailbreak aligned large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, 19, pages 21527--21536

    2024

  16. [24]

    https://www.reddit.com/ Reddit website

    Reddit. https://www.reddit.com/ Reddit website . Accessed: 2024-12-27

    2024

  17. [25]

    Erfan Shayegani, Yue Dong, and Nael Abu-Ghazaleh. 2023. Jailbreak in pieces: Compositional adversarial attacks on multi-modal language models. In The Twelfth International Conference on Learning Representations

    2023

  18. [26]

    do anything now

    Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, and Yang Zhang. 2023. " do anything now": Characterizing and evaluating in-the-wild jailbreak prompts on large language models. arXiv preprint arXiv:2308.03825

    2023 arXiv

  19. [27]

    do anything now

    Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, and Yang Zhang. 2024. " do anything now": Characterizing and evaluating in-the-wild jailbreak prompts on large language models. Proceedings of the 2024 ACM Conference on Computer and Communications Security

    2024

  20. [28]

    Llama Team. 2024. Meta llama. https: //github.com/meta-llama/PurpleLlama/blob/main/Llama-Guard2/MODEL\_CARD.md

    2024

  21. [29]

    Shubo Tian, Qiao Jin, Lana Yeganova, Po-Ting Lai, Qingqing Zhu, Xiuying Chen, Yifan Yang, Qingyu Chen, Won Kim, Donald C Comeau, et al. 2024. Opportunities and challenges for chatgpt and large language models in biomedicine and health. Briefings in Bioinformatics, 25(1):bbad493

    2024

  22. [30]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth \'e e Lacroix, Baptiste Rozi \`e re, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023 a . Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971

    2023 arXiv

  23. [31]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023 b . Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  24. [32]

    Siyuan Wang, Zhuohan Long, Zhihao Fan, and Zhongyu Wei. 2024. From llms to mllms: Exploring the landscape of multimodal jailbreaking. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 17568--17582

    2024

  25. [33]

    Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. 2022 a . Emergent abilities of large language models. Transactions on Machine Learning Research

    2022

  26. [34]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022 b . Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824--24837

    2022

  27. [35]

    Zeguan Xiao, Yan Yang, Guanhua Chen, and Yun Chen. 2024. Distract large language models for automatic jailbreak attack. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 16230--16244

    2024

  28. [36]

    Zihao Xu, Yi Liu, Gelei Deng, Yuekang Li, and Stjepan Picek. 2024. A comprehensive study of jailbreak attack versus defense for large language models. In Findings of the Association for Computational Linguistics ACL 2024, pages 7432--7449

    2024

  29. [37]

    Sibo Yi, Yule Liu, Zhen Sun, Tianshuo Cong, Xinlei He, Jiaxing Song, Ke Xu, and Qi Li. 2024. Jailbreak attacks and defenses against large language models: A survey. arXiv preprint arXiv:2407.04295

    2024 arXiv

  30. [38]

    Zhiyuan Yu, Xiaogeng Liu, Shunning Liang, Zach Cameron, Chaowei Xiao, and Ning Zhang. 2024. Don't listen to me: Understanding and exploring jailbreak prompts of large language models. arXiv preprint arXiv:2403.17336

    2024 arXiv

  31. [40]

    Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang, Chongxuan Li, Ngai-Man Man Cheung, and Min Lin. 2024 b . On evaluating adversarial robustness of large vision-language models. Advances in Neural Information Processing Systems, 36

    2024

  32. [41]

    Zhenhong Zhou, Jiuyang Xiang, Haopeng Chen, Quan Liu, Zherui Li, and Sen Su. 2024. Speak out of turn: Safety vulnerability of large language models in multi-turn dialogue. arXiv preprint arXiv:2402.17262

    2024 arXiv

  33. [42]

    Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson. 2023. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043

    2023 arXiv

  34. [43]

    Jailbreakbench: An open robustness benchmark for jailbreaking large language models

    Patrick Chao, Edoardo Debenedetti, Alexander Robey, Maksym Andriushchenko, Francesco Croce, Vikash Sehwag, Edgar Dobriban, Nicolas Flammarion, George J Pappas, Florian Tram \`e r, et al. Jailbreakbench: An open robustness benchmark for jailbreaking large language models. In IC...

    2024

  35. [44]

    Xingjun Ma, Yifeng Gao, Yixu Wang, Ruofan Wang, Xin Wang, Ye Sun, Yifan Ding, Hengyuan Xu, Yunhao Chen, Yunhan Zhao, et al. 2025. Safety at scale: A comprehensive survey of large model safety. arXiv preprint arXiv:2502.05206

    2025 arXiv

  36. [45]

    Yanxu Mao, Peipei Liu, Tiehan Cui, Congying Liu, and Datao You. 2025. Low-resource fast text classification based on intra-class and inter-class distance calculation. In Proceedings of the 31st International Conference on Computational Linguistics, pages 1045--1056

    2025

  37. [46]

    Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, et al. 2024. Harmbench: A standardized evaluation framework for automated red teaming and robust refusal. In ICML

    2024

  38. [47]

    George A Miller. 1995. Wordnet: a lexical database for english. Communications of the ACM, 38(11):39--41

    1995

  39. [48]

    Paul R \"o ttger, Hannah Kirk, Bertie Vidgen, Giuseppe Attanasio, Federico Bianchi, and Dirk Hovy. 2024. Xstest: A test suite for identifying exaggerated safety behaviours in large language models. In Proceedings of the 2024 Conference of the North American Chapter of the Asso...

    2024

  40. [49]

    Victor Sanh, L Debut, J Chaumond, and T Wolf. 2019. Distilbert, a distilled version of bert: Smaller, faster, cheaper and lighter. arxiv 2019. arXiv preprint arXiv:1910.01108

    2019 arXiv

  41. [50]

    A strongreject for empty jailbreaks

    Alexandra Souly, Qingyuan Lu, Dillon Bowen, Tu Trinh, Elvis Hsieh, Sana Pandey, Pieter Abbeel, Justin Svegliato, Scott Emmons, Olivia Watkins, et al. A strongreject for empty jailbreaks. In ICLR 2024 Workshop on Reliable and Responsible Foundation Models

    2024

  42. [51]

    Xuandong Zhao, Xianjun Yang, Tianyu Pang, Chao Du, Lei Li, Yu-Xiang Wang, and William Yang Wang. 2024. Weak-to-strong jailbreaking on large language models. arXiv preprint arXiv:2401.17256

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.