Pith. sign in

REVIEW 4 major objections 6 minor 135 references

A Multimodal Automatic Redteaming Evaluation based on Atomic Jailbreak Strategy Decoupling and Combination

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read An automated multimodal jailbreak method reports 95.48% average attack success on five mainstream multimodal large language models by decomposing attacks into six atomic strategy dimensions and recombining them.

desk verdict HACA's six-dimensional strategy taxonomy is a real contribution, but its headline 95.48% ASR is an in-sample number from the same GPT-4.1 judge that guides the attack — don't quote the absolute number without independent judging. read the letter →

arxiv 2608.04034 v1 pith:XWC6KSJ3 submitted 2026-08-03 cs.CR

Shiji Zhao , Yuxuan Zhou , Chen Xiong , Dongxian Wu , Yang Bai , Xun Chen This is my paper
classification cs.CR
keywords multimodaljailbreakredteamingMLLMsafetyatomicstrategytaxonomycross-modalplannerhierarchicalcombinationattacksuccessratetext-to-image
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that multimodal jailbreak attacks can be systematized: it decomposes text–image attack tricks into a six-dimensional atomic strategy space (structural, semantic, and syntactic layers, each applied to text and to images), then uses a planner to pick and combine one strategy from each dimension and a generative executor to turn the combination into a jailbreak prompt. On 350 harmful queries across seven categories, the resulting method, Hierarchical Atomic Combination Attack (HACA), is reported to achieve an average 95.48% attack success rate against five mainstream multimodal large language models, outperforming all compared baselines on every model and category. The claim matters because it shifts red-teaming from handcrafted, per-strategy code toward a reusable, searchable strategy space, and it would give safety evaluators a way to probe MLLMs at every processing layer at once. The authors also report that HACA needs far fewer queries than the best baseline, reaching near-saturated success in under two queries on average.

What carries the argument

The load-bearing object is the six-dimensional atomic strategy space: structural, semantic, and syntactic layers, each instantiated for text and for image modalities. Around it HACA builds a cross-modal joint planner that solves a constrained selection problem with two criteria — intra-layer cross-modal complementarity (text and image strategies at the same layer must reinforce each other) and inter-layer decoupling (strategies from different layers must not conflict) — followed by a unified generative executor that maps the selected strategy tuple to jailbreak text via a language model and to a jailbreak image via a text-to-image model. The success metric is a toxic-score judge with a threshold of 4, used both as feedback to the planner and as the evaluation criterion. The taxonomy is the part that carries the argument: it makes strategy combination systematic, and the ablations attribute the gains to all three layers and to the multimodal combination.

What would settle it

Run the exact HACA pipeline on the same five models and 350 queries but score every response with a second, independent harmfulness measure — human annotators or a different classifier — and compare ASR under that measure. If the independently judged success rate falls below the compared baselines, or if a substantial share of responses the GPT-4.1 judge scored 4 or higher are refusals or generic text, then the central result is an artifact of the judge rather than a property of the attack.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that the jailbreak problem for multimodal LLMs can be formulated as discrete optimization over a hierarchical strategy space. A strategy unit is a tuple specifying its modality, layer, name, execution method, and example; the taxonomy covers structural, semantic, and syntactic layers in both text and image modalities, giving thirty atomic units. The cross-modal joint planner enforces intra-layer cross-modal complementarity and inter-layer non-conflict, selecting six strategies (one per dimension) that jointly maximize the judged harmfulness of the target model's output under a query budget. A unified executor then renders the plan: a language model writes the jailbreak text and a text-to-image model renders the jailbreak image. The reported result is that this pipeline outperforms all compared baselines on all five MLLMs, with average attack success rate of 95.48% and near saturation on open-source models.

Load-bearing premise

The load-bearing premise is that the GPT-4.1 toxic-score judge with a threshold of 4 correctly identifies responses that are genuinely harmful and fully answer the malicious query; if that judge is lenient or gameable, the reported 95.48% attack success rate overstates true jailbreak capability.

Editorial extensions

If this is right

  • HACA saturates the attack success metric on vulnerable open-source MLLMs (99.14% on InternVL-3.5, 99.71% on LLaVA-1.6-Mistral) and stays strong on safety-tuned models (83.14% on VLGuard, 97.71% on GPT-5).
  • The planner reduces query overhead: average ASR near saturation in 1.72 queries on InternVL-3.5 and 1.90 on GPT-5, versus 3.15 and 3.82 for the strongest baseline Ideator.
  • Ablation shows each layer contributes: structural alone reaches 86% ASR, structural+semantic 98%, and all three layers 100% on the tested subset, while text+image together beat either modality alone.
  • The per-strategy analysis shows the planner reliably avoids low-performing strategies and prefers a small set of high-yield units, supporting the claim that the taxonomy separates useful from ineffective attack logic.
  • Because the selected strategies cover all six dimensions, an attack produced this way probes layout parsing, intent understanding, and low-level encoding simultaneously, which the paper argues targets the underdeveloped cross-layer defenses of current MLLMs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct implication the paper does not draw: the same strategy space could be inverted into a defense audit tool, where each atomic unit is a probe for which cognitive layer of a given MLLM is weakest, enabling layer-specific hardening rather than whole-model retraining.
  • Because the planner's feedback loop and the final metric both use the same judge, the 95.48% figure may partly reflect the planner learning to please that judge; an independent judge or human evaluation would be needed to confirm real-world harmfulness. This is an inference about the evaluation design, not a claim the paper makes.
  • If the toxic-score judge is stable across thresholds, the near-saturation results suggest that query-budget automated attacks on MLLMs may be approaching an upper ceiling on these benchmark categories, so future red-teaming should move to harder or dynamically updated safety categories.
  • The per-strategy table suggests that a much smaller subset of atomic units drives most of the success; a testable extension is to run HACA with the low-yield strategies removed and check whether ASR stays near 95%, which would tell whether the full thirty-unit taxonomy or only a handful of units matter.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes a multimodal jailbreak framework, HACA, built on a taxonomy of atomic jailbreak strategies decomposed along three layers (structural, semantic, syntactic) and two modalities (text, image), giving a six-dimensional strategy space. A cross-modal planner selects one strategy per layer-modality cell, and a unified generative executor (DeepSeek-V1 for text, Flux.1-dev for images) produces the jailbreak instruction. The authors report an average attack success rate of 95.48% across five MLLMs, with HACA outperforming all compared baselines on every model and category in Table 1. The paper also reports query-efficiency gains, ablations of the strategy layers and modalities, and per-strategy selection statistics in the appendix.

Significance. If the reported attack success rates are accurate and independently verified, HACA would represent a strong automated multimodal jailbreak method and the taxonomy could be a useful organizing device for red-teaming research. The paper ships internally consistent tables: the per-model ASR averages in Table 1 reproduce the reported 95.48% mean, and the ablation results are plausibly ordered. The six-dimensional atomic strategy taxonomy is a genuinely useful contribution, and the unified executor is a pragmatic engineering improvement over per-strategy pipelines. However, the significance of the headline claim depends entirely on the validity of the harmfulness judge, because the same judge is used both to guide the planner and to measure success; no human agreement or independent evaluation is reported. The paper's own Limitation section candidly acknowledges benchmark scope and dependence on external generators, but it does not address this evaluator feedback loop.

major comments (4)
  1. [Section 4.1, Eq. (2); Section 5.1; Appendix A.2] The central claim of 95.48% ASR is an in-sample measure. The optimization objective in Eq. (2) is the harmful judge J, and Section 4.1 states that HACA queries the target and stops when J labels the output an attack success. Appendix A.2 identifies J as GPT-4.1 with a threshold of 4, and Section 5.1 reports ASR using exactly this same judge. Thus the planner is optimized against the same criterion used for evaluation, and the reported ASR measures agreement with J rather than an independently established notion of harmfulness. The rubric's requirement that a response 'fully match the harmful intention' without an operational definition of actionable completion makes the threshold potentially permissive. Please report inter-annotator agreement between J and human raters on a sample of responses, include a second independent judge that was not used in the optimization loop, and provide examples of responses that received scores of 4 and 5 so readers can assess whether the criterion is too easy to satisfy. This is load-bearing for the headline result.
  2. [Section 5.1, Table 1] No variance, seeds, or statistical tests are reported. Each category contains only 50 queries, and many HACA cells are at 94-100% ASR, so the claims that HACA 'consistently achieves the strongest attack performance' and 'substantially outperforms' baselines are not supported with uncertainty quantification. For example, the difference between HACA's 100% and Ideator's 90% on InternVL-3.5 for several categories could be within sampling noise at n=50. Please provide confidence intervals or bootstrap estimates for the ASR means and a paired significance test across the 350 queries, or across repeated runs with different seeds, before drawing strong cross-model and cross-category conclusions.
  3. [Section A.4, Table 5 vs Section 4.2, Eq. (3)] There is an internal inconsistency about the enforced six-dimensional selection. Section 4.2 states that the planner selects exactly six atomic strategies, one from each of the three layers in each of the two modalities. Yet Table 5 shows text syntactic strategies 'underlying encoding shift encryption' and 'character isolation noise filling' with no selections, and the text states that some syntactic text strategies are 'not involved in any combination.' If every attack must include one text syntactic strategy, at least one text syntactic strategy must be selected in every combination. Additionally, the 'selected nums' column sums to 947 for text strategies and 947 for image strategies, which does not reconcile with 350 queries and an average of 1.90 queries per attack (about 665 total queries). Please clarify what the counts represent and correct the contradiction, or revise the description of the selection constraint.
  4. [Section 5.4, Table 4] The planner ablation is not persuasive as presented. Table 4 shows that random selection already achieves 94% ASR and the planner reaches 100%, while the no-planner baseline is 8%. The claimed benefit of the planner over random selection is modest and is reported without error bars or significance testing. Given that the planner is a core contribution, please report repeated runs, confidence intervals, and a comparison against random selection on more than one model or category, so the reader can judge whether the planner's advantage is reliable.
minor comments (6)
  1. [Section 4.2] The text says 'the prompt can be viewed in Appendix,' but the appendix contains only the toxic-score judge prompt; please provide the full planner prompt and the image-executor prompt, or state explicitly that they are withheld for safety reasons.
  2. [Section 5.3] There are typos: 'ACA achieves hightest success rates' should be 'HACA achieves highest success rates.'
  3. [Abstract] The phrase 'a atomic multi-modal strategy space' contains a grammatical error ('a' before 'atomic'); please correct.
  4. [Table 1] The model name 'LLaV A-1.6-Mistral' has inconsistent spacing, and the table header 'Toxic ASR' is ambiguous; clarify whether 'Toxic' refers to the toxic score and 'ASR' to the attack success rate.
  5. [Ethical Considerations] The statement that code and full prompts are withheld limits reproducibility, especially because Appendix A.2 provides the judge prompt; please clarify the exact disclosure policy and explain how the results can be independently reproduced without the withheld components.
  6. [References] Several references are dated 2026 and appear to be preprints (e.g., Azulay et al., Li et al. 2026a, Li et al. 2026b); please verify the versions and ensure the citations (including the apparent duplication of Li et al. 2024b and 2024c) are correct.

Circularity Check

2 steps flagged · score 6.0 of 10

The 95.48% ASR is the in-sample value of HACA's optimization objective (the GPT-4.1 toxic-score judge), making the headline claim self-referential; the judge prompt is a load-bearing self-citation that is withheld.

  1. self definitional [Section 4.1, Eq. (2); Section 5.1, Evaluation Metric; Appendix A.2]
    "arg max_{S*} J(M(X_T, X_I)) ... Here we apply GPT-4.1-2025-04-14 (OpenAI, 2023) as the judge model J. ... if the toxic score is no lower than the toxic threshold 4 ... can be applied to judge both the comprehension ability and safety ability for MLLMs."

    The attack success rate (ASR) is defined by thresholding the toxic-score judge J at 4, and Equation (2) sets the optimization objective as maximizing this same J. Therefore, the reported ASR is exactly the value of the objective that HACA is designed to maximize via query feedback. The 'prediction' of 95.48% is thus not an independent measure of harmfulness but an in-sample measure of how well HACA satisfies its own judge, reducing the result to the optimization target by construction.

  2. self citation load bearing [Appendix A.2]
    "Following (Zhao et al., 2025a), we apply the automatic identification prompt for toxic score evaluation of the MLLMs' responses ... The toxic score is obtained by the toxicity judge model ... if the toxic score is no lower than the toxic threshold 4 following (Zhao et al., 2025a)."

    The judge prompt, which drives both HACA's planning feedback and the final ASR measurement, is adopted from prior work by the same first author (Zhao et al., 2025a). The exact prompt is withheld (Ethical Considerations: 'we deliberately omit complete implementation code and detailed executable attack workflows'), so the central evaluation rests on an unverified self-citation. No human second judgment or inter-annotator agreement is reported to independently corroborate the toxicity scores.

full rationale

The paper's strategy taxonomy and the HACA planner/executor pipeline are not circular: the atomic strategies are extracted from the published literature, and the planner uses explicit complementarity heuristics rather than fitting to the test data. The core circularity is in the evaluation loop. Equation (2) defines the optimization objective as maximizing the judge function J, while Section 5.1 defines ASR and Toxic Score using the same GPT-4.1 judge J with a threshold of 4 (Appendix A.2). Thus the reported 95.48% ASR is the in-sample objective value, not an independent measurement of jailbreak capability; the evaluation metric is identical to the optimization target by construction. Additionally, the judge prompt is imported from the authors' own prior work (Zhao et al., 2025a) and the exact prompt is withheld, making this self-citation load-bearing for the central empirical claim. The comparison against baselines is fair in the sense that all methods use the same judge, but the absolute headline number does not provide independent evidence of harmfulness. Score 6 reflects that one or more reported predictions (the ASR) reduce by construction to the optimization target, while the rest of the method remains independently defined.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central claim rests on three assumptions: the layered cognitive model of MLLMs, the validity of the LLM judge, and the completeness of the hand-built 30-strategy taxonomy. No numeric parameters are fitted to data; the combination size of six and the taxonomy size of 30 are design choices.

free parameters (2)
  • combination size constraint (six strategies) = 6
    The planner always selects exactly one atomic strategy per layer per modality, giving six total. This number is imposed rather than searched over, and it directly shapes the attack structure.
  • strategy library size = 30
    The paper says 'we select 30 strategies in this paper' after LLM-assisted extraction. This count is a hand-chosen scope, not derived from external criteria.
assumptions (3)
  • domain assumption MLLMs process inputs through distinct structural, semantic, and syntactic layers, and cross-layer defenses are underdeveloped.
    Invoked in Section 3.2 to justify why layer-combination attacks should succeed. No direct measurement of internal layer processing is provided.
  • domain assumption GPT-4.1 toxic scores with a threshold of 4 are a valid measure of jailbreak success and harmfulness.
    Used in Section 5.1 and Appendix A.2 as the only success criterion; no human agreement study is reported.
  • ad hoc to paper The six-dimensional atomic strategy set is sufficiently complete to cover effective multimodal jailbreaks.
    The taxonomy is built from a selected set of 30 prior strategies; completeness is asserted rather than proven, and some listed strategies are never selected in Table 5.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Multimodal Automatic Redteaming Evaluation based on Atomic Jailbreak Strategy Decoupling and Combination." pith.science (2026). https://pith.science/paper/XWC6KSJ3

@misc{pith2026260804034,
  author       = {Pith},
  title        = {Pith review of: A Multimodal Automatic Redteaming Evaluation based on Atomic Jailbreak Strategy Decoupling and Combination},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XWC6KSJ3}},
  note         = {Machine review of arXiv:2608.04034}
}
read the original abstract

Multimodal Large Language Models (MLLMs) have achieved impressive progress in image-text comprehension and generation, yet they remain susceptible to jailbreak attacks that can trigger harmful outputs and pose serious safety concerns. Existing multimodal jailbreak attacks have shown the feasibility of such attacks, but they still face two fundamental challenges: the lack of a atomic multi-modal strategy space, the absence of a concise and efficient executable framework beyond human-craft experience. To address these challenges, we first decompose the text-image jailbreak strategy space into three levels: structural, semantic, and syntactic, constructing a jailbreak strategy set encompassing both text and image modalities to systematically achieve combined coverage of different attack types. Then we propose a multimodal automated red team jailbreak method named Hierarchical Atomic Combination Attack (HACA). Specifically, based on a six-dimensional strategy space, a cross-modal joint planner is used to select and combine the different atomic jailbreak strategy for subsequent jailbreak command generation. Finally, at the implementation level, we explore to apply a unified generate executor to directly generate jailbreak instructions based on the selected multi-modal strategies. A series of experiments show that our automated red team method can achieve an attack success rate of average 95.48\% against five mainstream MLLMs.

Figures

Figures reproduced from arXiv: 2608.04034 by the authors.

Figure 1
Figure 1. The pipeline of HACA. We decompose the existing jailbreak strategies to obtain the atomic taxonomy. And then we selected the strategies by cross-modal joint planner and uniformly generated jailbreak instruction. the optimization towards the selection and combi￾nation of multimodal jailbreaking strategies, this paper formulates this problem as a discrete con￾strained optimization problem. Its core goal is to search f… view at source ↗
Figure 2
Figure 2. The query num and ASR for two MLLMs. Our HACA achieves highest ASR with least query num. making it the most effective attack among all com￾pared methods. 5.3 Attack Overhead We compare the query efficiency of HACA with mainstream baseline methods. As shown in [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. The GPT-5-2025-0807 responses (Case1) for the harmful instructions optimized by HACA. [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Case2: The GPT-5-2025-0807 responses (Case2) for the harmful instructions optimized by HACA. [PITH_FULL_IMAGE:figures/full_fig_p015_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

135 extracted references · 17 canonical work pages

  1. [1]

    arXiv preprint arXiv:2307.01952 , year=

    Sdxl: Improving latent diffusion models for high-resolution image synthesis , author=. arXiv preprint arXiv:2307.01952 , year=

    arXiv

  2. [3]

    2024 IEEE Security and Privacy Workshops (SPW) , pages=

    Exploiting programmatic behavior of llms: Dual-use through standard security attacks , author=. 2024 IEEE Security and Privacy Workshops (SPW) , pages=. 2024 , organization=

    2024

  3. [4]

    International conference on machine learning , pages=

    Learning transferable visual models from natural language supervision , author=. International conference on machine learning , pages=. 2021 , organization=

    2021

  4. [5]

    arXiv preprint arXiv:2507.21540 , year =

    PRISM: Programmatic Reasoning with Image Sequence Manipulation for LVLM Jailbreaking , author =. arXiv preprint arXiv:2507.21540 , year =

    arXiv

  5. [6]

    Proceedings of the IEEE/CVF International Conference on Computer Vision , year =

    Heuristic-Induced Multimodal Risk Distribution Jailbreak Attack for Multimodal Large Language Models , author =. Proceedings of the IEEE/CVF International Conference on Computer Vision , year =

  6. [7]

    2024 , eprint=

    DeepSeek-VL: Towards Real-World Vision-Language Understanding , author=. 2024 , eprint=

    2024

  7. [9]

    arXiv preprint arXiv:2411.11496 , year=

    Safe+ safe= unsafe? exploring how safe images can be exploited to jailbreak large vision-language models , author=. arXiv preprint arXiv:2411.11496 , year=

    arXiv

  8. [11]

    arXiv preprint arXiv:2509.01444 , year=

    Strata-Sword: A Hierarchical Safety Evaluation towards LLMs based on Reasoning Complexity of Jailbreak Instructions , author=. arXiv preprint arXiv:2509.01444 , year=

    arXiv

Show all 135 references
  1. [12]

    arXiv preprint arXiv:2412.16720 , year=

    Openai o1 system card , author=. arXiv preprint arXiv:2412.16720 , year=

    arXiv

  2. [13]

    arXiv preprint arXiv:2510.18234 , year=

    DeepSeek-OCR: Contexts Optical Compression , author=. arXiv preprint arXiv:2510.18234 , year=

    arXiv

  3. [14]

    arXiv preprint arXiv:2505.21277 , year=

    Breaking the Ceiling: Exploring the Potential of Jailbreak Attacks through Expanding Strategy Space , author=. arXiv preprint arXiv:2505.21277 , year=

    arXiv

  4. [15]

    2023 , url =

    OpenAI , title =. 2023 , url =

    2023

  5. [16]

    arXiv preprint arXiv:2502.11054 , year=

    Reasoning-augmented conversation for multi-turn jailbreak attacks on large language models , author=. arXiv preprint arXiv:2502.11054 , year=

    arXiv

  6. [17]

    Introducing Claude Sonnet 4.5 , year =

  7. [18]

    arXiv preprint arXiv:2412.10302 , year=

    DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding , author=. arXiv preprint arXiv:2412.10302 , year=

    arXiv

  8. [19]

    arXiv preprint arXiv:2402.09177 , year=

    Leveraging the context through multi-round interactions for jailbreaking attacks , author=. arXiv preprint arXiv:2402.09177 , year=

    arXiv

  9. [20]

    arXiv preprint arXiv:2402.17262 , year=

    Speak out of turn: Safety vulnerability of large language models in multi-turn dialogue , author=. arXiv preprint arXiv:2402.17262 , year=

    arXiv

  10. [21]

    arXiv preprint , year =

    Hugo Touvron and Louis Martin and Kevin Stone and Peter Albert and Amjad Almahairi and Yasmine Babaei and Nikolay Bashlykov and Soumya Batra and Prajjwal Bhargava and Shruti Bhosale and others , title =. arXiv preprint , year =

  11. [22]

    arXiv preprint , year =

    Xianjun Yang and Xiao Wang and Qi Zhang and Linda Petzold and William Yang Wang and Xun Zhao and Dahua Lin , title =. arXiv preprint , year =

  12. [23]

    arXiv preprint arXiv:2310.08419 , year=

    Jailbreaking black box large language models in twenty queries , author=. arXiv preprint arXiv:2310.08419 , year=

    arXiv

  13. [29]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Visual adversarial examples jailbreak aligned large language models , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  14. [31]

    Advances in Neural Information Processing Systems , volume=

    On evaluating adversarial robustness of large vision-language models , author=. Advances in Neural Information Processing Systems , volume=

  15. [32]

    The Twelfth International Conference on Learning Representations , year=

    Jailbreak in pieces: Compositional adversarial attacks on multi-modal language models , author=. The Twelfth International Conference on Learning Representations , year=

  16. [34]

    arXiv preprint arXiv:2303.08774 , year=

    Gpt-4 technical report , author=. arXiv preprint arXiv:2303.08774 , year=

    arXiv

  17. [35]

    https://openai.com/index/hello-gpt-4o/ , year=

    Hello gpt-4o , author=. https://openai.com/index/hello-gpt-4o/ , year=

  18. [36]

    https://chatgpt.com/ , year=

    ChatGPT , author=. https://chatgpt.com/ , year=

  19. [37]

    https://gemini.google.com/app/ , year=

    Gemini , author=. https://gemini.google.com/app/ , year=

  20. [38]

    https://claude.ai/ , year=

    The claude 3 model family: Opus, sonnet, haiku , author=. https://claude.ai/ , year=

  21. [39]

    https://www.perspectiveapi.com/ , year=

    PerspectiveAPI , author=. https://www.perspectiveapi.com/ , year=

  22. [40]

    https://platform.openai.com/ , year=

    ModerationAPI , author=. https://platform.openai.com/ , year=

  23. [41]

    Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Models , journal =

    Yifan Li and Hangyu Guo and Kun Zhou and Wayne Xin Zhao and Ji. Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Models , journal =

  24. [44]

    arXiv preprint arXiv:2312.10766 , year=

    A mutation-based method for multi-modal jailbreaking attack detection , author=. arXiv preprint arXiv:2312.10766 , year=

    arXiv

  25. [45]

    arXiv preprint arXiv:2407.21659 , year=

    Defending jailbreak attack in vlms via cross-modality information detector , author=. arXiv preprint arXiv:2407.21659 , year=

    arXiv

  26. [46]

    arXiv preprint arXiv:2403.09037 , year=

    The First to Know: How Token Distributions Reveal Hidden Knowledge in Large Vision-Language Models? , author=. arXiv preprint arXiv:2403.09037 , year=

    arXiv

  27. [48]

    arXiv preprint arXiv:2406.02575 , year=

    Cross-Modal Safety Alignment: Is textual unlearning all you need? , author=. arXiv preprint arXiv:2406.02575 , year=

  28. [49]

    arXiv preprint arXiv:2312.06674 , year=

    Llama guard: Llm-based input-output safeguard for human-ai conversations , author=. arXiv preprint arXiv:2312.06674 , year=

    arXiv

  29. [50]

    arXiv preprint arXiv:2405.07788 , year=

    DEPTH: Discourse Education through Pre-Training Hierarchically , author=. arXiv preprint arXiv:2405.07788 , year=

  30. [51]

    arXiv preprint arXiv:2406.09967 , year=

    Bag of Lies: Robustness in Continuous Pre-training BERT , author=. arXiv preprint arXiv:2406.09967 , year=

    arXiv

  31. [53]

    How effective is BERT without word ordering? implications for language understanding and data privacy , author=. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (...

  32. [54]

    arXiv preprint arXiv:2402.19464 , year=

    Curiosity-driven red-teaming for large language models , author=. arXiv preprint arXiv:2402.19464 , year=

    arXiv

  33. [55]

    arXiv preprint arXiv:1908.10084 , year=

    Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks , author=. arXiv preprint arXiv:1908.10084 , year=

    1908 arXiv

  34. [56]

    arXiv preprint arXiv:2308.12966 , year=

    Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond , author=. arXiv preprint arXiv:2308.12966 , year=

    arXiv

  35. [57]

    arXiv preprint arXiv:2210.01936 , year=

    When and why vision-language models behave like bags-of-words, and what to do about it? , author=. arXiv preprint arXiv:2210.01936 , year=

    arXiv

  36. [59]

    arXiv preprint arXiv:2304.10592 , year=

    MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models , author=. arXiv preprint arXiv:2304.10592 , year=

    arXiv

  37. [61]

    arXiv preprint arXiv:2404.16821 , year=

    How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites , author=. arXiv preprint arXiv:2404.16821 , year=

    arXiv

  38. [62]

    arXiv preprint arXiv:2406.06302 , year=

    Unveiling the Safety of GPT-4o: An Empirical Study using Jailbreak Attacks , author=. arXiv preprint arXiv:2406.06302 , year=

    arXiv

  39. [63]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    High-resolution image synthesis with latent diffusion models , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  40. [65]

    arXiv preprint arXiv:2310.04451 , year=

    Autodan: Generating stealthy jailbreak prompts on aligned large language models , author=. arXiv preprint arXiv:2310.04451 , year=

    arXiv

  41. [66]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Mma-diffusion: Multimodal attack on diffusion models , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  42. [67]

    arXiv preprint arXiv:2210.04610 , year=

    Red-teaming the stable diffusion safety filter , author=. arXiv preprint arXiv:2210.04610 , year=

    arXiv

  43. [68]

    Forty-first International Conference on Machine Learning , year=

    On prompt-driven safeguarding for large language models , author=. Forty-first International Conference on Machine Learning , year=

  44. [70]

    arXiv preprint arXiv:2310.06474 , year=

    Multilingual jailbreak challenges in large language models , author=. arXiv preprint arXiv:2310.06474 , year=

    arXiv

  45. [71]

    arXiv preprint arXiv:2303.16634 , year=

    G-eval: Nlg evaluation using gpt-4 with better human alignment , author=. arXiv preprint arXiv:2303.16634 , year=

    arXiv

  46. [72]

    2024 IEEE symposium on security and privacy (SP) , pages=

    Sneakyprompt: Jailbreaking text-to-image generative models , author=. 2024 IEEE symposium on security and privacy (SP) , pages=. 2024 , organization=

    2024

  47. [73]

    Advances in Neural Information Processing Systems , volume=

    Jailbroken: How does llm safety training fail? , author=. Advances in Neural Information Processing Systems , volume=

  48. [74]

    arXiv preprint:1706.06083 , year=

    Towards deep learning models resistant to adversarial attacks , author=. arXiv preprint:1706.06083 , year=

    arXiv

  49. [75]

    arXiv preprint arXiv:2309.00614 , year=

    Baseline defenses for adversarial attacks against aligned language models , author=. arXiv preprint arXiv:2309.00614 , year=

    arXiv

  50. [76]

    arXiv preprint arXiv:2308.14132 , year=

    Detecting language model attacks with perplexity , author=. arXiv preprint arXiv:2308.14132 , year=

    arXiv

  51. [77]

    arXiv preprint arXiv:2407.21783 , year=

    The llama 3 herd of models , author=. arXiv preprint arXiv:2407.21783 , year=

    arXiv

  52. [78]

    IEEE transactions on pattern analysis and machine intelligence , volume=

    Faster R-CNN: Towards real-time object detection with region proposal networks , author=. IEEE transactions on pattern analysis and machine intelligence , volume=. 2016 , publisher=

    2016

  53. [79]

    2023 , url =

    Ben Wang and Aran Komatsuzaki and others , title =. 2023 , url =

    2023

  54. [80]

    arXiv preprint arXiv:2302.13971 , year =

    Hugo Touvron and Thibaut Lavril and Gautier Izacard and Xavier Martinet and Marie-Anne Lachaux and Timothée Lacroix and Baptiste Rozière and Naman Goyal and Eric Hambro and Faisal Azhar and Aurélien Rodriguez and Armand Joulin and Edouard Grave and Guillaume Lample , title =. ...

    arXiv

  55. [81]

    2025 , url =

    QwenLM Team , title =. 2025 , url =

    2025

  56. [82]

    2023 , url =

    AQ Jiang and others , title =. 2023 , url =

    2023

  57. [83]

    arXiv preprint arXiv:2509.00391 , year =

    Yuting Tan and Xuying Li and Zhuo Li and Huizhen Shu and Peikang Hu , title =. arXiv preprint arXiv:2509.00391 , year =

    arXiv

  58. [84]

    Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , pages=

    Red teaming language models with language models , author=. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , pages=

    2022

  59. [87]

    Proceedings of the 32nd ACM International Conference on Multimedia , pages=

    Arondight: Red teaming large vision language models with auto-generated multi-modal jailbreak prompts , author=. Proceedings of the 32nd ACM International Conference on Multimedia , pages=

  60. [92]

    Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

    Ideator: Jailbreaking and benchmarking large vision-language models using themselves , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

  61. [93]

    FLUX.1-dev: Foundation Model for Image Generation , author =

  62. [94]

    System Card: Claude Opus 4.6 , year =

  63. [95]

    34th USENIX Security Symposium (USENIX Security 25) , pages=

    Great, now write an article about that: The crescendo \ Multi-Turn \ \ LLM \ jailbreak attack , author=. 34th USENIX Security Symposium (USENIX Security 25) , pages=

  64. [96]

    Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    Queryattack: Jailbreaking aligned large language models using structured non-natural query language , author=. Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    2025

  65. [97]

    arXiv preprint arXiv:2502.12893 , year=

    H-cot: Hijacking the chain-of-thought safety reasoning mechanism to jailbreak large reasoning models, including openai o1/o3, deepseek-r1, and gemini 2.0 flash thinking , author=. arXiv preprint arXiv:2502.12893 , year=

    arXiv

  66. [98]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Jailbreaking? One Step Is Enough! , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  67. [99]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    from benign import toxic: Jailbreaking the language model via adversarial metaphors , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  68. [105]

    International conference on learning representations , volume=

    Jailbreak in pieces: Compositional adversarial attacks on multi-modal language models , author=. International conference on learning representations , volume=

  69. [106]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    TOMBRAIDER: Entering the Vault of History to Jailbreak Large Language Models , author=. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    2025

  70. [109]

    Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024) , pages=

    Tricking llms into disobedience: Formalizing, analyzing, and detecting jailbreaks , author=. Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024) , pages=

    2024

  71. [110]

    Forty-second International Conference on Machine Learning , year=

    Textural or textual: How vision-language models read text in images , author=. Forty-second International Conference on Machine Learning , year=

  72. [112]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    Playing the fool: Jailbreaking llms and multimodal llms with out-of-distribution strategy , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

  73. [113]

    Maksym Andriushchenko and Nicolas Flammarion. 2024. Does refusal training in llms generalize to the past tense?, 2024. URL https://arxiv. org/abs/2407.11969

    2024 arXiv

  74. [114]

    Aharon Azulay, Jan Dubi \'n ski, Zhuoyun Li, Atharv Mittal, and Yossi Gandelsman. 2026. Jailbreaking vision-language models through the visual modality. arXiv preprint arXiv:2605.00583

    2026 arXiv

  75. [115]

    Saketh Bachu, Erfan Shayegani, Rohit Lal, Trishna Chakraborty, Arindam Dutta, Chengyu Song, Yue Dong, Nael Abu-Ghazaleh, and Amit K Roy-Chowdhury. 2024. Layer-wise alignment: Examining safety alignment across image encoder layers in vision language models. arXiv preprint arXiv...

    2024 arXiv

  76. [116]

    Luke Bailey, Euan Ong, Stuart Russell, and Scott Emmons. 2023. Image hijacks: Adversarial images can control generative models at runtime. arXiv preprint arXiv:2309.00236

    2023 arXiv

  77. [117]

    Black Forest Labs . 2024. Flux.1-dev: Foundation model for image generation. https://blackforestlabs.ai/

    2024

  78. [118]

    Zhaorun Chen, Xun Liu, Mintong Kang, Jiawei Zhang, Minzhou Pan, Shuang Yang, and Bo Li. 2025. Arms: Adaptive red-teaming agent against multimodal models with plug-and-play attacks. arXiv preprint arXiv:2510.02677

    2025

  79. [119]

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, Bin Li, Ping Luo, Tong Lu, Yu Qiao, and Jifeng Dai. 2023. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. arXiv ...

    2023 arXiv

  80. [120]

    Junchen Ding, Jiahao Zhang, Yi Liu, Ziqi Ding, Gelei Deng, and Yuekang Li. 2025. Tombraider: Entering the vault of history to jailbreak large language models. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 5478--5493

    2025

  81. [121]

    Yichen Gong, Delong Ran, Jinyuan Liu, Conglei Wang, Tianshuo Cong, Anyu Wang, Sisi Duan, and Xiaoyun Wang. 2023. Figstep: Jailbreaking large vision-language models via typographic visual prompts. arXiv preprint arXiv:2311.05608

    2023 arXiv

  82. [122]

    Google. 2024. Gemini. https://gemini.google.com/app/

    2024

  83. [123]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948

    2025 arXiv

  84. [124]

    Joonhyun Jeong, Seyun Bae, Yeonsung Jung, Jaeryong Hwang, and Eunho Yang. 2025. Playing the fool: Jailbreaking llms and multimodal llms with out-of-distribution strategy. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 29937--29946

    2025

  85. [125]

    Daniel Kang, Xuechen Li, Ion Stoica, Carlos Guestrin, Matei Zaharia, and Tatsunori Hashimoto. 2024. Exploiting programmatic behavior of llms: Dual-use through standard security attacks. In 2024 IEEE Security and Privacy Workshops (SPW), pages 132--143. IEEE

    2024

  86. [126]

    Chenxi Li, Xianggan Liu, Dake Shen, Yaosong Du, Zhibo Yao, Hao Jiang, Linyi Jiang, Chengwei Cao, Jingzhe Zhang, RanYi Peng, and 1 others. 2026 a . Models as lego builders: Assembling malice from benign blocks via semantic blueprints. arXiv preprint arXiv:2603.07590

    2026

  87. [127]

    Chunxiao Li, Lijun Li, and Jing Shao. 2026 b . Treeteaming: Autonomous red-teaming of vision-language models via hierarchical strategy exploration. arXiv preprint arXiv:2603.22882

    2026

  88. [128]

    Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang, Bo Li, Wei Li, Zejun Ma, and Chunyuan Li. 2024 a . Llava-next-interleave: Tackling multi-image, video, and 3d in large multimodal models. arXiv preprint arXiv:2407.07895

    2024 arXiv

  89. [129]

    Yifan Li, Hangyu Guo, Kun Zhou, Wayne Xin Zhao, and Ji - Rong Wen. 2024 b . Images are achilles' heel of alignment: Exploiting visual vulnerabilities for jailbreaking multimodal large language models. ECCV

    2024

  90. [130]

    Yifan Li, Hangyu Guo, Kun Zhou, Wayne Xin Zhao, and Ji-Rong Wen. 2024 c . Images are achilles' heel of alignment: Exploiting visual vulnerabilities for jailbreaking multimodal large language models. arXiv preprint arXiv:2403.09792

    2024 arXiv

  91. [131]

    Mingrui Liu, Sixiao Zhang, Cheng Long, and Kwok Yan Lam. 2025. The trojan example: Jailbreaking llms through template filling and unsafety reasoning. arXiv preprint arXiv:2510.21190

    2025

  92. [132]

    X Liu, Y Zhu, J Gu, Y Lan, C Yang, and Y Qiao. 2023. Mm-safetybench: A benchmark for safety evaluation of multimodal large language models. arXiv preprint arXiv:2311.17600

    2023 arXiv

  93. [133]

    Xiaogeng Liu, Peiran Li, Edward Suh, Yevgeniy Vorobeychik, Zhuoqing Mao, Somesh Jha, Patrick McDaniel, Huan Sun, Bo Li, and Chaowei Xiao. 2024 a . Autodan-turbo: A lifelong agent for strategy self-exploration to jailbreak llms. arXiv preprint arXiv:2410.05295

    2024 arXiv

  94. [134]

    Yi Liu, Chengjun Cai, Xiaoli Zhang, Xingliang Yuan, and Cong Wang. 2024 b . Arondight: Red teaming large vision language models with auto-generated multi-modal jailbreak prompts. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 3578--3586

    2024

  95. [135]

    Teng Ma, Xiaojun Jia, Ranjie Duan, Xinfeng Li, Yihao Huang, Xiaoshuang Jia, Zhixuan Chu, and Wenqi Ren. 2025. Heuristic-induced multimodal risk distribution jailbreak attack for multimodal large language models. In Proceedings of the IEEE/CVF International Conference on Comput...

    2025

  96. [136]

    Zhenxing Niu, Haodong Ren, Xinbo Gao, Gang Hua, and Rong Jin. 2024. Jailbreaking attack against multimodal large language model. arXiv preprint arXiv:2402.02309

    2024 arXiv

  97. [137]

    OpenAI. 2023. Chatgpt. https://chatgpt.com/

    2023

  98. [138]

    OpenAI. 2024. Hello gpt-4o. https://openai.com/index/hello-gpt-4o/

    2024

  99. [139]

    OpenAI . 2025. Gpt-5

    2025

  100. [140]

    Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, and Geoffrey Irving. 2022. Red teaming language models with language models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing...

    2022

  101. [141]

    Xiangyu Qi, Kaixuan Huang, Ashwinee Panda, Peter Henderson, Mengdi Wang, and Prateek Mittal. 2024. Visual adversarial examples jailbreak aligned large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 21527--21536

    2024

  102. [142]

    Salman Rahman, Liwei Jiang, James Shiffer, Genglin Liu, Sheriff Issaka, Md Rizwan Parvez, Hamid Palangi, Kai-Wei Chang, Yejin Choi, and Saadia Gabriel. 2025. X-teaming: Multi-turn jailbreaks and defenses with adaptive multi-agents. arXiv preprint arXiv:2504.13203

    2025 arXiv

  103. [143]

    Abhinav Sukumar Rao, Atharva Roshan Naik, Sachin Vashistha, Somak Aditya, and Monojit Choudhury. 2024. Tricking llms into disobedience: Formalizing, analyzing, and detecting jailbreaks. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Lan...

    2024

  104. [144]

    Mark Russinovich, Ahmed Salem, and Ronen Eldan. 2025. Great, now write an article about that: The crescendo \ Multi-Turn \ \ LLM \ jailbreak attack. In 34th USENIX Security Symposium (USENIX Security 25), pages 2421--2440

    2025

  105. [145]

    Ephraiem Sarabamoun. 2025. Special-character adversarial attacks on open-source language model. arXiv preprint arXiv:2508.14070

    2025

  106. [146]

    Erfan Shayegani, Yue Dong, and Nael Abu-Ghazaleh. 2023. Jailbreak in pieces: Compositional adversarial attacks on multi-modal language models. In The Twelfth International Conference on Learning Representations

    2023

  107. [147]

    Erfan Shayegani, Yue Dong, and Nael Abu-Ghazaleh. 2024. Jailbreak in pieces: Compositional adversarial attacks on multi-modal language models. In International conference on learning representations, volume 2024, pages 30853--30885

    2024

  108. [148]

    do anything now

    Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, and Yang Zhang. 2023. " do anything now": Characterizing and evaluating in-the-wild jailbreak prompts on large language models. arXiv preprint arXiv:2308.03825

    2023 arXiv

  109. [149]

    Ma Teng, Jia Xiaojun, Duan Ranjie, Li Xinfeng, Huang Yihao, Jia Xiaoshuang, Chu Zhixuan, and Ren Wenqi. 2024. Heuristic-induced multimodal risk distribution jailbreak attack for multimodal large language models. arXiv preprint arXiv:2412.05934

    2024 arXiv

  110. [150]

    Fengxiang Wang, Ranjie Duan, Peng Xiao, Xiaojun Jia, Shiji Zhao, Cheng Wei, YueFeng Chen, Chongwen Wang, Jialing Tao, Hang Su, and 1 others. 2024. Mrj-agent: An effective jailbreak agent for multi-round dialogue. arXiv preprint arXiv:2411.03814

    2024 arXiv

  111. [151]

    Hanzhang Wang and Qingyuan Ma. 2025. Textural or textual: How vision-language models read text in images. In Forty-second International Conference on Machine Learning

    2025

  112. [152]

    Ruofan Wang, Juncheng Li, Yixu Wang, Bo Wang, Xiaosen Wang, Yan Teng, Yingchun Wang, Xingjun Ma, and Yu-Gang Jiang. 2025. Ideator: Jailbreaking and benchmarking large vision-language models using themselves. In Proceedings of the IEEE/CVF International Conference on Computer V...

    2025

  113. [153]

    Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. 2023. Jailbroken: How does llm safety training fail? Advances in Neural Information Processing Systems, 36:80079--80110

    2023

  114. [154]

    Fenghua Weng, Yue Xu, Chengyan Fu, and Wenjie Wang. 2024. Mmj-bench: A comprehensive study on jailbreak attacks and defenses for vision language models. arXiv preprint arXiv:2408.08464

    2024 arXiv

  115. [155]

    Yuan Xiong, Ziqi Miao, Lijun Li, Chen Qian, Jie Li, and Jing Shao. 2025. Contextual image attack: How visual context exposes multimodal safety vulnerabilities. arXiv preprint arXiv:2512.02973

    2025

  116. [156]

    Yu Yan, Sheng Sun, Zenghao Duan, Teli Liu, Min Liu, Zhiyi Yin, LeiJingyu LeiJingyu, and Qi Li. 2025. from benign import toxic: Jailbreaking the language model via adversarial metaphors. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics ...

    2025

  117. [157]

    Yi Zeng, Hongpeng Lin, Jingwen Zhang, Diyi Yang, Ruoxi Jia, and Weiyan Shi. 2024. How johnny can persuade llms to jailbreak them: Rethinking persuasion to challenge ai safety by humanizing llms. arXiv preprint arXiv:2401.06373

    2024 arXiv

  118. [158]

    Yichi Zhang, Yao Huang, Yitong Sun, Chang Liu, Zhe Zhao, Zhengwei Fang, Yifan Wang, Huanran Chen, Xiao Yang, Xingxing Wei, and 1 others. 2024. Benchmarking trustworthiness of multimodal large language models: A comprehensive study. arXiv preprint arXiv:2406.07057

    2024 arXiv

  119. [159]

    Ziyi Zhang, Zhen Sun, Zongmin Zhang, Jihui Guo, and Xinlei He. 2025. Fc-attack: Jailbreaking multimodal large language models via auto-generated flowcharts. arXiv preprint ArXiv:2502.21059

    2025

  120. [160]

    Shiji Zhao, Ranjie Duan, Fengxiang Wang, Chi Chen, Caixin Kang, Shouwei Ruan, Jialing Tao, YueFeng Chen, Hui Xue, and Xingxing Wei. 2025 a . Jailbreaking multimodal large language models via shuffle inconsistency. arXiv preprint arXiv:2501.04931

    2025 arXiv

  121. [161]

    Shiji Zhao, Shukun Xiong, Yao Huang, Yan Jin, Zhenyu Wu, Jiyang Guan, Ranjie Duan, Jialing Tao, Hui Xue, and Xingxing Wei. 2025 b . Vrsa: Jailbreaking multimodal large language models through visual reasoning sequential attack. arXiv preprint arXiv:2512.05853

    2025

  122. [162]

    Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang, Chongxuan Li, Ngai-Man Man Cheung, and Min Lin. 2024. On evaluating adversarial robustness of large vision-language models. Advances in Neural Information Processing Systems, 36

    2024

  123. [163]

    Weixiong Zheng, Peijian Zeng, Yiwei Li, Hongyan Wu, Nankai Lin, Junhao Chen, Aimin Yang, and Yongmei Zhou. 2025. Jailbreaking? one step is enough! In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11623--11642

    2025

  124. [164]

    Andy Zhou, Kevin Wu, Francesco Pinto, Zhaorun Chen, Yi Zeng, Yu Yang, Shuang Yang, Sanmi Koyejo, James Zou, and Bo Li. 2025. Autoredteamer: Autonomous red teaming with lifelong attack integration. arXiv preprint arXiv:2503.15754

    2025 arXiv

  125. [165]

    Yongshuo Zong, Ondrej Bohdal, Tingyang Yu, Yongxin Yang, and Timothy Hospedales. 2024. Safety fine-tuning at (almost) no cost: A baseline for vision large language models. arXiv preprint arXiv:2402.02207

    2024 arXiv

  126. [166]

    Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson. 2023. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043

    2023 arXiv

  127. [167]

    Qingsong Zou, Jingyu Xiao, Qing Li, Zhi Yan, Yuhang Wang, Li Xu, Wenxuan Wang, Kuofeng Gao, Ruoyu Li, and Yong Jiang. 2025. Queryattack: Jailbreaking aligned large language models using structured non-natural query language. In Findings of the Association for Computational Lin...

    2025

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.