Pith. sign in

REVIEW 3 major objections 4 minor 45 references

Watch, Listen, Understand, Mislead: Tri-modal Adversarial Attacks on Short Videos for Content Appropriateness Evaluation

T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A paper claims that short-video MLLM moderation can be flipped by coordinated adversarial descriptions of video, audio, and meaning, introducing SVMA and ChimeraBreak.

desk verdict Real new dataset and a clever prompt-based attack, but the two-step protocol in Eq. 6 means the 90%+ ASR mostly measures self-consistency, not a genuine tri-modal vulnerability. read the letter →

arxiv 2507.11968 v1 pith:R26JHEFS submitted 2025-07-16 cs.CV

Sahid Hossain Mustakim , S M Jishanul Islam , Ummay Maria Muna , Montasir Chowdhury , Mohammed Jawwadul Islam , Sadia Ahmmed , Tashfia Sikder , Syed Tasdid Azam Dhrubo
show 1 more author
Swakkhar Shatabda
This is my paper
classification cs.CV
keywords adversarialattackmultimodallargelanguagemodelsshort-formvideocontentmoderationjailbreaktri-modalLLM-as-a-judgesafetyevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that short-video content moderation by multimodal large language models can be defeated by coordinated adversarial descriptions of what a video shows, what its audio says, and what the content means. It introduces a dataset (SVMA) of 1,009 YouTube short videos with human-refined adversarial prompts, and an attack pipeline (ChimeraBreak) that leaves the original video and audio streams untouched. The paper reports attack success rates above 90% on nearly all evaluated models, so benign content is often mislabeled as harmful while harmful content is often accepted as appropriate. The authors also recommend the two-step reasoning-then-classification format as a general evaluation strategy for adversarial probing of MLLMs.

What carries the argument

The load-bearing machinery is ChimeraBreak's staged prompt structure. First, five keyframes plus audio transcription and descriptions are shown alongside adversarial prompts $\mathbf{adv}_i = (\mathrm{adv}^v_i, \mathrm{adv}^a_i, \mathrm{adv}^p_i)$ targeting video, audio, and perception reasoning; the model is primed to accept that internet content can be misleading and is asked to produce a rationale $R'_i$. Second, with no frames or audio, the rationale alone is re-prompted to produce the label $y'_i$, forming the chain $y'_i \leftarrow \mathcal{M}(R'_i) \leftarrow R'_i \leftarrow \mathcal{M}(c_i, \mathbf{adv}_i)$. The SVMA dataset supplies the adversarial prompts, with scene-level video and audio attacks plus a holistic perception attack, originally generated by a Gemini model and manually refined by the authors.

What would settle it

Re-run the attack on the SVMA samples with Stage 2 removed: ask each model for the label directly in the same prompt that contains the frames, audio transcription, and adversarial descriptions, instead of first generating a rationale and then classifying from it; if the one-step attack success rate drops substantially while the two-step labels are predictable from the rationales, then the reported vulnerability is an artifact of the model agreeing with its own earlier narrative.

Watch

Extended reading notes

Core claim

The central claim is that current MLLMs can be manipulated by narratives rather than by pixel or waveform perturbations. By injecting contextually aligned but semantically misleading descriptions across three channels, video, audio, and perception, an attacker shifts the model's generated rationale $R'_i$ and, through that rationale, its binary appropriateness label $y'_i$, with attack success rates above 90% on almost all tested models including GPT-4o mini, LLaMA 4 variants, Qwen2.5-VL variants, Gemma 3 variants, and LLaVA variants. The paper reports asymmetric failure modes: some models are more easily convinced to flag benign content as harmful, while others are more easily convinced to approve genuinely harmful content. It also reports that the attacks degrade ethical reasoning dimensions, especially awareness of vulnerable populations, and that models stay highly confident even when wrong.

Load-bearing premise

The load-bearing premise is that making the model classify content from its own generated rationale isolates true multimodal understanding; if the final label simply follows the already-manipulated narrative, then the high attack success rates measure self-consistency rather than a genuine tri-modal vulnerability.

Editorial extensions

If this is right

  • If ChimeraBreak's reported attack success rates hold, current MLLM-based moderation pipelines can be flipped by narrative override without altering the uploaded video or audio.
  • Tri-modal attacks surpass every single-modality variant in consistency across the baseline models, so safety evaluations that test only one modality at a time will underestimate real-world risk.
  • The two-step rationale-then-classify format gives evaluators a reusable protocol for checking whether a model's label follows from its own reasoning rather than from an inconsistent shortcut.
  • The content-type asymmetry shows that defenses must be calibrated per model: some systems need protection against false flags on benign content, while others need protection against approving harmful content.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: because ChimeraBreak never modifies the raw video or audio stream, its measured success may mostly reflect how much weight current foundation models give to text guidance over sensory evidence; a direct test would compare labels produced from raw frames and audio with labels produced from the model's own rationale.
  • Editorial inference: the same narrative-flip mechanism likely transfers to longer videos and to moderation pipelines that first generate a text summary and then classify that summary, so the risk may extend beyond short-form reels.
  • Editorial inference: a defender could add a grounding check that forces the model to point at the exact frames or audio segments supporting its label; ChimeraBreak's success suggests such grounding checks, rather than simply larger models, are the critical mitigation to test.
  • Editorial inference: the hallucination factor and confidence ratings come from an LLM judge, which shares the same class of failure modes as the attacked models; having independent human annotators rescore a sample of rationales would show whether the reported confusion is genuine or a style artifact.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces SVMA, a 1,009-video dataset of short-form videos with human- and machine-generated adversarial descriptions, and ChimeraBreak, a two-stage attack that first asks an MLLM to generate a rationale under adversarial text prompts describing video, audio, and perception, and then asks the same model to classify the video from that rationale alone. The authors report high Attack Success Rates (mostly above 90%) across ten open and closed MLLMs, content-type asymmetries, and LLM-as-a-judge scores for ethical reasoning and hallucination resistance. The claimed contribution is a coordinated tri-modal vulnerability in current MLLMs used for content appropriateness evaluation.

Significance. If the reported ASR is a genuine measure of multimodal classification robustness, the paper would be a useful benchmark contribution: it is the first short-video adversarial dataset of its kind, it uses unmodified source videos, it evaluates ten models including large open and closed variants, and it releases code and data. The content-type asymmetry findings and the observation that larger variants are often more resistant are interesting and falsifiable. However, the central claim rests on the two-step decision procedure, and the current evaluation does not separate the attack's ability to flip the model's narrative from the model's willingness to label consistently with that narrative. With the requested ablations, the contribution can be made solid; without them, the headline 'tri-modal vulnerability' is not supported.

major comments (3)
  1. [Section 4.2, Eq. (6)] The final label is produced by M(R'_i), that is, by feeding the model only its own previously generated rationale, with no video frames, audio track, or adversarial prompt. Since R'_i has already been influenced by the Stage 1 attack, a high ASR can be explained entirely by the model agreeing with its own manipulated narrative. This is not merely a stylistic choice: the authors report that the one-step version in Eq. (5) produced rationales that contradicted labels, and the two-step version was adopted precisely because it removes those contradictions. Consequently Eq. (6) makes ASR a measure of the model's self-consistency under a narrative-flipping attack, not directly a measure of robustness of multimodal input classification. A control is needed: run Stage 2 on the ground-truth rationale R_i; run the full one-step classification M(c_i, adv_i); and run a text-only baseline that uses the original descriptions and adversarial prompts but no frames or audio. If the current high ASR is driven by the narrative-flip effect, the tri-modal claim must be substantially revised; if the one-step classification is also highly vulnerable, the attack is real but should be reported with that evidence.
  2. [Section 4.1 and Section 4.2, Stage 1] The attack is called 'tri-modal,' yet neither the video nor the audio stream is modified: Stage 1 supplies five keyframes, transcriptions, and high-level audio descriptions, while all adversarial components advv_i, adva_i, advp_i are natural-language text. For models such as LLaVA 7B/13B, which do not accept raw audio input, the 'audio' attack is necessarily a text-only attack on a textual description. The paper should state explicitly which evaluated models actually received raw audio, and should include a text-only ablation in which the model receives only the original text descriptions and adversarial prompts, without video frames. Without this ablation, the comparison 'tri-modal > uni-modal' in Table 1 may be comparing differently phrased text prompts rather than demonstrating that all three input modalities are necessary for the attack.
  3. [Section 4.3 and Tables 2-3] The LLM-as-a-judge results are presented as evidence of 'attack reasoning efficacy' and hallucination resistance, but the judge is a single Mistral 7B model with no reported human agreement, no inter-judge variability, and no qualitative examples of judge outputs. Moreover, the hallucination factor near 1.0 is nearly tautological under the two-step protocol: the same model that writes the rationale is then asked to classify from that rationale, so strong alignment between rationale and prediction is expected by construction. The ethical-reasoning scores in Table 3 are likewise averages from an unvalidated rubric; they should be treated as exploratory qualitative signals unless the judge is calibrated against human annotations.
minor comments (4)
  1. [Section 3.1] The dataset is described as containing 'diverse cultural, thematic, and perceptual boundaries,' but the only reported distribution is a binary appropriateness split; the paper would benefit from a more detailed breakdown of video sources, content types, and annotation guidelines.
  2. [Section 5.1] The experimental setup does not report exact model versions, API access dates, or the specific video preprocessing pipeline (e.g., frame extraction rate, audio handling) for each model; these details are needed for reproducibility.
  3. [Throughout] There are several typographical artifacts, such as 'LLaV A' and 'VIDEOJAIL', and inconsistent spacing in citations; a careful proofreading pass is needed.
  4. [Section 4.2] The claim that the model is instructed that 'internet content can be misleading' before the attack is an important design detail; its contribution should be isolated in an ablation, since it may alone increase compliance with the injected narrative.

Circularity Check

2 steps flagged · score 6.0 of 10

The headline ASR is produced by re-feeding the model its own manipulated rationale (Eq. 6), so the metric largely measures self-consistency rather than tri-modal vulnerability; a text-only control is missing.

  1. self definitional [Section 4.2, Stage 2 (Two-step Attack), Eq. (6)]
    "Next, we take this generated rationale and, without any video frames or audio transcriptions, re-prompt the same model to produce a binary classification: appropriate or inappropriate. The entire pipeline now restructures to: y′i ← M(R′i) ← R′i ← M(ci, advi) (6)"

    The final label y′i is a function only of the model's own prior rationale R′i, which the attack has already manipulated in Stage 1 via advi and via the priming instruction that 'internet content can be misleading.' Therefore the ASR definition y′i≠yi measures whether the model labels consistently with its own flipped narrative, not whether the visual/audio content was overpowered. The paper presents Eq. 6 as isolating 'multimodal understanding from the final decision,' but it actually removes the multimodal input from the decision stage entirely. Without a text-only control (classifying unmanipulated rationales) or a single-step comparison, the reported 90%+ ASR cannot be attributed to a tri-modal vulnerability.

  2. other [Section 4.3, Evaluation Metrics and Judging Criteria (Hallucination Factor)]
    "For prediction confidence with hallucination resistance (which we term in our work as the hallucination factor), we pass the prediction and the reasoning, along with the meaning of the labels, to determine whether they align on a scale of 0-1 with the judge identifying mismatches."

    The 'hallucination resistance' score is defined as alignment between the prediction and the reasoning, but both are produced by the same attack pipeline: the prediction is M(R′i) and the reasoning is R′i. The two-step design was explicitly chosen because it 'proved to be highly effective' at reducing inconsistencies, so near-1.0 hallucination factors in Table 2 are built into the protocol and cannot serve as evidence that the model is not hallucinating about the video/audio content.

full rationale

The central result (90%+ ASR, Table 1) is not fully self-contained as a tri-modal robustness finding because the evaluation protocol in Eq. (6) makes the final label a function of the model's own generated rationale. Stage 1 already injects adversarial prompts and primes the model with 'internet content can be misleading' and an instruction to 'be firm in its judgment'; Stage 2 then asks the same model to classify solely from that rationale, with no video frames or audio. Thus the attack success metric y′i≠yi largely measures whether the model agrees with its own manipulated narrative. The paper's claim that the decomposition 'isolates multimodal understanding from the final decision' is inverted: the decision stage no longer has any multimodal input. The uni-modal baselines in Table 1 also use the two-step ChimeraBreak protocol (the paper states the formulation is 'adopted throughout ChimeraBreak'), so the tri-modal versus uni-modal comparison reflects influences on rationale generation rather than direct modality robustness. The hallucination factor similarly scores alignment between prediction and reasoning, which the two-step protocol was designed to maximize. These are construction-level issues rather than self-citation: the dataset excludes Gemini used in generation, and judgments use Mistral 7B. However, because the headline metric is coupled to the attacked rationale by definition, the paper's central vulnerability claim is partially circular; a text-only or single-step control is needed.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the appropriateness labels, on representing audio as text, and on trusting the LLM judge. No new physical or mathematical entities are introduced.

assumptions (3)
  • domain assumption The definition of inappropriateness is based on platform community standards and prior hate speech work, and is treated as ground truth for binary labeling.
    Used in Section 3.1 to label the 1,009 videos as appropriate or inappropriate, forming the labels y_i against which ASR is measured.
  • domain assumption Audio content is adequately represented by transcriptions and high-level descriptions for the model's auditory understanding.
    The models do not receive raw audio; they receive text versions, so claims about auditory pathway attacks rely on this equivalence.
  • domain assumption Mistral 7B as LLM-as-a-judge provides reliable ethical alignment and hallucination scores without calibration against human judgments.
    Used in Section 4.3 to produce Tables 2 and 3; the paper cites general LLM-as-a-judge literature but provides no human validation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Watch, Listen, Understand, Mislead: Tri-modal Adversarial Attacks on Short Videos for Content Appropriateness Evaluation." pith.science (2026). https://pith.science/paper/R26JHEFS

@misc{pith2026250711968,
  author       = {Pith},
  title        = {Pith review of: Watch, Listen, Understand, Mislead: Tri-modal Adversarial Attacks on Short Videos for Content Appropriateness Evaluation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/R26JHEFS}},
  note         = {Machine review of arXiv:2507.11968}
}
read the original abstract

Multimodal Large Language Models (MLLMs) are increasingly used for content moderation, yet their robustness in short-form video contexts remains underexplored. Current safety evaluations often rely on unimodal attacks, failing to address combined attack vulnerabilities. In this paper, we introduce a comprehensive framework for evaluating the tri-modal safety of MLLMs. First, we present the Short-Video Multimodal Adversarial (SVMA) dataset, comprising diverse short-form videos with human-guided synthetic adversarial attacks. Second, we propose ChimeraBreak, a novel tri-modal attack strategy that simultaneously challenges visual, auditory, and semantic reasoning pathways. Extensive experiments on state-of-the-art MLLMs reveal significant vulnerabilities with high Attack Success Rates (ASR). Our findings uncover distinct failure modes, showing model biases toward misclassifying benign or policy-violating content. We assess results using LLM-as-a-judge, demonstrating attack reasoning efficacy. Our dataset and findings provide crucial insights for developing more robust and safe MLLMs.

Figures

Figures reproduced from arXiv: 2507.11968 by the authors.

Figure 1
Figure 1. Graphical abstract illustrating the overall pipeline. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Warning. Sensitive words. An example of inappropriate and appropriate short-video contents from the SVMA dataset tematically bypass safety protocols in several LLMs [45]. Moreover, overwhelming the model was pushed further by a new concept, Distraction Hypothesis, by providing com￾plicated visual inputs to divert their attention from detect￾ing the harmful query. The visual complexity of an input is created with num… view at source ↗
Figure 3
Figure 3. The t-SNE distribution of the original descriptions with [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Overview of ChimeraBreak demonstrating the two-step process of Reasoning Generation followed by Decision Making. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

45 extracted references · 25 canonical work pages

  1. [1]

    Reference-guided ver- dict: Llms-as-judges in automatic evaluation of free-form text

    Sher Badshah and Hassan Sajjad. Reference-guided ver- dict: Llms-as-judges in automatic evaluation of free-form text. arXiv preprint arXiv:2408.09235, 2024. 6

    arXiv 2024

  2. [2]

    Qwen technical report

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309.16609, 2023. 2

    arXiv 2023

  3. [3]

    Qwen2.5-vl technical report, 2025

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhao- hai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report, 2025. 7

    2025

  4. [4]

    Mind the trojan horse: Image prompt adapter enabling scalable and decep- tive jailbreaking

    Junxi Chen, Junhao Dong, and Xiaohua Xie. Mind the trojan horse: Image prompt adapter enabling scalable and decep- tive jailbreaking. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 23785–23794, 2025. 3

    work page 2025

  5. [5]

    Cross-modal causal relation alignment for video question grounding

    Weixing Chen, Yang Liu, Binglin Chen, Jiandong Su, Yongsen Zheng, and Liang Lin. Cross-modal causal relation alignment for video question grounding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025. 5

    work page 2025

  6. [6]

    From compliance to exploitation: Jailbreak prompt attacks on multimodal llms

    Chun Wai Chiu, Linghan Huang, Bo Li, and Huaming Chen. From compliance to exploitation: Jailbreak prompt attacks on multimodal llms. arXiv preprint arXiv:2502.00735, 2025. 2

    arXiv 2025

  7. [7]

    Automated hate speech detection and the prob- lem of offensive language

    Thomas Davidson, Dana Warmsley, Michael Macy, and Ing- mar Weber. Automated hate speech detection and the prob- lem of offensive language. In Proceedings of the interna- tional AAAI conference on web and social media, pages 512– 515, 2017. 4

    work page 2017

  8. [8]

    Adversaflow: Visual red team- ing for large language models with multi-level adversarial flow

    Dazhen Deng, Chuhan Zhang, Huawei Zheng, Yuwen Pu, Shouling Ji, and Yingcai Wu. Adversaflow: Visual red team- ing for large language models with multi-level adversarial flow. IEEE Transactions on Visualization and Computer Graphics, 2024. 3

    work page 2024

Show all 45 references
  1. [9]

    BERT: Pre-training of deep bidirectional trans- formers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional trans- formers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the As- sociation for Computational Linguistics: Human L...

    2019

  2. [10]

    Figstep: Jailbreaking large vision-language models via typo- graphic visual prompts

    Yichen Gong, Delong Ran, Jinyuan Liu, Conglei Wang, Tianshuo Cong, Anyu Wang, Sisi Duan, and Xiaoyun Wang. Figstep: Jailbreaking large vision-language models via typo- graphic visual prompts. In Proceedings of the AAAI Con- ference on Artificial Intelligence, pages 23951–23959...

    2025

  3. [11]

    Agent smith: a single image can jailbreak one million multimodal llm agents exponentially fast

    Xiangming Gu, Xiaosen Zheng, Tianyu Pang, Chao Du, Qian Liu, Ye Wang, Jing Jiang, and Min Lin. Agent smith: a single image can jailbreak one million multimodal llm agents exponentially fast. JMLR.org, 2024. 2

    2024

  4. [12]

    Exploring visual vulnerabilities via multi-loss adversarial search for jailbreaking vision- language models

    Shuyang Hao, Bryan Hooi, Jun Liu, Kai-Wei Chang, Zi Huang, and Yujun Cai. Exploring visual vulnerabilities via multi-loss adversarial search for jailbreaking vision- language models. arXiv preprint arXiv:2411.18000 , 2024. 2

    2024 arXiv

  5. [13]

    Llm-as-a-judge: Reassessing the performance of llms in ex- tractive qa

    Xanh Ho, Jiahao Huang, Florian Boudin, and Akiko Aizawa. Llm-as-a-judge: Reassessing the performance of llms in ex- tractive qa. arXiv preprint arXiv:2504.11972, 2025. 6

    2025 arXiv

  6. [14]

    Curiosity-driven red-teaming for large language models

    Zhang-Wei Hong, Idan Shenfeld, Tsun-Hsuan Wang, Yung- Sung Chuang, Aldo Pareja, James Glass, Akash Srivastava, and Pulkit Agrawal. Curiosity-driven red-teaming for large language models. arXiv preprint arXiv:2402.19464, 2024. 3

    2024 arXiv

  7. [15]

    Videojail: Exploiting video-modality vulnerabilities for jail- break attacks on multimodal large language models

    Wenbo Hu, Shishen Gu, Youze Wang, and Richang Hong. Videojail: Exploiting video-modality vulnerabilities for jail- break attacks on multimodal large language models. InICLR 2025 Workshop on Building Trust in Language Models and Applications, 2025. 2

    2025

  8. [16]

    Gpt-4o system card

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perel- man, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Weli- hinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024. 2

    2024 arXiv

  9. [17]

    Playing the fool: Jailbreaking llms and multimodal llms with out-of-distribution strategy

    Joonhyun Jeong, Seyun Bae, Yeonsung Jung, Jaeryong Hwang, and Eunho Yang. Playing the fool: Jailbreaking llms and multimodal llms with out-of-distribution strategy. In Proceedings of the Computer Vision and Pattern Recogni- tion Conference, pages 29937–29946, 2025. 2

    2025

  10. [18]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lam- ple, Lucile Saulnier, L ´elio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Tho...

    2023

  11. [19]

    Mixtral of experts

    Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Deven- dra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. Mixtral of experts. arXiv preprint arXiv:2401.04088, 2024. 7

    2024 arXiv

  12. [20]

    Trust or esca- late: Llm judges with provable guarantees for human agree- ment

    Jaehun Jung, Faeze Brahman, and Yejin Choi. Trust or esca- late: Llm judges with provable guarantees for human agree- ment. arXiv preprint arXiv:2407.18370, 2024. 6

    2024 arXiv

  13. [21]

    The hateful memes challenge: Detecting hate speech in multimodal memes

    Douwe Kiela, Hamed Firooz, Aravind Mohan, Vedanuj Goswami, Amanpreet Singh, Pratik Ringshia, and Davide Testuggine. The hateful memes challenge: Detecting hate speech in multimodal memes. Advances in neural informa- tion processing systems, 33:2611–2624, 2020. 4

    2020

  14. [22]

    Learning diverse at- tacks on large language models for robust red-teaming and safety tuning

    Seanie Lee, Minsu Kim, Lynn Cherif, David Dobre, Juho Lee, Sung Ju Hwang, Kenji Kawaguchi, Gauthier Gidel, Yoshua Bengio, Nikolay Malkin, et al. Learning diverse at- tacks on large language models for robust red-teaming and safety tuning. arXiv preprint arXiv:2405.18540, 2024. 3

    2024 arXiv

  15. [23]

    Fmm-attack: A flow-based multi- modal adversarial attack on video-based llms.arXiv preprint arXiv:2403.13507, 2024

    Jinmin Li, Kuofeng Gao, Yang Bai, Jingyun Zhang, Shu- tao Xia, and Yisen Wang. Fmm-attack: A flow-based multi- modal adversarial attack on video-based llms.arXiv preprint arXiv:2403.13507, 2024. 2

    2024 arXiv

  16. [24]

    Red teaming visual language models

    Mukai Li, Lei Li, Yuwei Yin, Masood Ahmed, Zhenguang Liu, and Qi Liu. Red teaming visual language models. arXiv preprint arXiv:2401.12915, 2024. 2 9

    2024 arXiv

  17. [25]

    Images are achilles’ heel of alignment: Exploit- ing visual vulnerabilities for jailbreaking multimodal large language models

    Yifan Li, Hangyu Guo, Kun Zhou, Wayne Xin Zhao, and Ji- Rong Wen. Images are achilles’ heel of alignment: Exploit- ing visual vulnerabilities for jailbreaking multimodal large language models. In European Conference on Computer Vi- sion, pages 174–189. Springer, 2024. 2

    2024

  18. [26]

    GroundingGPT: Language en- hanced multi-modal grounding model

    Zhaowei Li, Qi Xu, Dong Zhang, Hang Song, YiQing Cai, Qi Qi, Ran Zhou, Junting Pan, Zefeng Li, Vu Tu, Zhida Huang, and Tao Wang. GroundingGPT: Language en- hanced multi-modal grounding model. In Proceedings of the 62nd Annual Meeting of the Association for Computa- tional Ling...

    2024

  19. [27]

    Video-llava: Learning united visual rep- resentation by alignment before projection

    Bin Lin, Yang Ye, Bin Zhu, Jiaxi Cui, Munan Ning, Peng Jin, and Li Yuan. Video-llava: Learning united visual rep- resentation by alignment before projection. arXiv preprint arXiv:2311.10122, 2023. 2

    2023 arXiv

  20. [28]

    Visual instruction tuning, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, 2023. 7

    2023

  21. [29]

    Prompt injection attack against llm- integrated applications

    Yi Liu, Gelei Deng, Yuekang Li, Kailong Wang, Zihao Wang, Xiaofeng Wang, Tianwei Zhang, Yepang Liu, Haoyu Wang, Yan Zheng, et al. Prompt injection attack against llm- integrated applications. arXiv preprint arXiv:2306.05499 ,

    arXiv

  22. [30]

    Arondight: Red teaming large vision language models with auto-generated multi-modal jailbreak prompts

    Yi Liu, Chengjun Cai, Xiaoli Zhang, Xingliang Yuan, and Cong Wang. Arondight: Red teaming large vision language models with auto-generated multi-modal jailbreak prompts. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 3578–3586, 2024. 3

    2024

  23. [31]

    LLaMA 4: Advancing Multimodal Intelli- gence

    Meta AI. LLaMA 4: Advancing Multimodal Intelli- gence. https://ai.meta.com/blog/llama- 4- multimodal-intelligence/, 2024. Accessed: 2025- 06-27. 7

    2024

  24. [32]

    Jailbreaking attack against multimodal large lan- guage model

    Zhenxing Niu, Haodong Ren, Xinbo Gao, Gang Hua, and Rong Jin. Jailbreaking attack against multimodal large lan- guage model. arXiv preprint arXiv:2402.02309, 2024. 2

    2024 arXiv

  25. [33]

    Gpt-4o system card, 2024

    OpenAI. Gpt-4o system card, 2024. 7

    2024

  26. [34]

    Cross-modal attention congruence regularization for vision-language relation align- ment

    Rohan Pandey, Rulin Shao, Paul Pu Liang, Ruslan Salakhut- dinov, and Louis-Philippe Morency. Cross-modal attention congruence regularization for vision-language relation align- ment. In Proceedings of the 61st Annual Meeting of the Asso- ciation for Computational Linguistics (...

    2023

  27. [35]

    Vision-llms can fool themselves with self-generated typographic attacks

    Maan Qraitem, Nazia Tasnim, Piotr Teterwak, Kate Saenko, and Bryan A Plummer. Vision-llms can fool themselves with self-generated typographic attacks. arXiv preprint arXiv:2402.00626, 2024. 2

    2024 arXiv

  28. [36]

    On the adversarial robustness of multi-modal foundation models

    Christian Schlarmann and Matthias Hein. On the adversarial robustness of multi-modal foundation models. In Proceed- ings of the IEEE/CVF International Conference on Com- puter Vision, pages 3677–3685, 2023. 2

    2023

  29. [37]

    Gemini: A family of highly capable multi- modal models, 2025

    Gemini Team. Gemini: A family of highly capable multi- modal models, 2025. 4

    2025

  30. [38]

    Gemma 3 technical report, 2025

    Gemma Team. Gemma 3 technical report, 2025. 7

    2025

  31. [39]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth´ee Lacroix, Baptiste Rozi`ere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023. 2

    2023 arXiv

  32. [40]

    Stop reasoning! when multimodal llm with chain-of-thought rea- soning meets adversarial image, 2024

    Zefeng Wang, Zhen Han, Shuo Chen, Fan Xue, Zifeng Ding, Xun Xiao, V olker Tresp, Philip Torr, and Jindong Gu. Stop reasoning! when multimodal llm with chain-of-thought rea- soning meets adversarial image, 2024. 2

    2024

  33. [41]

    Audio is the achilles’ heel: Red teaming audio large multimodal models

    Hao Yang, Lizhen Qu, Ehsan Shareghi, and Gholamreza Haffari. Audio is the achilles’ heel: Red teaming audio large multimodal models. arXiv preprint arXiv:2410.23861, 2024. 2

    2024 arXiv

  34. [42]

    Distraction is all you need for multimodal large language model jailbreaking

    Zuopeng Yang, Jiluan Fan, Anli Yan, Erdun Gao, Xin Lin, Tao Li, Kanghua Mo, and Changyu Dong. Distraction is all you need for multimodal large language model jailbreaking. In Proceedings of the Computer Vision and Pattern Recogni- tion Conference, pages 9467–9476, 2025. 3

    2025

  35. [43]

    Gptfuzzer: Red teaming large language models with auto-generated jailbreak prompts

    Jiahao Yu, Xingwei Lin, Zheng Yu, and Xinyu Xing. Gptfuzzer: Red teaming large language models with auto-generated jailbreak prompts. arXiv preprint arXiv:2309.10253, 2023. 3

    2023 arXiv

  36. [44]

    Judging llm-as-a-judge with mt-bench and chatbot arena.Advances in Neural Information Processing Systems, 36:46595–46623, 2023

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena.Advances in Neural Information Processing Systems, 36:46595–46623, 2023. 6

    2023

  37. [45]

    Universal and transferable adversarial attacks on aligned lan- guage models, 2023

    Andy Zou, Zifan Wang, J Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned lan- guage models, 2023. URL https://arxiv. org/abs/2307.15043, 19, 2024. 3 10

    2023 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.