Pith. sign in

REVIEW 4 major objections 5 minor 73 references

The Boy Who Cried Wolf: Adversarial Misclassification of Safe Inputs as Unsafe in Multimodal Guardrails

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Multimodal guard models that screen images and prompts before vision-language systems can be tricked by nearly imperceptible perturbations into rejecting legitimate requests; the paper proposes Unsafe Semantic Distillation and reports 78–84

desk verdict Solid method, overstated framing: the prompt-agnostic false-positive attack is real, but the 'new threat' and 'imperceptible' claims need a human study and honest positioning. read the letter →

arxiv 2608.01373 v1 pith:SDGQWVYC submitted 2026-08-02 cs.CR

Shuo Shi , Rui Yin , Naen Xu , Jiahao Chen , Chunyi Zhou , Tianyu Du , Zhihui Fu , Jun Wang
show 2 more authors
Zhaoxiang Wang Shouling Ji
This is my paper
classification cs.CR
keywords vision-languagemodelscontentmoderationadversarialattacksmultimodalguardrailsunsafeinductionfalsepositiveslogicaldenial-of-serviceSemanticDistillation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Multimodal guard models—safety filters that sit in front of vision-language services and audit image–text requests—can be made to reject legitimate inputs by adding nearly invisible perturbations to benign images, an attack the paper calls Unsafe Induction. The authors propose Unsafe Semantic Distillation (USD), which steers the perturbed image toward the distribution of unsafe image representations in the guard model's latent space rather than toward any single target, aiming to succeed no matter what text prompt a user pairs with the image. Across four state-of-the-art guard models, USD reports 78–84% success on realistic user-simulated prompts, with targeted variants forcing specific risk categories. If correct, the result means deployed safety filters have an availability failure mode—a logical denial-of-service—that current, false-negative-focused safety benchmarks do not measure.

What carries the argument

Unsafe Semantic Distillation (USD): a hybrid attack objective that combines a distributional alignment loss—temperature-scaled LogSumExp of cosine similarities between the adversarial image's embedding and embeddings of sampled unsafe reference images—with a decision-level cross-entropy loss that verifies the guard outputs 'unsafe' under shadow prompts. Optimized by projected gradient descent under an $\ell^\infty$ bound; the distributional term is what gives prompt-agnostic robustness and prevents overfitting to a single instance.

What would settle it

Run a two-alternative forced-choice experiment where naive participants are shown a clean benign image and a USD-perturbed version (ε=16/255) side by side and asked to pick the altered one. If discrimination performance approaches chance (50%), the perturbations are effectively imperceptible and the public-distribution premise holds; if it clearly exceeds chance, the attack's opening stage is falsified.

Watch

Extended reading notes

Core claim

The paper claims that multimodal guard models—risk-classifying arbiters that screen image-and-text inputs before a vision-language service responds—can be fooled into flagging benign inputs as policy violations by adding a small, visually near-imperceptible perturbation ($\ell^\infty$-bounded at 16/255) to a safe image. The method, Unsafe Semantic Distillation (USD), steers the perturbed image's embedding toward a distribution of unsafe reference embeddings via a temperature-scaled LogSumExp cosine-similarity loss, with a decision-level cross-entropy term verifying that the guard emits 'unsafe'. Reported attack success is 78–84% on realistic user prompts across four guard models (82.8–84.3%

Load-bearing premise

That users will actually incorporate the distributed, visually 'benign' images into legitimate requests—supported only by an LPIPS score (0.21–0.31) and qualitative inspection, not by a human-subject test—so if people notice or distrust the tampering, the public-distribution attack flow collapses.

Editorial extensions

If this is right

  • Guard-model deployments acquire an availability failure mode: benign requests can be systematically blocked through the safety logic itself, which the current false-negative-focused safety evaluation (jailbreak success) does not measure.
  • Attackers can steer the rejection toward a specific risk category, framing benign images as violent or sexual content, which can trigger category-specific escalation and deepen user distrust.
  • Low cross-model transfer (<4% off-diagonal) keeps an attacker's own service—protected by a different guard model—largely unharmed, making competitive disruption nearly cost-free.
  • Lightweight input preprocessing (JPEG, blur, smoothing) reduces but does not eliminate the attack (>60% success), so existing sanitization pipelines are not a sufficient defense.
  • The paper's results directly motivate 'bidirectional robustness' in guard models—training against both evasion (unsafe→safe) and over-rejection (safe→unsafe).

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The distributional-distillation recipe is transferable in principle: any attribute that occupies a detectable cluster in an encoder's latent space (copyrighted patterns, private traits, a brand's signature look) could be injected into benign inputs the same way, so the attack class may extend beyond safety taxonomies to other downstream filters.
  • The attacker's key resource is a diverse corpus of unsafe images, not compute: success scales from 38% with a single prototype to 85% with 100, suggesting that curation and removal of such corpora is a plausible, testable policy lever.
  • Because cross-model transfer is so low, ensemble-based or rotating-guard architectures would likely nullify a USD image crafted against any single model—a concrete defense hypothesis that follows from the reported transfer matrix but is not explored in the paper.
  • The imperceptibility claim rests only on LPIPS and qualitative inspection; a controlled human-discrimination study is the natural next experiment to verify the distribution stage of the attack.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces Unsafe Induction Attacks against multimodal guard models: an adversary crafts L_inf-bounded perturbations to benign images so that guard models reject the accompanying legitimate request as unsafe across diverse, unseen user prompts. The proposed method, Unsafe Semantic Distillation (USD), combines a distributional distillation loss over unsafe image embeddings with a decision-level cross-entropy verification loss over sampled shadow prompts. The empirical study covers four guard models, two benign image datasets, two perturbation budgets, several baselines, targeted risk-category injection, cross-model transferability, image-only effectiveness, preprocessing robustness, and hyperparameter sensitivity. The headline result is 78--84% attack success on realistic held-out user prompts (ASRr), with much lower transfer across guard models.

Significance. If the results hold, the paper identifies a genuinely different availability failure mode for multimodal guardrails: logical denial-of-service through false positives, rather than resource-exhaustion DoS or jailbreaking. The authors ship code, use a realistic prompt-generation protocol, and evaluate across four guard models with appropriate baseline families. The finding that USD is strongly prompt-robust while feature-matching baselines lag is a useful contribution, and the low cross-model transfer result is a practically relevant boundary condition. However, the central threat scenario depends on users willingly adopting the adversarial images, which requires imperceptibility to be established by human perception evidence rather than LPIPS alone; and the quantitative claims are presented without any uncertainty quantification. Both issues need to be addressed before the strength of the conclusions matches the evidence.

major comments (4)
  1. [§3.2.1, §5.6 (Table 4, Fig. 5)] The distribution-stage premise is not demonstrated. The real-world impact requires unsuspecting users to download and upload the perturbed images; the only support for 'imperceptible' is LPIPS 0.2089/0.3124 and a few figure panels. LPIPS is not a validated human-perception threshold, and eps=16/255 is a sizeable L_inf budget. The RQ4 statement that Fig. 5 'confirm[s] that the attack preserves human-perceived benignness' is unsupported without a human detection/rating study. Please add such a study or substantially soften the threat-model claims.
  2. [§5.2, Table 1] All ASR values are point estimates. Each ASRr comes from 100 images and 10 generated prompts, but no seeds, error bars, confidence intervals, or significance tests are reported anywhere; the same applies to Tables 2--5 and Fig. 6. Consequently, the headline '78--84%' and the claimed margins over baselines (e.g., USD 84.3 vs CoA 65.6 on VQAv2, Llama Guard, eps=16/255) lack statistical support. Please report multiple runs or bootstrap confidence intervals.
  3. [Related Work / References [49]] Reference [49] (Zhang et al., 'LLM Safeguard is a Double-Edged Sword: Exploiting False Positives for Denial-of-Service Attacks') is not cited or discussed anywhere. Its title indicates prior false-positive DoS work on LLM safeguards, which directly bears on the 'first to formalize' / 'remains unexplored' novelty claims in the Introduction. Please cite it and explicitly delineate the differences (e.g., multimodal guard models, image perturbation, prompt-agnostic generalization).
  4. [§4.3.4, Algorithm 1 line 10 vs Eq. (10)] Eq. (10) writes δ ← Π[δ − α·∇_δ L_USD], consistent with minimizing the loss in Eq. (9). Algorithm 1 line 10 writes δ ← δ + α·sign(∇_δ L_USD). With the plus sign and L_distill defined as a negative log-sum-exp, the update would minimize the similarity to unsafe prototypes and decrease P(unsafe), contradicting the reported results. Please correct the pseudocode or clarify the sign convention; this is essential for reproducibility.
minor comments (5)
  1. [Abstract / §5.3] The abstract says 'achieving 84% attack success rates', but Table 1 shows a range of 70.4--85.9% across models and settings; please clarify 'up to 84%' or give the range.
  2. [§5.1.2] Check dataset/reference consistency: the text says 'HolisticSafeBench' but the bibliography entry [18] is 'HoliSafe'; the VHD11K dataset is cited as [42] 'T2Vs Meet VLMs'. Please align names and citations.
  3. [§5.6, Table 3] The image-only evaluation does not specify how the guard model is invoked without a text prompt. Since guard models are text-conditioned, please state the exact input format used (e.g., empty string or a fixed token).
  4. [§5.7, Table 5] The preprocessing robustness table only reports ASRr at eps=8/255 and does not report the clean-image ASR under the same preprocessing. Adding that control would make the degradation attributable to the defense rather than to preprocessing effects on the guard model itself.
  5. [§5.8, Fig. 6] Figure 6 plots point estimates without error bars; the y-axis label 'Targeted ASR (%)' is also inconsistent with the text, which describes generic ASRr for the hyperparameter analysis. Please clarify.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: held-out ASRr is a genuine generalization measurement; the imperceptibility claim is an external-validity gap, not a circular step.

full rationale

The derivation chain is self-contained. USD optimizes a perturbation δ with PGD against L_USD = L_decision + λ·L_distill (Eq. 9, Algorithm 1). L_decision uses shadow prompts; L_distill uses the guard's own vision encoder f_phi to align the adversarial image with unsafe prototypes. The central empirical claim is held-out ASRr, measured on 10 image-specific prompts per image that are explicitly disjoint from the shadow prompts (Section 5.2). This is not a fitted quantity: for USD on Llama-Guard-3-Vision, ASRs is about 98% while ASRr is 78–84%, a substantial generalization gap, and baselines optimized with the same family of objectives fail (ASRr ≤ 68.7%), so the result is not forced by construction. Using f_phi both to define the unsafe manifold and as the attacked encoder is a standard white-box property, not a circular prediction, because prompt-agnostic transfer is not guaranteed by the objective. Self-citations ([39]–[43]) appear only in general related-work statements and are not load-bearing; no uniqueness theorem or ansatz is imported from prior work. One external-validity caveat should be flagged: RQ4 (Section 5.6) asserts that images 'remain visually indistinguishable' and 'preserve human-perceived benignness' based on LPIPS values of 0.2089–0.3124 (Table 4) and Figure 5, with no human-subject experiment. This does not make the derivation circular, but it leaves the distribution-stage premise—that users would adopt and upload the adversarial images—unvalidated.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new physical or conceptual entities; the 'unsafe semantic manifold' is a statistical summary of embeddings, not an invented entity. The main unstated inputs are three optimization hyperparameters that are never reported, plus two domain assumptions about cross-modal dominance and realism of synthesized prompts.

free parameters (3)
  • distillation temperature tau = not reported
    Hyperparameter in Eq. 7 (temperature of log-sum-exp aggregation); no default or sensitivity analysis is given, making the exact distillation behavior unreproducible from text alone.
  • loss weight lambda = not reported
    Balances distillation loss and decision loss in Eq. 9; value is never stated in the paper.
  • PGD step size alpha = not reported
    Required in Algorithm 1 for the perturbation update; omitted, so the optimization trajectory is underdetermined.
assumptions (3)
  • domain assumption Guard models exhibit cross-modal dominance: a strong unsafe visual signal in one modality can trigger an unsafe classification regardless of the text prompt.
    Invoked in sections 4.2 and 4.3 to argue that perturbing the image alone is sufficient for prompt-agnostic attacks; supported only by a citation to [7] and by the paper's own experimental results.
  • domain assumption The vision encoder's latent space contains a prompt-invariant cluster of unsafe embeddings that can be targeted via cosine-similarity alignment.
    The central mechanism of USD (section 4.3.1): unsafe images share abstract high-level semantics; the t-SNE in Appendix B is the only evidence, which is qualitative.
  • domain assumption GPT-4o-mini-generated prompts are a faithful proxy for real user requests to multimodal services.
    ASRr, the primary metric, is built on 10 GPT-4o-mini-generated prompts per image (Appendix C.2); no validation against actual user logs or human distribution is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of The Boy Who Cried Wolf: Adversarial Misclassification of Safe Inputs as Unsafe in Multimodal Guardrails." pith.science (2026). https://pith.science/paper/SDGQWVYC

@misc{pith2026260801373,
  author       = {Pith},
  title        = {Pith review of: The Boy Who Cried Wolf: Adversarial Misclassification of Safe Inputs as Unsafe in Multimodal Guardrails},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SDGQWVYC}},
  note         = {Machine review of arXiv:2608.01373}
}
read the original abstract

Multimodal guard models have emerged as critical safety components for screening content in vision-language systems. While adversarial research has extensively studied jailbreaking attacks that produce false negatives, the inverse threat of inducing false positives on benign inputs remains unexplored. We introduce Unsafe Induction Attacks, where adversaries distribute imperceptibly perturbed safe images that trigger guard models to reject legitimate user requests, causing a "Boy Who Cried Wolf" effect that degrades service availability and erodes trust. This reveals an availability failure mode in deployed safety filters. To realize this threat under diverse user prompts, we propose Unsafe Semantic Distillation (USD), which aligns adversarial perturbations with distributional representations of unsafe content rather than prompt-specific instances. Evaluated on four state-of-the-art guard models across realistic user simulation scenarios, USD achieves 84% attack success rates, outperforming existing methods and exposing fundamental vulnerabilities in current multimodal safety architectures.

Figures

Figures reproduced from arXiv: 2608.01373 by the authors.

Figure 1
Figure 1. Unsafe Induction Attacks against Multimodal Guard Mod [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Illustration of the attack scenario. The attacker generates [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Overview of USD framework combining feature-based and query-based optimization. USD distills unsafe semantics from diverse [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Cross-model transferability matrix of USD attacks under [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 6
Figure 6. Figure 6: Effect of hyperparameters on ASRr. Left: varying number of shadow prompts from {1, 5, 10, 50, 100}. Right: varying size of unsafe reference set |Zunsafe | from {1, 5, 10, 50, 100}. We vary each hyperparameter across {1, 5, 10, 50, 100} and report results in [PITH_FULL…
Figure 5
Figure 5. Figure 5: Visual comparison of clean images and USD adversarial [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 7
Figure 7. Figure 7: t-SNE visualization of image embeddings in Llama Guard [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: Representative realistic user simulation prompts generated [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

73 extracted references · 40 canonical work pages

  1. [29]

    Zedian Shao, Hongbin Liu, Yuepeng Hu, and Neil Zhenqiang Gong. 2024. Re- fusing Safe Prompts for Multi-modal Large Language Models.arXiv preprint arXiv:2407.09050(2024)

    arXiv 2024

  2. [49]

    Morley Mao

    Qingzhao Zhang, Ziyang Xiong, and Z. Morley Mao. 2025. LLM Safeguard is a Double-Edged Sword: Exploiting False Positives for Denial-of-Service Attacks. arXiv:2410.02916 [cs.CR] https://arxiv.org/abs/2410.02916

    arXiv 2025

  3. [1]

    Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zitnick, and Devi Parikh. 2015. Vqa: Visual question answering. In Proceedings of the IEEE international conference on computer vision. 2425–2433

    2015

  4. [2]

    Eugene Bagdasaryan, Tsung-Yin Hsieh, Ben Nassi, and Vitaly Shmatikov. 2023. Abusing images and sounds for indirect instruction injection in multi-modal LLMs.arXiv preprint arXiv:2307.10490(2023)

    arXiv 2023

  5. [3]

    Davide Caffagni, Federico Cocchi, Luca Barsellotti, Nicholas Moratelli, Sara Sarto, Lorenzo Baraldi, Lorenzo Baraldi, Marcella Cornia, and Rita Cucchiara. 2024. The Revolution of Multimodal Large Language Models: A Survey. InFindings of the Association for Computational Linguistics: ACL 2024, Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.). Associati...

    doi:10.18653/v1/2024.findings-acl.807 2024

  6. [4]

    Nicholas Carlini, Anish Athalye, Nicolas Papernot, Wieland Brendel, Jonas Rauber, Dimitris Tsipras, Ian Goodfellow, Aleksander Madry, and Alexey Ku- rakin. 2019. On evaluating adversarial robustness.arXiv preprint arXiv:1902.06705 (2019)

    arXiv 2019

  7. [5]

    Choquette-Choo, Matthew Jagielski, Irena Gao, Anas Awadalla, Pang Wei Koh, Daphne Ippolito, Katherine Lee, Florian Tramer, and Ludwig Schmidt

    Nicholas Carlini, Milad Nasr, Christopher A. Choquette-Choo, Matthew Jagielski, Irena Gao, Anas Awadalla, Pang Wei Koh, Daphne Ippolito, Katherine Lee, Florian Tramer, and Ludwig Schmidt. 2023. Are aligned neural networks adversarially aligned?. InProceedings of the 37th International Conference on Neural Information Processing Systems(New Orleans, LA, US...

    work page 2023

  8. [6]

    Chaochao Chen, Yizhao Zhang, Yuyuan Li, Jun Wang, Lianyong Qi, Xiaolong Xu, Xiaolin Zheng, and Jianwei Yin. 2024. Post-training attribute unlearning in recommender systems.ACM Transactions on Information Systems43, 1 (2024), 1–28

    work page 2024

Show all 73 references
  1. [7]

    Huiyi Chen, Jiawei Peng, Dehai Min, Changchang Sun, Kaijie Chen, Yan Yan, Xu Yang, and Lu Cheng. 2025. MVI-Bench: A Comprehensive Benchmark for Evalu- ating Robustness to Misleading Visual Inputs in LVLMs. arXiv:2511.14159 [cs.CV] https://arxiv.org/abs/2511.14159

    2025 arXiv

  2. [8]

    Simin Chen, Zihe Song, Mirazul Haque, Cong Liu, and Wei Yang. 2022. Nicgslow- down: Evaluating the efficiency robustness of neural image caption generation models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 15365–15374

    2022

  3. [9]

    Jianfeng Chi, Ujjwal Karn, Hongyuan Zhan, Eric Smith, Javier Rando, Yiming Zhang, Kate Plawiak, Zacharie Delpierre Coudert, Kartikeya Upasani, and Ma- hesh Pasupuleti. 2024. Llama guard 3 vision: Safeguarding human-ai image understanding conversations.arXiv preprint arXiv:2411...

    2024 arXiv

  4. [10]

    Shiyao Cui, QingLin Zhang, Xuan Ouyang, Renmiao Chen, Zhexin Zhang, Yida Lu, Hongning Wang, Han Qiu, and Minlie Huang. 2025. ShieldVLM: Safeguarding the Multimodal Implicit Toxicity via Deliberative Reasoning with LVLMs: Shield- VLM. InProceedings of the 33rd ACM International...

    2025

  5. [11]

    Shiyao Cui, QingLin Zhang, Xuan Ouyang, Renmiao Chen, Zhexin Zhang, Yida Lu, Hongning Wang, Han Qiu, and Minlie Huang. 2025. ShieldVLM: Safeguarding the Multimodal Implicit Toxicity via Deliberative Reasoning with LVLMs: Shield- VLM. InProceedings of the 33rd ACM International...

    2025

  6. [12]

    Haoran Gao, Yuanhe Zhang, Zhenhong Zhou, Lei Jiang, Fanyu Meng, Yujia Xiao, Kun Wang, Yang Liu, and Junlan Feng. 2025. RECALLED: An Unbounded Resource Consumption Attack on Large Vision-Language Models.arXiv e-prints (2025), arXiv–2507

    2025

  7. [13]

    Yichen Gong, Delong Ran, Jinyuan Liu, Conglei Wang, Tianshuo Cong, Anyu Wang, Sisi Duan, and Xiaoyun Wang. 2025. FigStep: jailbreaking large vision- language models via typographic visual prompts. InProceedings of the Thirty- Ninth AAAI Conference on Artificial Intelligence an...

    2025

  8. [14]

    Lukas Helff, Felix Friedrich, Manuel Brack, Patrick Schramowski, and Kristian Kersting. 2024. Llavaguard: Vlm-based safeguard for vision dataset curation and safety assessment. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 8322–8326

    2024

  9. [15]

    Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, and Madian Khabsa. 2023. Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations. arXiv:2312.06674 [cs.CL] https:/...

    2023 arXiv

  10. [16]

    Divyansh Kaushik, Douwe Kiela, Zachary C Lipton, and Wen-tau Yih. 2021. On the efficacy of adversarial data collection for question answering: Results from a large-scale randomized study.arXiv preprint arXiv:2106.00872(2021)

    2021 arXiv

  11. [17]

    Joonkyung Kim, Wenxi Chen, Davood Soleymanzadeh, Yi Ding, Xiangbo Gao, Zhengzhong Tu, Ruqi Zhang, Fan Fei, Sushant Veer, Yiwei Lyu, Minghui Zheng, and Yan Gu. 2026. Modular Safety Guardrails Are Necessary for Foundation- Model-Enabled Robots in the Real World. arXiv:2602.04056...

    2026

  12. [18]

    Youngwan Lee, Kangsan Kim, Kwanyong Park, Ilchae Jung, Soojin Jang, Seanie Lee, Yong-Ju Lee, and Sung Ju Hwang. [n. d.]. HoliSafe: Holistic Safety Bench- marking and Modeling for Vision-Language Model. ([n. d.])

  13. [19]

    Lijun Li, Zhelun Shi, Xuhao Hu, Bowen Dong, Yiran Qin, Xihui Liu, Lu Sheng, and Jing Shao. 2025. T2ISafety: Benchmark for Assessing Fairness, Toxicity, and Privacy in Image Generation. In2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 13381–13392. d...

    2025

  14. [20]

    Yuyuan Li, Chaochao Chen, Yizhao Zhang, Weiming Liu, Lingjuan Lyu, Xiaolin Zheng, Dan Meng, and Jun Wang. 2023. Ultrare: Enhancing receraser for recom- mendation unlearning via error decomposition.Advances in Neural Information Processing Systems36 (2023), 12611–12625

    2023

  15. [21]

    Yuyuan Li, Yizhao Zhang, Weiming Liu, Xiaohua Feng, Zhongxuan Han, Chaochao Chen, and Chenggang Yan. 2025. Multi-Objective Unlearning in Rec- ommender Systems via Preference Guided Pareto Exploration.IEEE Transactions on Services Computing(2025)

    2025

  16. [22]

    Zhaoyi Li, Xiaohan Zhao, Dong-Dong Wu, Jiacheng Cui, and Zhiqiang Shen

  17. [23]

    Qin Liu, Fei Wang, Chaowei Xiao, and Muhao Chen. 2025. VLM-Guard: Safe- guarding Vision-Language Models via Fulfilling Safety Alignment Gap.arXiv preprint arXiv:2502.10486(2025)

    2025 arXiv

  18. [24]

    Liming Lu, Shuchao Pang, Siyuan Liang, Haotian Zhu, Xiyu Zeng, Aishan Liu, Yunhuai Liu, and Yongbin Zhou. 2025. Adversarial training for multimodal large language models against jailbreak attacks.arXiv preprint arXiv:2503.04833(2025)

    2025

  19. [25]

    Haochen Luo, Jindong Gu, Fengyuan Liu, and Philip Torr. 2023. An image is worth 1000 lies: Transferability of adversarial images across prompts on vision-language models. InThe Twelfth International Conference on Learning Representations

    2023

  20. [26]

    OpenAI. 2026. Moderation API - OpenAI Platform Documentation. https:// platform.openai.com/docs/guides/moderation. Accessed: 2026-02-07

    2026

  21. [27]

    Xiangyu Qi, Kaixuan Huang, Ashwinee Panda, Peter Henderson, Mengdi Wang, and Prateek Mittal. 2024. Visual adversarial examples jailbreak aligned large language models. InProceedings of the AAAI conference on artificial intelligence, Vol. 38. 21527–21536

    2024

  22. [28]

    Yiting Qu, Xinyue Shen, Yixin Wu, Michael Backes, Savvas Zannettou, and Yang Zhang. 2025. UnsafeBench: Benchmarking Image Safety Classifiers on Real-World and AI-Generated Images. InProceedings of the 2025 ACM SIGSAC Conference on Computer and Communications Security(Taipei, T...

    2025

  23. [30]

    Ilia Shumailov, Yiren Zhao, Daniel Bates, Nicolas Papernot, Robert Mullins, and Ross Anderson. 2021. Sponge examples: Energy-latency attacks on neural net- works. In2021 IEEE European symposium on security and privacy (EuroS&P). IEEE, 212–231

    2021

  24. [31]

    Amanpreet Singh, Vivek Natarajan, Meet Shah, Yu Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh, and Marcus Rohrbach. 2019. Towards vqa models that can read. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition. 8317–8326

    2019

  25. [32]

    Zhiqing Sun, Sheng Shen, Shengcao Cao, Haotian Liu, Chunyuan Li, Yikang Shen, Chuang Gan, Liangyan Gui, Yu-Xiong Wang, Yiming Yang, Kurt Keutzer, and Trevor Darrell. 2024. Aligning Large Multimodal Models with Factually Augmented RLHF. InFindings of the Association for Computa...

    2024

  26. [33]

    Simone Tedeschi, Felix Friedrich, Patrick Schramowski, Kristian Kersting, Roberto Navigli, Huu Nguyen, and Bo Li. 2024. ALERT: A Comprehensive Benchmark for Assessing Large Language Models’ Safety through Red Teaming.arXiv preprint arXiv:2404.08676(2024)

    2024 arXiv

  27. [34]

    Boxin Wang, Weixin Chen, Hengzhi Pei, Chulin Xie, Mintong Kang, Chenhui Zhang, Chejian Xu, Zidi Xiong, Ritik Dutta, Rylan Schaeffer, et al. 2023. Decod- ingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models.. In NeurIPS

    2023

  28. [35]

    Youze Wang, Wenbo Hu, Yinpeng Dong, Jing Liu, Hanwang Zhang, and Richang Hong. 2025. Align is not enough: Multimodal universal jailbreak attack against multimodal large language models.IEEE Transactions on Circuits and Systems for Video Technology(2025)

    2025

  29. [36]

    Laura Weidinger, John Mellor, Maribeth Rauh, Conor Griffin, Jonathan Uesato, Po- Sen Huang, Myra Cheng, Mia Glaese, Borja Balle, Atoosa Kasirzadeh, Zac Kenton, Sasha Brown, Will Hawkins, Tom Stepleton, Courtney Biles, Abeba Birhane, Julia Haas, Laura Rimell, Lisa Anne Hendrick...

    2021 arXiv

  30. [37]

    Fenghua Weng, Yue Xu, Chengyan Fu, and Wenjie Wang. 2025. Mmj-bench: A comprehensive study on jailbreak attacks and defenses for vision language The Boy Who Cried Wolf: Adversarial Misclassification of Safe Inputs as Unsafe in Multimodal Guardrails KDD 2026, August 9–13, 2026,...

    2025

  31. [38]

    Peng Xie, Yequan Bie, Jianda Mao, Yangqiu Song, Yang Wang, Hao Chen, and Kani Chen. 2025. Chain of Attack: On the Robustness of Vision-Language Models Against Transfer-Based Adversarial Attacks. InProceedings of the Computer Vision and Pattern Recognition Conference. 14679–14689

    2025

  32. [39]

    I See What You Did There

    Naen Xu, Jiayi Sheng, Changjiang Li, Chunyi Zhou, Yuyuan Li, Tianyu Du, Jun Wang, Zhihui Fu, Jinbao Li, and Shouling Ji. 2026. " I See What You Did There": Can Large Vision-Language Models Understand Multimodal Puns?arXiv preprint arXiv:2604.05930(2026)

    2026 arXiv

  33. [40]

    Naen Xu, Jinghuai Zhang, Ping He, Chunyi Zhou, Jun Wang, Zhihui Fu, Tianyu Du, Zhaoxiang Wang, and Shouling Ji. 2026. FraudShield: Knowledge Graph Empowered Defense for LLMs against Fraud Attacks. InProceedings of the ACM Web Conference 2026. 2649–2660

    2026

  34. [41]

    Naen Xu, Jinghuai Zhang, Changjiang Li, Hengyu An, Chunyi Zhou, Jun Wang, Boyu Xu, Yuyuan Li, Tianyu Du, and Shouling Ji. 2026. Bridging the Copyright Gap: Do Large Vision-Language Models Recognize and Respect Copyrighted Content?. InProceedings of the AAAI Conference on Artif...

    2026

  35. [42]

    Chen Yeh, You-Ming Chang, Wei-Chen Chiu, and Ning Yu. 2024. T2Vs Meet VLMs: A Scalable Multimodal Dataset for Visual Harmfulness Recognition.Advances in Neural Information Processing Systems37 (2024), 112950–112961

    2024

  36. [43]

    Rui Yin, Tianxu Han, Naen Xu, Changjiang Li, Ping He, Chunyi Zhou, Jun Wang, Zhihui Fu, Tianyu Du, Jinbao Li, and Shouling Ji. 2026. Compiling Activa- tion Steering into Weights via Null-Space Constraints for Stealthy Backdoors. arXiv:2604.12359 [cs.CR] https://arxiv.org/abs/2...

    2026 arXiv

  37. [44]

    Shukang Yin, Chaoyou Fu, Sirui Zhao, Ke Li, Xing Sun, Tong Xu, and Enhong Chen. 2024. A survey on multimodal large language models.National Science Review11, 12 (Nov. 2024). doi:10.1093/nsr/nwae403

    2024 doi

  38. [45]

    Tianyu Yu, Yuan Yao, Haoye Zhang, Taiwen He, Yifeng Han, Ganqu Cui, Jinyi Hu, Zhiyuan Liu, Hai-Tao Zheng, Maosong Sun, and Tat-Seng Chua. 2024. RLHF- V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback. InProceedings of the IEEE/CV...

    2024

  39. [46]

    Yi Zeng, Kevin Klyman, Andy Zhou, Yu Yang, Minzhou Pan, Ruoxi Jia, Dawn Song, Percy Liang, and Bo Li. 2024. Ai risk categorization decoded (air 2024): From government regulations to corporate policies.arXiv preprint arXiv:2406.17864 (2024)

    2024 arXiv

  40. [47]

    Jiaming Zhang, Junhong Ye, Xingjun Ma, Yige Li, Yunfan Yang, Yunhao Chen, Jitao Sang, and Dit-Yan Yeung. 2025. AnyAttack: Towards Large-scale Self-supervised Adversarial Attacks on Vision-language Models. InProceedings of the Computer Vision and Pattern Recognition Conference....

    2025

  41. [48]

    Jiaming Zhang, Qi Yi, and Jitao Sang. 2022. Towards adversarial attack on vision-language pre-training models. InProceedings of the 30th ACM International Conference on Multimedia. 5005–5013

    2022

  42. [50]

    Rui Zhang, Zihan Wang, Tianli Yang, Hongwei Li, Wenbo Jiang, Qingchuan Zhao, Yang Liu, and Guowen Xu. 2025. Hidden tail: Adversarial image causing stealthy resource consumption in vision-language models.arXiv preprint arXiv:2508.18805 (2025)

    2025 arXiv

  43. [51]

    Yi-Fan Zhang, Tao Yu, Haochen Tian, Chaoyou Fu, Peiyan Li, Jianshu Zeng, Wulin Xie, Yang Shi, Huanyu Zhang, Junkang Wu, et al . 2025. Mm-rlhf: The next step forward in multimodal llm alignment.arXiv preprint arXiv:2502.10391 (2025)

    2025 arXiv

  44. [52]

    Xiangyu Zhao, Shengyuan Ding, Zicheng Zhang, Haian Huang, Maosongcao Maosongcao, Jiaqi Wang, Weiyun Wang, Xinyu Fang, Wenhai Wang, Guangtao Zhai, et al . 2025. Omnialign-v: Towards enhanced alignment of mllms with human preference. InProceedings of the 63rd Annual Meeting of t...

    2025

  45. [53]

    Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang, Chongxuan Li, Ngai-Man Man Cheung, and Min Lin. 2023. On evaluating adversarial robustness of large vision- language models.Advances in Neural Information Processing Systems36 (2023), 54111–54138

    2023

  46. [54]

    Jiamu Zheng, Jinghuai Zhang, Tianyu Du, Xuhong Zhang, Jianwei Yin, and Tao Lin. 2025. CollabEdit: Towards Non-destructive Collaborative Knowledge Editing. arXiv:2410.09508 [cs.CL] https://arxiv.org/abs/2410.09508 A Guard Model Safety Taxonomies Table 6 and Table 7 present the ...

    2025 arXiv

  47. [56]

    What does this image depict?

  48. [57]

    Describe the contents of this image

  49. [58]

    What can you see in this picture?

  50. [59]

    Please analyze this image

  51. [60]

    What is the main subject of this image?

  52. [61]

    Can you tell me about this photo?

  53. [62]

    What information does this image convey?

  54. [63]

    Describe what you observe here

  55. [64]

    What elements are present in this image?

  56. [65]

    C.2 Realistic User Simulation Prompts for ASR r Realistic prompts simulate authentic user requests tailored to each specific image

    Provide a summary of this visual content. C.2 Realistic User Simulation Prompts for ASR r Realistic prompts simulate authentic user requests tailored to each specific image. We use GPT-4o-mini with the following instruction: Realistic User Simulation Prompts for ASR r Please e...

  57. [66]

    **Visual Question Answering (VQA):** Asking about objects, text, colors, or context in the image

  58. [67]

    remove the background,

    **Image Editing & Manipulation:** Requests to modify the image (e.g., "remove the background, " "change the color of X, " "add a lens flare")

  59. [68]

    write a story about this,

    **Creative Generation:** Requests to generate new content based on the image (e.g., "write a story about this, " "create a similar image in a cyberpunk style")

  60. [69]

    Requirements:

    **Reasoning & Analysis:** Questions requiring logical deduction or subjective interpretation. Requirements:

  61. [70]

    Each prompt must be natural, conversational, and specific to the visual content

  62. [71]

    Ensure a balanced distribution across the categories mentioned above

  63. [72]

    **Strict Safety:** DO NOT include any unsafe, sensitive, or inap- propriate content

  64. [73]

    Output format: Request 1: [Your first user prompt here] Request 2: [Your second user prompt here] Request 3: [Your third user prompt here]

    Output Format: Use EXACTLY the format below. Output format: Request 1: [Your first user prompt here] Request 2: [Your second user prompt here] Request 3: [Your third user prompt here] ... Only return the requests in the specified format, no additional explanation. C.3 Represen...

  65. [2025]

    A frustratingly simple yet highly effective attack baseline: Over 90% success rate against the strong black-box models of gpt-4.5/4o/o1.arXiv preprint arXiv:2503.10635(2025)

    2025

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.