Pith. sign in

REVIEW 3 major objections 3 minor 1 cited by

Red-Teaming Text-to-Image Systems by Rule-based Preference Modeling

T0 review · 3 major / 3 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that an iterative loop of prompt rewriting, coarse feedback typing, and rule-based preference fine-tuning lets a red-team agent attack unknown black-box text-to-image defenses, achieving at least twice the attack success…

desk verdict Genuinely new red-teaming mechanism with broad empirical results, but the fine-grained preference ordering is under-validated and the evaluation lacks statistical rigor; worth peer review. read the letter →

arxiv 2505.21074 v1 pith:ZWBV4WDQ submitted 2025-05-27 cs.LG cs.AIcs.CRcs.CVstat.ML

Yichuan Cao , Yibo Miao , Xiao-Shan Gao , Yinpeng Dong This is my paper · ORCID
classification cs.LGcs.AIcs.CRcs.CVstat.ML
keywords red-teamingtext-to-imagesafetyblack-boxattackpreferencemodelingdirectoptimizationCLIPembeddingdecompositionLLMagentdiffusionmodel
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Commercial text-to-image APIs keep their safety filters and alignment strategies hidden, so an auditor cannot know which defense a prompt will hit. This paper claims a red-team agent does not need that knowledge: it can learn the target system's behavior by observing only its outputs — rejection messages, generated images flagged safe, and generated images flagged unsafe. The proposed RPG-RT loops an LLM that rewrites prompts through the black-box system, converts the coarse feedback into fine-grained preference rankings using a learned decomposition of CLIP embeddings into a harmfulness score and a benign-semantics score, and then fine-tunes the LLM with direct preference optimization. Across nineteen defended systems, three commercial APIs, and a text-to-video model, the paper reports attack success rates well above existing white-box and black-box baselines while keeping rewrites semantically close and prompt-level stealthy. If correct, this makes routine safety testing of closed generative systems practical without white-box access.

What carries the argument

The load-bearing object is the learned scoring transform f=(fn, fs), a single-layer network on top of a frozen CLIP image encoder that splits each image embedding into a harmfulness scalar fn and a benign-semantics vector fs, trained with four losses (harm ranking, innocuous-semantics invariance, benign-semantics similarity alignment, and reconstruction). Paired with the three-fold feedback typing (TYPE-1, TYPE-2, TYPE-3), the score function score(P)=fn(CLIP(M(P)))+c·SCORE_Sim(M(P), refs) produces a partial order over prompt rewrites that direct preference optimization uses to fine-tune the LLM agent. This machinery converts a coarse, binary safety label into a graded preference signal, which is what lets the agent adapt without knowing the defense.

What would settle it

Take a set of NSFW images that a single T2I system produces from RPG-RT rewrites, have independent human raters rank them by expected harmfulness, and compare that order with fn's scores using a rank correlation such as Kendall's tau; if the correlation is low, the fine-grained intra-type preference is not measuring what the method assumes. A complementary check is to ablate the scoring model entirely, keeping only the coarse TYPE-1 < TYPE-3 and TYPE-2 < TYPE-3 rules, and see whether the reported ASR advantage on the online APIs narrows.

Watch

Extended reading notes

Core claim

The paper argues that black-box red-teaming of T2I systems does not require knowing the defense; the defense reveals itself through the outputs it allows and blocks. RPG-RT formalizes each query's outcome as one of three types — rejection (TYPE-1), a generated image the detector calls safe (TYPE-2), or a generated image the detector calls NSFW (TYPE-3) — and converts these into a binary partial order over prompt rewrites. To make the ordering fine-grained, a small learned transformation f=(fn, fs) of CLIP embeddings separates harmfulness (fn) from other semantics (fs); the score fn(M(P)) + c·COSSim(fs(M(P)), refs) ranks rewrites within a type. DPO then fine-tunes the LLM to prefer higher-scored rewrites, and the loop repeats. The reported conclusion is that this procedure adapts to unknown defenses and achieves attack success rates that, on the three commercial APIs, are at least twice those of the strongest baselines.

Load-bearing premise

The method assumes that its learned harmfulness score orders images that are already NSFW by true severity, but the training only enforces that safe images score lower than unsafe ones; the within-NSFW ordering is never tested, and if it is wrong the fine-grained preference signal is mostly noise.

Editorial extensions

If this is right

  • On the commercial DALL-E 3, Leonardo.ai, and SDXL APIs, RPG-RT reports attack success rates at least twice those of the best baselines, with DALL-E 3 reaching 31.33% ASR while all baselines fall below 5%.
  • The fine-tuned LLM agent transfers to previously unseen prompts without re-optimization, requiring only a single inference forward pass per prompt.
  • The same framework generalizes across detection-based, removal-based, safety-aligned, and stacked defenses, and also to text-to-video generation.
  • Modified prompts achieve the lowest perplexity among compared methods, indicating that the rewrites are inconspicuous while preserving semantic similarity to the original images.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same three-type feedback loop (rejection, safe image, unsafe image) could extend to other closed generative services, such as audio or 3D model generators, wherever a coarse safety label is visible.
  • The trained fn/fs decomposition is itself a transferable artifact: it provides a graded per-image harmfulness score that could be reused as a reward signal in other LLM-agent optimization loops, or as an audit tool to characterize whether a defense is text-based or image-based.
  • The paper's fine-grained advantage rests on an untested monotonicity assumption on fn within the NSFW class; if that assumption fails, the reported gains might derive from the coarse inter-type rules alone, and the scoring model's contribution could be checked by ablating it entirely.
  • The high ASR on commercial APIs suggests that current safety filters are relatively easy to bypass with adaptive prompt rewriting, which may motivate more output-side and image-side filtering rather than prompt-side blocking.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The paper proposes RPG-RT, an iterative black-box red-teaming framework for text-to-image (T2I) systems. An unaligned LLM modifies prompts, the target T2I system responds with images or rejections, and feedback is converted into pairwise preferences. A learned CLIP-based scoring model decomposes image embeddings into an NSFW score component and a benign semantic component, and DPO is used to fine-tune the LLM on rule-based preferences (TYPE-3 over TYPE-1/TYPE-2, plus within-type ordering by the learned score). Experiments are reported on nineteen T2I systems, three online APIs, and one text-to-video model, with attack success rate (ASR), semantic similarity, FID, and perplexity metrics.

Significance. If the results hold, the framework is practically valuable: it treats closed systems as black boxes, adapts to unknown stacked defenses, and requires only a single LLM forward pass when transferring to unseen prompts. The paper's strengths include the breadth of evaluated defenses, a transfer experiment to unseen prompts (Table 3), online API results (Table 4), a text-to-video extension (Table 10), computational-cost analysis (Table 13), and ablations of the scoring-model losses (Table 8). However, the paper's central conceptual contribution, fine-grained rule-based preference modeling, rests on an untested monotonicity assumption about the NSFW score, and the empirical evaluation lacks uncertainty quantification. The contribution is plausible but not yet fully supported.

major comments (3)
  1. [§2.4, Eq. (1); §2.5, Eq. (7)] The within-TYPE-3 preference rule uses score(P_i) = f_n(CLIP(M(P_i))) + c·SCORE_Sim as an ordinal measure of NSFW severity. The training objective L_harm in Eq. (1) only enforces f_n(X^S_i) < f_n(X^N_i) for paired SFW/NSFW images; it does not constrain the relative ordering of f_n among different NSFW images. Since f is a single linear layer followed by a sigmoid (Sec. B.2), any linear functional that separates the paired SFW/NSFW sets satisfies Eq. (1), and the induced order among TYPE-3 images may be an artifact of initialization and optimization rather than a learned severity ranking. Appendix D reports an F1 of 0.9118 computed by thresholding f_n against binary detector labels, which validates binary discrimination, not within-NSFW monotonicity. Consequently, the fine-grained preferences injected into DPO may be noisy, and the claimed advantage of rule-based scoring over coarse TYPE-3 > TYPE-1/TYPE-2 rules is unsupported. Please add validation of within-NSFW ordinal quality (e.g., rank correlation with human severity ratings) or an ablation that replaces the fine-grained ordering with random or coarse ordering to show that score(P_i) is actually load-bearing.
  2. [§3.1–§3.5, Tables 1 and 4] All ASR values are point estimates based on 30 stochastic generations per prompt, but the paper reports no confidence intervals, standard deviations, or significance tests. In Table 4, the online API results are based on only 10 prompts, so the headline claim that RPG-RT attains at least twice the ASR of other methods has large uncertainty: a single prompt's outcome changes ASR by 10 percentage points. Please report per-prompt breakdowns, bootstrap confidence intervals, or repeated API runs on different days and with different random seeds to support the phrase 'significantly higher' in the Introduction.
  3. [§3.2 and §3.4, Tables 1 and 3] The main results in Table 1 are reported on the same set of 95 prompts that the LLM agent is fine-tuned on over 10 iterations; only Table 3 reports held-out prompts. As a result, the high ASRs partly measure adaptation to the training prompts rather than general red-teaming ability. The transfer experiment narrows this gap but covers only three systems and a restricted nudity range (30–50%). Please report the main evaluation on a held-out prompt split for all 19 systems, or explicitly restate the central claim as effective on the adaptively attacked prompt set.
minor comments (3)
  1. [Throughout] There are several typos and infelicities: 'noval' in the Conclusion should be 'novel'; 'utlize' appears in §3.1 and Appendix C.1; the Fig. 1 caption contains 'blaomk-box'; and 'FLRIT' appears in Appendix C.1.
  2. [§2.4, Eq. (3)] The target COS Sim(X^S_i, X^S_j) is a global CLIP similarity, while the decoupling motivation says global CLIP similarity confuses harmful and benign semantics; please clarify why aligning f_s to global CLIP similarity on SFW pairs is consistent with the decoupling objective.
  3. [§3.5 and Appendix H] The description of the online API experiment does not state how the 10 prompts were selected or whether API responses varied over time; please add this information. Also, ASR-30 in Table 14 is only defined in the appendix; consider defining it in the main text.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the feedback loop is the intended attack objective, and the only flagged issue is an unvalidated severity-ordering assumption in the scoring model, not a circular reduction.

full rationale

The paper's derivation chain is an empirical attack pipeline rather than a formal derivation, and I find no step in which an output is equivalent to an input by construction. The feedback loop (querying the target, reading rejection/detector labels, constructing TYPE-1/2/3 preferences, fine-tuning the LLM with DPO, and measuring ASR with the same detector) is the intended attack objective: optimizing against the evaluation detector is what a red-team method is supposed to do, and the baselines are compared under the same metric. The scoring model in Sec. 2.4 is trained on detector-labeled SFW/NSFW pairs via Lharm, and Eq. (7) then uses fn to score modified prompts; this introduces an unvalidated assumption that fn orders images monotonically by severity within the NSFW class, since Lharm only enforces cross-pair inequalities. That is a correctness/validity gap (Appendix D's F1 with per-prompt Youden thresholds validates binary separation, not within-class severity), but it is not a circular reduction: the reported ASR gains do not logically require the within-TYPE-3 ordering to be correct, and the transfer-to-unseen-prompts (Table 3) and online-API (Table 4) experiments provide out-of-sample checks that are not forced by the training objective. The self-citations in Related Work A.1 are survey-style references to the authors' prior adversarial-machine-learning papers and are not load-bearing; no uniqueness theorem or ansatz is imported via citation. Accordingly, there is no significant circularity, and the score is 0.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the existence of a reference undefended model for semantic similarity, an external NSFW detector for labels, and the decomposability of CLIP representations; these are stated assumptions, not derived. The only numerical hyperparameter fit to data is c=2.0. The scoring model f is trained from query data rather than postulated a priori, so it is a learned component rather than a free constant.

free parameters (1)
  • c = 2.0
    Weight balancing NSFW score and semantic similarity in score(P_i) (Eq. 7); chosen by ablation on the test systems (Appendix C.4, Table 9), and it directly changes which modifications are preferred during DPO training.
assumptions (5)
  • domain assumption There exists an undefended reference T2I model M0 that can generate harmful images from the original prompt, and reference images from M0 serve as semantic ground truth.
    Invoked in Sec 2.1 to define the similarity objective Sim(M(A(P)), M0(P)) and used in Eq. 6 to compute SCORE_Sim against reference images.
  • domain assumption The external detector D accurately labels NSFW content and is a valid ground truth for both preference construction and ASR evaluation.
    Used throughout Sec 2.3 and Sec 3.1 to classify TYPE-2 vs TYPE-3 feedback and to compute attack success rates.
  • ad hoc to paper CLIP embeddings can be decomposed by a single-layer transformation f=(fn, fs) into an independent harmful component and a benign semantic component.
    Introduced in Sec 2.4 and Fig. 2b as the key insight for the scoring model; the four losses in Eqs. 1-4 presuppose this separability and faithful invertibility.
  • domain assumption Direct preference optimization with LoRA can fine-tune an unaligned LLM (Vicuna-7B) to improve prompt modification behavior from pairwise preference data.
    Assumed in Sec 2.6; the paper provides empirical loss curves but no derivation or ablation over base LLM choices.
  • domain assumption The unaligned Vicuna-7B model will follow the red-team prompt template and produce diverse modified prompts without safety refusal.
    Prompt template in Appendix B.1 instructs the model to disregard ethical constraints; this is a behavioral assumption about an existing model, not a proved property.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Red-Teaming Text-to-Image Systems by Rule-based Preference Modeling." pith.science (2026). https://pith.science/paper/ZWBV4WDQ

@misc{pith2026250521074,
  author       = {Pith},
  title        = {Pith review of: Red-Teaming Text-to-Image Systems by Rule-based Preference Modeling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZWBV4WDQ}},
  note         = {Machine review of arXiv:2505.21074}
}
read the original abstract

Text-to-image (T2I) models raise ethical and safety concerns due to their potential to generate inappropriate or harmful images. Evaluating these models' security through red-teaming is vital, yet white-box approaches are limited by their need for internal access, complicating their use with closed-source models. Moreover, existing black-box methods often assume knowledge about the model's specific defense mechanisms, limiting their utility in real-world commercial API scenarios. A significant challenge is how to evade unknown and diverse defense mechanisms. To overcome this difficulty, we propose a novel Rule-based Preference modeling Guided Red-Teaming (RPG-RT), which iteratively employs LLM to modify prompts to query and leverages feedback from T2I systems for fine-tuning the LLM. RPG-RT treats the feedback from each iteration as a prior, enabling the LLM to dynamically adapt to unknown defense mechanisms. Given that the feedback is often labeled and coarse-grained, making it difficult to utilize directly, we further propose rule-based preference modeling, which employs a set of rules to evaluate desired or undesired feedback, facilitating finer-grained control over the LLM's dynamic adaptation process. Extensive experiments on nineteen T2I systems with varied safety mechanisms, three online commercial API services, and T2V models verify the superiority and practicality of our approach.

Figures

Figures reproduced from arXiv: 2505.21074 by the authors.

Figure 1
Figure 1. Overview of our RPG-RT framework. a) Stage 1: The LLM generates multiple different modifications of the prompt, then inputs them into the target T2I blaomk-box system and obtains the outputs. b) Stage 2: A binary partial order is constructed to model the preferences of the T2I system. Rule-based scoring is utilized to enable fine-grained control over the LLM’s exploration of the commercial black-box system. c) Stage… view at source ↗
Figure 2
Figure 2. Overview of our scoring model. a): Motivation: the presence of harmful or semantically identical non-harmful semantics can lead to a high CLIP similarity between two images, causing confusion that cannot be resolved by a straightforward CLIP similarity measure. b): Our key insight is to decouple the CLIP representation using a transformation f = (fn, fs), where fn captures harmful content, and fs captures other inno… view at source ↗
Figure 3
Figure 3. Qualitative visualization results of baselines and our RPG-RT. Our RPG-RT can [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Full qualitative visualization results of baselines and our RPG-RT in generating images with nudity [PITH_FULL_IMAGE:figures/full_fig_p018_4.png]
Figure 5
Figure 5. Figure 5: Full qualitative visualization results of baselines and our RPG-RT across various NSFW categories. [PITH_FULL_IMAGE:figures/full_fig_p020_5.png]
Figure 6
Figure 6. Figure 6: Examples of successful and failed modifications by RPG-RT against different defenses. [PITH_FULL_IMAGE:figures/full_fig_p022_6.png]
Figure 7
Figure 7. Figure 7: Loss curves for DPO training of the LLM and the training of the scoring model. [PITH_FULL_IMAGE:figures/full_fig_p023_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Dynamic Defense Profiling Enables Cognitive Jailbreak of Text-to-Image Models

    cs.AI 2026-07 conditional novelty 6.0 of 10

    MIND learns a 'defense profile' of a T2I model from fine-grained feedback, then uses it to guide an evolutionary search, achieving 95.62% ASR across six defenses and 91.58% on Wan-2.5.

Reference graph

Works this paper leans on

69 extracted references · 44 canonical work pages · cited by 1 Pith paper

  1. [1]

    Celebrity classifier, 2024

    Tony Assi. Celebrity classifier, 2024

    work page 2024

  2. [2]

    Surrogateprompt: Bypassing the safety filter of text-to-image models via substitution

    Zhongjie Ba, Jieming Zhong, Jiachen Lei, Peng Cheng, Qinglong Wang, Zhan Qin, Zhibo Wang, and Kui Ren. Surrogateprompt: Bypassing the safety filter of text-to-image models via substitution. InProceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, pages 1166–1180, 2024

    work page 2024

  3. [3]

    Rank analysis of incomplete block designs: I

    Ralph Allan Bradley and Milton E Terry. Rank analysis of incomplete block designs: I. the method of paired comparisons.Biometrika, 39(3/4):324–345, 1952

    1952

  4. [4]

    Towards evaluating the robustness of neural networks

    Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In2017 IEEE symposium on security and privacy (SP), pages 39–57. IEEE, 2017

    work page 2017

  5. [5]

    Efficient black-box adversarial attacks via bayesian optimization guided by a function prior

    Shuyu Cheng, Yibo Miao, Yinpeng Dong, Xiao Yang, Xiao-Shan Gao, and Jun Zhu. Efficient black-box adversarial attacks via bayesian optimization guided by a function prior. InInternational Conference on Machine Learning, pages 8163–8183, 2024

    work page 2024

  6. [6]

    Nsfw classifier, 2020

    Lakshay Chhabra. Nsfw classifier, 2020

    work page 2020

  7. [7]

    Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality.See https://vicuna

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality.See https://vicuna. lmsys. org (accessed 14 April 2023), 2(3):6, 2023

    2023

  8. [8]

    Prompt- ing4debugging: Red-teaming text-to-image diffusion models by finding problematic prompts

    Zhi-Yi Chin, Chieh Ming Jiang, Ching-Chun Huang, Pin-Yu Chen, and Wei-Chen Chiu. Prompt- ing4debugging: Red-teaming text-to-image diffusion models by finding problematic prompts. InIn- ternational Conference on Learning Representations, 2024

    work page 2024

Show all 69 references
  1. [9]

    Ai, governance and ethics: Global perspectives.Deakin University, 2022

    Angela Daly, Thilo Hagendorff, Li Hui, Monique Mann, Vidushi Marda, Ben Wagner, and Wayne Wei Wang. Ai, governance and ethics: Global perspectives.Deakin University, 2022

    2022

  2. [10]

    DiffZOO: A purely query-based black-box attack for red-teaming text-to-image generative model via zeroth order optimization

    Pucheng Dang, Xing Hu, Dong Li, Rui Zhang, Qi Guo, and Kaidi Xu. DiffZOO: A purely query-based black-box attack for red-teaming text-to-image generative model via zeroth order optimization. In Luis Chiruzzo, Alan Ritter, and Lu Wang, editors,Findings of the Association for Com...

    2025

  3. [11]

    Divide-and-conquer attack: Harnessing the power of llm to bypass the censorship of text-to-image generation model.arXiv preprint arXiv:2312.07130, 2023

    Yimo Deng and Huangxun Chen. Divide-and-conquer attack: Harnessing the power of llm to bypass the censorship of text-to-image generation model.arXiv preprint arXiv:2312.07130, 2023

    2023 arXiv

  4. [12]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. InAdvances in Neural Information Processing Systems, volume 34, pages 8780–8794, 2021

    2021

  5. [13]

    Fuzz-testing meets llm-based agents: An automated and efficient framework for jailbreaking text-to-image generation models

    Yingkai Dong, Xiangtao Meng, Ning Yu, Zheng Li, and Shanqing Guo. Fuzz-testing meets llm-based agents: An automated and efficient framework for jailbreaking text-to-image generation models. In2025 IEEE Symposium on Security and Privacy (SP), pages 336–354. IEEE Computer Society, 2025

    2025

  6. [14]

    Boosting adversarial attacks with momentum

    Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boosting adversarial attacks with momentum. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9185–9193, 2018

    2018

  7. [15]

    Scaling rectified flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. InForty-first International Conference on Machine L...

    2024

  8. [16]

    Erasing concepts from diffusion models

    Rohit Gandikota, Joanna Materzynska, Jaden Fiotto-Kaufman, and David Bau. Erasing concepts from diffusion models. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 2426–2436, 2023. 10

    2023

  9. [17]

    Rt-attack: Jailbreaking text-to-image models via random token.arXiv preprint arXiv:2408.13896, 2024

    Sensen Gao, Xiaojun Jia, Yihao Huang, Ranjie Duan, Jindong Gu, Yang Liu, and Qing Guo. Rt-attack: Jailbreaking text-to-image models via random token.arXiv preprint arXiv:2408.13896, 2024

    2024 arXiv

  10. [18]

    Nsfw words list, 2020

    Rojit George. Nsfw words list, 2020

    2020

  11. [19]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. InInternational Conference on Learning Representations, 2022

    2022

  12. [20]

    Safetext: Safe text-to-image models via aligning the text encoder.arXiv preprint arXiv:2502.20623, 2025

    Yuepeng Hu, Zhengyuan Jiang, and Neil Zhenqiang Gong. Safetext: Safe text-to-image models via aligning the text encoder.arXiv preprint arXiv:2502.20623, 2025

    2025 arXiv

  13. [21]

    Perception-guided jailbreak against text-to-image models

    Yihao Huang, Le Liang, Tianlin Li, Xiaojun Jia, Run Wang, Weikai Miao, Geguang Pu, and Yang Liu. Perception-guided jailbreak against text-to-image models. InProceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 26238–26247, 2025

    2025

  14. [22]

    Safety alignment backfires: Preventing the re-emergence of suppressed concepts in fine-tuned text-to-image diffusion models.arXiv preprint arXiv:2412.00357, 2024

    Sanghyun Kim, Moonseok Choi, Jinwoo Shin, and Juho Lee. Safety alignment backfires: Preventing the re-emergence of suppressed concepts in fine-tuned text-to-image diffusion models.arXiv preprint arXiv:2412.00357, 2024

    2024 arXiv

  15. [23]

    Adam: A method for stochastic optimization

    Diederik P Kingma. Adam: A method for stochastic optimization. InInternational Conference on Learning Representations, 2015

    2015

  16. [24]

    Ablating concepts in text-to-image diffusion models

    Nupur Kumari, Bingliang Zhang, Sheng-Yu Wang, Eli Shechtman, Richard Zhang, and Jun-Yan Zhu. Ablating concepts in text-to-image diffusion models. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 22691–22702, 2023

    2023

  17. [25]

    Clip-based nsfw detector, 2023

    LAION-AI. Clip-based nsfw detector, 2023

    2023

  18. [26]

    Leonardo.ai, 2022

    Leonardo.ai. Leonardo.ai, 2022

    2022

  19. [27]

    ART: Automatic red-teaming for text-to-image models to protect benign users

    Guanlin Li, Kangjie Chen, Shudong Zhang, Jie Zhang, and Tianwei Zhang. ART: Automatic red-teaming for text-to-image models to protect benign users. InAdvances in Neural Information Processing Systems, 2024

    2024

  20. [28]

    Nsfw text classifier, 2022

    Michelle Li. Nsfw text classifier, 2022

    2022

  21. [29]

    Safegen: Mitigating sexually explicit content generation in text-to-image models

    Xinfeng Li, Yuchen Yang, Jiangyi Deng, Chen Yan, Yanjiao Chen, Xiaoyu Ji, and Wenyuan Xu. Safegen: Mitigating sexually explicit content generation in text-to-image models. InProceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, pages 4807–4821, 2024

    2024

  22. [30]

    Token-level constraint boundary search for jailbreaking text-to-image models.arXiv preprint arXiv:2504.11106, 2025

    Jiangtao Liu, Zhaoxin Wang, Handing Wang, Cong Tian, and Yaochu Jin. Token-level constraint boundary search for jailbreaking text-to-image models.arXiv preprint arXiv:2504.11106, 2025

    2025

  23. [31]

    Jailbreaking the text-to-video generative models.arXiv preprint arXiv:2505.06679, 2025

    Jiayang Liu, Siyuan Liang, Shiqian Zhao, Rongcheng Tu, Wenbo Zhou, Xiaochun Cao, Dacheng Tao, and Siew Kei Lam. Jailbreaking the text-to-video generative models.arXiv preprint arXiv:2505.06679, 2025

    2025 arXiv

  24. [32]

    Safetydpo: Scalable safety alignment for text-to-image generation.arXiv preprint arXiv:2412.10493, 2024

    Runtao Liu, Chen I Chieh, Jindong Gu, Jipeng Zhang, Renjie Pi, Qifeng Chen, Philip Torr, Ashkan Khakzar, and Fabio Pizzati. Safetydpo: Scalable safety alignment for text-to-image generation.arXiv preprint arXiv:2412.10493, 2024

    2024 arXiv

  25. [33]

    Jailbreaking prompt attack: A controllable adversarial attack against diffusion models.arXiv preprint arXiv:2404.02928, 2024

    Jiachen Ma, Anda Cao, Zhiqing Xiao, Yijiang Li, Jie Zhang, Chao Ye, and Junbo Zhao. Jailbreaking prompt attack: A controllable adversarial attack against diffusion models.arXiv preprint arXiv:2404.02928, 2024

    2024 arXiv

  26. [34]

    FLIRT: Feedback loop in-context red teaming

    Ninareh Mehrabi, Palash Goyal, Christophe Dupuy, Qian Hu, Shalini Ghosh, Richard Zemel, Kai-Wei Chang, Aram Galstyan, and Rahul Gupta. FLIRT: Feedback loop in-context red teaming. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 7...

    2024

  27. [35]

    Improving robustness of 3d point cloud recognition from a fourier perspective.Advances in Neural Information Processing Systems, 37:68183–68210, 2024

    Yibo Miao, Yinpeng Dong, Jinlai Zhang, Lijia Yu, Xiao Yang, and Xiao-Shan Gao. Improving robustness of 3d point cloud recognition from a fourier perspective.Advances in Neural Information Processing Systems, 37:68183–68210, 2024

    2024

  28. [36]

    Isometric 3d adversarial examples in the physical world

    Yibo Miao, Yinpeng Dong, Jun Zhu, and Xiao-Shan Gao. Isometric 3d adversarial examples in the physical world. InAdvances in Neural Information Processing Systems, volume 35, pages 19716–19731, 2022

    2022

  29. [37]

    T2vsafetybench: Evaluating the safety of text-to-video generative models

    Yibo Miao, Yifan Zhu, Lijia Yu, Jun Zhu, Xiao-Shan Gao, and Yinpeng Dong. T2vsafetybench: Evaluating the safety of text-to-video generative models. InAdvances in Neural Information Processing Systems, volume 37, pages 63858–63872, 2024. 11

    2024

  30. [38]

    Midjourney, 2022

    Midjourney. Midjourney, 2022

    2022

  31. [39]

    Google chief admits ‘biased’ ai tool’s photo diversity offended users, 2024

    Dan Milmo and Alex Hern. Google chief admits ‘biased’ ai tool’s photo diversity offended users, 2024

    2024

  32. [40]

    Simple open- vocabulary object detection

    Matthias Minderer, Alexey Gritsenko, Austin Stone, Maxim Neumann, Dirk Weissenborn, Alexey Doso- vitskiy, Aravindh Mahendran, Anurag Arnab, Mostafa Dehghani, Zhuoran Shen, et al. Simple open- vocabulary object detection. InEuropean Conference on Computer Vision, pages 728–755....

    2022

  33. [41]

    Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741, 2021

    Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741, 2021

    2021 arXiv

  34. [42]

    Nudenet, 2024

    notAI.tech. Nudenet, 2024

    2024

  35. [43]

    Dall-e 3, 2023

    OpenAI. Dall-e 3, 2023

    2023

  36. [44]

    Gpt-4o, 2024

    OpenAI. Gpt-4o, 2024

    2024

  37. [45]

    Direct unlearning optimization for robust and safe text-to-image models

    Yong-Hyun Park, Sangdoo Yun, Jin-Hwa Kim, Junho Kim, Geonhui Jang, Yonghyun Jeong, Junghyo Jo, and Gayoung Lee. Direct unlearning optimization for robust and safe text-to-image models. InAdvances in Neural Information Processing Systems, 2024

    2024

  38. [46]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. In International Conference on Learning Representations, 2024

    2024

  39. [47]

    Unsafe diffusion: On the generation of unsafe images and hateful memes from text-to-image models

    Yiting Qu, Xinyue Shen, Xinlei He, Michael Backes, Savvas Zannettou, and Yang Zhang. Unsafe diffusion: On the generation of unsafe images and hateful memes from text-to-image models. InProceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security, pages...

    2023

  40. [48]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. InInternational conference on machine learning, pag...

    2021

  41. [49]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. InAdvances in Neural Information Processing Systems, volume 36, pages 53728–53741, 2023

    2023

  42. [50]

    Red-teaming the stable diffusion safety filter.arXiv preprint arXiv:2210.04610, 2022

    Javier Rando, Daniel Paleka, David Lindner, Lennart Heim, and Florian Tramèr. Red-teaming the stable diffusion safety filter.arXiv preprint arXiv:2210.04610, 2022

    2022 arXiv

  43. [51]

    Classification algorithm for skin color (casco): A new tool to measure skin color in social science research.Social Science Quarterly, 104(2):168–179, 2023

    René Alejandro Rejón Piña and Chenglong Ma. Classification algorithm for skin color (casco): A new tool to measure skin color in social science research.Social Science Quarterly, 104(2):168–179, 2023

    2023

  44. [52]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022

    2022

  45. [53]

    Safe latent diffusion: Mitigating inappropriate degeneration in diffusion models

    Patrick Schramowski, Manuel Brack, Björn Deiseroth, and Kristian Kersting. Safe latent diffusion: Mitigating inappropriate degeneration in diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22522–22531, 2023

    2023

  46. [54]

    Patrick Schramowski, Christopher Tauchmann, and Kristian Kersting. Can machines help us answering question 16 in datasheets, and in turn reflecting on inappropriate content? InProceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, pages 1350–1361, 2022

    2022

  47. [55]

    Intriguing properties of neural networks.arXiv preprint arXiv:1312.6199, 2013

    Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks.arXiv preprint arXiv:1312.6199, 2013

    2013 arXiv

  48. [56]

    Ring-a-bell! how reliable are concept removal methods for diffusion models? In International Conference on Learning Representations, 2024

    Yu-Lin Tsai, Chia-yi Hsu, Chulin Xie, Chih-hsun Lin, Jia You Chen, Bo Li, Pin-Yu Chen, Chia-Mu Yu, and Chun-ying Huang. Ring-a-bell! how reliable are concept removal methods for diffusion models? In International Conference on Learning Representations, 2024

    2024

  49. [57]

    The stronger the diffusion model, the easier the backdoor: Data poisoning to induce copyright breacheswithout adjusting finetuning pipeline

    Haonan Wang, Qianli Shen, Yao Tong, Yang Zhang, and Kenji Kawaguchi. The stronger the diffusion model, the easier the backdoor: Data poisoning to induce copyright breacheswithout adjusting finetuning pipeline. InForty-first International Conference on Machine Learning, 2024. 12

    2024

  50. [58]

    Mma-diffusion: Multimodal attack on diffusion models

    Yijun Yang, Ruiyuan Gao, Xiaosen Wang, Tsung-Yi Ho, Nan Xu, and Qiang Xu. Mma-diffusion: Multimodal attack on diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7737–7746, 2024

    2024

  51. [59]

    Guardt2i: Defending text-to-image models from adversarial prompts

    Yijun Yang, Ruiyuan Gao, Xiao Yang, Jianyuan Zhong, and Qiang Xu. Guardt2i: Defending text-to-image models from adversarial prompts. InAdvances in neural information processing systems, 2024

    2024

  52. [60]

    Sneakyprompt: Jailbreaking text-to-image generative models

    Yuchen Yang, Bo Hui, Haolin Yuan, Neil Gong, and Yinzhi Cao. Sneakyprompt: Jailbreaking text-to-image generative models. In2024 IEEE symposium on security and privacy (SP), pages 897–912. IEEE, 2024

    2024

  53. [61]

    Safree: Training-free and adaptive guard for safe text-to-image and video generation

    Jaehong Yoon, Shoubin Yu, Vaidehi Patil, Huaxiu Yao, and Mohit Bansal. Safree: Training-free and adaptive guard for safe text-to-image and video generation. InInternational Conference on Learning Representations, 2025

    2025

  54. [62]

    Generalization bound and new algorithm for clean-label backdoor attack

    Lijia Yu, Shuang Liu, Yibo Miao, Xiao-Shan Gao, and Lijun Zhang. Generalization bound and new algorithm for clean-label backdoor attack. InInternational Conference on Machine Learning, pages 57559–57596, 2024

    2024

  55. [63]

    Reason2attack: Jailbreaking text-to-image models via llm reasoning.arXiv preprint arXiv:2503.17987, 2025

    Chenyu Zhang, Lanjun Wang, Yiwen Ma, Wenhui Li, and An-An Liu. Reason2attack: Jailbreaking text-to-image models via llm reasoning.arXiv preprint arXiv:2503.17987, 2025

    2025

  56. [64]

    Forget-me-not: Learning to forget in text-to-image diffusion models

    Gong Zhang, Kai Wang, Xingqian Xu, Zhangyang Wang, and Humphrey Shi. Forget-me-not: Learning to forget in text-to-image diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1755–1764, 2024

    2024

  57. [65]

    Defensive unlearning with adversarial training for robust concept erasure in diffusion models

    Yimeng Zhang, Xin Chen, Jinghan Jia, Yihua Zhang, Chongyu Fan, Jiancheng Liu, Mingyi Hong, Ke Ding, and Sijia Liu. Defensive unlearning with adversarial training for robust concept erasure in diffusion models. InAdvances in neural information processing systems, 2024

    2024

  58. [66]

    To generate or not? safety-driven unlearned diffusion models are still easy to generate unsafe images

    Yimeng Zhang, Jinghan Jia, Xin Chen, Aochuan Chen, Yihua Zhang, Jiancheng Liu, Ke Ding, and Sijia Liu. To generate or not? safety-driven unlearned diffusion models are still easy to generate unsafe images... for now. InEuropean Conference on Computer Vision, pages 385–403. Spr...

    2025

  59. [67]

    Open-sora: Democratizing efficient video production for all.arXiv preprint arXiv:2412.20404, 2024

    Zangwei Zheng, Xiangyu Peng, Tianji Yang, Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. Open-sora: Democratizing efficient video production for all.arXiv preprint arXiv:2412.20404, 2024

    2024 arXiv

  60. [68]

    Toward availability attacks in 3d point clouds

    Yifan Zhu, Yibo Miao, Yinpeng Dong, and Xiao-Shan Gao. Toward availability attacks in 3d point clouds. InInternational Conference on Machine Learning, pages 62510–62530, 2024

    2024

  61. [69]

    A pilot study of query-free adversarial attack against stable diffusion

    Haomin Zhuang, Yihua Zhang, and Sijia Liu. A pilot study of query-free adversarial attack against stable diffusion. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2385–2392, 2023. 13 A Related Work A.1 Jailbreak on Text-to-Image Mode...

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.