Pith. sign in

REVIEW 4 major objections 7 minor 49 references

Structure Disruption: Subverting Malicious Diffusion-Based Inpainting via Self-Attention Query Perturbation

T0 review · 4 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Perturbing self-attention queries in the first denoising step makes diffusion inpainting collapse into incoherent output.

desk verdict Specific, cheap attack idea that is currently undermined by a seed-overfitting risk in the evaluation. read the letter →

arxiv 2505.19425 v1 pith:EGN6R323 submitted 2025-05-26 cs.CV cs.CRcs.LG

Yuhao He , Jinyu Tian , Haiwei Wu , Jianqing Li This is my paper · ORCID
classification cs.CVcs.CRcs.LG
keywords adversarialperturbationdiffusioninpaintingself-attentionimageprotectionprivacydefenseStablemask-guidededitingstructuredisruption
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that diffusion-based inpainting can be reliably blocked by attacking a single, early part of the model: the self-attention queries in the first denoising step. The proposed protection, Structure Disruption Attack (SDA), adds a small perturbation to the image that maximizes the $\ell^2$ distance between the original and perturbed self-attention queries at that step. Because early denoising establishes object contours, this interference prevents the model from ever forming a coherent structure, and the failure cascades into misaligned text conditioning and incomplete generation. On face and instance datasets, SDA reports stronger protection than existing inpainting-specific defenses, along with robustness to augmentations, unseen masks, and transfer across Stable Diffusion checkpoints. The paper matters because it points to self-attention, not the loss or cross-attention, as the most efficient lever for protecting images from malicious editing.

What carries the argument

The load-bearing object is the self-attention query $Q$ inside the U-Net of a latent diffusion model, transformed by $\phi$ (latent mapping) and $q$ (linear projection). The paper's central identity is the adversarial objective of Eq. 2: maximize $\sum_l \|\hat{Q}^l_s - Q^l_s\|$ with $\hat{Q}^l_s = q^l_s(\phi^l_T(I+\delta))$ and $Q^l_s = q^l_s(\phi^l_T(I))$, under $\|\delta\|_2\le\eta$. This objective encodes the paper's causal story: early denoising steps set contours, self-attention governs contours, so pushing the queries apart at step $T$ removes the model's structural anchor and triggers a cascade failure. Because only the initial step is backpropagated, the attack costs about 1.5 minutes per image versus roughly 2.5–3 minutes for full-chain baselines.

What would settle it

Take a fixed SDA perturbation and ablate the mechanism: after optimizing $\delta$ on the initial step's self-attention queries, re-run the inpainting forward pass with those query differences randomly scrambled or zeroed while keeping the perturbation applied to the input; if the output still fails to form a coherent image, the collapse is not caused by query disruption. A second check is to compare SDA against same-budget high-frequency noise: if noise alone reproduces the degradation, the specific self-attention targeting is not necessary.

Watch

Extended reading notes

Core claim

The central claim is that disrupting self-attention queries during the initial denoising step is sufficient to collapse the entire generative process of an inpainting diffusion model. The authors define the perturbation objective as $\delta = \arg\max_{\|\delta\|_2\le\eta}\sum_l \|\hat{Q}^l_s - Q^l_s\|$, where $Q^l_s$ and $\hat{Q}^l_s$ are self-attention queries of the original and perturbed image in the $l$-th U-Net layer at the initial step $T$. Optimizing this objective makes the model lose its attention to object contours at early timesteps and, as a cascade, breaks cross-attention alignment with the text prompt; the final inpainted region is decoupled from the protected content rather than coherently merged. The paper supports this by visualizing attention maps during generation and by quantitative comparisons against Photoguard, DiffusionGuard, and DDD, reporting the best VIF, SSIM, PSNR, FID, LPIPS, CLIP Score, and PIQE on the face and instance datasets. It also shows the same perturbation remains effective when masks change, when images are augmented, and when the attack transfers to a different Stable Diffusion version.

Load-bearing premise

The load-bearing premise is that maximizing the discrepancy of self-attention queries at the first denoising step is what actually causes the generation collapse, but the paper supports this mainly with attention-map visualizations rather than a causal ablation that rules out the perturbation acting as generic high-frequency noise.

Editorial extensions

If this is right

  • SDA protects sensitive image regions against inpainting with arbitrary malicious prompts, not only against attacks that use the same mask as the optimization.
  • The protective perturbation transfers across Stable Diffusion checkpoints, working in a black-box setting on v1.5 after being optimized on v2.0.
  • The protection remains effective under common image corruptions, including Gaussian noise, resizing-based cropping, and JPEG compression.
  • Because only the initial denoising step is optimized, SDA is faster than full-chain attacks: about 1 minute 29 seconds per image versus 2 minutes 52 seconds for DiffusionAttack and 2 minutes 33 seconds for DDD.
  • SDA maintains strong protection under unseen masks produced by mask augmentation, a scenario where some baselines such as DDD fail completely.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the query-disruption mechanism is causal, the same initial-step perturbation recipe could plausibly extend beyond inpainting to other attention-based editing pipelines, though the paper explicitly leaves instruction-driven editing methods untested.
  • The paper's success suggests that monitoring self-attention query coherence at early denoising steps could serve as a cheap detector of adversarial protection, a defensive countermeasure the paper does not discuss.
  • A testable extension would be to optimize the same objective at randomly selected later denoising steps; if those also collapse generation, the claimed special status of the initial step would be weakened.
  • The comparison against same-budget high-frequency noise would clarify whether the effect is truly due to query disruption or to generic perturbation energy, an experiment the paper does not report.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes Structure Disruption Attack (SDA), a protection method against malicious diffusion-based inpainting. SDA optimizes an L2-bounded perturbation by maximizing the discrepancy between self-attention query vectors of the original and perturbed images at the initial denoising step (Eq. 2), with the rationale that early-step self-attention controls contours and structural coherence. Experiments on face and COCO instance datasets compare against Photoguard, DiffusionGuard, and DDD under fixed image-mask-prompt-seed pairs and report improvements across VIF, SSIM, PSNR, FID, LPIPS, CLIP Score, and PIQE, plus robustness to augmentations, transfer to a different Stable Diffusion version, and unseen masks. The authors claim state-of-the-art protection performance and strong practical robustness.

Significance. If confirmed, SDA is a significant contribution: it is an efficient protection method that targets a single denoising step, avoiding full-chain backpropagation; it shows consistent gains over four published baselines on two datasets; and it demonstrates transfer to a different checkpoint and robustness to common augmentations and mask changes. The paper also makes an interesting empirical observation that early self-attention disruption cascades into cross-attention and text-alignment failure. However, the experimental design does not currently rule out seed overfitting, omits the closest baseline AdvPaint, and reports no variance or significance testing. These gaps must be closed before the state-of-the-art claim is fully supported.

major comments (4)
  1. [Section 4.1 and Section 4.2] The evaluation protocol assigns a unique fixed random seed to each image-mask pair and maintains a strict one-to-one correspondence between image, mask, prompt, and random seed. Because Eq. (2) optimizes perturbations through the latent at the initial denoising step, and that latent includes the masked-region noise determined by the seed, the reported protection metrics may be measured under exactly the noise realization used during optimization. The paper does not state whether optimization and evaluation share the same seed, and it reports no experiment that varies the inpainting seed. Please clarify the protocol and add a seed-variation experiment; without this, the claimed robustness of SDA to real-world attackers who choose their own seeds is not established.
  2. [Section 2 and Table 1] The related work section cites AdvPaint [30] as an attention-based inpainting attack, but the quantitative comparisons in Table 1, Table 3, Figure 7, and the robustness experiments include only EncoderAttack, DiffusionAttack, DiffusionGuard, and DDD. AdvPaint is the closest comparator to SDA because it also disrupts attention mechanisms for inpainting. Omitting it from the quantitative evaluation means the state-of-the-art claim is not fully supported; please include it or explicitly justify its exclusion.
  3. [Tables 1, 2, and 3] All quantitative claims are based on aggregate scores over 100 samples with no error bars, confidence intervals, or significance tests. Some of the reported margins are modest, for example in Table 3 under unseen masks SDA achieves FID 126.65 versus DDD's 117.28 and PIQE 33.32 versus DDD's 32.32. Without variance information or paired statistical tests, the claimed improvements over the strongest baseline may not be robust. Please report per-method variance or perform significance testing.
  4. [Section 3.2 and Figure 5] The central causal claim that maximizing self-attention query discrepancy is the mechanism of generation collapse is supported only by qualitative attention-map visualization. No ablation isolates query perturbation from a generic perturbation of the same budget, such as random noise or key/value perturbation, and Figure 5 does not quantify the loss of contour attention. Please add a causal ablation that controls for perturbation strength and target component, or soften the mechanism claim accordingly.
minor comments (7)
  1. [Section 3.1, Eq. (2)] The notation for the query vectors is ambiguous: the text uses Q^l_s, Ql_s, and the perturbed query without fully specifying the norm, the range of l, and how layers are selected. Please define these quantities precisely and state the norm used in the objective.
  2. [Section 3.1] There are typos in the text, for example 'crucail' in 'The self-attention mechanism are crucail in stable diffusion models.' The sentence should also agree in number ('mechanism is crucial').
  3. [Table 1] The FID value '150.131' has three decimal places while adjacent entries have two; please standardize the number of decimals across all tables.
  4. [Section 4.1 and Table 2] The Stable Diffusion version is referred to as 'v2' in Section 4.1 and as 'v2.0' in the transferability discussion; please use consistent naming.
  5. [Appendix A.1] The statement that SDA 'directly employs null-text prompts' introduces a detail that is not defined or motivated in the main text; please clarify what null-text prompts are and how they are used in the optimization.
  6. [Figure 7] The figure relies on color-coded bars, which may be difficult to distinguish in grayscale or for color-blind readers; please add pattern labels or a legend with direct value annotations.
  7. [Appendix A.2] The limitation statement honestly notes that instruction-driven editing methods such as DiffEdit and MasaCtrl are not evaluated, but the abstract and conclusion claim general 'robustness'; please temper the broader claims or add a sentence acknowledging this scope limitation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the SDA objective is defined on self-attention queries while evaluation uses independent image-quality metrics, and no load-bearing self-citations are present.

full rationale

The paper's central derivation is self-contained. The SDA objective (Eq. 2) maximizes the L2 distance between the self-attention queries of the original and perturbed latents at the initial denoising step; this objective is not defined in terms of any evaluation metric (VIF/SSIM/PSNR/FID/LPIPS/CLIP/PIQE), so the reported protection scores are not forced by construction. The paper makes no load-bearing use of self-citations: the citations invoked for the coarse-to-fine generation pattern and the structural role of self-attention ([25], [26], [32]) are independent prior works, and no uniqueness theorem or author-specific prior result is used to exclude alternatives. The causal claim that query disruption at early steps collapses generation is supported by attention-map visualizations (Figure 5) and by transfer experiments across model versions and masks; even if that causal story were incomplete, the evaluation remains independent of the optimization objective. The fixed-seed protocol described in Sec. 4.1 is a robustness and generalization concern, not a circularity, because it does not make the evaluation metric equal to the fitted objective. The stated limitation that instruction-driven editing scenarios were not evaluated is an acknowledged scope restriction, not a circular step. No self-definitional, fitted-input, self-citation, or renaming pattern is present, so the paper warrants a non-circular verdict.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method relies on several domain assumptions about diffusion model behavior (coarse-to-fine generation, self-attention's role in structure). These are taken from prior work and qualitative visualizations. The key ad-hoc assumption is that query-space disruption in the first step causes full generation collapse. No new physical entities are introduced.

free parameters (4)
  • L2 perturbation budget eta = 12
    Set to 12 in all experiments; chosen by hand and directly controls the attack strength.
  • Optimization iterations = 300
    Fixed at 300 for all methods; no ablation on iteration count is provided.
  • U-Net layers l in Eq. 2
    The objective sums over layers, but the paper does not state which layers are included or their weights.
  • Initial denoising step T
    The method targets the initial step, but the exact timestep index is not given; this affects what contours are present.
assumptions (4)
  • domain assumption Diffusion models generate images in a coarse-to-fine manner: early denoising steps establish contours, later steps refine textures.
    Invoked in Section 3.1 to justify targeting the initial denoising step; supported by citation [25] and Figure 2.
  • domain assumption Self-attention in Stable Diffusion primarily governs structural contours and global coherence, while cross-attention handles text alignment.
    Invoked in Section 3.1 to motivate the choice of self-attention as the attack surface; cited from [26] and illustrated in Figure 3.
  • ad hoc to paper Disrupting self-attention queries in the initial step is sufficient to trigger complete generation collapse, including loss of text alignment.
    This is the paper's core causal hypothesis; supported only by attention map visualizations in Figure 5, with no quantitative causality check.
  • domain assumption An adversary will perform inpainting on a known sensitive region using a mask and Stable Diffusion v2, and the protector knows the model and mask during optimization.
    Threat model in Section 3; constrains the evaluation to a white-box setting with mask augmentation for transfer.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Structure Disruption: Subverting Malicious Diffusion-Based Inpainting via Self-Attention Query Perturbation." pith.science (2026). https://pith.science/paper/EGN6R323

@misc{pith2026250519425,
  author       = {Pith},
  title        = {Pith review of: Structure Disruption: Subverting Malicious Diffusion-Based Inpainting via Self-Attention Query Perturbation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EGN6R323}},
  note         = {Machine review of arXiv:2505.19425}
}
read the original abstract

The rapid advancement of diffusion models has enhanced their image inpainting and editing capabilities but also introduced significant societal risks. Adversaries can exploit user images from social media to generate misleading or harmful content. While adversarial perturbations can disrupt inpainting, global perturbation-based methods fail in mask-guided editing tasks due to spatial constraints. To address these challenges, we propose Structure Disruption Attack (SDA), a powerful protection framework for safeguarding sensitive image regions against inpainting-based editing. Building upon the contour-focused nature of self-attention mechanisms of diffusion models, SDA optimizes perturbations by disrupting queries in self-attention during the initial denoising step to destroy the contour generation process. This targeted interference directly disrupts the structural generation capability of diffusion models, effectively preventing them from producing coherent images. We validate our motivation through visualization techniques and extensive experiments on public datasets, demonstrating that SDA achieves state-of-the-art (SOTA) protection performance while maintaining strong robustness.

Figures

Figures reproduced from arXiv: 2505.19425 by the authors.

Figure 1
Figure 1. Protected vs. Unprotected Image Resistance: (Top) Malicious inpainting alters contextual [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Denoising process of inpainting diffusion models. We visualize intermediate denoising [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The attention map during the denoising process of Inpainting [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: The inpainting diffusion pipeline and the protective perturbation update process. The black [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Comparison of attention maps during the generation process between original and protected [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Comparison of our method with baseline approaches. Face protection case with prompt "A [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Performance comparison of protection methods under data augmentation. We compute FID [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: The visualization performance of our SDA under seen and unseen mask conditions. Seen [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]
Figure 9
Figure 9. Figure 9: Time cost of different protection methods. [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: Performance variation of protection methods with inpainting strength. Evaluation con [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

49 extracted references · 35 canonical work pages

  1. [30]

    Advpaint: Protecting images from inpainting manipulation via adversarial attention disruption

    Joonsung Jeon, Woo Jae Kim, Suhyeon Ha, Sooel Son, and Sung eui Yoon. Advpaint: Protecting images from inpainting manipulation via adversarial attention disruption. InThe Thirteenth International Conference on Learning Representations, 2025. URL https://openreview. net/forum?id=m73tETvFkX

    work page 2025

  2. [1]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780–8794, 2021

    2021

  3. [2]

    High- resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. InThe IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022

    work page 2022

  4. [3]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. InThe IEEE/CVF international conference on computer vision, pages 3836–3847, 2023

    work page 2023

  5. [4]

    Instructpix2pix: Learning to follow image editing instructions

    Tim Brooks, Aleksander Holynski, and Alexei A Efros. Instructpix2pix: Learning to follow image editing instructions. InThe IEEE/CVF conference on computer vision and pattern recognition, pages 18392–18402, 2023

    work page 2023

  6. [5]

    An image is worth one word: Personalizing text-to-image generation using textual inversion

    Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit Haim Bermano, Gal Chechik, and Daniel Cohen-or. An image is worth one word: Personalizing text-to-image generation using textual inversion. InThe Eleventh International Conference on Learning Representations, 2023. URLhttps://openreview.net/forum?id=NAQvF08TcyG

    work page 2023

  7. [6]

    LoRA: Low-rank adaptation of large language models

    Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022. URL https://openreview. net/forum?id=nZeVKeeFYf9

    2022

  8. [7]

    T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models

    Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, and Ying Shan. T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. InThe AAAI conference on artificial intelligence, volume 38, pages 4296–4304, 2024

    work page 2024

Show all 49 references
  1. [8]

    Inversion-based style transfer with diffusion models

    Yuxin Zhang, Nisha Huang, Fan Tang, Haibin Huang, Chongyang Ma, Weiming Dong, and Changsheng Xu. Inversion-based style transfer with diffusion models. InThe IEEE/CVF conference on computer vision and pattern recognition, pages 10146–10156, 2023

    2023

  2. [9]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. InThe IEEE/CVF conference on computer vision and pattern recognition, pages 22500–22510, 2023

    2023

  3. [10]

    Multi- concept customization of text-to-image diffusion

    Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shechtman, and Jun-Yan Zhu. Multi- concept customization of text-to-image diffusion. InThe IEEE/CVF conference on computer vision and pattern recognition, pages 1931–1941, 2023

    1931

  4. [11]

    Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models.arXiv preprint arXiv:2308.06721, 2023

    Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models.arXiv preprint arXiv:2308.06721, 2023

    2023 arXiv

  5. [12]

    How to continually adapt text-to-image diffusion models for flexible customization?Advances in Neural Information Processing Systems, 37:130057– 130083, 2024

    Jiahua Dong, Wenqi Liang, Hongliu Li, Duzhen Zhang, Meng Cao, Henghui Ding, Salman H Khan, and Fahad Shahbaz Khan. How to continually adapt text-to-image diffusion models for flexible customization?Advances in Neural Information Processing Systems, 37:130057– 130083, 2024

    2024

  6. [13]

    Repaint: Inpainting using denoising diffusion probabilistic models

    Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. InThe IEEE/CVF conference on computer vision and pattern recognition, pages 11461–11471, 2022

    2022

  7. [14]

    Smartbrush: Text and shape guided object inpainting with diffusion model

    Shaoan Xie, Zhifei Zhang, Zhe Lin, Tobias Hinz, and Kun Zhang. Smartbrush: Text and shape guided object inpainting with diffusion model. InThe IEEE/CVF conference on computer vision and pattern recognition, pages 22428–22437, 2023

    2023

  8. [15]

    Personalized face inpainting with diffusion models by parallel visual attention

    Jianjin Xu, Saman Motamed, Praneetha Vaddamanu, Chen Henry Wu, Christian Haene, Jean- Charles Bazin, and Fernando De la Torre. Personalized face inpainting with diffusion models by parallel visual attention. InThe IEEE/CVF Winter Conference on Applications of Computer Vision, ...

    2024

  9. [16]

    Uni-paint: A unified framework for multimodal image inpainting with pretrained diffusion model

    Shiyuan Yang, Xiaodong Chen, and Jing Liao. Uni-paint: A unified framework for multimodal image inpainting with pretrained diffusion model. InThe 31st ACM International Conference on Multimedia, pages 3190–3199, 2023

    2023

  10. [17]

    Applying current copyright law to artificial intelligence image generators in the context of anderson v

    Matthew Lindberg. Applying current copyright law to artificial intelligence image generators in the context of anderson v. stability ai, ltd.Cybaris Intell. Prop. L. Rev., 15:37, 2024

    2024

  11. [18]

    Artists are alarmed by ai—and they’re fighting back.The Washington Post

    M Cavna. Artists are alarmed by ai—and they’re fighting back.The Washington Post. Retrieved May, 5:2023, 2023

    2023

  12. [19]

    Anti-dreambooth: Protecting users from personalized text-to-image synthesis

    Thanh Van Le, Hao Phung, Thuan Hoang Nguyen, Quan Dao, Ngoc N Tran, and Anh Tran. Anti-dreambooth: Protecting users from personalized text-to-image synthesis. InThe IEEE/CVF International Conference on Computer Vision, pages 2116–2127, 2023

    2023

  13. [20]

    Adversarial example does good: Preventing painting imitation from diffusion models via adversarial examples

    Chumeng Liang, Xiaoyu Wu, Yang Hua, Jiaru Zhang, Yiming Xue, Tao Song, Zhengui Xue, Ruhui Ma, and Haibing Guan. Adversarial example does good: Preventing painting imitation from diffusion models via adversarial examples. InInternational Conference on Machine Learning, pages 20...

    2023

  14. [21]

    Toward effective protection against diffusion-based mimicry through score distillation

    Haotian Xue, Chumeng Liang, Xiaoyu Wu, and Yongxin Chen. Toward effective protection against diffusion-based mimicry through score distillation. InThe Twelfth International Con- ference on Learning Representations, 2024. URL https://openreview.net/forum?id= NzxCMe88HX

    2024

  15. [22]

    Glaze: Protecting artists from style mimicry by {Text-to-Image} models

    Shawn Shan, Jenna Cryan, Emily Wenger, Haitao Zheng, Rana Hanocka, and Ben Y Zhao. Glaze: Protecting artists from style mimicry by {Text-to-Image} models. In32nd USENIX Security Symposium (USENIX Security 23), pages 2187–2204, 2023

    2023

  16. [23]

    Raising the cost of malicious ai-powered image editing

    Hadi Salman, Alaa Khaddaj, Guillaume Leclerc, Andrew Ilyas, and Aleksander Madry. Raising the cost of malicious ai-powered image editing. InThe 40th International Conference on Machine Learning, ICML’23. JMLR.org, 2023

    2023

  17. [24]

    Diffusionguard: A robust defense against malicious diffusion-based image editing

    June Suk Choi, Kyungmin Lee, Jongheon Jeong, Saining Xie, Jinwoo Shin, and Kimin Lee. Diffusionguard: A robust defense against malicious diffusion-based image editing. InThe Thirteenth International Conference on Learning Representations, 2025. URL https:// openreview.net/foru...

    2025

  18. [25]

    Diffusion probabilistic model made slim

    Xingyi Yang, Daquan Zhou, Jiashi Feng, and Xinchao Wang. Diffusion probabilistic model made slim. InThe IEEE/CVF Conference on computer vision and pattern recognition, pages 22552–22562, 2023

    2023

  19. [26]

    Towards understanding cross and self-attention in stable diffusion for text-guided image editing

    Bingyan Liu, Chengyu Wang, Tingfeng Cao, Kui Jia, and Jun Huang. Towards understanding cross and self-attention in stable diffusion for text-guided image editing. InThe IEEE/CVF conference on computer vision and pattern recognition, pages 7817–7826, 2024

    2024

  20. [27]

    Perturbing attention gives you more bang for the buck: Subtle imaging perturbations that efficiently fool customized diffusion models

    Jingyao Xu, Yuetong Lu, Yandong Li, Siyang Lu, Dongdong Wang, and Xiang Wei. Perturbing attention gives you more bang for the buck: Subtle imaging perturbations that efficiently fool customized diffusion models. InThe IEEE/CVF Conference on Computer Vision and Pattern Recognit...

    2024

  21. [28]

    Mist: Towards improved adversarial examples for diffusion models.arXiv preprint arXiv:2305.12683, 2023

    Chumeng Liang and Xiaoyu Wu. Mist: Towards improved adversarial examples for diffusion models.arXiv preprint arXiv:2305.12683, 2023

    2023 arXiv

  22. [29]

    Disrupting diffusion-based inpainters with semantic digression.arXiv preprint arXiv:2407.10277, 2024

    Geonho Son, Juhun Lee, and Simon S Woo. Disrupting diffusion-based inpainters with semantic digression.arXiv preprint arXiv:2407.10277, 2024

    2024 arXiv

  23. [31]

    Decent deepfakes? professional deepfake developers’ ethical considerations and their governance potential.AI and Ethics, pages 1–26, 2024

    Maria Pawelec. Decent deepfakes? professional deepfake developers’ ethical considerations and their governance potential.AI and Ethics, pages 1–26, 2024. 12

    2024

  24. [32]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors,Advances in Neural Information Processing Systems, volume 33, pages 6840–6851. Curran Associates, Inc., 2020

    2020

  25. [33]

    Photorealistic text-to-image diffusion models with deep language understanding.Advances in neural information processing systems, 35:36479–36494, 2022

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding.Advances in neural information ...

    2022

  26. [34]

    Perceiver: General perception with iterative attention

    Andrew Jaegle, Felix Gimeno, Andy Brock, Oriol Vinyals, Andrew Zisserman, and Joao Carreira. Perceiver: General perception with iterative attention. InInternational conference on machine learning, pages 4651–4664. PMLR, 2021

    2021

  27. [35]

    Attention is all you need.Advances in neural information processing systems, 30, 2017

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need.Advances in neural information processing systems, 30, 2017

    2017

  28. [36]

    Auto-encoding variational bayes, 2022

    Diederik P Kingma and Max Welling. Auto-encoding variational bayes, 2022. URL https: //arxiv.org/abs/1312.6114

    2022 arXiv

  29. [37]

    Multi-class face segmentation, 2022

    Ashish Goswami. Multi-class face segmentation, 2022. URL https://www.kaggle.com/ datasets/ashish2001/multiclass-face-segmentation

    2022

  30. [38]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. InComputer vision–ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings,...

    2014

  31. [39]

    Chatgpt (march 2025 version).https://chat.openai.com/, 2025

    OpenAI. Chatgpt (march 2025 version).https://chat.openai.com/, 2025

    2025

  32. [40]

    Image information and visual quality.IEEE transactions on image processing, 15(2):430–444, 2006

    Hamid R Sheikh and Alan C Bovik. Image information and visual quality.IEEE transactions on image processing, 15(2):430–444, 2006

    2006

  33. [41]

    Image quality assessment: from error visibility to structural similarity.IEEE transactions on image processing, 13(4): 600–612, 2004

    Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity.IEEE transactions on image processing, 13(4): 600–612, 2004

    2004

  34. [42]

    Springer Science & Business Media, 2005

    Bernd Jähne.Digital image processing. Springer Science & Business Media, 2005

    2005

  35. [43]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium.Advances in neural information processing systems, 30, 2017

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium.Advances in neural information processing systems, 30, 2017

    2017

  36. [44]

    The unreason- able effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreason- able effectiveness of deep features as a perceptual metric. InThe IEEE conference on computer vision and pattern recognition, pages 586–595, 2018

    2018

  37. [45]

    Clipscore: A reference - free evaluation metric for image captioning

    Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference - free evaluation metric for image captioning. InEMNLP, 2021

    2021

  38. [46]

    No-reference image quality assessment in the spatial domain.IEEE transactions on image processing, 21(12):4695–4708, 2012

    Anish Mittal, Anush Krishna Moorthy, and Alan Conrad Bovik. No-reference image quality assessment in the spatial domain.IEEE transactions on image processing, 21(12):4695–4708, 2012

    2012

  39. [47]

    IQA-PyTorch: Pytorch toolbox for image quality assessment

    Chaofeng Chen and Jiadi Mo. IQA-PyTorch: Pytorch toolbox for image quality assessment. [Online]. Available:https://github.com/chaofengc/IQA-PyTorch, 2022

    2022

  40. [48]

    Diffedit: Diffusion- based semantic image editing with mask guidance

    Guillaume Couairon, Jakob Verbeek, Holger Schwenk, and Matthieu Cord. Diffedit: Diffusion- based semantic image editing with mask guidance. InThe Eleventh International Confer- ence on Learning Representations, 2023. URL https://openreview.net/forum?id= 3lge0p5o-M-

    2023

  41. [49]

    Masactrl: Tuning-free mutual self-attention control for consistent image synthesis and editing

    Mingdeng Cao, Xintao Wang, Zhongang Qi, Ying Shan, Xiaohu Qie, and Yinqiang Zheng. Masactrl: Tuning-free mutual self-attention control for consistent image synthesis and editing. InThe IEEE/CVF International Conference on Computer Vision (ICCV), pages 22560–22570, October 2023...

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.