Pith. sign in

REVIEW 2 major objections 6 minor 64 references

Nearly Zero-Cost Protection Against Mimicry by Personalized Diffusion Models

T0 review · 2 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read FastProtect achieves near-zero-cost protection against diffusion-model mimicry: a pre-trained mixture of perturbations runs in 0.04 s per 512 × 512 image, 200–3500× faster than iterative methods, with matched efficacy and better…

desk verdict A fast and practical protection method, but the invisibility/efficacy claim rests on a possibly unfair budget comparison. read the letter →

arxiv 2412.11423 v2 pith:FUZTZGGM submitted 2024-12-16 cs.CV

Namhyuk Ahn , KiYoon Yoo , Wonhyuk Ahn , Daesik Kim , Seung-Hun Nam This is my paper · ORCID
classification cs.CV
keywords imageprotectionadversarialperturbationdiffusionmodelspersonalizationuniversalmixture-of-perturbationslatentinferencelatency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to remove the main obstacle to practical image protection: latency. Existing defenses against mimicry by personalized diffusion models run iterative optimization on every image at protection time, taking 7–225 seconds on an A100 GPU and minutes to hours on a CPU. FastProtect instead pre-trains a small set of perturbations and, at inference, picks one per image from the image's VAE latent code, turning protection into a lookup plus one forward pass. The paper argues that this mixture-of-perturbations recovers the protection efficacy lost by a single universal perturbation, and that adaptive target selection plus a perceptual mask make it match iterative methods' protection level while running 200–3500× faster and usually more invisible. If that holds, protection becomes cheap enough to deploy at scale, including on low-end devices and for high-resolution artwork.

What carries the argument

The load-bearing object is the mixture-of-perturbations (MoP): a global perturbation plus K=4 perturbations, each trained to disrupt personalized diffusion mimicry, together with an assignment function A that maps an image's VAE latent code to one of the K perturbations, learned by K-means++ clustering of 80,000 training latents from four domains (object, face, painting, cartoon). Because A selects the perturbation at inference, protection becomes semi-image-specific without iteration. The surrounding machinery: a multi-layer protection loss evaluated on intermediate VAE encoder features, adaptive targeted protection that chooses among three pattern-repetition target images by L1 entropy distance, and adaptive protection strength that rescales the perturbation spatially according to an inverted LPIPS perceptual map.

What would settle it

Protect images from a genuinely novel domain — for example medical scans or satellite imagery, which share little texture with the four trained domains — with FastProtect, fine-tune LoRA on the protected images, and measure FID against unprotected mimicry. If efficacy falls to the single-universal-perturbation level (about 207.6 FID in the paper's own numbers, versus 227.6 for iterative protection), the generalization claim for the assignment function is refuted; the paper's domain-generalization experiment only varies the four trained domains, so it does not settle this case.

Watch

Extended reading notes

Core claim

The paper's central claim is that near-zero-cost protection need not sacrifice efficacy: a pre-trained mixture of K perturbations, routed by a K-means assignment function over VAE latent codes, provides semi-image-specific protection that closes the gap between image-agnostic universal perturbations and slow per-image optimization. Contributing claims: computing the protection loss across multiple VAE encoder feature layers (not just the final latent) strengthens the perturbation; matching the target image's pattern repetition to the input's texture complexity through an entropy-based rule yields near-optimal protection across domains; and masking the perturbation with an inverted LPIPS distance map improves invisibility at negligible extra cost. In matched-strength comparisons FastProtect reports the best invisibility on object, face, and cartoon benchmarks and second-best on paintings, with FID-measured protection efficacy on par with or above the iterative baselines, and it transfers to unseen diffusion backbones and personalization methods.

Load-bearing premise

The load-bearing premise is that the K-means router trained on 80,000 images from four domains chooses a perturbation that still works on images it has never seen; if routing fails on a truly novel domain, protection efficacy drops toward the weak single-perturbation baseline.

Editorial extensions

If this is right

  • Protection becomes real-time: 0.04 s on an A100 GPU and about 2.9 s on a CPU for a 512 × 512 image, with latency nearly flat up to 2048 × 2048, so high-resolution artwork can be protected on demand.
  • In comparisons where protection strength is matched across methods, FastProtect achieves the best invisibility on object, face, and cartoon benchmarks and second-best on paintings, with FID protection efficacy equal to or better than the iterative baselines.
  • The pre-trained perturbations survive Gaussian noise, JPEG compression, and arbitrary input resolutions, and they transfer in black-box settings to unknown diffusion backbones (SD v2.1, SD-XL) and unknown personalization methods (Textual Inversion, DreamStyler).
  • FastProtect's output is a better warm start for iterative refinement: PGD from the FastProtect perturbation converges in about 25 steps with higher efficacy than 100 steps from a clean image, so users with spare compute can push protection further.
  • Inference requires only 1.7 GB of VRAM, versus more than 8 GB for existing methods, making CPU-only and low-memory deployment realistic.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • My inference: the same pre-trained-mixture recipe should transfer to other latency-bound protection tasks, such as video frames, batch uploads, or on-device mobile protection, because the marginal cost per image is one VAE encode plus one perturbation addition; the paper does not test these settings.
  • My inference: the entropy-based target-selection rule suggests that protection difficulty is predictable from an image's texture statistics, so a learned difficulty predictor could set the perturbation budget per image — the paper floats this direction as future work — which would improve invisibility on easy-to-protect images.
  • My inference: the finding that FastProtect initializations accelerate PGD convergence suggests pre-trained perturbation mixtures could serve as generic warm starts for iterative adversarial perturbation methods beyond diffusion-model protection, though the paper demonstrates this only for its own task.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. FastProtect is an image-protection method against personalized diffusion models. It replaces per-image iterative adversarial optimization with a pretrained mixture of perturbations: the image is encoded with the SD VAE, an assignment function selects one of several K-means-clustered perturbations, and an additional global perturbation is added. At inference, the method also adaptively chooses among low-, mid-, and high-repetition target images using an entropy distance and applies an LPIPS-based spatial mask to improve invisibility. The paper reports near-constant latency (0.04 s on A100 and about 2.9 s on CPU at 512x512), comparable or better protection efficacy measured by FID, and improved invisibility measured by DISTS relative to six iterative baselines, together with robustness, black-box, and ablation results across object, face, painting, and cartoon domains.

Significance. If the empirical claims hold, the main contribution is practical: FastProtect removes the expensive inference-time optimization that dominates the latency of existing protection methods, and its 175x or larger speedup is supported by direct CPU/GPU measurements rather than by architectural speculation. The paper is also careful to evaluate multiple domains, countermeasures, unknown backbones, and unknown personalization methods, and it provides ablations for the MoP assignment, multi-layer loss, adaptive targeting, and adaptive strength components. Code and a demo are promised. The main caveat is that the reported invisibility/efficacy trade-off may rely on an effectively larger perturbation budget than the baselines are allowed, so the headline comparisons in Tables 2 and 7 need to be re-verified under an equal true L-infinity constraint.

major comments (2)
  1. [§3.2, Eq. (7)–(8); Appendix A, Eq. (8)] The final perturbation may exceed the nominal L∞ budget η that all baselines respect. The training algorithm clamps δg and each Δk to the (η/2)-ball, so their sum can reach η in a pixel before scaling. Equation (8) sets M′ = β_i α = 0.91 × 1.3 = 1.183 for the first three deciles of the reversed perceptual map, and the final image is x̂ = x + M′ ⊙ (δg + Δk) without a reported clamp or realized-norm check. Tables 2 and 7 therefore may compare η-bounded baselines against perturbations of up to 1.183η in low-perceptibility regions. Please report the realized per-pixel max-norm of the final protected images, clamp the applied perturbation to η (or justify a different budget definition), and re-run the strength-matched and fixed-strength comparisons under an equal true budget.
  2. [§4, 'Evaluation' paragraph; Table 2] The strength-matching protocol used to produce Table 2 is underspecified and the matching itself is not validated. The text states that protection strength is adjusted 'to match the protection level across methods' but gives no objective criterion, no per-method perturbation strengths, and no confidence intervals; entries such as Object FID 223.0 for both PhotoGuard and FastProtect suggest an unreported tuning procedure. Since the headline claim is 'comparable protection efficacy with best invisibility,' please state exactly how matching was performed, report the η values used for each method, repeat over multiple seeds/subject sets, and show variance or confidence intervals for both FID and invisibility metrics.
minor comments (6)
  1. [§3.2, Eq. (6)] The entropy distance H(z) is defined over a continuous latent variable; please specify how p(z) is estimated (e.g., histogram or soft binning).
  2. [Appendix A, Eq. (8)] The decile definition is ambiguous: 'from the highest value' plus the strict condition M < q_i should be made precise with inclusive/exclusive thresholds.
  3. [Table 1 and Figure 2] The PGD row in Table 1 is never identified as PhotoGuard; state that the iterative baseline is PhotoGuard and give its budget.
  4. [§1 and Figure 9] Please label the exact settings for the claimed 200–3500× speedup range (image size and CPU/GPU), since Figure 9 shows a wide range.
  5. [§4.2, Figure 6a] The domain-generalization analysis varies training-domain composition but does not include a truly unseen domain, so the claim of generalization should be phrased accordingly.
  6. [Affiliations] The affiliation 'NA VER WEBTOON AI' contains a spacing typo; it should read 'NAVER WEBTOON AI'.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the paper's claims are empirical comparisons, and its self-citations are not load-bearing for the central results.

full rationale

The paper's central claims—real-time latency, comparable protection efficacy, and improved invisibility—are empirical measurements, not quantities derived from fitted parameters by construction. Perturbation pre-training (Eq. 3), the MoP assignment (Eq. 4), the MLP loss (Eq. 5), adaptive target selection (Eq. 6), and LPIPS-based adaptive masking (Eqs. 7–8) are all proposed mechanisms whose contributions are assessed by ablations on held-out benchmark data (Tables 3–7, Figures 6–7). No equation reduces to the claimed outcome: the invisibility/efficacy trade-off is measured rather than implied by the loss definition, and the latency gain follows from pre-training plus lightweight inference, as directly measured in Figure 1a and Figure 9. The strength-matched protocol in Table 2 is explicitly disclosed as a design choice, so matching FID values are an experimental setting, not a hidden prediction. The self-citations (Ahn et al. [1] and [2]) are motivational or baseline-related and do not carry the method's validity. The most substantive concern—that Eq. 8's scaling (α=1.3, β=0.91, c=3) can push the applied perturbation beyond the nominal η used for baselines in Table 7—is an evaluation-fairness and correctness risk, not circularity: it does not make any claimed result equal to its input by definition. No fitted parameter is renamed as a prediction, no uniqueness theorem is imported, and no known result is merely relabeled.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on standard learning assumptions plus a few heuristic choices (entropy-based targeting, LPIPS masking) that are empirically validated but not theoretically justified.

free parameters (4)
  • K (number of perturbations) = 4
    Chosen from ablation; performance converges at K=4 (Figure 6b).
  • lambda (MLP loss weight) = 3.5e-5
    Set in Appendix A; controls the influence of intermediate feature losses.
  • alpha, beta, c (perceptual scaling) = 1.3, 0.91, 3
    Hand-tuned in Eq. 8 to balance efficacy and invisibility.
  • perturbation budget eta = adjusted per comparison
    Used to match protection levels across baselines; free parameter of the efficacy-invisibility trade-off.
assumptions (4)
  • domain assumption VAE latent codes are representative enough for clustering images by protection-relevant characteristics
    MoP assignment relies on K-means over E(x); invoked in Section 3.1.
  • domain assumption LPIPS distance map correlates with human perception for masking perturbations
    Adaptive protection strength uses LPIPS(x, x+delta) as a perceptual map (Section 3.2).
  • domain assumption Texture loss (minimizing latent distance to a target) transfers to protection against personalization such as LoRA
    The training loss Eq. 5 is the texture loss; evaluation uses LoRA fine-tuning.
  • ad hoc to paper Entropy of the latent code is a valid proxy for selecting the appropriate target pattern repetition
    Adaptive targeted protection compares H(z) with target entropies (Eq. 6); motivated by Figure 3 observations.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Nearly Zero-Cost Protection Against Mimicry by Personalized Diffusion Models." pith.science (2026). https://pith.science/paper/FUZTZGGM

@misc{pith2026241211423,
  author       = {Pith},
  title        = {Pith review of: Nearly Zero-Cost Protection Against Mimicry by Personalized Diffusion Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FUZTZGGM}},
  note         = {Machine review of arXiv:2412.11423}
}
read the original abstract

Recent advancements in diffusion models revolutionize image generation but pose risks of misuse, such as replicating artworks or generating deepfakes. Existing image protection methods, though effective, struggle to balance protection efficacy, invisibility, and latency, thus limiting practical use. We introduce perturbation pre-training to reduce latency and propose a mixture-of-perturbations approach that dynamically adapts to input images to minimize performance degradation. Our novel training strategy computes protection loss across multiple VAE feature spaces, while adaptive targeted protection at inference enhances robustness and invisibility. Experiments show comparable protection performance with improved invisibility and drastically reduced inference time. The code and demo are available at https://webtoon.github.io/impasto

Figures

Figures reproduced from arXiv: 2412.11423 by the authors.

Figure 1
Figure 1. (a) FastProtect shows unprecedented speed in protection against diffusion models. On an A100 GPU, FastProtect achieves [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Model overview. (a) Current iterative optimization approaches lack a training phase and perform optimization during inference, [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Relationship between target image’s pattern repetition [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (8 more)
Figure 5
Figure 5. Figure 5: Qualitative comparison of different protection frameworks. (Top) Protected image with a zoomed-in patch in the inset. (Bottom) [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Analysis of the proposed modules in the pre-training phase. [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Analysis of the proposed modules in the inference phase. [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Examples of target images used by FastProtect. In our [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]
Figure 9
Figure 9. Figure 9: Inference latency (log-scaled) vs. image size on both CPU and GPU environments. [PITH_FULL_IMAGE:figures/full_fig_p017_9.png]
Figure 10
Figure 10. Figure 10: Protection efficacy vs. invisibility comparison on the Object and Cartoon domains. [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]
Figure 11
Figure 11. Figure 11: Additional qualitative comparison. For each example, (top) protected image and (bottom) mimicry image generated by LoRA. [PITH_FULL_IMAGE:figures/full_fig_p018_11.png]
Figure 12
Figure 12. Figure 12: Additional qualitative comparison. For each example, (top) protected image and (bottom) mimicry image generated by LoRA. [PITH_FULL_IMAGE:figures/full_fig_p019_12.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

64 extracted references · 36 canonical work pages

  1. [1]

    Imperceptible protection against style imitation from diffusion models

    Namhyuk Ahn, Wonhyuk Ahn, KiYoon Yoo, Daesik Kim, and Seung-Hun Nam. Imperceptible protection against style imitation from diffusion models. arXiv preprint arXiv:2403.19254, 2024. 2, 3, 5, 6, 14, 15, 16, 17

    work page arXiv 2024

  2. [2]

    Dream- styler: Paint by style inversion with text-to-image diffusion models

    Namhyuk Ahn, Junsoo Lee, Chunggi Lee, Kunhee Kim, Daesik Kim, Seung-Hun Nam, and Kibeom Hong. Dream- styler: Paint by style inversion with text-to-image diffusion models. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 674–681, 2024. 1, 7

    work page 2024

  3. [3]

    k-means++: The advantages of careful seeding

    David Arthur, Sergei Vassilvitskii, et al. k-means++: The advantages of careful seeding. In Soda, pages 1027–1035,

  4. [4]

    Vggface2: A dataset for recognising faces across pose and age

    Qiong Cao, Li Shen, Weidi Xie, Omkar M Parkhi, and An- drew Zisserman. Vggface2: A dataset for recognising faces across pose and age. In 2018 13th IEEE international con- ference on automatic face & gesture recognition (FG 2018), pages 67–74. IEEE, 2018. 14

    work page 2018

  5. [5]

    Universal Adversarial Perturbations: A Survey

    Ashutosh Chaubey, Nikhil Agrawal, Kavya Barnwal, Keerat K Guliani, and Pramod Mehta. Universal adversarial perturbations: A survey. arXiv preprint arXiv:2005.08087,

    work page Pith review arXiv 2005

  6. [6]

    Topiq: A top-down approach from semantics to distortions for image quality assessment

    Chaofeng Chen, Jiadi Mo, Jingwen Hou, Haoning Wu, Liang Liao, Wenxiu Sun, Qiong Yan, and Weisi Lin. Topiq: A top-down approach from semantics to distortions for image quality assessment. IEEE Transactions on Image Processing,

  7. [7]

    Sparse and imperceiv- able adversarial attacks

    Francesco Croce and Matthias Hein. Sparse and imperceiv- able adversarial attacks. In Proceedings of the IEEE/CVF international conference on computer vision , pages 4724– 4732, 2019. 13

    work page 2019

  8. [8]

    Saliency attack: Towards imperceptible black-box adversarial attack

    Zeyu Dai, Shengcai Liu, Qing Li, and Ke Tang. Saliency attack: Towards imperceptible black-box adversarial attack. ACM Transactions on Intelligent Systems and Technology , 14(3):1–20, 2023. 13

    work page 2023

Show all 64 references
  1. [9]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009. 14

    2009

  2. [10]

    Image quality assessment: Unifying structure and texture similarity

    Keyan Ding, Kede Ma, Shiqi Wang, and Eero P Simoncelli. Image quality assessment: Unifying structure and texture similarity. IEEE transactions on pattern analysis and ma- chine intelligence, 44(5):2567–2581, 2020. 3, 6, 14

    2020

  3. [11]

    An image is worth one word: Personalizing text-to- image generation using textual inversion

    Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patash- nik, Amit H Bermano, Gal Chechik, and Daniel Cohen- Or. An image is worth one word: Personalizing text-to- image generation using textual inversion. arXiv preprint arXiv:2208.01618, 2022. 1, 2, 7

    2022 arXiv

  4. [12]

    Explaining and harnessing adversarial examples

    Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014. 1

    2014 arXiv

  5. [13]

    Low frequency adversarial perturbation

    Chuan Guo, Jared S Frank, and Kilian Q Weinberger. Low frequency adversarial perturbation. arXiv preprint arXiv:1809.08758, 2018. 13

    2018 arXiv

  6. [14]

    Learning universal adver- sarial perturbations with generative models

    Jamie Hayes and George Danezis. Learning universal adver- sarial perturbations with generative models. In 2018 IEEE Security and Privacy Workshops (SPW), pages 43–49. IEEE,

    2018

  7. [15]

    Gans trained by a two time-scale update rule converge to a local nash equilib- rium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilib- rium. Advances in neural information processing systems , 30, 2017. 3, 6, 14

    2017

  8. [16]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 1

    2020

  9. [17]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021. 3, 6, 14

    2021 arXiv

  10. [18]

    Composer: Creative and controllable im- age synthesis with composable conditions

    Lianghua Huang, Di Chen, Yu Liu, Yujun Shen, Deli Zhao, and Jingren Zhou. Composer: Creative and controllable im- age synthesis with composable conditions. arXiv preprint arXiv:2302.09778, 2023. 1

    2023 arXiv

  11. [19]

    A style-based generator architecture for generative adversarial networks

    Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 4401–4410, 2019. 14

    2019

  12. [20]

    One millisecond face alignment with an ensemble of regression trees

    Vahid Kazemi and Josephine Sullivan. One millisecond face alignment with an ensemble of regression trees. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 1867–1874, 2014. 14

    2014

  13. [21]

    Diffblender: Scalable and composable multimodal text-to-image diffusion models

    Sungnyun Kim, Junsoo Lee, Kibeom Hong, Daesik Kim, and Namhyuk Ahn. Diffblender: Scalable and composable multimodal text-to-image diffusion models. arXiv preprint arXiv:2305.15194, 2023. 1

    2023 arXiv

  14. [22]

    Adam: A method for stochastic optimization

    Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 ,

    arXiv

  15. [23]

    Imagenet classification with deep convolutional neural net- works

    Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural net- works. Advances in neural information processing systems , 25, 2012. 2

    2012

  16. [24]

    Multi-concept customization of text-to-image diffusion

    Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shechtman, and Jun-Yan Zhu. Multi-concept customization of text-to-image diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 1931–1941, 2023. 1

    1931

  17. [25]

    Perceptual adversarial robustness: Defense against unseen threat mod- els

    Cassidy Laidlaw, Sahil Singla, and Soheil Feizi. Perceptual adversarial robustness: Defense against unseen threat mod- els. arXiv preprint arXiv:2006.12655, 2020. 13

    2006 arXiv

  18. [26]

    Attentions help cnns see better: Attention-based hybrid image quality 9 assessment network

    Shanshan Lao, Yuan Gong, Shuwei Shi, Sidi Yang, Tianhe Wu, Jiahao Wang, Weihao Xia, and Yujiu Yang. Attentions help cnns see better: Attention-based hybrid image quality 9 assessment network. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognitio...

    2022

  19. [27]

    Mist: Towards improved adversarial examples for diffusion models

    Chumeng Liang and Xiaoyu Wu. Mist: Towards improved adversarial examples for diffusion models. arXiv preprint arXiv:2305.12683, 2023. 3, 4, 6, 14, 15, 17

    2023 arXiv

  20. [28]

    Adversarial example does good: preventing paint- ing imitation from diffusion models via adversarial exam- ples

    Chumeng Liang, Xiaoyu Wu, Yang Hua, Jiaru Zhang, Yim- ing Xue, Tao Song, Zhengui Xue, Ruhui Ma, and Haibing Guan. Adversarial example does good: preventing paint- ing imitation from diffusion models via adversarial exam- ples. In Proceedings of the 40th International Conferenc...

    2023

  21. [29]

    Universal adversarial per- turbation via prior driven uncertainty approximation

    Hong Liu, Rongrong Ji, Jie Li, Baochang Zhang, Yue Gao, Yongjian Wu, and Feiyue Huang. Universal adversarial per- turbation via prior driven uncertainty approximation. InPro- ceedings of the IEEE/CVF international conference on com- puter vision, pages 2941–2949, 2019. 12

    2019

  22. [30]

    Metacloak: Preventing unauthorized subject-driven text-to-image diffusion-based synthesis via meta-learning

    Yixin Liu, Chenrui Fan, Yutong Dai, Xun Chen, Pan Zhou, and Lichao Sun. Metacloak: Preventing unauthorized subject-driven text-to-image diffusion-based synthesis via meta-learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2421...

    2024

  23. [31]

    Towards imperceptible and robust adversarial example attacks against neural networks

    Bo Luo, Yannan Liu, Lingxiao Wei, and Qiang Xu. Towards imperceptible and robust adversarial example attacks against neural networks. In Proceedings of the AAAI conference on artificial intelligence, 2018. 13

    2018

  24. [32]

    Frequency-driven imperceptible adversarial attack on semantic similarity

    Cheng Luo, Qinliang Lin, Weicheng Xie, Bizhu Wu, Jin- heng Xie, and Linlin Shen. Frequency-driven imperceptible adversarial attack on semantic similarity. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15315–15324, 2022. 13

    2022

  25. [33]

    Towards deep learn- ing models resistant to adversarial attacks

    Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learn- ing models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083, 2017. 1, 2

    2017 arXiv

  26. [34]

    Sdedit: Guided image synthesis and editing with stochastic differential equa- tions

    Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jia- jun Wu, Jun-Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equa- tions. arXiv preprint arXiv:2108.01073, 2021. 1

    2021 arXiv

  27. [35]

    Sparsefool: a few pixels make a big dif- ference

    Apostolos Modas, Seyed-Mohsen Moosavi-Dezfooli, and Pascal Frossard. Sparsefool: a few pixels make a big dif- ference. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9087–9096,

  28. [36]

    Universal adversarial perturba- tions

    Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Omar Fawzi, and Pascal Frossard. Universal adversarial perturba- tions. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 1765–1773, 2017. 2, 3, 4, 7, 12

    2017

  29. [37]

    Generalizable data-free objective for crafting univer- sal adversarial perturbations

    Konda Reddy Mopuri, Aditya Ganeshan, and R Venkatesh Babu. Generalizable data-free objective for crafting univer- sal adversarial perturbations. IEEE transactions on pattern analysis and machine intelligence, 41(10):2452–2465, 2018. 12

    2018

  30. [38]

    Nag: Network for adversary generation

    Konda Reddy Mopuri, Utkarsh Ojha, Utsav Garg, and R Venkatesh Babu. Nag: Network for adversary generation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 742–751, 2018. 12

    2018

  31. [39]

    Ask, acquire, and attack: Data-free uap generation using class impressions

    Konda Reddy Mopuri, Phani Krishna Uppala, and R Venkatesh Babu. Ask, acquire, and attack: Data-free uap generation using class impressions. In Proceedings of the European Conference on Computer Vision (ECCV) , pages 19–34, 2018. 12

    2018

  32. [40]

    T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models

    Chong Mou, Xintao Wang, Liangbin Xie, Jian Zhang, Zhon- gang Qi, Ying Shan, and Xiaohu Qie. T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. arXiv preprint arXiv:2302.08453, 2023. 1

    2023 arXiv

  33. [41]

    Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023. 1, 7

    2023 arXiv

  34. [42]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  35. [43]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 1, 2, 7

    2022

  36. [44]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22500...

    2023

  37. [45]

    Large-scale classifica- tion of fine-art paintings: Learning the right metric on the right feature

    Babak Saleh and Ahmed Elgammal. Large-scale classifica- tion of fine-art paintings: Learning the right metric on the right feature. arXiv preprint arXiv:1505.00855, 2015. 14

    2015 arXiv

  38. [46]

    Raising the cost of malicious ai-powered image editing

    Hadi Salman, Alaa Khaddaj, Guillaume Leclerc, Andrew Ilyas, and Aleksander Madry. Raising the cost of malicious ai-powered image editing. arXiv preprint arXiv:2302.06588,

    arXiv

  39. [47]

    Colorfool: Semantic adversarial coloriza- tion

    Ali Shahin Shamsabadi, Ricardo Sanchez-Matilla, and An- drea Cavallaro. Colorfool: Semantic adversarial coloriza- tion. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 1151–1160,

  40. [48]

    Glaze: Protecting artists from style mimicry by text-to-image models

    Shawn Shan, Jenna Cryan, Emily Wenger, Haitao Zheng, Rana Hanocka, and Ben Y Zhao. Glaze: Protecting artists from style mimicry by text-to-image models. arXiv preprint arXiv:2302.04222, 2023. 1, 3

    2023 arXiv

  41. [49]

    Styledrop: Text-to-image generation in any style

    Kihyuk Sohn, Nataniel Ruiz, Kimin Lee, Daniel Castro Chin, Irina Blok, Huiwen Chang, Jarred Barber, Lu Jiang, Glenn Entis, Yuanzhen Li, et al. Styledrop: Text-to-image generation in any style. arXiv preprint arXiv:2306.00983 ,

    arXiv

  42. [50]

    Score-based generative modeling through stochastic differential equa- tions

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equa- tions. arXiv preprint arXiv:2011.13456, 2020. 1 10

    2011 arXiv

  43. [51]

    Anti-dreambooth: Pro- tecting users from personalized text-to-image synthesis

    Thanh Van Le, Hao Phung, Thuan Hoang Nguyen, Quan Dao, Ngoc N Tran, and Anh Tran. Anti-dreambooth: Pro- tecting users from personalized text-to-image synthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2116–2127, 2023. 1, 3, 6, 14, 15, 16, 17

    2023

  44. [52]

    Prompt-agnostic adversarial perturbation for customized dif- fusion models

    Cong Wan, Yuhang He, Xiang Song, and Yihong Gong. Prompt-agnostic adversarial perturbation for customized dif- fusion models. Advances in Neural Information Processing Systems, 37:136576–136619, 2024. 3

    2024

  45. [53]

    Demiguise attack: Crafting invisible semantic adversarial perturbations with perceptual similarity

    Yajie Wang, Shangbo Wu, Wenyi Jiang, Shengang Hao, Yu- an Tan, and Quanxin Zhang. Demiguise attack: Crafting invisible semantic adversarial perturbations with perceptual similarity. arXiv preprint arXiv:2107.01396, 2021. 13

    2021 arXiv

  46. [54]

    Q-align: Teaching lmms for visual scoring via discrete text-defined levels

    Haoning Wu, Zicheng Zhang, Weixia Zhang, Chaofeng Chen, Liang Liao, Chunyi Li, Yixuan Gao, Annan Wang, Erli Zhang, Wenxiu Sun, et al. Q-align: Teaching lmms for visual scoring via discrete text-defined levels. arXiv preprint arXiv:2312.17090, 2023. 14

    2023 arXiv

  47. [55]

    Perturbing attention gives you more bang for the buck: Subtle imaging perturbations that effi- ciently fool customized diffusion models

    Jingyao Xu, Yuetong Lu, Yandong Li, Siyang Lu, Dongdong Wang, and Xiang Wei. Perturbing attention gives you more bang for the buck: Subtle imaging perturbations that effi- ciently fool customized diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision an...

    2024

  48. [56]

    Toward effective protection against diffusion-based mimicry through score distillation

    Haotian Xue, Chumeng Liang, Xiaoyu Wu, and Yongxin Chen. Toward effective protection against diffusion-based mimicry through score distillation. In The Twelfth Interna- tional Conference on Learning Representations, 2023. 2, 3, 6, 14, 15, 17

    2023

  49. [57]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3836–3847, 2023. 1

    2023

  50. [58]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shecht- man, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 586–595, 2018. 5, 13, 14

    2018

  51. [59]

    Towards large yet imperceptible adversarial image perturbations with perceptual color distance

    Zhengyu Zhao, Zhuoran Liu, and Martha Larson. Towards large yet imperceptible adversarial image perturbations with perceptual color distance. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 1039–1048, 2020. 13

    2020

  52. [60]

    Un- learnable examples for diffusion models: Protect data from unauthorized exploitation

    Zhengyue Zhao, Jinhao Duan, Xing Hu, Kaidi Xu, Chenan Wang, Rui Zhang, Zidong Du, Qi Guo, and Yunji Chen. Un- learnable examples for diffusion models: Protect data from unauthorized exploitation. arXiv preprint arXiv:2306.01902,

    arXiv

  53. [61]

    down 1”, “down 2

    Boyang Zheng, Chumeng Liang, Xiaoyu Wu, and Yan Liu. Understanding and improving adversarial attacks on latent diffusion model. arXiv preprint arXiv:2310.04687, 2023. 2, 3, 4, 5 11 A. Implementation Details Training. We set the number of perturbations K in MoP to four. When ca...

    2023

  54. [63]

    We utilize three patterned target im- ages, each representing low, mid, and high pattern repeti- tion, as illustrated in Figure 8. To implement adaptive tar- geted protection, for each protection strength (budget)η, we obtain three MoP models (low, mid, and high), each corre- ...

  55. [2023]

    1, 3, 6, 7, 8, 14, 15, 16, 17

  56. [5122]

    A paint- ing in <sks> style

    The Painting dataset is compiled by randomly sam- pling (but not overlap to the training dataset) 20 artists from the WikiArt dataset, each artist represented by 10 artworks, all resized to 5122. For the Cartoon domain, we randomly sample 20 Webtoon works (but also no overlap ...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.