Pith. sign in

REVIEW 6 major objections 5 minor 4 cited by

Anti-Reference: Universal and Immediate Defense Against Reference-Based Generation

T0 review · 6 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Anti-Reference claims that a single imperceptible adversarial perturbation can block personalized diffusion generation across fine-tuning, adapter, and reference-network methods in one pass.

desk verdict Useful first attack on tuning-free reference-based generation with a fast encoder, but the universality claim is undercut by in-sample evaluation and qualitative API tests. read the letter →

arxiv 2412.05980 v1 pith:7F6LL5FH submitted 2024-12-08 cs.CV

Yiren Song , Shengtao Lou , Xiaokang Liu , Hai Ci , Pei Yang , Jiaming Liu , Mike Zheng Shou This is my paper · ORCID
classification cs.CV
keywords adversarialperturbationdiffusionmodelreference-basedgenerationimageprotectioncustomizedgray-boxtransfernoiseencoderhumananimation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Anti-Reference is a defense method that tries to make photos unusable as reference inputs for personalized diffusion generation. The authors claim that one imperceptible adversarial perturbation, generated either by a trained neural encoder in a single forward pass or by iterative Projected Gradient Descent (PGD) optimization, is enough to disrupt the outputs of fine-tuning methods like DreamBooth and LoRA, tuning-free methods like IP-Adapter, and human-animation methods like Magic Animate and Ecomimic. The point of the paper is that protection can be universal, not tied to one specific customization method, and fast enough for real-world uploads, which would matter because anyone publishing an image online currently has no practical way to stop it from being cloned into fake photos or videos. The method reports transfer to closed-source APIs through white-box proxy models and robustness to JPEG compression, cropping, noise, and color changes.

What carries the argument

The central machinery is a weighted unified loss $L_{\text{total}} = w_{\text{adv}} L_{\text{adv}} + \sum_i w_{\text{con},i} L_{\text{con adv},i} + w_{\text{reg}} L_{\text{reg}}$, where $L_{\text{adv}}$ maximizes the denoising U-Net's noise-prediction error on the protected image and $L_{\text{con adv},i}$ does the same when conditional features from one of four modules (IP-Adapter, Reference-only, Magic Animate, Ecomimic) are injected. The loss is used either to train a ViT-based Adversarial Noise Encoder that outputs pixel-space noise in one pass, or to run PGD iterations that refine the noise directly. A differentiable data-augmentation module and a two-phase training schedule (fixed conditional modules first, then randomized module weights and LoRA styling) are what push the noise toward generality rather than overfitting to a single vulnerability.

What would settle it

Run the released Adversarial Noise Encoder on a held-out set of face images, upload the protected images to a closed-source face-animation API, and compute identity similarity between generated frames and the original faces; if most outputs remain readily identifiable, the gray-box transfer claim fails.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central discovery is that protecting an image against reference-based generation can be framed as a single optimization problem: maximize the diffusion noise-prediction error both unconditionally in the denoising U-Net and conditionally when reference features extracted by an adapter or a ReferenceNet are injected into the U-Net. Maximizing both errors at once makes the protected image fail as a reference for adapter-style, reference-network-style, and fine-tuning-based generators, because the same conditional features are what a fine-tuned model would learn from. The authors further claim that a Vision-Transformer-based Adversarial Noise Encoder can learn to produce this noise in one forward pass, cutting protection time from hundreds of seconds to about a second with only a modest drop in effectiveness relative to their PGD version. The result is presented as the first universal, immediate defense covering fine-tuning, tuning-free reference generation, and human-centric driving tasks.

Load-bearing premise

The universal gray-box protection claim rests on the assumption that white-box proxy models share enough architecture and latent-space similarity with closed-source services that noise optimized on the proxies still disrupts the targets.

Editorial extensions

If this is right

  • If the central claim holds, a user can protect an image right before uploading, with a one-pass encoder that adds noise in under a second, making real-time protection feasible on phones or social platforms.
  • Because the same noise jointly attacks fine-tuning, adapter-based, and ReferenceNet-based methods, an attacker cannot evade protection simply by switching from DreamBooth to IP-Adapter or to a human-animation pipeline.
  • Noise trained with augmentations should remain effective after JPEG compression, cropping and resizing, noise addition, and color transforms, covering the preprocessing steps social platforms typically apply.
  • Gray-box transfer via white-box proxies means closed-source services built on the same architecture family may also produce visibly corrupted outputs even though their weights are hidden.
  • The PGD variant offers a stronger protection option when a few minutes of computation are acceptable, giving users a speed-versus-strength trade-off.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • As an inference beyond the paper, the defense's effectiveness is tied to the SD1.5 base family; if the community shifts to SDXL, SD3, or autoregressive generators, the same noise architecture would likely need retraining, and the paper's stated limitation makes that a concrete next test.
  • As an inference beyond the paper, any future closed-source service that inherits SD1.5's latent space or conditioning layout may be vulnerable to the same proxy attack even if its own interface and weights are new.
  • As an inference beyond the paper, the reported 'critical oversight' result implies that earlier protection benchmarks may overstate Anti-DreamBooth and SimAC when the text encoder is not fine-tuned; re-evaluating those baselines under that standard practice could change their standing.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

6 major / 5 minor

Summary. The paper proposes Anti-Reference, a method for protecting images from misuse by reference-based generation and fine-tuning-based customization of diffusion models. The method adds imperceptible adversarial noise to images, optimized via a unified loss that combines an unconditional diffusion adversarial loss with conditional adversarial losses computed over several conditional modules (IP-Adapter, ReferenceNet-based methods, and human-animation models). The authors introduce an Adversarial Noise Encoder (ANE) based on a ViT/DiT architecture that predicts the noise in a single forward pass, as well as a slower PGD-based variant. Quantitative experiments cover seven customization tasks (DreamBooth, LoRA, Textual Inversion, IP-Adapter, Reference-only, Ecomimic, Magic Animate) with ISM, Aesthetics, and CLIP-IQA metrics; timing and invisibility comparisons are also provided. The paper further reports qualitative robustness to image transformations and gray-box transfer to closed-source APIs (EMO, Animate Anyone).

Significance. If the universality claim is substantiated, this is a practically relevant contribution: the ANE reduces per-image protection time from hundreds of seconds to about a second, and the unified loss is a reasonable extension of prior adversarial approaches such as Anti-DreamBooth and SimAC. The open-source code (GitHub) and the explicit acknowledgment of the SD1.5 scope limitation are positive aspects. However, the current evidence is not yet sufficient for the claimed universality: the main quantitative evaluation is performed on the exact conditional modules used to train the ANE, the face/body evaluation shares data sources with the training set, and the only out-of-sample evidence is qualitative. These issues are load-bearing for the paper's central claim, and additional experiments with held-out models and disjoint data are needed.

major comments (6)
  1. [Sec. 4.3 and Sec. 5.2] The universal-attack claim is supported almost entirely by in-sample evaluation. Section 4.3 states that joint training is performed across IP-Adapter, Reference-only, Magic Animate, and Ecomimic's ReferenceNet, and Section 5.2 evaluates exactly those four conditional modules. The ANE's effectiveness on these tasks therefore partly reflects fitting to the same modules that it was trained against, rather than generalization to unseen reference-based methods. The only out-of-sample quantitative evidence is the gray-box API test (Fig. 5), which is qualitative. Please add a held-out evaluation, e.g., omit one of the four modules from training and test on it, or evaluate on unseen reference-based adapters (e.g., InstantID, PhotoMaker) to support the universality claim.
  2. [Sec. 5.1 and Sec. 5.2] There appears to be train/test identity leakage for the human-centric tasks. Section 5.1 states that the training set includes the Celeb-A dataset (200K) and the TikTok dataset (30K), while Section 5.2 evaluates on 10 identities from CelebA-HQ and on TikTok data. Since CelebA-HQ is a high-quality subset of CelebA, and the TikTok evaluation appears to draw from the same dataset used in training, the reported ISM, Aesthetics, and CLIP-IQA improvements on the face- and body-driven tasks may be inflated by identity or clip overlap. The authors should use identity-disjoint splits or external evaluation datasets and describe the split explicitly.
  3. [Tables 1 and 3] The statement in Sec. 5.2 that "our method achieved leading results" on ISM is contradicted by Table 1: in the Magic Animate row, PhotoGuard achieves ISM 0.134, which is lower (better) than Ours(PGD) at 0.163 and Ours(ANE) at 0.221. Similarly, in Table 3 (CLIP-IQA, where lower is better), SimAC achieves 0.184 on Magic Animate, which is better than both our methods (0.225 and 0.202). The claim should be refined to specify the tasks and metrics where the method actually ranks first, and the discrepancies should be discussed.
  4. [Tables 1-3] No error bars or statistical significance tests are reported anywhere in the quantitative evaluation. All numbers are point estimates, even though some differences between methods are small (e.g., Echomimic ISM: Ours(PGD) 0.655 vs. SimAC 0.673). With 200 generated images per condition, bootstrapped confidence intervals or paired significance tests are needed to support the claimed advantages, especially for the human-animation rows where the ordering is inconsistent across metrics.
  5. [Eq. (2)] Equation (2) defines the diffusion adversarial loss L_adv as a function of x0, which the text explicitly calls "the original data." As written, this loss does not depend on the protected image Iadv and therefore cannot serve as an attack objective on Iadv. For the total loss in Eq. (5) to attack the protected image, the diffusion loss must be computed on x_t constructed from Iadv. Please clarify the notation and make explicit that x0 in Eq. (2) is Iadv (or otherwise correct the formulation), since this is central to the method.
  6. [Sec. 5.2 and Supplementary A] The "Critical Oversight" claim that Anti-DreamBooth and SimAC lose effectiveness when Bob does not fine-tune the CLIP text encoder is supported only by a single qualitative figure in the supplementary material. This claim is used to dismiss the performance of the baselines, so it should be backed by a controlled quantitative comparison of baseline ISM/Aesthetics/CLIP-IQA under both text-encoder fine-tuning and no-fine-tuning settings, with the same evaluation protocol as Table 1.
minor comments (5)
  1. [Abstract] The phrase "a Adversarial Noise Encoder" should be "an Adversarial Noise Encoder."
  2. [Sec. 2.1] The section heading "Custmized Diffusion Model" is a typo; it should be "Customized Diffusion Model."
  3. [Tables 1-3 and text] The name of the method is inconsistently spelled as "Ecomimic" in the text and "Echomimic" in the tables; please standardize the spelling throughout.
  4. [Sec. 4.2] The sentence "We utilized a pre-trained DiT-S/8 model with the same architecture as ANE for the Noise Encoder" is ambiguous; it should say that ANE is initialized from or uses a DiT-S/8 architecture, not that DiT-S/8 has the same architecture as ANE.
  5. [Abstract and Sec. 1] The claim of being the "first" universal method should be positioned more carefully relative to existing universal approaches such as DUAW (Ref. [41]), even if the threat model differs.

Circularity Check

1 steps flagged · score 4.0 of 10

The universal-claim evaluation is partly in-sample: the ANE is trained on the same four conditional modules it is then tested on, so the reported quantitative effectiveness is partly fit rather than independent evidence of universality.

  1. fitted input called prediction [Sec. 4.3 (Loss Function) and Sec. 5.2 (Quantitative Evaluation, Tables 1-3)]
    "This paper conducts joint training across four conditional modules: IP-Adapter [40], Reference-only [31], Magic Animate [39], and Ecomimic's ReferenceNet [6]. ... Our PGD method effectively protects images from the threats of 7 customized generation methods, and our ANE method also demonstrates effectiveness across all tasks."

    Four of the seven evaluated methods in Tables 1-3 are exactly the four conditional modules used to train the Adversarial Noise Encoder in Sec. 4.3. The reported ANE effectiveness on IP-Adapter, Reference-only, Magic Animate, and Ecomimic therefore measures how well the encoder fit its own training objective (maximizing the conditional adversarial loss on those modules), rather than providing independent evidence of a universal, transferable attack. For the PGD variant, Sec. 5.2 states that 'the results of our two methods used for calculating quantitative metrics are all obtained through joint optimization,' meaning those numbers are direct white-box attacks on the same task-specific models, not a demonstration of universality.

full rationale

No equation-level circularity is present: the losses in Eqs. (2)-(5) are standard diffusion adversarial objectives, and the method's equations do not reduce to the evaluation metrics. There is no load-bearing self-citation chain or imported uniqueness theorem; the authors' prior works appear only as background or as target models. The main circularity concern is methodological: the paper trains ANE jointly on IP-Adapter, Reference-only, Magic Animate, and Ecomimic (Sec. 4.3), then uses those same four modules among the seven tasks in the quantitative evaluation (Sec. 5.2) to support the claim of universal effectiveness. This is a fitted-input-presented-as-generalization issue rather than a formal derivation identity, so it warrants a moderate score rather than 6 or above. The method still has independent content, including the fast single-pass ANE design, robustness augmentations, and qualitative gray-box transfer to closed-source APIs, which prevents the central claim from being entirely forced by construction.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method introduces no new physical or conceptual entities. The Adversarial Noise Encoder is a machine learning model trained from data, not a postulated entity. The load-bearing assumptions are about architecture prevalence, the perceptual effect of the noise-prediction loss, proxy transferability, and data representativeness.

free parameters (4)
  • loss weights (wadv, wcon1-4, wreg) = ANE: 30, 50, 60, 30, 30, 200; PGD: 3, 5, 5, 2, 2, 0
    Hand-chosen in Sec. 5.1 to balance attack strength across tasks; they directly determine the trade-off between protection and invisibility.
  • PGD perturbation bound epsilon = 0.05 (13/255)
    Chosen in Sec. 5.1 to keep noise imperceptible; this is the fidelity budget of the attack.
  • PGD step size alpha and iterations T = alpha=1e-3, T=300
    Optimization schedule in Sec. 5.1; affects convergence, runtime, and final noise strength.
  • phase training schedule for ANE = switch conditional models every 1000 steps in phase two
    Ad hoc design choice in Sec. 4.2 intended to avoid overfitting to a single conditional model; not justified by ablation.
assumptions (4)
  • domain assumption Stable Diffusion 1.5 UNet is the dominant base for mainstream reference-based customization methods.
    Sec. 1 and Sec. 4.1: the attack is trained on SD1.5 and the authors state that most reference-based methods rely on SD1.5; the universality claim depends on this prevalence.
  • domain assumption Maximizing the noise prediction error in the diffusion denoiser translates into visible degradation in generated images.
    Eq. (2) and Eq. (3) borrow from Anti-DreamBooth; the paper does not prove the link between the loss and perceptual artifacts, it is an empirical assumption from the adversarial attack literature.
  • domain assumption White-box proxy models share enough structural and latent-space similarity with closed-source target APIs for noise transfer.
    Sec. 4.5: the gray-box attack relies on structural similarity and shared latent spaces; failure of this assumption would invalidate the API transfer claims.
  • domain assumption The training data (LAION, CelebA, TikTok) is representative of images that users will actually protect.
    Sec. 5.1: the encoder's generalization to unseen user photos and diverse content depends on this coverage.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Anti-Reference: Universal and Immediate Defense Against Reference-Based Generation." pith.science (2026). https://pith.science/paper/7F6LL5FH

@misc{pith2026241205980,
  author       = {Pith},
  title        = {Pith review of: Anti-Reference: Universal and Immediate Defense Against Reference-Based Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7F6LL5FH}},
  note         = {Machine review of arXiv:2412.05980}
}
read the original abstract

Diffusion models have revolutionized generative modeling with their exceptional ability to produce high-fidelity images. However, misuse of such potent tools can lead to the creation of fake news or disturbing content targeting individuals, resulting in significant social harm. In this paper, we introduce Anti-Reference, a novel method that protects images from the threats posed by reference-based generation techniques by adding imperceptible adversarial noise to the images. We propose a unified loss function that enables joint attacks on fine-tuning-based customization methods, non-fine-tuning customization methods, and human-centric driving methods. Based on this loss, we train a Adversarial Noise Encoder to predict the noise or directly optimize the noise using the PGD method. Our method shows certain transfer attack capabilities, effectively challenging both gray-box models and some commercial APIs. Extensive experiments validate the performance of Anti-Reference, establishing a new benchmark in image security.

Figures

Figures reproduced from arXiv: 2412.05980 by the authors.

Figure 1
Figure 1. Malicious attackers can collect users’ images as reference images and use diffusion models to achieve malicious purposes. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Illustration of Anti-reference. We propose a loss function to protect images from the threats of customized generation methods, [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Results of different image protection methods in safeguarding images from the threats of customized generation tasks. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Qualitative Evaluation of Method Robustness. Our method is Robustness under prompt mismatch and image transformation. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Gray-box attack results on Tongyi APIs. which attacks multiple customized generation methods, faces greater convergence challenges than single-task meth￾ods, making comparable invisibility difficult to achieve. 5.3. Qualitative Evaluation 5.3.1. Gray-Box Performance In…
Figure 6
Figure 6. Figure 6: We have identified a critical oversight in the current [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 7
Figure 7. Figure 7: More robustness test results: Our method (ANE) is robust against common image transformations. [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: More robustness test results: Our method (PGD) is robust against common image transformations. [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. LayerTracer: Cognitive-Aligned Layered SVG Synthesis via Diffusion Transformer

    cs.CV 2025-02 conditional novelty 6.0 of 10

    A diffusion transformer trained on SVG construction sequences generates and vectorizes layered SVG graphics, breaking creation into editable steps.

  2. Any2AnyTryon: Leveraging Adaptive Position Embeddings for Versatile Virtual Clothing Tasks

    cs.CV 2025-01 conditional novelty 6.0 of 10

    A single DiT-based model with adaptive position embeddings performs virtual try-on, garment reconstruction, model-free try-on, and layered try-on from text and variable-size image inputs.

  3. IDProtector: An Adversarial Noise Encoder to Protect Against ID-Preserving Image Generation

    cs.CV 2024-12 conditional novelty 6.0 of 10

    IDProtector adds imperceptible adversarial noise to a portrait in a single forward pass, disrupting identity-preserving generation by InstantID, IP-Adapter, IP-Adapter-Plus, and PhotoMaker.

  4. RelationAdapter: Learning and Transferring Visual Relation with Diffusion Transformers

    cs.CV 2025-06 conditional novelty 5.0 of 10

    A decoupled-attention adapter transfers image-pair edits to new photos in diffusion transformers, trained with a new 218-task visual editing dataset.

Reference graph

Works this paper leans on

47 extracted references · 16 canonical work pages · cited by 4 Pith papers

  1. [1]

    https://civitai.com/

    Civitai. https://civitai.com/. Accessed: 2024-11-

    work page 2024

  2. [2]

    https://tongyi.aliyun.com/ , 2024

    Tongyi. https://tongyi.aliyun.com/ , 2024. Ac- cessed: 2024-11-15. 8

    work page 2024

  3. [3]

    Aesthetic predictor

    LAION AI. Aesthetic predictor. https://github. com/LAION- AI/aesthetic- predictor, 2023. 7, 1

    work page 2023

  4. [4]

    In- structpix2pix: Learning to follow image editing instructions

    Tim Brooks, Aleksander Holynski, and Alexei A Efros. In- structpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18392–18402, 2023. 2

    2023

  5. [5]

    Livephoto: Real image animation with text-guided motion control

    Xi Chen, Zhiheng Liu, Mengting Chen, Yutong Feng, Yu Liu, Yujun Shen, and Hengshuang Zhao. Livephoto: Real image animation with text-guided motion control. arXiv preprint arXiv:2312.02928, 2023. 2, 3

    arXiv 2023

  6. [6]

    Echomimic: Lifelike audio-driven por- trait animations through editable landmark conditions

    Zhiyuan Chen, Jiajiong Cao, Zhiquan Chen, Yuming Li, and Chenguang Ma. Echomimic: Lifelike audio-driven por- trait animations through editable landmark conditions. arXiv preprint arXiv:2407.08136, 2024. 1, 2, 3, 4, 5, 6

    arXiv 2024

  7. [7]

    Idguard: Ro- bust, general, identity-centric poi proactive defense against face editing abuse

    Yunshu Dai, Jianwei Fei, and Fangjun Huang. Idguard: Ro- bust, general, identity-centric poi proactive defense against face editing abuse. In 2024 IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), pages 11934– 11943. IEEE, 2024. 3

    work page 2024

  8. [8]

    Arcface: Additive angular margin loss for deep face recognition

    Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Arcface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 4690–4699, 2019. 3

    2019

Show all 47 references
  1. [9]

    Restricted black-box adversarial attack against deepfake face swapping

    Junhao Dong, Yuan Wang, Jianhuang Lai, and Xiaohua Xie. Restricted black-box adversarial attack against deepfake face swapping. IEEE Transactions on Information Forensics and Security, 18:2596–2608, 2023. 3

    2023

  2. [10]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020. 2, 4

    2010 arXiv

  3. [11]

    An image is worth one word: Personalizing text-to- image generation using textual inversion

    Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patash- nik, Amit H Bermano, Gal Chechik, and Daniel Cohen- Or. An image is worth one word: Personalizing text-to- image generation using textual inversion. arXiv preprint arXiv:2208.01618, 2022. 1, 2

    2022 arXiv

  4. [12]

    Id-animator: Zero-shot identity-preserving human video generation.arXiv preprint arXiv:2404.15275, 2024

    Xuanhua He, Quande Liu, Shengju Qian, Xin Wang, Tao Hu, Ke Cao, Keyu Yan, Man Zhou, and Jie Zhang. Id-animator: Zero-shot identity-preserving human video generation.arXiv preprint arXiv:2404.15275, 2024. 1, 2

    2024 arXiv

  5. [13]

    Prompt-to-prompt im- age editing with cross attention control

    Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-prompt im- age editing with cross attention control. arXiv preprint arXiv:2208.01626, 2022. 2

    2022 arXiv

  6. [14]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 2

    2020

  7. [15]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021. 1, 2, 5

    2021 arXiv

  8. [16]

    Animate anyone: Consistent and controllable image- to-video synthesis for character animation

    Li Hu. Animate anyone: Consistent and controllable image- to-video synthesis for character animation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8153–8163, 2024. 1, 2, 3, 6, 8

    2024

  9. [17]

    Multi-concept customization of text-to-image diffusion

    Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shechtman, and Jun-Yan Zhu. Multi-concept customization of text-to-image diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 1931–1941, 2023. 1, 2

    1931

  10. [18]

    Photomaker: Customizing re- alistic human photos via stacked id embedding

    Zhen Li, Mingdeng Cao, Xintao Wang, Zhongang Qi, Ming- Ming Cheng, and Ying Shan. Photomaker: Customizing re- alistic human photos via stacked id embedding. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8640–8650, 2024. 3

    2024

  11. [19]

    Adversarial example does good: Preventing paint- ing imitation from diffusion models via adversarial exam- ples

    Chumeng Liang, Xiaoyu Wu, Yang Hua, Jiaru Zhang, Yim- ing Xue, Tao Song, Zhengui Xue, Ruhui Ma, and Haibing Guan. Adversarial example does good: Preventing paint- ing imitation from diffusion models via adversarial exam- ples. arXiv preprint arXiv:2302.04578, 2023. 1, 3, 6

    2023 arXiv

  12. [20]

    Towards deep learning models resis- tant to adversarial attacks

    Aleksander Madry. Towards deep learning models resis- tant to adversarial attacks. arXiv preprint arXiv:1706.06083,

    arXiv

  13. [21]

    Realcus- tom++: Representing images as real-word for real-time cus- tomization

    Zhendong Mao, Mengqi Huang, Fei Ding, Mingcong Liu, Qian He, Xiaojun Chang, and Yongdong Zhang. Realcus- tom++: Representing images as real-word for real-time cus- tomization. arXiv preprint arXiv:2408.09744, 2024. 3

    2024

  14. [22]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 4195–4205,

  15. [23]

    Dreamfusion: Text-to-3d using 2d diffusion

    Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Milden- hall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209.14988, 2022. 2

    2022 arXiv

  16. [24]

    Scal- able universal adversarial watermark defending against fa- cial forgery

    Tong Qiao, Bin Zhao, Ran Shi, Meng Han, Mahmoud Hassaballah, Florent Retraint, and Xiangyang Luo. Scal- able universal adversarial watermark defending against fa- cial forgery. IEEE Transactions on Information Forensics and Security, 2024. 3

    2024

  17. [25]

    Learning transferable visual models from natural language supervision, 2021

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision, 2021. 3

    2021

  18. [26]

    High-resolution image syn- thesis with latent diffusion models, 2021

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models, 2021. 2

    2021

  19. [27]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 2250...

    2023

  20. [28]

    Raising the cost of malicious ai-powered image editing

    Hadi Salman, Alaa Khaddaj, Guillaume Leclerc, Andrew Ilyas, and Aleksander Madry. Raising the cost of malicious ai-powered image editing. arXiv preprint arXiv:2302.06588,

    arXiv

  21. [29]

    Glaze: Protecting artists from style mimicry by {Text-to-Image} models

    Shawn Shan, Jenna Cryan, Emily Wenger, Haitao Zheng, Rana Hanocka, and Ben Y Zhao. Glaze: Protecting artists from style mimicry by {Text-to-Image} models. In 32nd USENIX Security Symposium (USENIX Security 23) , pages 2187–2204, 2023. 3

    2023

  22. [30]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020. 2

    2010 arXiv

  23. [31]

    Stable diffusion reference implementa- tion, 2023

    Hugging Face Team. Stable diffusion reference implementa- tion, 2023. Available online. 2, 4, 5, 6

    2023

  24. [32]

    Emo: Emote portrait alive-generating expressive portrait videos with audio2video diffusion model under weak conditions

    Linrui Tian, Qi Wang, Bang Zhang, and Liefeng Bo. Emo: Emote portrait alive-generating expressive portrait videos with audio2video diffusion model under weak conditions. arXiv preprint arXiv:2402.17485, 2024. 1, 2, 3, 6, 8

    2024 arXiv

  25. [33]

    Anti-dreambooth: Pro- tecting users from personalized text-to-image synthesis

    Thanh Van Le, Hao Phung, Thuan Hoang Nguyen, Quan Dao, Ngoc N Tran, and Anh Tran. Anti-dreambooth: Pro- tecting users from personalized text-to-image synthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2116–2127, 2023. 1, 2, 3, 5, 7, 8

    2023

  26. [34]

    Simac: A simple anti-customization method for pro- tecting face privacy against text-to-image synthesis of diffu- sion models

    Feifei Wang, Zhentao Tan, Tianyi Wei, Yue Wu, and Qidong Huang. Simac: A simple anti-customization method for pro- tecting face privacy against text-to-image synthesis of diffu- sion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, ...

    2024

  27. [35]

    Ex- ploring clip for assessing the look and feel of images

    Jianyi Wang, Kelvin CK Chan, and Chen Change Loy. Ex- ploring clip for assessing the look and feel of images. InPro- ceedings of the AAAI Conference on Artificial Intelligence , pages 2555–2563, 2023. 7, 1

    2023

  28. [36]

    Instantid: Zero-shot identity-preserving gener- ation in seconds

    Qixun Wang, Xu Bai, Haofan Wang, Zekui Qin, and An- thony Chen. Instantid: Zero-shot identity-preserving gener- ation in seconds. arXiv preprint arXiv:2401.07519, 2024. 1, 3

    2024 arXiv

  29. [37]

    Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation

    Jay Zhangjie Wu, Yixiao Ge, Xintao Wang, Weixian Lei, Yuchao Gu, Wynne Hsu, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation. arXiv preprint arXiv:2212.11565, 2022. 2

    2022 arXiv

  30. [38]

    X-portrait: Expressive portrait anima- tion with hierarchical motion attention

    You Xie, Hongyi Xu, Guoxian Song, Chao Wang, Yichun Shi, and Linjie Luo. X-portrait: Expressive portrait anima- tion with hierarchical motion attention. In ACM SIGGRAPH 2024 Conference Papers, pages 1–11, 2024. 1, 2, 3

    2024

  31. [39]

    Magicanimate: Temporally consistent human im- age animation using diffusion model

    Zhongcong Xu, Jianfeng Zhang, Jun Hao Liew, Hanshu Yan, Jia-Wei Liu, Chenxu Zhang, Jiashi Feng, and Mike Zheng Shou. Magicanimate: Temporally consistent human im- age animation using diffusion model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Reco...

    2024

  32. [40]

    Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models

    Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models. arXiv preprint arXiv:2308.06721,

    arXiv

  33. [41]

    Duaw: Data-free universal adversarial watermark against stable dif- fusion customization

    Xiaoyu Ye, Hao Huang, Jiaqi An, and Yongtao Wang. Duaw: Data-free universal adversarial watermark against stable dif- fusion customization. arXiv preprint arXiv:2308.09889 ,

    arXiv

  34. [42]

    I2vgen-xl: High-quality image-to-video synthesis via cascaded diffusion models

    Shiwei Zhang, Jiayu Wang, Yingya Zhang, Kang Zhao, Hangjie Yuan, Zhiwu Qin, Xiang Wang, Deli Zhao, and Jingren Zhou. I2vgen-xl: High-quality image-to-video synthesis via cascaded diffusion models. arXiv preprint arXiv:2311.04145, 2023. 2, 3

    2023 arXiv

  35. [43]

    Ssr-encoder: Encoding selective subject representation for subject-driven generation

    Yuxuan Zhang, Yiren Song, Jiaming Liu, Rui Wang, Jinpeng Yu, Hao Tang, Huaxia Li, Xu Tang, Yao Hu, Han Pan, et al. Ssr-encoder: Encoding selective subject representation for subject-driven generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Rec...

    2024

  36. [44]

    Stable- makeup: When real-world makeup transfer meets diffusion model

    Yuxuan Zhang, Lifu Wei, Qing Zhang, Yiren Song, Jiaming Liu, Huaxia Li, Xu Tang, Yao Hu, and Haibo Zhao. Stable- makeup: When real-world makeup transfer meets diffusion model. arXiv preprint arXiv:2403.07764, 2024. 2

    2024 arXiv

  37. [45]

    Stable-hair: Real-world hair transfer via diffusion model

    Yuxuan Zhang, Qing Zhang, Yiren Song, and Jiaming Liu. Stable-hair: Real-world hair transfer via diffusion model. arXiv preprint arXiv:2407.14078, 2024. 2

    2024 arXiv

  38. [46]

    Understanding and improving adversarial attacks on latent diffusion model

    Boyang Zheng, Chumeng Liang, Xiaoyu Wu, and Yan Liu. Understanding and improving adversarial attacks on latent diffusion model. arXiv preprint arXiv:2310.04687, 2023. 3, 5 Anti-Reference: Universal and Immediate Defense Against Reference-Based Generation Supplementary Material...

    2023

  39. [47]

    For human body and face-driven tasks, we calculated quantitative metrics across all frames

    and CLIP-IQA [35]. For human body and face-driven tasks, we calculated quantitative metrics across all frames. • ISM (Identity Score Matching): Measures the cosine sim- ilarity between the features of the generated face and the original face to evaluate how well the generated ...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.