Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

Concept Replacer: Replacing Sensitive Concepts in Diffusion Models via Precision Localization

T0 review · 4 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Concept Replacer localizes a diffusion model's target concept and swaps it out, leaving the rest of the image intact.

desk verdict A plausible localized concept-replacement method whose localization half is solidly evaluated on real-image segmentation, but whose central generation-time replacement claim lacks direct evidence. read the letter →

arxiv 2412.01244 v2 pith:BFZE7RJI submitted 2024-12-02 cs.CV

Lingyun Zhang , Yu Xie , Yanwei Fu , Ping Chen This is my paper · ORCID
classification cs.CV
keywords conceptreplacementdiffusionmodelscross-attentionlocalizationfew-shotlearningerasureimagesafetytext-to-imagegenerationdualprompts
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that unwanted concepts in text-to-image diffusion outputs can be removed by first pinpointing the exact region where the concept appears during the denoising process, then substituting that region with a replacement prompt, all without retraining the base model. If correct, the method would give content-safety tools a surgical alternative to existing global concept-erasure techniques, which tend to alter unrelated parts of the image. The authors demonstrate the approach on sensitive concepts like nudity and on named individuals, showing that non-target areas remain visually consistent with the original generation.

What carries the argument

The Dual Prompts Cross-Attention (DPCA) module is the central mechanism: at each cross-attention layer it computes Z = Softmax(Q·K_R^T/√d)·V_R·M + Softmax(Q·K_P^T/√d)·V_P·(1−M), where M is the concept mask, (K_R,V_R) come from the replacement prompt, and (K_P,V_P) from the original prompt. The concept localizer produces M by fusing low- and high-resolution cross-attention scores (Across = A_l_cross + A_l_cross·A_h_cross) and multiplying them with self-attention scores, after few-shot fine-tuning of the W_k and W_v projection matrices in the U-Net.

What would settle it

Generate images from prompts containing a known object (e.g., 'a dog'), apply the concept localizer at the early denoising timesteps it uses, then compare the predicted mask against the ground-truth segmentation of the final image (e.g., using an off-the-shelf segmenter or human annotation). If the IoU of those masks is substantially lower than the IoU reported on real-image benchmarks, the claim that precise localization transfers to generation-time concepts is falsified.

Watch

Extended reading notes

Core claim

The core discovery is that a few-shot fine-tuned concept localizer can extract a precise spatial mask of a target concept from cross-attention and self-attention maps at early denoising timesteps, and that this mask can feed a training-free Dual Prompts Cross-Attention (DPCA) module. The DPCA module computes attention with two prompt conditionings: the original input prompt outside the mask and a replacement prompt inside the mask, so that only the localized region changes. The paper reports that this preserves non-target regions better than methods that guide or fine-tune the whole diffusion model.

Load-bearing premise

The few-shot fine-tuned concept localizer, validated only on segmenting real photos, is assumed to also produce accurate masks for concepts generated from scratch during denoising, including abstract concepts like nudity, with no training labels or generation-time mask accuracy provided for those cases.

Editorial extensions

If this is right

  • Sensitive content could be removed or replaced in generated images without retraining or fine-tuning the diffusion U-Net, preserving the model's original distribution elsewhere.
  • The approach could make content moderation more user-friendly: instead of blocking an entire image, only the problematic concept is swapped out, leaving the composition and style intact.
  • Few-shot localization suffices to identify concepts, so the method could be adapted to new unwanted concepts with only a handful of labeled examples.
  • Because the replacement is prompt-driven, users could choose any substitute concept, not just a fixed safe placeholder.
  • The mask-guided cross-attention mechanism could be reused for other localized image manipulations beyond safety, such as targeted object editing within a generated scene.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's claim that early-timestep masks transfer to generated concepts would be directly tested by comparing localizer masks produced at denoising timesteps against ground-truth segmentations of the final generated image; this is an inference from the methods section, not something the paper quantifies for sensitive concepts.
  • If the localizer's masks are accurate enough, the same DPCA machinery could be extended to simultaneous replacement of multiple distinct concepts by assigning each mask its own replacement prompt, a natural extension the paper does not explore.
  • A practical limitation not discussed is the dependence on the localizer's few-shot tuning: the paper validates few-shot segmentation only on real-image benchmarks, so the transfer to abstract and high-level concepts like nudity remains an untested assumption that a reader should keep in mind.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes Concept Replacer, a method for replacing a user-specified concept in text-to-image diffusion models at generation time. The method has two components: a few-shot trained concept localizer that fuses low- and high-resolution cross-attention maps with self-attention to predict a mask for the target concept during denoising, and a training-free Dual Prompts Cross-Attention (DPCA) module that applies the original prompt outside the mask and a replacement prompt inside the mask. The authors evaluate localization on CelebAMask-HQ and Pascal-Car segmentation benchmarks against SLiMe and other baselines, and evaluate replacement on the I2P prompts with NudeNet detection, plus FID/CLIP on COCO 30k, together with qualitative examples of replacing nudity, celebrities, and other concepts.

Significance. If the central claims hold, the paper offers a useful alternative to global concept-erasure methods: a few-shot localizer that runs during denoising and a training-free attention gate that confines the edit to the target region. The idea of combining a fine-tuned localizer with dual-prompt cross-attention is original, and the authors provide code and compare against several prior methods. However, the current evidence does not fully establish the two load-bearing claims of 'precise localization during generation' and 'minimal impact on non-target areas.' The localization evaluation is confined to real-image segmentation, while the replacement experiments report no generation-time mask accuracy and no quantitative outside-mask fidelity metric. The training protocol for the sensitive-concept localizer is also unspecified. These gaps are fixable with additional experiments, so the manuscript merits revision rather than rejection.

major comments (4)
  1. [Section 4.2, Figure 7] The central claim of precise concept localization during generation is not quantitatively validated. Section 4.1 evaluates the localizer only on real-image segmentation benchmarks (CelebAMask-HQ and Pascal-Car), but the replacement experiments in Section 4.2 apply the localizer to concepts such as nudity inside latents during denoising. No generation-time mask IoU, pixel accuracy, or other localization metric is reported for these generated-image cases, and no error bars are given for any of the quantitative results. Please add a quantitative evaluation of mask quality on generated images for the target concepts used in replacement, ideally with manual labels or a pseudo-label protocol, and report variability across seeds.
  2. [Section 4.2, Eq. (9), Table 3] The claim that non-target areas remain 'minimal[ly] impact[ed]' is asserted but never directly measured. Eq. (9) gates the cross-attention by the mask M, but later denoising steps, self-attention, and the mask boundary can propagate changes outside M, so even a perfect mask does not guarantee unchanged non-target regions. Table 3 reports only global FID and CLIP on COCO 30k, which cannot detect localized leakage, and Figure 7 reports only NudeNet-based nudity reduction. Please add a quantitative preservation metric, such as LPIPS/SSIM computed only on the non-masked region between the original and replaced images (or between paired generations with and without DPCA), to substantiate the minimal-impact claim.
  3. [Section 3.1, Section 4.2] The training protocol for the concept localizer used in the replacement experiments is unspecified. Section 4.1 describes training on CelebAMask-HQ and Pascal-Car with dense segmentation labels, but no information is given about how the localizer is trained for concepts such as nudity, blood, skull, knife, zombie, ghost, gun, or monster used in Figures 5 and 6: what training images, what labels, what location-prompt wording, how many shots, what fine-tuning steps, and what loss weighting. Without this information, the results are not reproducible, and the transfer of the real-image localizer to abstract or under-specified concepts during generation is an unverified premise. Please provide the full training setup for the sensitive-concept localizers.
  4. [Section 5, Figure 9] The choice of replacement timestep T=666 and the averaging timesteps T=666, 726, 766 is made qualitatively from a single face-replacement example (Brad Pitt to Leonardo DiCaprio), with no quantitative criterion. This hyperparameter directly controls the trade-off between replacement strength and preservation of the original structure, which is central to the paper's non-target-preservation claim. Please provide a quantitative sensitivity analysis over timesteps, including a preservation metric and a replacement-effectiveness metric (e.g., NudeNet reduction or CLIP-based target-concept score), and justify the chosen value on that basis.
minor comments (6)
  1. [Eqs. (5)-(7), (9)] The notation for the mask M is inconsistent: Eq. (5) defines M as vec(Across) * Aself, which is a vector, while Eq. (9) uses M as a spatial mask. Please clarify the shapes and how the losses in Eqs. (6)-(7) align with the spatial segmentation labels.
  2. [Figure 7] The bar chart in Figure 7 reports percentage reductions in nudity content without numeric values or error bars. Please provide the underlying numbers in a table (with standard deviations over multiple seeds or prompt subsets) so readers can verify the claimed improvements over baselines.
  3. [Section 5, Figure 9] The time-index terminology is confusing: the text says 'For T = 0, it refers to the initial image generated using the prompt,' which conflicts with the standard diffusion convention where T=0 is the final clean image, and with the caption's statement that high T corresponds to the early diffusion phase. Please fix the notation and the wording to make the direction of time clear.
  4. [Section 2.2] The related-work paragraph cites 'Ablating concepts [38]' but reference [38] is the SLD paper; Ablating Concepts is reference [23]. Please correct the citation.
  5. [Section 2.2] The sentence 'it is challenging to get an accurate Classifier also returning a meaningless image is not user-friendly' is ungrammatical and appears to be missing a clause break. Please revise.
  6. [Tables 1 and 2] There are minor formatting inconsistencies: 'ReGAn' appears with inconsistent capitalization, and the table captions for supervised vs. few-shot rows could be clearer about which rows use which training regime.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the localization and replacement claims rest on external training data and external evaluators, not on self-referential definitions.

full rationale

The paper's derivation chain is self-contained against external evidence. The concept localizer is trained with cross-entropy and MSE losses against dense segmentation labels on CelebAMask-HQ and Pascal-Car (Eqs. 6-7), and its localization precision is measured with mIoU against those external datasets. The DPCA module (Eq. 9) is a training-free construction that gates cross-attention by the predicted mask; it is not a fitted quantity and does not define the target concept in terms of the output. Replacement effectiveness is evaluated with the external NudeNet classifier on I2P prompts and with FID/CLIP on COCO. The choice of replacement timestep T=666 and the 32x32 resolution split are manual hyperparameter choices motivated by qualitative structure preservation, not parameters fitted to the evaluation metrics, so they do not make any reported quantity equivalent by construction. The main weakness is empirical rather than circular: the localizer is validated on real-image segmentation but used for generation-time localization of abstract concepts, and non-target consistency is not quantitatively measured. Those are validity/evidence gaps, not circular reductions.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central claim rests on assumptions about attention-map semantics in the pre-trained U-Net, the validity of the mask fusion rule, and the coherence of training-free dual-prompt attention replacement. Two operating points, the resolution threshold and the replacement timestep, are hand-picked. No new entities are introduced. The localizer is trained with external segmentation labels, and evaluation uses external datasets and NudeNet, so the result is not definitionally circular.

free parameters (2)
  • Low/high cross-attention resolution threshold = 32 x 32 spatial dimension
    In Section 3.1, the authors state they 'empirically divided' cross-attention layers with spatial dimension less than 32x32 into low resolution and others into high resolution. The threshold affects the mask refinement in Eq. 4.
  • Replacement and averaging timesteps = T=666, with localizer averaged over T=666, 726, 766
    In Section 5 and Figure 9, T=666 is selected as the 'optimal moment' based on qualitative inspection of replacing Brad Pitt with Leonardo DiCaprio. This choice affects both localization and replacement quality and is not derived from a principle.
assumptions (3)
  • domain assumption Pre-trained Stable Diffusion U-Net attention maps contain sufficient spatial and semantic information for few-shot localization of arbitrary concepts.
    The concept localizer is built by fine-tuning Wk and Wv projection matrices in the existing U-Net attention layers and extracting attention scores. The paper relies on this property in Section 3.1, citing SLiMe and DiffSegmenter.
  • ad hoc to paper Cross-attention maps at low and high resolutions can be combined multiplicatively to produce a mask that is both semantically accurate and detail-preserving.
    Equation 4 defines Across = Al_cross + Al_cross * Ah_cross as a design choice supported only by qualitative observation in Figure 3, with no ablations comparing alternative fusion rules.
  • ad hoc to paper Applying replacement-prompt conditioning only inside the localized mask yields a coherent image without visible seams or semantic artifacts.
    The DPCA module is training-free and is presented without a derivation; the claim that it produces minimal disruption to non-target areas is supported only by qualitative examples in Figure 6 and global FID and CLIP scores.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Concept Replacer: Replacing Sensitive Concepts in Diffusion Models via Precision Localization." pith.science (2026). https://pith.science/paper/BFZE7RJI

@misc{pith2026241201244,
  author       = {Pith},
  title        = {Pith review of: Concept Replacer: Replacing Sensitive Concepts in Diffusion Models via Precision Localization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BFZE7RJI}},
  note         = {Machine review of arXiv:2412.01244}
}
read the original abstract

As large-scale diffusion models continue to advance, they excel at producing high-quality images but often generate unwanted content, such as sexually explicit or violent content. Existing methods for concept removal generally guide the image generation process but can unintentionally modify unrelated regions, leading to inconsistencies with the original model. We propose a novel approach for targeted concept replacing in diffusion models, enabling specific concepts to be removed without affecting non-target areas. Our method introduces a dedicated concept localizer for precisely identifying the target concept during the denoising process, trained with few-shot learning to require minimal labeled data. Within the identified region, we introduce a training-free Dual Prompts Cross-Attention (DPCA) module to substitute the target concept, ensuring minimal disruption to surrounding content. We evaluate our method on concept localization precision and replacement efficiency. Experimental results demonstrate that our method achieves superior precision in localizing target concepts and performs coherent concept replacement with minimal impact on non-target areas, outperforming existing approaches.

Figures

Figures reproduced from arXiv: 2412.01244 by the authors.

Figure 1
Figure 1. Given a specified concept for replacement, our method precisely identifies its location during the generation phase and seamlessly [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Framework of Our Method. Our approach is designed to replace a specified target concept during image generation within [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Visualization of Cross-Attention Maps at Different Spa [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Segmentation Results on CelebAMask-HQ and Pascal [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Concept Localization Results with the Proposed Concept [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Nudity Concept Replacement Results. Unlike other methods, our approach identifies the target concept during image generation, [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Results of Nudity Concept Removal. We present percentage reductions in nudity content relative to original Stable Diffusion [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: Concept Replacement with Various Replacing Prompts. Our method accurately identifies the specified concept and seamlessly [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: Impact of Replacing Timesteps. Replacing Brad Pitt [PITH_FULL_IMAGE:figures/full_fig_p009_9.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. NSFW-Classifier Guided Prompt Sanitization for Safe Text-to-Image Generation

    cs.CV 2025-06 conditional novelty 6.0 of 10

    PromptSan reduces NSFW image generation by sanitizing input prompts through classifier-guided token edits or a learned safety suffix, with quantitative validation limited to nudity on the I2P benchmark.

Reference graph

Works this paper leans on

49 extracted references · 12 canonical work pages · cited by 1 Pith paper

  1. [1]

    ediffi: Text-to-image diffusion models with an ensemble of expert denoisers.arXiv preprint arXiv:2211.01324, 2022

    Yogesh Balaji, Seungjun Nah, Xun Huang, Arash Vahdat, Jiaming Song, Karsten Kreis, Miika Aittala, Timo Aila, Samuli Laine, Bryan Catanzaro, et al. ediffi: Text-to-image diffusion models with an ensemble of expert denoisers.arXiv preprint arXiv:2211.01324, 2022. 2

    arXiv 2022

  2. [2]

    Label-efficient se- mantic segmentation with diffusion models

    Dmitry Baranchuk, Ivan Rubachev, Andrey V oynov, Valentin Khrulkov, and Artem Babenko. Label-efficient se- mantic segmentation with diffusion models. arXiv preprint arXiv:2112.03126, 2021. 7

    arXiv 2021

  3. [3]

    Nudenet: Neural nets for nudity detec- tion and censoring,

    Praneeth Bedapudi. Nudenet: Neural nets for nudity detec- tion and censoring,. 2022. 8

    work page 2022

  4. [4]

    In- structpix2pix: Learning to follow image editing instructions

    Tim Brooks, Aleksander Holynski, and Alexei A Efros. In- structpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18392–18402, 2023. 3

    2023

  5. [5]

    Muse: Text-to-image generation via masked generative transform- ers

    Huiwen Chang, Han Zhang, Jarred Barber, AJ Maschinot, Jose Lezama, Lu Jiang, Ming-Hsuan Yang, Kevin Mur- phy, William T Freeman, Michael Rubinstein, et al. Muse: Text-to-image generation via masked generative transform- ers. arXiv preprint arXiv:2301.00704, 2023. 2

    arXiv 2023

  6. [6]

    Conceptual 12m: Pushing web-scale image-text pre- training to recognize long-tail visual concepts

    Soravit Changpinyo, Piyush Sharma, Nan Ding, and Radu Soricut. Conceptual 12m: Pushing web-scale image-text pre- training to recognize long-tail visual concepts. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3558–3568, 2021. 2

    2021

  7. [7]

    Attend-and-excite: Attention-based se- mantic guidance for text-to-image diffusion models

    Hila Chefer, Yuval Alaluf, Yael Vinker, Lior Wolf, and Daniel Cohen-Or. Attend-and-excite: Attention-based se- mantic guidance for text-to-image diffusion models. ACM Transactions on Graphics (TOG), 42(4):1–10, 2023. 3

    2023

  8. [8]

    Detect what you can: Detecting and representing objects using holistic mod- els and body parts

    Xianjie Chen, Roozbeh Mottaghi, Xiaobai Liu, Sanja Fi- dler, Raquel Urtasun, and Alan Yuille. Detect what you can: Detecting and representing objects using holistic mod- els and body parts. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 1971–1978,

    1971

Show all 49 references
  1. [9]

    Diffedit: Diffusion-based seman- tic image editing with mask guidance

    Guillaume Couairon, Jakob Verbeek, Holger Schwenk, and Matthieu Cord. Diffedit: Diffusion-based seman- tic image editing with mask guidance. arXiv preprint arXiv:2210.11427, 2022. 3

    2022 arXiv

  2. [10]

    Cogview2: Faster and better text-to-image generation via hierarchical transformers

    Ming Ding, Wendi Zheng, Wenyi Hong, and Jie Tang. Cogview2: Faster and better text-to-image generation via hierarchical transformers. Advances in Neural Information Processing Systems, 35:16890–16902, 2022. 2

    2022

  3. [11]

    Training-free structured diffusion guidance for compositional text-to-image synthesis

    Weixi Feng, Xuehai He, Tsu-Jui Fu, Varun Jampani, Arjun Akula, Pradyumna Narayana, Sugato Basu, Xin Eric Wang, and William Yang Wang. Training-free structured diffusion guidance for compositional text-to-image synthesis. arXiv preprint arXiv:2212.05032, 2022. 3

    2022 arXiv

  4. [12]

    Make-a-scene: Scene- based text-to-image generation with human priors

    Oran Gafni, Adam Polyak, Oron Ashual, Shelly Sheynin, Devi Parikh, and Yaniv Taigman. Make-a-scene: Scene- based text-to-image generation with human priors. In Eu- ropean Conference on Computer Vision , pages 89–106. Springer, 2022. 2

    2022

  5. [13]

    Erasing concepts from diffusion models

    Rohit Gandikota, Joanna Materzynska, Jaden Fiotto- Kaufman, and David Bau. Erasing concepts from diffusion models. arXiv preprint arXiv:2303.07345, 2023. 2, 3, 8

    2023 arXiv

  6. [14]

    Unified concept editing in diffusion models

    Rohit Gandikota, Hadas Orgad, Yonatan Belinkov, Joanna Materzy´nska, and David Bau. Unified concept editing in diffusion models. In Proceedings of the IEEE/CVF Win- ter Conference on Applications of Computer Vision , pages 5111–5120, 2024. 3

    2024

  7. [15]

    Generative adversarial nets

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014. 7

    2014

  8. [16]

    Focus on your instruction: Fine-grained and multi-instruction image editing by atten- 10 tion modulation

    Qin Guo and Tianwei Lin. Focus on your instruction: Fine-grained and multi-instruction image editing by atten- 10 tion modulation. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition , pages 6986–6996, 2024. 3

    2024

  9. [17]

    Selective amnesia: A contin- ual learning approach to forgetting in deep generative mod- els

    Alvin Heng and Harold Soh. Selective amnesia: A contin- ual learning approach to forgetting in deep generative mod- els. Advances in Neural Information Processing Systems, 36,

  10. [18]

    Prompt-to-prompt im- age editing with cross attention control

    Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-prompt im- age editing with cross attention control. arXiv preprint arXiv:2208.01626, 2022. 3

    2022 arXiv

  11. [19]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 3, 7

    2020

  12. [20]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021. 3

    2021 arXiv

  13. [21]

    Imagic: Text-based real image editing with diffusion models

    Bahjat Kawar, Shiran Zada, Oran Lang, Omer Tov, Huiwen Chang, Tali Dekel, Inbar Mosseri, and Michal Irani. Imagic: Text-based real image editing with diffusion models. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6007–6017, 2023. 3

    2023

  14. [22]

    Slime: Seg- ment like me

    Aliasghar Khani, Saeid Asgari Taghanaki, Aditya Sanghi, Ali Mahdavi Amiri, and Ghassan Hamarneh. Slime: Seg- ment like me. arXiv preprint arXiv:2309.03179, 2023. 2, 3, 5, 7

    2023 arXiv

  15. [23]

    Ablating con- cepts in text-to-image diffusion models

    Nupur Kumari, Bingliang Zhang, Sheng-Yu Wang, Eli Shechtman, Richard Zhang, and Jun-Yan Zhu. Ablating con- cepts in text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 22691–22702, 2023. 3

    2023

  16. [24]

    Maskgan: Towards diverse and interactive facial image ma- nipulation

    Cheng-Han Lee, Ziwei Liu, Lingyun Wu, and Ping Luo. Maskgan: Towards diverse and interactive facial image ma- nipulation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5549–5558,

  17. [25]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceeding...

    2014

  18. [26]

    Tf-icon: Diffusion-based training-free cross-domain image composi- tion

    Shilin Lu, Yanzhu Liu, and Adams Wai-Kin Kong. Tf-icon: Diffusion-based training-free cross-domain image composi- tion. In Proceedings of the IEEE/CVF International Confer- ence on Computer Vision, pages 2294–2305, 2023. 2

    2023

  19. [27]

    Mace: Mass concept erasure in diffu- sion models

    Shilin Lu, Zilan Wang, Leyang Li, Yanzhu Liu, and Adams Wai-Kin Kong. Mace: Mass concept erasure in diffu- sion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6430– 6440, 2024. 3

    2024

  20. [28]

    Midjourney: An AI Art Generator

    Midjourney. Midjourney: An AI Art Generator. https: //www.midjourney.com. 2

  21. [29]

    Watch your steps: Local image and scene editing by text instructions

    Ashkan Mirzaei, Tristan Aumentado-Armstrong, Marcus A Brubaker, Jonathan Kelly, Alex Levinshtein, Konstantinos G Derpanis, and Igor Gilitschenski. Watch your steps: Local image and scene editing by text instructions. In European Conference on Computer Vision , pages 111–129. Springer,

  22. [30]

    Localizing object-level shape variations with text-to-image diffusion models

    Or Patashnik, Daniel Garibi, Idan Azuri, Hadar Averbuch- Elor, and Daniel Cohen-Or. Localizing object-level shape variations with text-to-image diffusion models. In Proceed- ings of the IEEE/CVF International Conference on Com- puter Vision, pages 23051–23061, 2023. 3

    2023

  23. [31]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  24. [32]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International Confer- ence on Machine Learning, pages 8821–8831. PMLR, 2021. 2, 3

    2021

  25. [33]

    Hierarchical text-conditional image gener- ation with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gener- ation with clip latents. arXiv preprint arXiv:2204.06125, 1 (2):3, 2022. 2

    2022 arXiv

  26. [34]

    Red-teaming the stable diffusion safety filter

    Javier Rando, Daniel Paleka, David Lindner, Lennart Heim, and Florian Tram`er. Red-teaming the stable diffusion safety filter. arXiv preprint arXiv:2210.04610, 2022. 2

    2022 arXiv

  27. [35]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 2

    2022

  28. [36]

    U- net: Convolutional networks for biomedical image segmen- tation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U- net: Convolutional networks for biomedical image segmen- tation. In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Pa...

    2015

  29. [37]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in Neural Information...

    2022

  30. [38]

    Safe latent diffusion: Mitigating inappro- priate degeneration in diffusion models

    Patrick Schramowski, Manuel Brack, Bj ¨orn Deiseroth, and Kristian Kersting. Safe latent diffusion: Mitigating inappro- priate degeneration in diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22522–22531, 2023. 2, 3, 8

    2023

  31. [39]

    Laion-5b: An open large-scale dataset for training next generation image-text models

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Worts- man, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in Neural In- f...

    2022

  32. [40]

    Stable Diffusion 2.0: A Text-to-Image Diffu- sion Model

    Stability AI. Stable Diffusion 2.0: A Text-to-Image Diffu- sion Model. https://stability.ai. 2, 3, 8

  33. [41]

    Repurposing gans for one-shot se- 11 mantic part segmentation

    Nontawat Tritrong, Pitchaporn Rewatbowornwong, and Su- pasorn Suwajanakorn. Repurposing gans for one-shot se- 11 mantic part segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 4475–4485, 2021. 7

    2021

  34. [42]

    Plug-and-play diffusion features for text-driven image-to-image translation

    Narek Tumanyan, Michal Geyer, Shai Bagon, and Tali Dekel. Plug-and-play diffusion features for text-driven image-to-image translation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 1921–1930, 2023. 3

    1921

  35. [43]

    Attention is all you need

    A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017. 6

    2017

  36. [44]

    Diffusion model is secretly a training-free open vocabulary semantic segmenter

    Jinglong Wang, Xiawei Li, Jing Zhang, Qingyuan Xu, Qin Zhou, Qian Yu, Lu Sheng, and Dong Xu. Diffusion model is secretly a training-free open vocabulary semantic segmenter. arXiv preprint arXiv:2309.02773, 2023. 2, 3, 5

    2023 arXiv

  37. [45]

    Seggpt: Segmenting ev- erything in context

    Xinlong Wang, Xiaosong Zhang, Yue Cao, Wen Wang, Chunhua Shen, and Tiejun Huang. Seggpt: Segmenting ev- erything in context. arXiv preprint arXiv:2304.03284, 2023. 7

    2023 arXiv

  38. [46]

    Harnessing the spatial- temporal attention of diffusion models for high-fidelity text- to-image synthesis

    Qiucheng Wu, Yujian Liu, Handong Zhao, Trung Bui, Zhe Lin, Yang Zhang, and Shiyu Chang. Harnessing the spatial- temporal attention of diffusion models for high-fidelity text- to-image synthesis. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision, page...

  39. [47]

    Versatile diffusion: Text, images and variations all in one diffusion model

    Xingqian Xu, Zhangyang Wang, Gong Zhang, Kai Wang, and Humphrey Shi. Versatile diffusion: Text, images and variations all in one diffusion model. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 7754–7765, 2023. 2

    2023

  40. [48]

    Scaling autoregres- sive models for content-rich text-to-image generation

    Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gun- jan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yin- fei Yang, Burcu Karagol Ayan, et al. Scaling autoregres- sive models for content-rich text-to-image generation. arXiv preprint arXiv:2206.10789, 2(3):5, 2022. 2, 3

    2022 arXiv

  41. [49]

    Forget-me-not: Learning to forget in text-to-image diffusion models

    Gong Zhang, Kai Wang, Xingqian Xu, Zhangyang Wang, and Humphrey Shi. Forget-me-not: Learning to forget in text-to-image diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1755–1764, 2024. 3 12

    2024

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.