Pith. sign in

REVIEW 3 major objections 5 minor 43 references

Prompt Augmentation for Self-supervised Text-guided Image Manipulation

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Expanding one prompt into several targets teaches a diffusion model where to edit and what to keep.

desk verdict A genuinely new training-time recipe for mask-free localised editing, but the evidence is too thin to support the state-of-the-art claim. read the letter →

arxiv 2412.13081 v1 pith:HM3BYNC7 submitted 2024-12-17 cs.CV

Rumeysa Bodur , Binod Bhattarai , Tae-Kyun Kim This is my paper · ORCID
classification cs.CV
keywords text-guidedimageeditingpromptaugmentationdiffusionmodelscontrastivelosssoftself-supervisedlearninglatentmanipulation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a single input prompt can be expanded into several target prompts, and that comparing a diffusion model's noise predictions for those prompts reveals which parts of the image should change. On that basis, the authors fine-tune a latent diffusion model with a contrastive loss that pushes the to-be-edited regions apart while pulling the preserved regions together, plus a soft version that scales the push by how similar the prompts are. If this works, localized text-guided editing can be learned from unlabeled image-caption pairs, with no paired training data and no mask at inference time. The paper reports editing results competitive with state-of-the-art methods on public and generated images, and ablation studies show each added component improves the trade-off between translation and preservation.

What carries the argument

The load-bearing mechanism is the noise-difference mask: with $N$ augmented prompts $y_1,\ldots,y_N$, the model computes $\Delta\epsilon=\frac{1}{N_p}\sum_{i,j}|\epsilon_\theta(z_t,t,\tau_\theta(y_i))-\epsilon_\theta(z_t,t,\tau_\theta(y_j))|$ and thresholds it to obtain $M$. The same augmented prompts produce latent images $z_i$, and the Contrastive Loss with $L_p$ and $L_d$ pulls the $(1-M)$ regions together while pushing the $M$ regions apart; the Soft Contrastive Loss scales the push by $\gamma(y_i,y_j)$, the CLIP cosine similarity between prompts. This machinery lets the training mask and the supervision signal both come from the same unlabeled data.

What would settle it

Measure the mask from Eq. 2 against ground-truth edit regions on a set of single-object color-change edits and compute intersection-over-union; if the average IoU is near chance across object categories, the localization premise is false.

Watch

Extended reading notes

Core claim

The paper's central claim is that prompt augmentation turns an unlabeled image-caption pair into a self-supervised editing curriculum. The authors clean noisy web captions with a captioning model, mask a noun or adjective, and use a masked language model plus word-relation pools to generate several target prompts. For a corrupted latent $z_t$, the averaged absolute difference of noise estimates for the augmented prompts, thresholded at 0.4 of its range, defines a binary mask $M$ of the manipulation region. The Contrastive Loss $L_{CL}=L_p+\beta L_d$ then makes masked latent regions diverge from each other while unmasked regions converge to the source, and the Soft Contrastive Loss replaces the hard push with one weighted by the cosine distance between CLIP embeddings of the prompts, with soft prompt augmentation selecting prompts at different similarity levels. The paper argues this yields localized manipulation that preserves context, supported by qualitative comparisons, automatic metrics, a human study, and an ablation.

Load-bearing premise

The method stands or falls on whether the thresholded average difference of noise predictions for augmented prompts marks exactly the region that should change; if that map is noisy or misses the target object, the contrastive loss will preserve or displace the wrong pixels.

Editorial extensions

If this is right

  • Localized image editing can be trained from unlabeled image-caption pairs, without paired datasets or inference-time masks.
  • One fine-tuned model handles edits of different magnitudes, since similar prompts naturally produce smaller displacements under the soft loss.
  • The method is tied to a standard latent diffusion backbone, so it can be dropped into existing text-to-image pipelines without per-image optimization.
  • Cleaning captions with a captioning model makes large noisy web-scale data usable as editing supervision.
  • Greater prompt diversity improves both translation coherence and background preservation, per the ablation study.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the mask from Eq. 2 could be evaluated directly against human-annotated edit regions to separate mask quality from loss design; this would show how much of the gain comes from localization versus contrastive training.
  • Editorial inference: the approach should transfer to other conditional diffusion models and to edits beyond single word swaps, such as style or layout changes, because the mask and loss only require multiple target prompts.
  • Editorial inference: replacing masked-language-model and word-relation prompt generation with a large language model that produces paraphrases and attribute changes could widen mask coverage and improve complex edits.
  • Editorial inference: weighting contrastive pushes by prompt-pair similarity is a general recipe that could be applied to other generation or translation losses.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces prompt augmentation for self-supervised text-guided image manipulation. A single source prompt is expanded into multiple target prompts using BLIP captioning, BERT masked-language modeling, and NLTK word relations. These augmented prompts are fed through a latent diffusion model, and the average absolute difference of their noise predictions (Eq. 2) is thresholded to obtain a binary mask localizing the intended edit. The model is fine-tuned with a contrastive loss (Eqs. 3-6) that pulls unmasked regions together and pushes masked regions apart, plus a soft variant (Eq. 7) that weights the dissimilarity term by CLIP-based prompt similarity. The method is evaluated on 135 images against SDEdit, DiffEdit, DALL-E 2, and InstructPix2Pix using CLIPScore, FID, SSIM-M, CLIP-R-Precision, and a user study, with additional ablations and a hyperparameter study.

Significance. If the localization mechanism works, the contribution is meaningful: it offers a training-time mask and contrastive objective that avoid paired datasets and inference-time masks. Strengths include a clearly specified training objective, component-wise ablations (Table 2), a hyperparameter study (Table 3), and a human evaluation. The main risk is that the noise-difference mask, on which the entire method depends, is never directly validated; combined with the small evaluation set and mixed automatic metrics, the evidence for the central claim is currently incomplete. The paper is therefore a useful step, but its central claim needs stronger empirical support before acceptance.

major comments (3)
  1. [Section 3.2, Eq. (2)] The binary mask M computed by thresholding Δε is the sole localization signal for both the contrastive loss and the soft contrastive loss, but the paper never measures the quality of these masks. Since Eq. (2) averages differences only among augmented target prompts and not between the source and target prompts, global or multi-region edits (e.g., style or background changes) may produce masks that do not align with the user-intended manipulation; the empirically set threshold of 0.4 of the range is not justified by any mask-fidelity experiment. Please add a direct mask evaluation (e.g., IoU or recall against known edit regions, a threshold sweep, and analysis across timesteps) to validate the training signal.
  2. [Section 4.3, Table 1] The quantitative evidence for the abstract's 'improved or competitive' claim is mixed: SSIM-M is 70.39, below DiffEdit (85.14), DALL-E 2 (96.74), and InstructPix2Pix (81.71), while FID is 133, worse than DiffEdit (85) and InstructPix2Pix (123). With only 135 evaluation images and no confidence intervals for CLIPScore, FID, or SSIM-M, the paper should either provide bootstrap confidence intervals or multiple-seed results, or restrict the claim to the metrics where the method is actually competitive (CLIPScore, CLIP-R-Precision, and human preference).
  3. [Sections 4.4-4.5, Tables 2-3] The ablation study and the α hyperparameter study are performed on the same 135-image set as the final comparison, with no separate validation or test split described. Because the mask threshold, α, β, and prompt-augmentation choices are effectively tuned on the evaluation set, the reported improvements in Tables 2 and 3 may reflect fitting to this small set; please report results on a held-out split or explicitly designate and justify the 135 images as a validation set.
minor comments (5)
  1. [Section 3.4, Eq. (7)] The soft contrastive loss divides by γ(y_i,y_j), the cosine distance between CLIP embeddings, which can be zero or arbitrarily small for semantically identical prompts; specify a lower bound (e.g., ε) or a clamped similarity to avoid unbounded loss values.
  2. [Section 4.3] The metric SSIM-M is used but never defined; please state how the masked SSIM is computed, which mask is used, and how it differs from standard SSIM.
  3. [Section 1 and Figure 2] The mask generation is described as an 'attention mask' in the introduction, but the method actually thresholds noise-prediction differences; please use consistent terminology throughout.
  4. [Figure 3 caption] There is a typo 'along wtih heatmaps' in the caption; it should be 'along with heatmaps'.
  5. [Section 3.2, Eq. (2)] The summation over i,j with Np = N(N-1)/2 should specify the summation range (i<j) to avoid ambiguity about double-counting pairs.

Circularity Check

0 steps flagged · score 1.0 of 10

No equation-level circularity; the training mask is a self-referential pseudo-label but not a reduction of the claimed result to its inputs.

full rationale

The paper's derivation chain is not circular in the sense of the claimed result being equivalent to an input by construction. The mask M in Eqs. 2-3 is computed from the model's own noise estimates for augmented prompts and then used to weight the contrastive losses (Eqs. 3-7); this is a self-training/bootstrap scheme rather than an identity, because the final edited images are generated by the fine-tuned diffusion model and are not equal to M or to the noise differences. The similarity weight gamma in Eq. 7 uses CLIP embeddings while evaluation uses CLIPScore, but the loss does not directly optimize CLIPScore and the paper also reports SSIM, FID, and human preference, so this is a mild overlap rather than circularity. Self-citations ([3]-[7]) appear in related-work and augmentation context and are not load-bearing for the central claim; no uniqueness theorem or ansatz is imported from the authors' prior work. The main scientific risk, that noise-difference masks may not localize the intended edit region, is an empirical validity concern about the pseudo-label rather than a circularity, and it would be addressed by a quantitative mask-fidelity evaluation.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The method rests on three families of assumptions: clean and manipulable captions from BLIP, prompt-variant noise differences localizing edits, and CLIP prompt distance measuring edit size. The main free choices are the mask threshold, loss weights, and number of augmented prompts, none of which are rigorously justified.

free parameters (4)
  • Mask threshold for noise-difference binarization = 0.4 of absolute difference range
    Section 3.2: 'we employ a threshold empirically set at 0.4 of the range of absolute differences in noise estimates.' This threshold directly controls which regions are edited and preserved.
  • Weight alpha of contrastive loss in Eqn 6 = 1.0
    Chosen by hyperparameter study in Table 3; CLIPScore drops sharply at alpha=2.0, so 1.0 is selected on the same evaluation metrics.
  • Weight beta of dissimilarity term Ld in Eqn 5 = Not reported
    Eqn 5 combines preservation and dissimilarity with beta, but no value or ablation is given, despite this balance being central to the method.
  • Number of augmented prompts N = Not specified
    Eqn 2 and the losses average over N prompts; N determines mask quality and contrastive pairs but is never reported.
assumptions (5)
  • domain assumption Noise estimates for different text prompts differ mainly in the region to be edited, so averaged absolute differences localize manipulation.
    Section 3.2, Eqn 2: the training mask is built entirely from the pairwise differences of noise predictions. If this assumption fails, the mask and both loss terms are wrong.
  • domain assumption BLIP-generated captions are clean and manipulable enough that replacing one noun or adjective yields a valid target prompt for the depicted image.
    Section 3.1: caption cleaning with BLIP is called 'imperative' for subsequent modifications, yet the validity of the generated target prompts is not validated.
  • domain assumption BERT masked-language replacements plus NLTK synonyms, antonyms, and co-hyponyms produce prompt variations of appropriate difficulty.
    Section 3.1: the candidate pool is assembled from BERT and NLTK, and words are randomly selected; augmented prompts are assumed to be semantically sound without quantitative verification.
  • domain assumption Elementwise multiplication of the binary mask in latent space corresponds to localized spatial editing in pixels.
    Section 3.3: losses apply elementwise masks in latent space, but the paper does not analyze whether latent-space masks align with image-space regions under the autoencoder.
  • domain assumption CLIP embedding distance between prompts is a valid continuous measure of the required edit magnitude.
    Section 3.4: gamma(y_i,y_j) is defined as cosine distance between CLIP embeddings and used to weight the dissimilarity loss; no evidence shows this distance is monotonic in perceptual edit size.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Prompt Augmentation for Self-supervised Text-guided Image Manipulation." pith.science (2026). https://pith.science/paper/HM3BYNC7

@misc{pith2026241213081,
  author       = {Pith},
  title        = {Pith review of: Prompt Augmentation for Self-supervised Text-guided Image Manipulation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HM3BYNC7}},
  note         = {Machine review of arXiv:2412.13081}
}
read the original abstract

Text-guided image editing finds applications in various creative and practical fields. While recent studies in image generation have advanced the field, they often struggle with the dual challenges of coherent image transformation and context preservation. In response, our work introduces prompt augmentation, a method amplifying a single input prompt into several target prompts, strengthening textual context and enabling localised image editing. Specifically, we use the augmented prompts to delineate the intended manipulation area. We propose a Contrastive Loss tailored to driving effective image editing by displacing edited areas and drawing preserved regions closer. Acknowledging the continuous nature of image manipulations, we further refine our approach by incorporating the similarity concept, creating a Soft Contrastive Loss. The new losses are incorporated to the diffusion model, demonstrating improved or competitive image editing results on public datasets and generated images over state-of-the-art approaches.

Figures

Figures reproduced from arXiv: 2412.13081 by the authors.

Figure 1
Figure 1. Text-guided image manipulation. Illustrative exam￾ples generated by our method (bottom row) with localised manip￾ulations based on given text prompts and input images (top row). concurrently. Many pioneering text-guided image editing methods rely on text-guided image generation models [11, 16, 18, 27, 30, 41], which tend to be ill-equipped to address the nuanced challenges of image manipulation. Additionally, method… view at source ↗
Figure 2
Figure 2. Overview of our method. (a) Prompt Augmentation: In order to augment the prompts to facilitate localised image editing we start by refining textual descriptions for source images using the BLIP captioning model [22], resulting in cleaner captions suitable for further processing. Subsequently, we augment this input prompt by generating a range of target prompts using masked language modeling and exploiting word relat… view at source ↗
Figure 3
Figure 3. Qualitative comparison of our method against SDEdit [ [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Qualitative comparison of ablation study results. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 33 canonical work pages

  1. [1]

    Blended diffusion for text-driven editing of natural images

    Omri Avrahami, Dani Lischinski, and Ohad Fried. Blended diffusion for text-driven editing of natural images. Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 18208–18218, 2022. 1, 2

    work page 2022

  2. [2]

    Text2live: Text-driven layered image and video editing

    Omer Bar-Tal, Dolev Ofri-Amar, Rafail Fridman, Yoni Kas- ten, and Tali Dekel. Text2live: Text-driven layered image and video editing. In Proceedings of the European Confer- ence on Computer Vision (ECCV), pages 707–723. Springer,

    work page

  3. [3]

    Inducing optimal at- tribute representations for conditional gans

    Binod Bhattarai and Tae-Kyun Kim. Inducing optimal at- tribute representations for conditional gans. In Proceedings of the European Conference on Computer Vision (ECCV) ,

    work page

  4. [4]

    Auglabel: Exploiting word representations to augment labels for face attribute classification

    Binod Bhattarai, Rumeysa Bodur, and Tae-Kyun Kim. Auglabel: Exploiting word representations to augment labels for face attribute classification. International Conference on Acoustics, Speech, and Signal Processing (ICASSP) , pages 2308–2312, 2020. 1

    work page 2020

  5. [5]

    3d dense geometry-guided facial expression synthesis by adver- sarial learning

    Rumeysa Bodur, Binod Bhattarai, and Tae-Kyun Kim. 3d dense geometry-guided facial expression synthesis by adver- sarial learning. In Proceedings of the Winter Conference on Applicationn of Computer Vision (WACV), 2021. 1

    work page 2021

  6. [6]

    Joint training of hierarchical gans and semantic segmentation for expression translation

    Rumeysa Bodur, Binod Bhattarai, and Tae-Kyun Kim. Joint training of hierarchical gans and semantic segmentation for expression translation. International Conference on Acous- tics, Speech, and Signal Processing (ICASSP) , pages 2308– 2312, 2023. 1

    work page 2023

  7. [7]

    iedit: Localised text-guided image editing with weak supervision

    Rumeysa Bodur, Erhan Gundogdu, Binod Bhattarai, Tae- Kyun Kim, Michael Donoser, and Loris Bazzani. iedit: Localised text-guided image editing with weak supervision. arXiv preprint arXiv:2305.05947, 2023. 2

    arXiv 2023

  8. [8]

    Tim Brooks, Aleksander Holynski, and Alexei A. Efros. In- structpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023. 1, 2, 5, 6, 7

    work page 2023

Show all 43 references
  1. [9]

    Remix: Towards image-to-image translation with limited data

    Jie Cao, Luanxuan Hou, Ming-Hsuan Yang, Ran He, and Zhenan Sun. Remix: Towards image-to-image translation with limited data. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR) ,

  2. [10]

    Masactrl: Tuning-free mu- tual self-attention control for consistent image synthesis and editing

    Mingdeng Cao, Xintao Wang, Zhongang Qi, Ying Shan, Xi- aohu Qie, and Yinqiang Zheng. Masactrl: Tuning-free mu- tual self-attention control for consistent image synthesis and editing. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision (ICCV) , pages 22...

  3. [11]

    DiffEdit: Diffusion-based Semantic Image Editing with Mask Guidance

    Guillaume Couairon, Jakob Verbeek, Holger Schwenk, and Matthieu Cord. DiffEdit: Diffusion-based Semantic Image Editing with Mask Guidance. In International Conference on Learning Representations (ICLR), 2023. 1, 2, 5, 6, 7

    2023

  4. [12]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2009. 5

    2009

  5. [13]

    Bert: Pre-training of deep bidirectional trans- formers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional trans- formers for language understanding. In Proceedings of the 2019 Conference of the North . Association for Computa- tional Linguistics, 2019. 3

    2019

  6. [14]

    Deep em- bedding learning with discriminative sampling policy

    Yueqi Duan, Lei Chen, Jiwen Lu, and Jie Zhou. Deep em- bedding learning with discriminative sampling policy. Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4959–4968, 2019. 3

    2019

  7. [15]

    Diffusion self-guidance for control- lable image generation

    Dave Epstein, Allan Jabri, Ben Poole, Alexei Efros, and Aleksander Holynski. Diffusion self-guidance for control- lable image generation. In Advances in Neural Information Processing Systems (NeurIPS), pages 16222–16239. Curran Associates, Inc., 2023. 2

    2023

  8. [16]

    Prompt-to-prompt image editing with cross attention control

    Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-prompt image editing with cross attention control. In International Confer- ence on Learning Representations (ICLR), 2022. 1, 2, 7

    2022

  9. [17]

    Clipscore: A reference-free evaluation met- ric for image captioning

    Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation met- ric for image captioning. In EMNLP, 2021. 5

    2021

  10. [18]

    Imagic: Text-based real image editing with diffusion models

    Bahjat Kawar, Shiran Zada, Oran Lang, Omer Tov, Huiwen Chang, Tali Dekel, Inbar Mosseri, and Michal Irani. Imagic: Text-based real image editing with diffusion models. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023. 1, 2, 7

    2023

  11. [19]

    Dif- fusionclip: Text-guided diffusion models for robust image manipulation

    Gwanghyun Kim, Taesung Kwon, and Jong Chul Ye. Dif- fusionclip: Text-guided diffusion models for robust image manipulation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2426–2435, 2022. 1, 2

    2022

  12. [20]

    Arbitrary-scale image generation and upsampling using latent diffusion model and implicit neural decoder

    Jinseok Kim and Tae-Kyun Kim. Arbitrary-scale image generation and upsampling using latent diffusion model and implicit neural decoder. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 1

    2024

  13. [21]

    Deep metric learning beyond binary supervi- sion

    Sungyeon Kim, Minkyo Seo, Ivan Laptev, Minsu Cho, and Suha Kwak. Deep metric learning beyond binary supervi- sion. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), pages 2288– 2297, 2019. 3

    2019

  14. [22]

    Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In Interna- tional Conference on Machine Learning (ICML) , 2022. 3, 4

    2022

  15. [23]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Proceedings of the European Conference on Computer Vi- sion (ECCV). Springer, 2014. 5

    2014

  16. [24]

    Regeneration learning of diffusion models with rich prompts for zero-shot image translation

    Yupei Lin, Sen Zhang, Xiaojun Yang, Xiao Wang, and Yukai Shi. Regeneration learning of diffusion models with rich prompts for zero-shot image translation. arXiv preprint arXiv:2305.04651, 2023. 2

    2023 arXiv

  17. [25]

    Towards equitable representation in text-to-image synthesis models with the cross-cultural un- derstanding benchmark (ccub) dataset

    Zhixuan Liu, Youeun Shin, Beverley-Claire Okogwu, Youngsik Yun, Lia Coleman, Peter Schaldenbrand, Jihie Kim, and Jean Oh. Towards equitable representation in text-to-image synthesis models with the cross-cultural un- derstanding benchmark (ccub) dataset. arXiv preprint arXiv:2...

    2023 arXiv

  18. [26]

    Spe- cialist diffusion: Plug-and-play sample-efficient fine-tuning of text-to-image diffusion models to learn any unseen style

    Haoming Lu, Hazarapet Tunanyan, Kai Wang, Shant Navasardyan, Zhangyang Wang, and Humphrey Shi. Spe- cialist diffusion: Plug-and-play sample-efficient fine-tuning of text-to-image diffusion models to learn any unseen style. In Proceedings of the IEEE/CVF Conference on Com- pute...

    2023

  19. [27]

    Sdedit: Guided image synthesis and editing with stochastic differential equa- tions

    Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jia- jun Wu, Jun-Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equa- tions. In International Conference on Learning Representa- tions (ICLR), 2022. 1, 2, 5, 6, 7

    2022

  20. [28]

    Benchmark for compositional text-to- image synthesis

    Dong Huk Park, Samaneh Azadi, Xihui Liu, Trevor Darrell, and Anna Rohrbach. Benchmark for compositional text-to- image synthesis. In Advances in Neural Information Pro- cessing Systems (NeurIPS) Track on Datasets and Bench- marks. Curran, 2021. 7

    2021

  21. [29]

    Efros, Richard Zhang, and Jun-Yan Zhu

    Taesung Park, Alexei A. Efros, Richard Zhang, and Jun-Yan Zhu. Contrastive learning for conditional image synthesis. In Proceedings of the European Conference on Computer Vi- sion (ECCV), 2020. 3

    2020

  22. [30]

    Zero-shot image-to-image translation

    Gaurav Parmar, Krishna Kumar Singh, Richard Zhang, Yijun Li, Jingwan Lu, and Jun-Yan Zhu. Zero-shot image-to-image translation. In Proceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference (SIGGRAPH). ACM, 2023. 1

    2023

  23. [31]

    Hierarchical text-conditional image gen- eration with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gen- eration with clip latents. arXiv preprint arXiv:2204.06125,

    arXiv

  24. [32]

    High-resolution image synthesis with latent diffusion models

    Robin Rombah, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022. 1, 3, 5

    2022

  25. [33]

    U-net: Convolutional networks for biomedical image segmentation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Inter- vention (MICCAI), pages 234–241, Cham, 2015. 3

    2015

  26. [34]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. arXiv preprint arxiv:2208.12242, 2022. 2

    2022 arXiv

  27. [35]

    Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Worts- man, Patrick Schramowski, Srivatsa R

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade W. Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Worts- man, Patrick Schramowski, Srivatsa R. Kundurthy, Kather- ine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, and Jenia Jit...

    2022

  28. [36]

    pytorch-fid: FID Score for PyTorch

    Maximilian Seitzer. pytorch-fid: FID Score for PyTorch. https://github.com/mseitzer/pytorch-fid ,

  29. [37]

    High- fidelity guided image synthesis with latent diffusion models

    Jaskirat Singh, Stephen Gould, and Liang Zheng. High- fidelity guided image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023. 1

    2023

  30. [38]

    Soft contrastive learning for visual localization

    Janine Thoma, Danda Pani Paudel, and Luc V Gool. Soft contrastive learning for visual localization. In Advances in Neural Information Processing Systems (NeurIPS), pages 11119–11130, 2020. 3

    2020

  31. [39]

    Plug-and-play diffusion features for text- driven image-to-image translation

    Narek Tumanyan, Michal Geyer, Shai Bagon, and Tali Dekel. Plug-and-play diffusion features for text- driven image-to-image translation. arXiv preprint arXiv:2211.12572, 2022. 2

    2022 arXiv

  32. [40]

    Fleet, Radu Soricut, Jason Baldridge, Mo- hammad Norouzi, Peter Anderson, and William Chan

    Su Wang, Chitwan Saharia, Ceslee Montgomery, Jordi Pont- Tuset, Shai Noy, Stefano Pellegrini, Yasumasa Onoe, Sarah Laszlo, David J. Fleet, Radu Soricut, Jason Baldridge, Mo- hammad Norouzi, Peter Anderson, and William Chan. Im- agen editor and editbench: Advancing and evaluati...

    2023

  33. [41]

    A latent space of stochastic diffusion models for zero-shot image editing and guidance

    Chen Henry Wu and Fernando De la Torre. A latent space of stochastic diffusion models for zero-shot image editing and guidance. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 7378–7387,

  34. [42]

    Cross-modal contrastive learning for text-to- image generation

    Han Zhang, Jing Yu Koh, Jason Baldridge, Honglak Lee, and Yinfei Yang. Cross-modal contrastive learning for text-to- image generation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021. 3

    2021

  35. [43]

    Image augmentations for gan training

    Zhengli Zhao, Zizhao Zhang, Ting Chen, Sameer Singh, and Han Zhang. Image augmentations for gan training. arXiv preprint arXiv:2006.02595, 2020. 3

    2006 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.