Pith. sign in

REVIEW 4 major objections 3 minor 1 cited by

Negative Token Merging: Image-based Adversarial Feature Guidance

T0 review · 4 major / 3 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Diffusion outputs can be steered away from reference images by negative token merging, a training-free module.

desk verdict NegToMe is a genuinely new, training-free token-level adversarial guidance trick with solid diversity results, but the headline 34.57% copyright reduction is not backed by the paper's own Table 1. read the letter →

arxiv 2412.01339 v2 pith:2HCJHFGN submitted 2024-12-02 cs.CV cs.AIcs.GRcs.LGstat.ML

Jaskirat Singh , Lindsey Li , Weijia Shi , Ranjay Krishna , Yejin Choi , Pang Wei Koh , Michael F. Cohen , Stephen Gould
show 2 more authors
Liang Zheng Luke Zettlemoyer
This is my paper · ORCID
classification cs.CVcs.AIcs.GRcs.LGstat.ML
keywords negativetokenmergingadversarialguidancediffusionmodelsoutputdiversitycopyrightmitigationtraining-freeimage-based
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes Negative Token Merging (NegToMe), a training-free module inserted into diffusion transformer blocks that steers generation away from a reference image by pushing each output token away from its best-matching reference token. It claims that this image-based adversarial guidance complements text-based negative prompts and works where negative prompts fail, such as models that do not natively support them. The two demonstrated applications are increased output diversity when other batch images are used as references and reduced visual similarity to copyrighted characters when copyrighted reference images are used. The authors report that NegToMe lowers DreamSim similarity to copyrighted content by 34.57% and adds under 4% inference overhead on SDXL and Flux.

What carries the argument

The load-bearing object is the cross-image token correspondence computed from noisy intermediate diffusion features. Given attention-block outputs $O_{\mathrm{src}}$ and reference tokens $O_{\mathrm{ref}}$, NegToMe normalizes them, forms the cosine-similarity matrix $S = \tilde{O}_{\mathrm{src}} \cdot \tilde{O}_{\mathrm{ref}}^T$, takes the argmax match for each source token, masks out matches below threshold $\tau$, and applies linear extrapolation $O_{\mathrm{merge}} = (1+\alpha_t) O_{\mathrm{src}} - \alpha_t O_{\mathrm{target}}$. This operation is inserted between the attention and MLP branches of each transformer block; a mask bias $\log(\tilde{M}_{\mathrm{ref}}+\epsilon)$ restricts guidance to chosen regions. The whole procedure is about ten lines of code and requires no training.

What would settle it

Run NegToMe with a reference image that has no semantic content, such as pure Gaussian noise, keeping the threshold fixed. If diversity or copyright-similarity metrics still shift as much as with a real reference image, the effect does not depend on semantic token correspondence; if the shift vanishes, the semantic-matching assumption is supported.

Watch

Extended reading notes

Core claim

NegToMe's central claim is that adversarial guidance can be performed directly in visual feature space, without retraining, by applying a linear extrapolation at every transformer block during reverse diffusion: $O_{\mathrm{merge}} = (1+\alpha_t) O_{\mathrm{src}} - \alpha_t O_{\mathrm{target}}$, where each source token is matched to its cosine-most-similar target token in the reference image and only tokens above a similarity threshold are moved. The paper argues that this pushes the generated image away from the reference in a semantically meaningful way, and that changing the reference changes the application: batch references increase diversity, copyrighted character references reduce similarity to those characters, blurry references improve aesthetics, and masked references allow targeting specific parts. The authors claim the mechanism is model-agnostic, including MM-DiT architectures like Flux that do not natively support negative prompts.

Load-bearing premise

The method assumes that cosine similarity between noisy intermediate diffusion features reliably identifies semantically matching tokens at every timestep, and that linearly extrapolating those matched tokens apart moves the final image in the intended direction.

Editorial extensions

If this is right

  • If NegToMe works as claimed, diffusion models can be steered adversarially without text, which matters for models like Flux that lack native negative-prompt support.
  • Using other batch images as references offers a training-free route to increase demographic and visual diversity without the inference cost of prompt rewriting.
  • Copyright mitigation can be combined with negative prompts, since the paper reports the best similarity reduction when both are used together.
  • The same module can be repurposed for style exclusion, aesthetic improvement, and object feature interpolation or extrapolation by changing the reference image.
  • Because the mechanism is architecture-agnostic, it could be applied to future transformer-based diffusion backbones without finetuning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension would be applying the same negative token merging to video diffusion, where references from other frames could increase temporal diversity; the paper does not discuss temporal consistency.
  • The method's dependence on the threshold $\tau$ and the schedule of $\alpha_t$ is not characterized in the paper; a sensitivity analysis across these hyperparameters would tell whether the reported gains are robust or require careful tuning.
  • The diversity improvements are measured with DreamSim and entropy scores; a direct perceptual study comparing paired images with and without NegToMe at matched inference budgets would clarify how much of the improvement is noticeable to humans.
  • One implicit consequence is that the reference image choice acts as a control knob for generation, which could be used for targeted interventions such as reducing specific demographic skews in a batch without retraining.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 3 minor

Summary. The paper introduces NegToMe, a training-free module for adversarial guidance in text-to-image diffusion models. At each transformer block, the module matches each generated (source) token to its most similar token in a reference image by cosine similarity, applies a threshold, and linearly extrapolates the source token away from the matched target token (Eq. 4). The same mechanism is used for several applications by changing the reference: other images in the batch for diversity, copyrighted RAG images for copyright mitigation, blurry images for aesthetic improvement, and interpolation/extrapolation of object features. Quantitative evaluations on SDXL and Flux report improvements in diversity metrics and a 34.57% reduction in visual similarity to copyrighted characters, with under 4% inference overhead, plus a human study.

Significance. The contribution is conceptually simple and potentially useful: it provides a training-free, architecture-agnostic way to use image features as adversarial guidance, and it demonstrates complementarity with negative prompting. The paper makes code available, tests on two architectures, includes a human study, and reports a diverse set of applications. If the quantitative claims hold after fixing the issues below, the method is a solid practical contribution; the current version, however, has two load-bearing problems in the copyright claim: the headline 34.57% figure is not derivable from Table 1, and the RAG evaluation protocol in Sec. 4.2 excludes the reference used for NegToMe from the similarity computation, biasing the comparison.

major comments (4)
  1. [Abstract and Sec. 4.2 (Table 1)] The abstract and Sec. 4.2 state that NegToMe 'reduces visual similarity to copyrighted content by 34.57%,' but no result in the paper supports this number. Table 1 reports maximum DreamSim values of 0.766 (base), 0.684 (NegPrompt), 0.703 (NegToMe alone), and 0.638 (NegPrompt + NegToMe). The relative reductions are (0.766−0.638)/0.766 = 16.7% for the combined method and (0.766−0.703)/0.766 = 8.2% for NegToMe alone. Since no other row, metric, or appendix table is reported, the 34.57% figure either needs a precise definition or the abstract and Sec. 4.2 need to be amended to match the evidence.
  2. [Sec. 4.2, Table 1] The copyright evaluation protocol is not apples-to-apples across methods. The text says that for each prompt NegToMe is applied 'using the best-matching RAG asset (asset with highest Dreamsim score),' and that the reported maximum DreamSim score is computed 'excluding the reference used for NegToMe.' Thus the base and NegPrompt rows take the maximum over all RAG assets, while the NegToMe rows take the maximum over the same set minus the most similar asset. This mechanically lowers the NegToMe maxima and inflates the apparent reduction. Please recompute all rows over an identical held-out RAG set (e.g., include the reference in the maximum for every method, or use a fixed evaluation subset disjoint from the reference selection).
  3. [Sec. 3, Eq. (4); Sec. 5, Fig. 12] Reproducibility of the main experiments is incomplete: the paper defines alpha_t as a time-dependent coefficient and a threshold tau, but it does not report the values or schedule used in Fig. 4 or Table 1. Fig. 12 shows only qualitative variation with alpha. Please specify the default schedule and threshold, and provide a quantitative sensitivity analysis or ablation for these hyperparameters.
  4. [Fig. 4 and Table 1] The quantitative claims rest on point estimates without uncertainty quantification. Fig. 4 plots diversity and quality curves with no error bars, and Table 1 reports single DreamSim/quality numbers with no confidence intervals or significance tests. Given that the headline DreamSim difference between base and combined mitigation is 0.066, the reader cannot assess whether the reported improvements are robust. Please add error bars or significance measures, and state the number of samples per point.
minor comments (3)
  1. [Fig. 11 and References] The caption of Fig. 11 contains typos ('V ariation', 'wtih'), and reference [47] still contains the placeholder '[insert date]'.
  2. [Sec. 3, Eq. (5)] The masked-guidance bias term log(M_ref + eps) is not fully specified: the paper does not state how M_ref is resized/interpolated to the token length N, and the additive bias for masked-out tokens is about -13.8 for eps=1e-6, which is a large and seemingly arbitrary shift that should be justified.
  3. [Sec. 4.2, Table 1] The table caption 'Dreamsim ↓' does not indicate that the reported values are maxima over RAG assets and over seeds; align the caption with the metric definition in the text so that readers can interpret the numbers correctly.

Circularity Check

1 steps flagged · score 6.0 of 10

Copyright-mitigation metric excludes the very RAG asset selected as the reference, forcing part of the reported reduction by construction.

  1. fitted input called prediction [Sec. 4.2, 'Dataset and Setup' and 'Quantitative Results' (Table 1)]
    "Masked negative token merging is then performed for each prompt, using the best-matching RAG asset (asset with highest Dreamsim score) from the reference dataset. ... For evaluation, we use the 1) maximum DreamSim score [14] across all RAG assets (excluding the reference used for NegToMe) for measuring visual similarity to copyrighted characters."

    The reference used by NegToMe is selected as the RAG asset with the highest DreamSim score, and that same asset is excluded when computing the evaluation metric. For the base and negative-prompt-only rows there is no NegToMe reference, so their max is over the full set including the highest-scoring asset. Since max(S \ {argmax S}) <= max(S) by definition, the reported drop (0.766 to 0.703 for NegToMe alone; 0.766 to 0.638 combined) is guaranteed in part by the evaluation protocol itself, independent of any actual change in generated images. The headline 34.57% reduction is therefore partly an arithmetic consequence of the metric definition rather than a purely measured effect of the method.

full rationale

The core NegToMe update (Eq. 4) is a fixed, training-free manipulation of diffusion features; alpha_t and tau are manual hyperparameters, not fitted to any reported metric. The diversity claims are tested against external metrics (pairwise DreamSim, entropy, FID, IS, CLIPScore/VQAScore, and a human study), so they are not circular by construction. The main circularity is in the copyright evaluation: the method selects the highest-DreamSim RAG asset as its reference and the evaluation then excludes precisely that asset from the max-DreamSim calculation, while the base and negative-prompt baselines do not have such an exclusion. This makes part of the measured improvement an unavoidable artifact of the evaluation design. I also flag a separate non-circularity reporting problem: the abstract's 34.57% reduction is not recoverable from Table 1, where base 0.766 to combined 0.638 is 16.7% and NegToMe alone gives 8.2%; this undermines the headline number but is a support problem, not a circularity. The paper's self-citations (e.g., [17], [44]) share authors but are used only as related work or baselines and are not load-bearing, so they do not raise the circularity score further.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The method rests on two domain assumptions about diffusion feature semantics and feature-space extrapolation, plus two free hyperparameters. No new physical or conceptual entities are introduced.

free parameters (2)
  • alpha_t = unspecified in main text (scheduled over timesteps, ablated in Fig. 12)
    Controls how far source tokens are pushed from matched target tokens (Eq. 4); the schedule is not given in the main text and behavior depends on it.
  • threshold tau = unspecified in main text
    Minimum source-to-target similarity for a token to be modified (Eq. 3). No default value is stated in the main text.
assumptions (3)
  • domain assumption Intermediate diffusion features carry reliable cross-image semantic correspondences.
    Sec. 3: 'we can leverage the rich semantic structure of intermediate diffusion features to compute cross-image token-token similarities using noisy features itself.' This underpins the argmax matching in Eq. 2.
  • domain assumption Pushing matched tokens apart in feature space steers the final output away from the reference without breaking text alignment.
    Eq. 4 is applied at every transformer block; the paper provides only empirical evidence, no formal reason why this extrapolation transfers to output-level avoidance.
  • domain assumption The hard threshold keeps unrelated tokens untouched and preserves image quality.
    Eq. 3 sets H based on tau; the threshold choice is not derived and is not specified in the main text.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Negative Token Merging: Image-based Adversarial Feature Guidance." pith.science (2026). https://pith.science/paper/2HCJHFGN

@misc{pith2026241201339,
  author       = {Pith},
  title        = {Pith review of: Negative Token Merging: Image-based Adversarial Feature Guidance},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2HCJHFGN}},
  note         = {Machine review of arXiv:2412.01339}
}
read the original abstract

Text-based adversarial guidance using a negative prompt has emerged as a widely adopted approach to steer diffusion models away from producing undesired concepts. While useful, performing adversarial guidance using text alone can be insufficient to capture complex visual concepts or avoid specific visual elements like copyrighted characters. In this paper, for the first time we explore an alternate modality in this direction by performing adversarial guidance directly using visual features from a reference image or other images in a batch. We introduce negative token merging (NegToMe), a simple but effective training-free approach which performs adversarial guidance through images by selectively pushing apart matching visual features between reference and generated images during the reverse diffusion process. By simply adjusting the used reference, NegToMe enables a diverse range of applications. Notably, when using other images in same batch as reference, we find that NegToMe significantly enhances output diversity (e.g., racial, gender, visual) by guiding features of each image away from others. Similarly, when used w.r.t. copyrighted reference images, NegToMe reduces visual similarity to copyrighted content by 34.57%. NegToMe is simple to implement using just few-lines of code, uses only marginally higher (<4%) inference time and is compatible with different diffusion architectures, including those like Flux, which don't natively support the use of a negative prompt. Code is available at https://negtome.github.io

Figures

Figures reproduced from arXiv: 2412.01339 by the authors.

Figure 1
Figure 1. We introduce NegToMe, a training-free approach for adversarial guidance directly using images instead of a negative prompt. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. ), we can directly use the visual features from a ref￾erence image to adversarially guide the generation process. For instance in [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Method Overview. (a) The core idea of NegToMe is to perform adversarial guidance directly using visual features from a reference image (or other images in the same batch). (b) NegToMe is simple and can be applied in any transformer block. (c) A simple three step process for performing adversarial guidance using NegToMe (refer Sec. 3 and Alg. 1 for the detailed implementation). database (Sec. 4.2), we observe that Ne… view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Quantitative Results for Output Diversity. NegToMe (ours) helps improve output diversity (lower DreamSim score and higher Entropy) while preserving or improving quality (lower FID and higher IS) across different CFG scales for both SDXL and FLUX. “a hyper-realistic dig…
Figure 5
Figure 5. Figure 5: Increasing Output Diversity. We observe that when performed w.r.t to images in the same batch (the first image of each batch in above), NegToMe significantly improves output diversity (racial, ethnic, visual) while preserving output image quality. CLIPScore [19] are us…
Figure 6
Figure 6. Figure 6: NegToMe helps improve output diversity both with [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]
Figure 7
Figure 7. Figure 7: Copyright Mitigation. When used w.r.t a copyright RAG image dataset, NegToMe helps reduce visual similarities with copy￾righted characters without sacrificing output image quality (Tab. 1). Complete prompts and further results are provided in the appendix [PITH_FULL_I…
Figure 8
Figure 8. Figure 8: Human User Study. NegToMe helps improve output diversity while preserving text-to-image alignment performance. and FLUX ( [PITH_FULL_IMAGE:figures/full_fig_p006_8.png]
Figure 9
Figure 9. Figure 9: Application to MM-DiT models (Flux). NegToMe is model-agnostic and also applicable to MM-DiT models like Flux [6]. NegToMe significantly increases the output diversity (top), and helps reduce copyright violation (bottom). increased inference time. Furthermore, some of …
Figure 10
Figure 10. Figure 10: Improving aesthetics. Using a blurry reference with NegToMe improves output aesthetics without any training [36]. and [PITH_FULL_IMAGE:figures/full_fig_p007_10.png]
Figure 11
Figure 11. Figure 11: Variation with cfg scale leads to improved output quality at the cost reduced diversity (left). NegToMe not only improves output quality at lower cfg values (by guiding away from poor-quality features, see [PITH_FULL_IMAGE:figures/full_fig_p008_11.png]
Figure 12
Figure 12. Figure 12: Variation with merging alpha. Increasing the value of α (refer to Sec. 3) for NegToMe gradually increases output diver￾sity in terms of gender, race, ethnicity, lighting, style etc. copyright mitigation strategies. For each prompt, we sam￾ple 50 images with 50 differe…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Manifold-Constrained Noise Optimization for Diverse Diffusion Sampling

    cs.CV 2026-07 conditional novelty 6.0 of 10

    By optimizing each new starting noise on a fixed-radius, low-frequency sphere, MoNO recovers per-prompt diversity in distilled text-to-image models while keeping image quality roughly stable.

Reference graph

Works this paper leans on

52 extracted references · 24 canonical work pages · cited by 1 Pith paper

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 ,

    arXiv

  2. [2]

    How to use negative prompts?, 2023

    Andrew. How to use negative prompts?, 2023. 3

    work page 2023

  3. [3]

    Re-imagine the negative prompt algorithm: Transform 2d diffusion into 3d, alleviate janus problem and beyond

    Mohammadreza Armandpour, Ali Sadeghian, Huangjie Zheng, Amir Sadeghian, and Mingyuan Zhou. Re-imagine the negative prompt algorithm: Transform 2d diffusion into 3d, alleviate janus problem and beyond. arXiv preprint arXiv:2304.04968, 2023. 2

    arXiv 2023

  4. [4]

    Understanding the im- pact of negative prompts: When and how do they take effect? arXiv preprint arXiv:2406.02965, 2024

    Yuanhao Ban, Ruochen Wang, Tianyi Zhou, Minhao Cheng, Boqing Gong, and Cho-Jui Hsieh. Understanding the im- pact of negative prompts: When and how do they take effect? arXiv preprint arXiv:2406.02965, 2024. 2, 3

    arXiv 2024

  5. [5]

    How well can text-to-image generative models un- derstand ethical natural language interventions? arXiv preprint arXiv:2210.15230, 2022

    Hritik Bansal, Da Yin, Masoud Monajatipoor, and Kai-Wei Chang. How well can text-to-image generative models un- derstand ethical natural language interventions? arXiv preprint arXiv:2210.15230, 2022. 3

    arXiv 2022

  6. [6]

    Scaling recti- fied flow transformers for high-resolution image synthe- sis

    Andreas Blattmann, Axel Sauer, Dominik Lorenz, Dustin Podell, Frederic Boesel, Harry Saini, Jonas M ¨uller, Kyle Lacey, Patrick Esser, Robin Rombach, Sumith Kulal, Tim Dockhorn, Yam Levi, and Zion English. Scaling recti- fied flow transformers for high-resolution image synthe- sis. https://github.com/black-forest-labs/ flux, 2024. Accessed: 2024-09-12. 2, 3, 4, 7

    work page 2024

  7. [7]

    To- ken merging: Your vit but faster

    Daniel Bolya, Cheng-Yang Fu, Xiaoliang Dai, Peizhao Zhang, Christoph Feichtenhofer, and Judy Hoffman. To- ken merging: Your vit but faster. arXiv preprint arXiv:2210.09461, 2022. 3

    arXiv 2022

  8. [8]

    Token merging for fast stable diffusion

    Daniel Bolya and Judy Hoffman. Token merging for fast stable diffusion. CVPR Workshop on Efficient Deep Learning for Computer Vision, 2023. 3

    work page 2023

Show all 52 references
  1. [9]

    Attend-and-excite: Attention-based se- mantic guidance for text-to-image diffusion models

    Hila Chefer, Yuval Alaluf, Yael Vinker, Lior Wolf, and Daniel Cohen-Or. Attend-and-excite: Attention-based se- mantic guidance for text-to-image diffusion models. ACM Transactions on Graphics (TOG), 42(4):1–10, 2023. 3

    2023

  2. [10]

    Emu: Enhanc- ing image generation models using photogenic needles in a haystack

    Xiaoliang Dai, Ji Hou, Chih-Yao Ma, Sam Tsai, Jialiang Wang, Rui Wang, Peizhao Zhang, Simon Vandenhende, Xi- aofang Wang, Abhimanyu Dubey, et al. Emu: Enhanc- ing image generation models using photogenic needles in a haystack. arXiv preprint arXiv:2309.15807, 2023. 2

    2023 arXiv

  3. [11]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020. 3

    2010 arXiv

  4. [12]

    Can: Creative adversarial networks, gen- erating “art” by learning about styles and deviating from style norms

    Ahmed Elgammal. Can: Creative adversarial networks, gen- erating “art” by learning about styles and deviating from style norms. arXiv preprint arXiv:1706.07068 , 6:2017,

    2017 arXiv

  5. [13]

    Scaling recti- fied flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M ¨uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling recti- fied flow transformers for high-resolution image synthesis. In Forty-first International Conference on Mach...

    2024

  6. [14]

    Dream- sim: Learning new dimensions of human visual similar- ity using synthetic data

    Stephanie Fu, Netanel Tamir, Shobhita Sundaram, Lucy Chai, Richard Zhang, Tali Dekel, and Phillip Isola. Dream- sim: Learning new dimensions of human visual similar- ity using synthetic data. arXiv preprint arXiv:2306.09344 ,

    arXiv

  7. [15]

    CPR: Retrieval Augmented Generation for Copyright Pro- tection

    Aditya Golatkar, Alessandro Achille, Luca Zancato, Yu- Xiang Wang, Ashwin Swaminathan, and Stefano Soatto. CPR: Retrieval Augmented Generation for Copyright Pro- tection. In CVPR, 2024. 3

    2024

  8. [16]

    Reliable and Efficient Concept Erasure of Text- to-Image Diffusion Models, 2024

    Chao Gong, Kai Chen, Zhipeng Wei, Jingjing Chen, and Yu- Gang Jiang. Reliable and Efficient Concept Erasure of Text- to-Image Diffusion Models, 2024. 3

    2024

  9. [17]

    Fantastic copyrighted beasts and how (not) to generate them

    Luxi He, Yangsibo Huang, Weijia Shi, Tinghao Xie, Hao- tian Liu, Yue Wang, Luke Zettlemoyer, Chiyuan Zhang, Danqi Chen, and Peter Henderson. Fantastic copyrighted beasts and how (not) to generate them. arXiv preprint arXiv:2406.14526, 2024. 2, 3

    2024 arXiv

  10. [18]

    Lemley, and Percy Liang

    Peter Henderson, Xuechen Li, Dan Jurafsky, Tatsunori Hashimoto, Mark A. Lemley, and Percy Liang. Foundation Models and Fair Use. ArXiv, abs/2303.15715, 2023. 3

    2023 arXiv

  11. [19]

    Clipscore: A reference-free evaluation met- ric for image captioning

    Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation met- ric for image captioning. arXiv preprint arXiv:2104.08718,

    arXiv

  12. [20]

    Gans trained by a two time-scale update rule converge to a local nash equilib- rium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilib- rium. Advances in neural information processing systems , 30, 2017. 4, 8

    2017

  13. [21]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022. 2, 3, 4, 5, 8

    2022 arXiv

  14. [22]

    Fairface: Face at- tribute dataset for balanced race, gender, and age

    Kimmo K ¨arkk¨ainen and Jungseock Joo. Fairface: Face at- tribute dataset for balanced race, gender, and age. ArXiv, abs/1908.04913, 2019. 5

    1908 arXiv

  15. [23]

    Segment anything in high quality

    Lei Ke, Mingqiao Ye, Martin Danelljan, Yifan Liu, Yu-Wing Tai, Chi-Keung Tang, and Fisher Yu. Segment anything in high quality. In NeurIPS, 2023. 7

    2023

  16. [24]

    9 Talkin”bout ai generation: Copyright and the generative-ai supply chain

    Katherine Lee, A Feder Cooper, and James Grimmelmann. 9 Talkin”bout ai generation: Copyright and the generative-ai supply chain. arXiv preprint arXiv:2309.08133, 2023. 3

    2023 arXiv

  17. [25]

    Feder Cooper, and James Grimmelmann

    Katherine Lee, A. Feder Cooper, and James Grimmelmann. Talkin’ ’Bout AI Generation: Copyright and the Generative- AI Supply Chain, 2024. 3

    2024

  18. [26]

    Aligning text- to-image models using human feedback

    Kimin Lee, Hao Liu, Moonkyung Ryu, Olivia Watkins, Yuqing Du, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, and Shixiang Shane Gu. Aligning text- to-image models using human feedback. arXiv preprint arXiv:2302.12192, 2023. 3

    2023 arXiv

  19. [27]

    Vidtome: Video token merging for zero-shot video editing

    Xirui Li, Chao Ma, Xiaokang Yang, and Ming-Hsuan Yang. Vidtome: Video token merging for zero-shot video editing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7486–7495, 2024. 3

    2024

  20. [28]

    Evaluating text-to-visual generation with image-to-text gen- eration

    Zhiqiu Lin, Deepak Pathak, Baiqi Li, Jiayao Li, Xide Xia, Graham Neubig, Pengchuan Zhang, and Deva Ramanan. Evaluating text-to-visual generation with image-to-text gen- eration. arXiv preprint arXiv:2404.01291, 2024. 4, 8

    2024 arXiv

  21. [29]

    Training diffusion models towards diverse image generation with reinforcement learning

    Zichen Miao, Jiang Wang, Ze Wang, Zhengyuan Yang, Li- juan Wang, Qiang Qiu, and Zicheng Liu. Training diffusion models towards diverse image generation with reinforcement learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 108...

    2024

  22. [30]

    Wordnet: a lexical database for english

    George A Miller. Wordnet: a lexical database for english. Communications of the ACM, 38(11):39–41, 1995. 5

    1995

  23. [31]

    SILO Language Models: Isolating Legal Risk In a Nonpara- metric Datastore

    Sewon Min, Suchin Gururangan, Eric Wallace, Weijia Shi, Hannaneh Hajishirzi, Noah A Smith, and Luke Zettlemoyer. SILO Language Models: Isolating Legal Risk In a Nonpara- metric Datastore. In ICLR, 2023. 3

    2023

  24. [32]

    Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741, 2021

    Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741, 2021. 2

    2021 arXiv

  25. [33]

    Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023. 3, 4

    2023 arXiv

  26. [34]

    Class-balancing diffusion models

    Yiming Qin, Huangjie Zheng, Jiangchao Yao, Mingyuan Zhou, and Ya Zhang. Class-balancing diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 18434–18443, 2023. 3

    2023

  27. [35]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  28. [36]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christo- pher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36, 2024. 3, 7, 8

    2024

  29. [37]

    Hierarchical text-conditional image gen- eration with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gen- eration with clip latents. arXiv preprint arXiv:2204.06125,

    arXiv

  30. [38]

    High-resolution image syn- thesis with latent diffusion models, 2021

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models, 2021. 2

    2021

  31. [39]

    The new legal landscape for text mining and machine learning

    Matthew Sag. The new legal landscape for text mining and machine learning. J. Copyright Soc’y USA, 66:291, 2018. 3

    2018

  32. [40]

    Copyright safety for generative ai

    Matthew Sag. Copyright safety for generative ai. Forthcom- ing in the Houston Law Review, 2023. 3

    2023

  33. [41]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S Sara Mahdavi, Rapha Gontijo Lopes, et al. Photorealistic text-to-image diffusion models with deep language understanding. arXiv preprint arXiv...

    2022 arXiv

  34. [42]

    Improved techniques for training gans

    Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. Advances in neural information processing systems, 29, 2016. 4

    2016

  35. [43]

    Laion-400m: Open dataset of clip-filtered 400 million image-text pairs

    Christoph Schuhmann, Richard Vencu, Romain Beaumont, Robert Kaczmarczyk, Clayton Mullis, Aarush Katta, Theo Coombes, Jenia Jitsev, and Aran Komatsuzaki. Laion-400m: Open dataset of clip-filtered 400 million image-text pairs. arXiv preprint arXiv:2111.02114, 2021. 4

    2021 arXiv

  36. [44]

    Detecting Pretraining Data from Large Language Models

    Weijia Shi, Anirudh Ajith, Mengzhou Xia, Yangsibo Huang, Daogao Liu, Terra Blevins, Danqi Chen, and Luke Zettle- moyer. Detecting Pretraining Data from Large Language Models. In The Twelfth International Conference on Learn- ing Representations, 2024. 3

    2024

  37. [45]

    Emergent correspondence from image diffusion

    Luming Tang, Menglin Jia, Qianqian Wang, Cheng Perng Phoo, and Bharath Hariharan. Emergent correspondence from image diffusion. Advances in Neural Information Pro- cessing Systems, 36:1363–1389, 2023. 3

    2023

  38. [46]

    Smith, Chiyuan Zhang, Luke Zettlemoyer, Kai Li, and Peter Hen- derson

    Boyi Wei, Weijia Shi, Yangsibo Huang, Noah A. Smith, Chiyuan Zhang, Luke Zettlemoyer, Kai Li, and Peter Hen- derson. Evaluating copyright takedown methods for lan- guage models, 2024. 3

    2024

  39. [47]

    Stable diffusion 2.0 and the importance of nega- tive prompts for good results, 2023

    Max Woolf. Stable diffusion 2.0 and the importance of nega- tive prompts for good results, 2023. Accessed: [insert date]. 3

    2023

  40. [48]

    Fairy: Fast parallelized instruction-guided video-to-video synthesis

    Bichen Wu, Ching-Yao Chuang, Xiaoyan Wang, Yichen Jia, Kapil Krishnakumar, Tong Xiao, Feng Liang, Licheng Yu, and Peter Vajda. Fairy: Fast parallelized instruction-guided video-to-video synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni...

    2024

  41. [49]

    Scaling autoregres- sive models for content-rich text-to-image generation

    Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gun- jan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yin- fei Yang, Burcu Karagol Ayan, et al. Scaling autoregres- sive models for content-rich text-to-image generation. arXiv preprint arXiv:2206.10789, 2022. 2

    2022 arXiv

  42. [50]

    Iti- gen: Inclusive text-to-image generation

    Cheng Zhang, Xuanbai Chen, Siqi Chai, Chen Henry Wu, Dmitry Lagun, Thabo Beeler, and Fernando De la Torre. Iti- gen: Inclusive text-to-image generation. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 3969–3980, 2023. 3

    2023

  43. [51]

    Forget-me-not: Learning to forget in text-to-image diffusion models

    Gong Zhang, Kai Wang, Xingqian Xu, Zhangyang Wang, and Humphrey Shi. Forget-me-not: Learning to forget in text-to-image diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1755–1764, 2024. 3 10

    2024

  44. [52]

    A tale of two features: Stable diffusion complements dino for zero-shot semantic correspondence

    Junyi Zhang, Charles Herrmann, Junhwa Hur, Luisa Pola- nia Cabrera, Varun Jampani, Deqing Sun, and Ming-Hsuan Yang. A tale of two features: Stable diffusion complements dino for zero-shot semantic correspondence. Advances in Neural Information Processing Systems, 36, 2024. 3 11

    2024

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.