Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

ReNeg: Learning Negative Embedding with Reward Guidance

T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read ReNeg learns a single negative embedding in the text encoder's space by maximizing a reward model on one-step denoised predictions, and shows that this vector outperforms null-text and handcrafted negative prompts across multiple…

desk verdict Simple, useful method for learning negative embeddings with reward feedback, but the 'seamless transfer' claim ignores text-encoder dimension changes and the headline metric is trained on the same reward; worth reviewing with fixes. read the letter →

arxiv 2412.19637 v3 pith:UW53XXJ3 submitted 2024-12-27 cs.CV

Xiaomin Li , Yixuan Liu , Takashi Isobe , Xu Jia , Qinpeng Cui , Dong Zhou , Dong Li , You He
show 3 more authors
Huchuan Lu Zhongdao Wang Emad Barsoum
This is my paper · ORCID
classification cs.CV
keywords negativeembeddingtext-to-imagegenerationrewardfeedbacklearningclassifier-freeguidancehumanpreferencealignmentdiffusionmodelstransferabilitypromptoptimization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that the negative embedding used in classifier-free guidance can be learned rather than handcrafted. The authors propose ReNeg, which optimizes a continuous vector in the text embedding space to maximize a human-preference reward model on predicted images. They incorporate classifier-free guidance into training so that gradients flow to the negative embedding, and they show that the learned global embedding improves generation quality over null-text and manual negative prompts on Stable Diffusion 1.4/1.5/2.1, and transfers to ControlNet and video models sharing the same text encoder. A per-sample variant that starts from the global embedding and adapts to each prompt yields further gains. If correct, this means a tiny, storable vector can replace prompt engineering for negative guidance.

What carries the argument

The central object is the negative embedding vector $n$, a point in the CLIP text-embedding space that replaces the null-text embedding in the CFG score combination. The argument is carried by the reward feedback loop: sample a prompt, denoise with CFG to a predicted image $\hat{x}_0$ via Eq. (6), score it with a pretrained reward model $R$, and backpropagate the reward gradient to $n$ while all other parameters are frozen. Two supporting elements are the parameter-efficiency criterion based on the Jacobian norm, which suggests $n$ changes the output distribution much faster than model weights or LoRA, and the use of the DDIM deterministic ODE solver to make the one-step prediction faithful to the full sampling trajectory.

What would settle it

Train ReNeg with HPSv2.1 as the reward model, then evaluate the final 30-step DDIM outputs on a different human-preference benchmark not used in training, such as a fresh user study or a held-out split of a preference dataset. If the learned negative embedding fails to show a consistent preference improvement over the handcrafted negative prompt on that held-out evaluation, the one-step reward proxy is likely being overfitted rather than reflecting true generation quality.

Watch

Extended reading notes

Core claim

The central claim is that a single learned negative embedding is a better substitute for null-text or handcrafted negative prompts in classifier-free guidance. ReNeg treats the negative embedding as a set of model parameters, initializes it from the null-text embedding, and optimizes it with gradient ascent on the reward objective $J_\theta(\mathcal{D}) = \mathbb{E}_{c\sim\mathcal{D}} R(c, \hat{x}_0)$, where $\hat{x}_0$ is a one-step denoising prediction obtained with CFG reparameterization. Training uses a deterministic DDIM ODE solver to make $\hat{x}_0$ accurate. The paper reports that this global embedding beats handcrafted negative prompts on HPSv2 and Parti-Prompts, rivals full finetuning methods like TextCraftor, and transfers seamlessly to other models using the same CLIP text encoder. Per-sample adaptation, initialized from the global embedding and run with early stopping, gives additional improvements in detail and text-image alignment.

Load-bearing premise

The reward score computed on a one-step predicted image is a reliable and non-exploitable proxy for human preference on the full multi-step denoising output.

Editorial extensions

If this is right

  • Any T2I or T2V model that shares the text encoder can be improved by inserting the same learned negative embedding, with no retraining and negligible storage cost (about $5\times10^{-4}$ times the full model's parameter budget).
  • Negative prompt engineering can be replaced by a one-time reward-guided optimization, removing the need for manual trial-and-error search over negative vocabulary.
  • Per-sample negative embeddings, initialized from the global one and tuned for a specific prompt, add a further quality gain with only a short adaptation run.
  • The method's gains are complementary to positive-prompt refinement, so it can be combined with methods that rewrite or optimize the positive prompt.
  • Because the learned embedding lives in the text encoder's output space, the same vector can serve different tasks (image and video) as long as the conditioning pipeline uses the same text encoder.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The success of ReNeg suggests that the negative embedding direction is a high-leverage parameter subspace of a frozen diffusion model; a similar reward-guided search may work for other conditioning modalities, such as image embeddings or control hints.
  • The choice of reward model likely steers the semantic content of the learned negative embedding: a reward model trained on photographic preferences would yield a different vector than one trained on animation aesthetics, which could be exploited to produce style-specific negative embeddings.
  • A testable extension is to apply the same CFG-in-training reward loop to learn negative embeddings for video models directly on video reward models like VBench, rather than transferring an image-trained vector.
  • The one-step prediction proxy used in training may fail when the noise schedule or guidance scale at inference differs substantially from the training setting; multi-step reward estimation could make the learned embedding more robust.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This paper proposes ReNeg, a method for learning a negative text embedding in the continuous CLIP text-embedding space for Stable Diffusion. A global embedding n is optimized by maximizing an image reward model R(c, \hat{x}_0) on one-step DDIM predictions, with classifier-free guidance (CFG) used during training; a per-sample variant adapts n to each prompt. The authors report improvements on HPSv2, Parti-Prompts, PickScore, and aesthetic metrics, and claim that the learned embedding transfers 'seamlessly' across SD1.4, SD1.5, SD2.1, ControlNet, ZeroScope, and VideoCrafter2 because they share the same CLIP text encoder.

Significance. If the reported effects are real, ReNeg is an attractive and extremely parameter-efficient alignment tool: a single vector in text-embedding space rather than LoRA or full fine-tuning, with plausible generalization to same-encoder models. The paper includes some independent validation (PickScore, aesthetic predictor, VBench) that shows real but modest gains. However, the main quantitative evidence is partly circular because HPSv2.1 is both the training objective and the headline evaluation metric, and the transfer claim conflates different CLIP encoders. The significance therefore currently rests on assumptions that need to be verified in revision.

major comments (4)
  1. [Sec. 4.2, Eq. (7) and Table 2] The training objective is J_theta(D) = E_c[R(c, \hat{x}_0)] with R = HPSv2.1, and the HPSv2.1 column in Table 2 is evaluated with the same reward model. The reported HPSv2.1 gains therefore measure optimization success on the training reward, not independent human-preference alignment. The independent metrics (PickScore: 19.90 vs 18.40; Aesthetic: 5.45 vs 5.26 for the global embedding) show smaller improvements and should be made the primary evidence, with confidence intervals and, ideally, a reward model not used for training as the headline validation.
  2. [Abstract and Sec. 5.4] The claim that the SD1.5-learned embedding transfers 'seamlessly' to SD2.1, ZeroScope, and VideoCrafter2 'using the same CLIP text encoder' is factually inconsistent. SD1.4/1.5 use OpenAI CLIP ViT-L/14 with 768-dimensional text embeddings, whereas SD2.1 and its derivatives use OpenCLIP ViT-H/14 with 1024-dimensional embeddings. A 768-dimensional vector cannot be substituted for a 1024-dimensional null embedding without an undocumented projection. Please state explicitly which text encoder and embedding dimension were used for each model in Tables 3-5, and either present the projection or revise the generalization claim.
  3. [Sec. 4.3, Algorithm 1 and Table 2] The per-sample negative embedding is optimized on each evaluation prompt, using the same reward for early stopping, and is then scored on that prompt. Table 2's per-sample row therefore reports test-time optimization, not out-of-sample generalization. Please clarify whether the HPSv2 evaluation prompts were used for per-sample training, and if so, label the row as test-time adaptation or evaluate on a separate prompt set with a fixed computational budget.
  4. [Sec. 4.2, Eqs. (5)-(6) and Fig. 3] The reward is applied to a one-step DDIM prediction \hat{x}_0 from a random timestep t in [0, 10], yet the final evaluation uses 30-step DDIM samples. The authors support this proxy with a single-prompt SSIM plot (Fig. 3). This is not sufficient evidence that maximizing reward on one-step predictions improves full-sample quality. Please add a per-prompt correlation analysis between one-step reward and full-sample metrics, and an ablation over the timestep t and the sampling solver.
minor comments (6)
  1. [Table 1] The computation of the Jacobian-based parameter efficiency E(θ) lacks details: the number of prompts N, latent dimensions, normalization, and standard errors. The numbers in Table 1 are hard to interpret without this information.
  2. [Tables 2-5] No error bars or significance tests are provided for any of the quantitative results. Given stochastic sampling, differences such as the win rates in Table 3 (e.g., 0.87 vs 0.93) may not be reliable.
  3. [Fig. 3] The claim that DDIM 'consistently outperforms' DDPM for \hat{x}_0 prediction is based on one prompt only; multi-prompt statistics are needed to support this statement.
  4. [Abstract / Section 1] The paper states that code and learned negative embeddings are released 'in this URL', but no URL appears in the manuscript text.
  5. [Algorithm 1] The per-sample optimization does not specify the learning rate, the number of samples per prompt, or whether multiple random seeds were used for the final reported images; these details are needed for reproducibility.
  6. [Sec. 4.1 and Eq. (4)] The notation E(θ) for parameter efficiency conflicts with the text encoder E in Eq. (4); consider renaming one of them to avoid confusion.

Circularity Check

2 steps flagged · score 6.0 of 10

HPSv2.1 is both the training objective and the headline evaluation metric; the per-sample variant optimizes the same scorer on the same prompts it is then measured with.

  1. fitted input called prediction [Section 4.2 (Learning objective, Eq. 7), Section 5.1 (Evaluation metrics), Table 2]
    "Specifically, using the human preference-based reward model HPSv2.1 [36], we optimize the negative embedding to maximize the reward score. ... The final learning objective is to maximize the expectation of reward scores over the prompt distribution D: Jθ(D) = Ec∼D(R(c, ˆx0)). ... Evaluation metrics. We adopt the Human Preference Score v2.1 (HPSv2.1) [36], PickScore [18], and an aesthetic predictor [28] to comprehensively evaluate our method."

    The global negative embedding is trained by gradient ascent on the HPSv2.1 reward R, and then Table 2 reports HPSv2.1 as the primary evidence of 'human preference alignment.' The HPSv2.1 column therefore measures optimization success against the same scorer used to fit n, not an independent validation. The gains over null-text and handcrafted baselines are expected whenever the reward model's scores respond to the learned vector, so the headline HPSv2.1 improvement reduces, in large part, to the training objective. Independent metrics (PickScore, aesthetic, VBench) provide separate evidence, but the central HPSv2.1 claim is self-referential.

  2. fitted input called prediction [Section 4.3 (Per-sample Negative Embedding), Algorithm 1, Table 2]
    "Algorithm 1: ... Jn(c) = R(c, ˆx0) ... if Jn(c) > Jbest then Jbest ← Jn(c) ... Update n using gradient descend. ... Section 4.3: we propose a search strategy that guarantees convergence to a solution that outperforms the global negative embedding."

    For each evaluation prompt, the per-sample negative embedding is optimized to maximize R(c, xhat0) on that exact prompt, with early stopping keyed to whether R increases. Table 2 then reports the per-sample HPSv2.1 score, i.e., the value of the very reward function being maximized. The 'guaranteed' improvement over the global embedding is true by construction because the selection criterion is R itself: the returned n is the one that did not decrease R. The per-sample HPSv2.1 gain is therefore a fitted quantity reported as a prediction, not independent evidence of quality. The non-circular evidence is limited to the small PickScore and aesthetic gains, and the method's own limitation that generation ability is still bounded by the base model.

full rationale

The clearest circularity is in the per-sample variant: Algorithm 1 optimizes the HPSv2.1 reward on each individual prompt, and Table 2 evaluates that variant with HPSv2.1 on the same benchmark prompts. The reported per-sample improvement is the optimized objective itself, so the HPSv2.1 row is not an independent test. The global embedding is less tightly circular because it is trained on ImageReward prompts while HPSv2.1 evaluation uses held-out prompts, but the objective (Eq. 7) and the headline metric are the same reward model, so the HPSv2.1 column mostly reconfirms the training signal. This warrants a 6 rather than a higher score because the paper also reports non-circular evidence: PickScore, an aesthetic predictor, and VBench metrics are not the training objective, and some of these improve. No self-citation load-bearing issue exists: the ReFL/ImageReward and HPSv2 references are external works, and no uniqueness theorem from the authors is invoked. A separate non-circular correctness concern, noted in the transfer claim, is that SD2.1, ZeroScope, and VideoCrafter2 use a different CLIP text encoder than SD1.5, so a single 768-dimensional embedding cannot be 'seamlessly' shared as asserted; this affects validity of the generalization results but is not a circularity. The paper's own concluding limitation (semantic deviations) is an honest caveat but does not change the circularity assessment.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central learned object n is itself a fitted parameter; the ledger also counts the hand-chosen hyperparameters (CFG scale, timestep window, optimizer settings) that the results depend on. The main domain axioms are the validity of HPSv2.1 as a preference proxy and the alignment between one-step training and full-step inference. No new physical or model entities are invented.

free parameters (5)
  • global negative embedding n = learned vector in CLIP text embedding space (SD1.5)
    Central optimized object; initialized with null-text embedding and updated by Eq. (7) to maximize HPSv2.1 reward.
  • per-sample negative embedding n_c = per-prompt vector from up to 10 extra steps with patience 3
    Further optimizes n for each prompt, initialized from the global embedding; used in Table 2 per-sample results.
  • CFG guidance scale gamma = 7.5 at inference, constant during training
    Standard hyperparameter; the paper tested learnable gamma and fixed it, and all quality improvements are measured at this scale.
  • training timestep window = T=30, t sampled in [0,10]
    Chosen for reward distinguishability and x_hat_0 prediction quality; affects the behavior of the learned embedding.
  • optimizer and schedule = AdamW, learning rate 5e-3, batch size 64, 4000 steps
    Hand-selected training hyperparameters with no sensitivity analysis; the per-sample stage uses 10 steps and patience 3.
assumptions (5)
  • domain assumption HPSv2.1 reward model is a differentiable and valid proxy for human aesthetic preference on generated images.
    Used as the training signal in Eq. (7) and as the headline metric in Table 2; no non-overlapping preference model or human study is used to validate the learned embedding.
  • domain assumption One-step predicted x_hat_0 from Eq. (6) with CFG reparameterization (Eq. 5) has a reward landscape aligned with full 30-step DDIM sampling.
    Backpropagation stops at x_hat_0; the paper supports the alignment claim with Fig. 3, a similarity plot on a single prompt, not with reward-transfer experiments.
  • domain assumption A single global vector in the CLIP text embedding space can improve generation quality across diverse prompts.
    The method assumes a universal negative direction exists near the null-text embedding; failure modes are not analyzed.
  • standard math Standard diffusion, DDIM, and CFG equations (Eqs. 1-5) correctly describe the pretrained SD1.5 generation process.
    Background from Ho et al. 2020, Song et al. 2020, and Ho and Salimans 2022; accepted prior literature.
  • domain assumption The 10k ImageReward training prompts are representative enough that the learned embedding generalizes to HPSv2 and Parti-Prompts.
    No cross-distribution analysis or per-category breakdown of where the embedding helps or hurts is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ReNeg: Learning Negative Embedding with Reward Guidance." pith.science (2026). https://pith.science/paper/UW53XXJ3

@misc{pith2026241219637,
  author       = {Pith},
  title        = {Pith review of: ReNeg: Learning Negative Embedding with Reward Guidance},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UW53XXJ3}},
  note         = {Machine review of arXiv:2412.19637}
}
read the original abstract

In text-to-image (T2I) generation applications, negative embeddings have proven to be a simple yet effective approach for enhancing generation quality. Typically, these negative embeddings are derived from user-defined negative prompts, which, while being functional, are not necessarily optimal. In this paper, we introduce ReNeg, an end-to-end method designed to learn improved Negative embeddings guided by a Reward model. We employ a reward feedback learning framework and integrate classifier-free guidance (CFG) into the training process, which was previously utilized only during inference, thus enabling the effective learning of negative embeddings. We also propose two strategies for learning both global and per-sample negative embeddings. Extensive experiments show that the learned negative embedding significantly outperforms null-text and handcrafted counterparts, achieving substantial improvements in human preference alignment. Additionally, the negative embedding learned within the same text embedding space exhibits strong generalization capabilities. For example, using the same CLIP text encoder, the negative embedding learned on SD1.5 can be seamlessly transferred to text-to-image or even text-to-video models such as ControlNet, ZeroScope, and VideoCrafter2, resulting in consistent performance improvements across the board.

Figures

Figures reproduced from arXiv: 2412.19637 by the authors.

Figure 1
Figure 1. We develop ReNeg, a versatile negative embedding seamlessly adaptable to text-to-image and even text-to-video models. Strik [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of the training pipeline of our ReNeg. We learn the negative embedding by integrating Classifier-Free Guidance into [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. Comparison of results using global negative embedding [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figures from the paper (2 more)
Figure 5
Figure 5. Figure 5: Qualitative comparisons. The prompts source from HPSv2 and Parti-Prompts benchmarks. All images are generated at a [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 7
Figure 7. Figure 7: Comparison of the win rates on Parti-Prompts with and [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. A Minimalist Method for Fine-tuning Text-to-Image Diffusion Models

    cs.LG 2025-05 conditional novelty 6.0 of 10

    A one-step RL method learns a prompt-conditioned initial noise distribution for a frozen diffusion model, improving scores on the training reward models, with the largest gains at low inference steps.

Reference graph

Works this paper leans on

41 extracted references · 21 canonical work pages · cited by 1 Pith paper

  1. [1]

    Understanding the im- pact of negative prompts: When and how do they take effect? arXiv preprint arXiv:2406.02965, 2024

    Yuanhao Ban, Ruochen Wang, Tianyi Zhou, Minhao Cheng, Boqing Gong, and Cho-Jui Hsieh. Understanding the im- pact of negative prompts: When and how do they take effect? arXiv preprint arXiv:2406.02965, 2024. 2

    arXiv 2024

  2. [2]

    Training diffusion models with reinforce- ment learning

    Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine. Training diffusion models with reinforce- ment learning. arXiv preprint arXiv:2305.13301, 2023. 3, 6

    arXiv 2023

  3. [3]

    Stable video diffusion: Scaling latent video diffusion models to large datasets

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023. 2

    arXiv 2023

  4. [4]

    Beautifulprompt: Towards auto- matic prompt engineering for text-to-image synthesis

    Tingfeng Cao, Chengyu Wang, Bingyan Liu, Ziheng Wu, Jinhui Zhu, and Jun Huang. Beautifulprompt: Towards auto- matic prompt engineering for text-to-image synthesis. arXiv preprint arXiv:2311.06752, 2023. 2, 6, 7

    arXiv 2023

  5. [5]

    https : / / huggingface

    cerspense. https : / / huggingface . co / cerspense/zeroscope_v2_576w, 2023. 2

    work page 2023

  6. [6]

    Videocrafter2: Overcoming data limitations for high-quality video diffusion models

    Haoxin Chen, Yong Zhang, Xiaodong Cun, Menghan Xia, Xintao Wang, Chao Weng, and Ying Shan. Videocrafter2: Overcoming data limitations for high-quality video diffusion models. In CVPR, 2024. 2

    2024

  7. [7]

    Deep reinforcement learn- ing from human preferences

    Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learn- ing from human preferences. NeurIPS, 2017. 3, 4

    work page 2017

  8. [8]

    Directly fine-tuning diffusion models on differentiable re- wards

    Kevin Clark, Paul Vicol, Kevin Swersky, and David J Fleet. Directly fine-tuning diffusion models on differentiable re- wards. arXiv preprint arXiv:2309.17400, 2023. 2

    arXiv 2023

Show all 41 references
  1. [9]

    Improving image syn- thesis with diffusion-negative sampling

    Alakh Desai and Nuno Vasconcelos. Improving image syn- thesis with diffusion-negative sampling. In ECCV, 2025. 2, 3, 6, 7

    2025

  2. [10]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. NeurIPS, 2021. 2

    2021

  3. [11]

    A systematic survey of prompt engineer- ing on vision-language foundation models

    Jindong Gu, Zhen Han, Shuo Chen, Ahmad Beirami, Bailan He, Gengyuan Zhang, Ruotong Liao, Yao Qin, V olker Tresp, and Philip Torr. A systematic survey of prompt engineer- ing on vision-language foundation models. arXiv preprint arXiv:2307.12980, 2023. 2

    2023 arXiv

  4. [12]

    Animatediff: Animate your personalized text- to-image diffusion models without specific tuning

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text- to-image diffusion models without specific tuning. arXiv preprint arXiv:2307.04725, 2023. 2

    2023 arXiv

  5. [13]

    Optimizing prompts for text-to-image generation

    Yaru Hao, Zewen Chi, Li Dong, and Furu Wei. Optimizing prompts for text-to-image generation. NeurIPS, 2024. 2, 6, 7

    2024

  6. [14]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022. 2, 3

    2022 arXiv

  7. [15]

    Denoising diffu- sion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models. NeurIPS, 33, 2020. 2, 3

    2020

  8. [16]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021. 4

    2021 arXiv

  9. [17]

    VBench: Com- prehensive benchmark suite for video generative models

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, Yaohui Wang, Xinyuan Chen, Limin Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. VBench: Com- prehensive benchmark suite for video generative models....

    2024

  10. [18]

    Pick-a-pic: An open dataset of user preferences for text-to-image generation

    Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Ma- tiana, Joe Penna, and Omer Levy. Pick-a-pic: An open dataset of user preferences for text-to-image generation. NeurIPS, 2023. 2, 6

    2023

  11. [19]

    Bloom: A 176b-parameter open-access multilingual language model

    Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ili ´c, Daniel Hesslow, Roman Castagn ´e, Alexandra Sasha Luccioni, Franc ¸ois Yvon, Matthias Gall´e, et al. Bloom: A 176b-parameter open-access multilingual language model. 2023. 2

    2023

  12. [20]

    Motrans: Customized motion transfer with text-driven video diffusion models

    Xiaomin Li, Xu Jia, Qinghe Wang, Haiwen Diao, Meng- meng Ge, Pengxiang Li, You He, and Huchuan Lu. Motrans: Customized motion transfer with text-driven video diffusion models. In ACM MM, 2024. 2

    2024

  13. [21]

    Textcraftor: Your text encoder can be image quality controller

    Yanyu Li, Xian Liu, Anil Kag, Ju Hu, Yerlan Idelbayev, Dhritiman Sagar, Yanzhi Wang, Sergey Tulyakov, and Jian Ren. Textcraftor: Your text encoder can be image quality controller. In CVPR, 2024. 2, 3, 6

    2024

  14. [22]

    Decoupled weight decay regularization

    I Loshchilov. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017. 5

    2017 arXiv

  15. [23]

    Training lan- guage models to follow instructions with human feedback

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Car- roll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training lan- guage models to follow instructions with human feedback. NeurIPS, 2022. 3, 4

    2022

  16. [24]

    Language models are unsu- pervised multitask learners

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsu- pervised multitask learners. OpenAI blog, 2019. 2

    2019

  17. [25]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christo- pher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. NeurIPS, 2024. 3

    2024

  18. [26]

    Hierarchical text-conditional image gen- eration with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gen- eration with clip latents. arXiv preprint arXiv:2204.06125,

    arXiv

  19. [27]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. In CVPR, 2022. 2, 4

    2022

  20. [28]

    Laion-5b: An open large-scale dataset for training next generation image-text models

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Worts- man, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. NeurIPS, 2022. 2, 6 9

    2022

  21. [29]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020. 2

    2010 arXiv

  22. [30]

    Score-based generative modeling through stochastic differential equa- tions

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equa- tions. arXiv preprint arXiv:2011.13456, 2020. 2

    2011 arXiv

  23. [31]

    Diffusion model align- ment using direct preference optimization

    Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. Diffusion model align- ment using direct preference optimization. In CVPR, 2024. 3, 6

    2024

  24. [32]

    On discrete prompt optimization for diffusion models

    Ruochen Wang, Ting Liu, Cho-Jui Hsieh, and Boqing Gong. On discrete prompt optimization for diffusion models. arXiv preprint arXiv:2407.01606, 2024. 2, 3, 7

    2024 arXiv

  25. [33]

    Investigating prompt engineering in diffusion models

    Sam Witteveen and Martin Andrews. Investigating prompt engineering in diffusion models. arXiv preprint arXiv:2211.15462, 2022. 2

    2022 arXiv

  26. [34]

    Stable diffusion 2.0 and the importance of negative prompts for good results, 2022

    Max Woolf. Stable diffusion 2.0 and the importance of negative prompts for good results, 2022. https : / / minimaxir.com/2022/11/stable- diffusion- negative-prompt/. 2

    2022

  27. [35]

    Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation

    Jay Zhangjie Wu, Yixiao Ge, Xintao Wang, Stan Weixian Lei, Yuchao Gu, Yufei Shi, Wynne Hsu, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation. In ICCV, 2023. 2

    2023

  28. [36]

    Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis

    Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis. arXiv preprint arXiv:2306.09341 ,

    arXiv

  29. [37]

    Fastcomposer: Tuning-free multi- subject image generation with localized attention

    Guangxuan Xiao, Tianwei Yin, William T Freeman, Fr ´edo Durand, and Song Han. Fastcomposer: Tuning-free multi- subject image generation with localized attention. Interna- tional Journal of Computer Vision, 2024. 2

    2024

  30. [38]

    Imagere- ward: Learning and evaluating human preferences for text- to-image generation

    Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagere- ward: Learning and evaluating human preferences for text- to-image generation. 2024. 2, 3, 4, 5, 6

    2024

  31. [39]

    Scaling autoregres- sive models for content-rich text-to-image generation

    Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gun- jan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yin- fei Yang, Burcu Karagol Ayan, et al. Scaling autoregres- sive models for content-rich text-to-image generation. arXiv preprint arXiv:2206.10789, 2022. 2, 5

    2022 arXiv

  32. [40]

    Show-1: Marrying pixel and latent dif- fusion models for text-to-video generation

    David Junhao Zhang, Jay Zhangjie Wu, Jia-Wei Liu, Rui Zhao, Lingmin Ran, Yuchao Gu, Difei Gao, and Mike Zheng Shou. Show-1: Marrying pixel and latent dif- fusion models for text-to-video generation. arXiv preprint arXiv:2309.15818, 2023. 2

    2023 arXiv

  33. [41]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In ICCV, 2023. 2 10

    2023

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.