Pith. sign in

REVIEW 4 major objections 5 minor 56 references

Diffusion-NPO: Negative Preference Optimization for Better Preference Aligned Generation of Diffusion Models

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Training a model to generate bad images makes diffusion models better

desk verdict The NPO trick is cheap and the gains look consistent across models and methods, but the paper never rules out that its negative branch works simply because it is a worse model. read the letter →

arxiv 2505.11245 v1 pith:FAFJLHRR submitted 2025-05-16 cs.CV

Fu-Yun Wang , Yunhao Shui , Jingtan Piao , Keqiang Sun , Hongsheng Li This is my paper · ORCID
classification cs.CV
keywords negativepreferenceoptimizationclassifier-freeguidancediffusionmodelsalignmenttext-to-imagetext-to-videodirectplug-and-playinference
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that preference-aligned diffusion models are held back by a neglected half of classifier-free guidance: the negative or unconditional branch. Existing alignment methods teach a model what humans like, but not what they dislike, so the guidance contrast is weak. The authors propose Negative Preference Optimization (NPO), which trains a second model on reversed preference pairs or a negated reward, then uses that model as the negative-conditional predictor during CFG. This requires no new datasets, reward models, or training strategies, only a sign flip in the existing objective. A sympathetic reading takes the central claim to be that NPO consistently improves preference scores across SD1.5, SDXL, video diffusion, and already preference-optimized models, acting as a plug-and-play inference enhancement. The paper also notes that NPO doubles the stored weight offsets, though LoRA training keeps the extra cost small.

What carries the argument

The central object is the negative weight offset δ, trained by negative preference optimization, combined with the classifier-free guidance formula ε^ω = (ω+1)ε_{θ_pos} - ω ε_{θ_neg}. Preference alignment produces a positive offset η from the base weight θ; NPO produces a negative offset δ. The authors recommend composing the negative model as θ_neg = θ + αη + βδ, keeping a controlled amount of the positive direction to avoid divergence. This weight-offset picture explains why the method works: CFG is only as strong as the contrast between the conditional and negative-conditional predictions, and a deliberately trained negative branch sharpens that contrast.

What would settle it

Run a controlled test where NPO is replaced by a model trained on randomized preference labels (noise instead of reversed preferences) at the same regularization strength; if the random-negative model yields the same score gains as genuine NPO on the Pick-a-pic benchmark, then the reported improvements do not depend on learning true negative preferences.

Watch

Extended reading notes

Core claim

The central discovery is that reversing the preference order in any existing diffusion preference optimization method yields a model whose CFG guidance sharply increases alignment with human preferences. Concretely, training on swapped pairs r_NPO = (x1, x0, c), or equivalently maximizing 1 - R(x,c) instead of R(x,c), produces a weight offset δ that points away from desirable outputs. When this offset is added to the base model to form θ_neg, and CFG is run as ε = (ω+1)ε_{θ_pos} - ωε_{θ_neg} with θ_pos the positively aligned weights, the negative branch serves as a learned 'anti-model'. The paper thereby converts a training-time oversight into a training-free benefit: without new data or algorithms, any DR, RL, or DPO pipeline can be dual-trained into a positive and a negative model, and the contrast between them improves detail, color, and composition.

Load-bearing premise

The approach assumes that a model trained on reversed preference pairs or a negated reward learns to produce genuinely bad outputs, not merely degenerate or miscalibrated ones, so that the negative branch in classifier-free guidance encodes a useful avoidance direction.

Editorial extensions

If this is right

  • Any existing preference optimization method (differentiable reward, RL, or DPO) can be converted to a negative-preference variant by reversing the image pair order or negating the reward, with no new datasets or training strategies.
  • NPO functions as a plug-and-play inference enhancement: it raises PickScore, HPSv2, ImageReward, and LAION-Aesthetic scores on Stable Diffusion 1.5, SDXL, DreamShaper, and preference-optimized variants of each.
  • The method transfers to text-to-video generation, where a VADER model plus NPO(HPSv2) improves both in-domain aesthetics and out-of-domain human-prompt benchmarks.
  • NPO's improvements persist relative to simply training DPO for ten times longer, indicating the negative offset captures something that scaling the positive direction does not.
  • The weight offsets can be flexibly scaled (α, β) to trade off between sharpness and naturalness, giving users a direct dial on the strength of negative guidance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • NPO's benefit may stem from implicit unlearning: the negative branch learns to suppress features that reward models over-weight, effectively acting as an anti-reward regularizer that reduces over-saturated or 'reward-hacked' outputs.
  • The same pair-reversal trick could apply to any alignment signal beyond visual preference, such as safety or style constraints, suggesting NPO as a general recipe for building contrastive guidance branches.
  • Because preference optimization is often trained with LoRA, the two-offset storage overhead is small, which lowers the practical barrier to retraining existing pipelines with this dual-branch scheme.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. Diffusion-NPO proposes to train an additional diffusion model that is aligned against human preferences by reversing preference pair order (Eq. 8) or negating the reward (Eq. 7), and then uses this model as the negative/unconditional branch in classifier-free guidance (Eq. 9). The paper evaluates the method as a plug-and-play enhancement for SD1.5, SDXL, and VideoCrafter2 across three preference optimization families (Diffusion-DPO, Diffusion-SPO, and VADER), reporting consistent improvements on PickScore, HPSv2, ImageReward, and LAION-Aesthetic, along with a 15-person user study. The core claim is that any existing preference optimization method can be converted into a negative-preference method at negligible extra cost, and that the resulting negative branch of CFG improves alignment with human preferences.

Significance. The proposed recipe is simple, training-data-free, and broadly applicable; the empirical coverage across three method families and two modalities is a real strength, and the release of code supports reproducibility. The paper also includes comparisons with training-free CFG-strengthening baselines such as Autoguidance and SEG. However, the evidence does not currently establish that reversing preference pairs is the mechanism behind the gains: the manuscript's own ablation (Table 7) shows that a model trained on generic Gaussian-blurred negative images performs equally well, and the inference rule (Eq. 9) evaluates the NPO branch on conditioning input c' that is not specified and may differ from the training conditioning c. If the mechanism is merely 'a worse version of the model yields better CFG contrast,' the contribution overlaps heavily with Autoguidance (Karras et al., 2024). The reported gains, if confirmed with significance tests, would still be practically useful, but the central interpretational claim needs substantial additional evidence.

major comments (4)
  1. [§3.2, Eq. (9) and §3.1, Eq. (8)] The negative branch is evaluated at conditioning input c', but the NPO model is trained on paired data (x0, x1, c) with the same prompt c. The paper never states what c' is at inference or whether the NPO model receives the prompt at all. If c' is the standard empty/unconditional conditioning, the NPO model is evaluated out of its training distribution and cannot implement prompt-specific avoidance; if c' = c, the rule reduces to contrasting two models on the same prompt, which is precisely the Autoguidance setup and does not require any notion of negative preference. The authors should specify c' in their experiments and include a condition that varies c' (empty vs. prompt) to determine which setting is responsible for the gains.
  2. [Appendix II.3, Table 7] Replacing the reversed-preference NPO training data with purely generic Gaussian-blurred negative images yields essentially the same or slightly better scores (Aesthetic 5.7676 vs. 5.7621, HPS 27.63 vs. 27.60, ImageReward 0.3222 vs. 0.3102, PickScore 21.60 vs. 21.58). Under the paper's hypothesis, the negative-preference content of the training pairs is the causal ingredient; under the alternative hypothesis, any sufficiently degraded predictor improves CFG contrast. The near-identical performance of the corruption-trained model is a direct test of the mechanism and currently supports the alternative. The authors need to add a condition that controls the training data while keeping the backbone, inference rule, and effective guidance strength fixed, and show that reversed-preference NPO outperforms a model trained to be generically worse (e.g., on blurred images) by a margin that cannot be explained by guidance-strength calibration.
  3. [§2, Eqs. (4)–(6) and Appendix II.1] The weight-merge motivating example establishes only that a model with a weakened positive offset η improves the negative branch; it does not establish that the NPO offset δ contains a component orthogonal to η (δ⊥) that is responsible for the gains. The appendix acknowledges δ⊥ conceptually but provides no measurement of it. The scaling-DPO comparisons in Appendix II.1 show that DPO trained 10x longer is worse than DPO+NPO, but NPO also alters the negative branch; a proper control would be to compare against DPO(k=1)+NPO with δ replaced by -aη (or by a corrupt-trained offset) and show that the reversed-preference training contributes beyond the scale and direction of the existing offset.
  4. [Tables 1–3 and Fig. 7] All quantitative comparisons report means and win ratios without variance, confidence intervals, or significance tests. Several headline differences are small (e.g., Table 2, SDXL vs. SDXL+NPO: PickScore 22.06 vs. 22.32, HPSv2 27.89 vs. 28.11; Table 1, Diff.-SPO vs. Diff.-SPO+NPO: HPSv2 26.85 vs. 27.09), and win ratios are reported without error bars despite being computed over a finite prompt set. The authors should report paired significance tests (e.g., bootstrap or Wilcoxon) and the number of prompts/samples per cell so the reader can judge whether the consistency claim is supported.
minor comments (5)
  1. [Appendix II.3, Table 7] The caption says 'WinRate represents the percentage of cases where the method outperforms the original NPO,' which is an unusual baseline; please clarify the comparison protocol and state the number of prompts used.
  2. [§3.1, Eq. (7)] The transform RNPO = 1 - R assumes R ∈ [0,1], but the paper applies it to HPSv2/ImageReward-style rewards without stating how non-bounded rewards are rescaled in practice; please specify the exact scaling used in the experiments.
  3. [Throughout] There are several typos: 'Fo methods' (page 6), 'emplogy' (page 5), 'pptimization' in the Appendix heading II.3, 'f the deviation' (page 10), and 'Shanghai Jiang Tong University' in the affiliation should be 'Shanghai Jiao Tong University'.
  4. [Abstract and Fig. 2] The text says NPO requires 'no new training strategies or datasets,' while Appendix II.3 explores modified corruptions of the dataset; please reconcile the wording, e.g., 'no new datasets are required for the core method, though data curation can further improve it.'
  5. [Appendix II.2] The relationship to Autoguidance (Karras et al., 2024) is discussed only in an appendix; given that the paper's own mechanism question turns on the distinction between 'negative preference' and 'a bad version of itself,' this comparison should be moved to the main text and expanded with the c'/c analysis requested above.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: NPO is an explicit label/reward reversal, and the claimed gains are empirical on external benchmarks.

full rationale

The paper's central construction is an explicit transformation of existing preference signals: Eq. 7 maps a reward R to 1−R, and Eq. 8 reverses the order of preference pairs. These equations define what 'negative preference' means rather than secretly importing the paper's target claim. The inference rule in Eq. 9 then combines the positive and negative branches through classifier-free guidance, and whether this combination improves preference metrics is an empirical, falsifiable claim tested on the held-out Pick-a-pic 'test unique' split and reported against external metrics (HPSv2, ImageReward, PickScore, LAION-Aesthetic). No parameter is fitted to the evaluation metric and then relabeled as a prediction: the hyper-parameter study explicitly varies α, β, and Beta, and the final comparisons use official weights under stated configurations. The motivating weight-merge argument is explicitly presented as an approximation, and the appendix concedes that the NPO offset δ has an orthogonal component not captured by scaling η, weakening any suggestion that the method reduces to a scaled version of the positive offset. Self-citations appear in the paper but are not load-bearing: no uniqueness theorem or prior result by the same authors is invoked to force the choice of the negative branch. The skeptical concerns about the c′ mismatch in Eq. 9 and the corrupted-data ablation in Table 7 are about mechanism identification and external validity—namely, whether the negative branch encodes prompt-specific avoidance rather than generic degradation—not about circularity of the derivation. Likewise, the use of preference metrics from the same family as the supervision is an evaluation-design consideration, not a reduction of the claimed result to its inputs. The derivation chain is therefore self-contained: the method is defined by explicit transforms, the improvement is measured empirically on separate data, and no step equates the conclusion with an input by construction.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper's contribution is empirical; its formal content is mostly algebraic rewriting of CFG. The load-bearing assumptions are that label-flipped preference optimization produces a useful negative model, that weight offsets combine linearly, and that CFG contrast is the right mechanism. Main free parameters are the mixing factor α, the training regularization β, and the scaling of δ; the reported headline numbers come after tuning these on validation.

free parameters (3)
  • alpha_mixing_factor = 0.6 or 1.0 for DreamShaper; ablated 0.0-1.0
    Eq. 10: controls how much of the preference-alignment offset η is added to the negative model. Selected per model after looking at validation metrics, so the reported numbers partly depend on this choice.
  • training_beta = 250 or 1000 for Diffusion-DPO based NPO; ablated 100-2500
    DPO regularization strength that controls how far the NPO weight offset δ moves from the base model. Tuned in Fig. 10, and the reported configurations differ by experiment.
  • delta_scaling_beta = not reported; presumably 1.0
    Eq. 10 includes a scaling factor β for the NPO offset δ, but the paper provides no explicit default or ablation for this value in the main text, so an unstated choice is load-bearing.
assumptions (4)
  • domain assumption Reversing preference pairs (Eq. 8) or negating a reward (Eq. 7) produces a model aligned with preferences opposite to human in a semantically useful direction.
    Central to NPO. The paper does not derive this; it is only probed indirectly through downstream metrics and a small user study.
  • domain assumption Weight offsets combine linearly, so θneg = θ + αη + βδ preserves the intended behavior of each offset.
    Eq. 10 assumes the trained offsets remain interpretable under linear interpolation. The motivating example uses a weight merge approximation, which the appendix itself calls an oversimplification.
  • domain assumption CFG contrast between conditional and negative-conditional predictors is the operative mechanism for preference alignment, and improving the negative branch improves final output.
    Borrowed from Ho and Salimans and Karras et al. The whole method depends on this mechanism being the right place to intervene.
  • domain assumption Preference metrics such as PickScore, HPSv2, ImageReward, and LAION-Aesthetic approximate human preference.
    Used for nearly all quantitative evaluation. These metrics overlap with the preference supervision used to train the baselines, so they are not fully independent evidence.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Diffusion-NPO: Negative Preference Optimization for Better Preference Aligned Generation of Diffusion Models." pith.science (2026). https://pith.science/paper/FAFJLHRR

@misc{pith2026250511245,
  author       = {Pith},
  title        = {Pith review of: Diffusion-NPO: Negative Preference Optimization for Better Preference Aligned Generation of Diffusion Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FAFJLHRR}},
  note         = {Machine review of arXiv:2505.11245}
}
read the original abstract

Diffusion models have made substantial advances in image generation, yet models trained on large, unfiltered datasets often yield outputs misaligned with human preferences. Numerous methods have been proposed to fine-tune pre-trained diffusion models, achieving notable improvements in aligning generated outputs with human preferences. However, we argue that existing preference alignment methods neglect the critical role of handling unconditional/negative-conditional outputs, leading to a diminished capacity to avoid generating undesirable outcomes. This oversight limits the efficacy of classifier-free guidance~(CFG), which relies on the contrast between conditional generation and unconditional/negative-conditional generation to optimize output quality. In response, we propose a straightforward but versatile effective approach that involves training a model specifically attuned to negative preferences. This method does not require new training strategies or datasets but rather involves minor modifications to existing techniques. Our approach integrates seamlessly with models such as SD1.5, SDXL, video diffusion models and models that have undergone preference optimization, consistently enhancing their alignment with human preferences.

Figures

Figures reproduced from arXiv: 2505.11245 by the authors.

Figure 1
Figure 1. Diffusion-NPO enhances high-frequency details, color and lighting, and low-frequency [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. High-level overview of negative preference optimization (NPO). (Training) NPO needs [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. NPO works as a plug-and-play inference enhancement strategy. It can be easily combined [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Plug-and-play NPO on DreamShaper. NPO not only works on the base Stable Diffusion [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Motivating example results of Section 2. Applying merged model as the orange component (i.e., predic￾tion for unconditional/negative-conditional inputs) effectively improves the human pref￾erence alignment. Analysis: the weight merge is an approximated NPO. What is the…
Figure 6
Figure 6. Figure 6: Video comparison. The videos are trained using 12 frames. For better visualization, we [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Quantitative winning ratios. the [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: User study analysis. Stable Diffusion v1 - 5 Diffusion -DPO Diffusion -DPO+NPO Beta = 100 Beta = 250 Beta = 500 Beta = 1000 Beta = 2500 𝛼 = 1.0 𝛼 = 0.8 𝛼 = 0.6 𝛼 = 0.4 𝛼 = 0.2 𝛼 = 0.0 [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 10
Figure 10. Figure 10: Heat map-based ablation study on hyper-parameter choice. [PITH_FULL_IMAGE:figures/full_fig_p010_10.png]
Figure 11
Figure 11. Figure 11: Comparison on Stable Diffusion 1.5. 5 [PITH_FULL_IMAGE:figures/full_fig_p020_11.png]
Figure 12
Figure 12. Figure 12: Comparison on DreamShaper. 6 [PITH_FULL_IMAGE:figures/full_fig_p021_12.png]
Figure 13
Figure 13. Figure 13: Comparison on Stable Diffusion XL. 7 [PITH_FULL_IMAGE:figures/full_fig_p022_13.png]
Figure 14
Figure 14. Figure 14: Comparison on Diffusion-DPO. 8 [PITH_FULL_IMAGE:figures/full_fig_p023_14.png]
Figure 15
Figure 15. Figure 15: Comparison on Diffusion-SPO. 9 [PITH_FULL_IMAGE:figures/full_fig_p024_15.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

56 extracted references · 12 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  3. [3]

    Self-rectifying diffusion sampling with perturbed-attention guidance

    Donghoon Ahn, Hyoungwon Cho, Jaewon Min, Wooseok Jang, Jungwoo Kim, SeonHwa Kim, Hyun Hee Park, Kyong Hwan Jin, and Seungryong Kim. Self-rectifying diffusion sampling with perturbed-attention guidance. arXiv preprint arXiv:2403.17377, 2024

    arXiv 2024

  4. [4]

    Gs-dit: Advancing video generation with pseudo 4d gaussian fields through efficient dense 3d point tracking

    Weikang Bian, Zhaoyang Huang, Xiaoyu Shi, Yijin Li, Fu-Yun Wang, and Hongsheng Li. Gs-dit: Advancing video generation with pseudo 4d gaussian fields through efficient dense 3d point tracking. arXiv preprint arXiv:2501.02690, 2025

    arXiv 2025

  5. [5]

    Training diffusion models with reinforcement learning

    Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine. Training diffusion models with reinforcement learning. arXiv preprint arXiv:2305.13301, 2023

    arXiv 2023

  6. [6]

    Videocrafter2: Overcoming data limitations for high-quality video diffusion models

    Haoxin Chen, Yong Zhang, Xiaodong Cun, Menghan Xia, Xintao Wang, Chao Weng, and Ying Shan. Videocrafter2: Overcoming data limitations for high-quality video diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 7310--7320, 2024

    2024

  7. [7]

    Directly fine-tuning diffusion models on differentiable rewards

    Kevin Clark, Paul Vicol, Kevin Swersky, and David J Fleet. Directly fine-tuning diffusion models on differentiable rewards. arXiv preprint arXiv:2309.17400, 2023

    arXiv 2023

  8. [8]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34: 0 8780--8794, 2021

    2021

Show all 56 references
  1. [9]

    badhand, 2024

    Euge. badhand, 2024. URL https://civitai.com/models/16993/badhandv4

    2024

  2. [10]

    Reinforcement learning for fine-tuning text-to-image diffusion models

    Ying Fan, Olivia Watkins, Yuqing Du, Hao Liu, Moonkyung Ryu, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, Kangwook Lee, and Kimin Lee. Reinforcement learning for fine-tuning text-to-image diffusion models. Advances in Neural Information Processing Systems, 36, 2024

    2024

  3. [11]

    An image is worth one word: Personalizing text-to-image generation using textual inversion

    Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H Bermano, Gal Chechik, and Daniel Cohen-Or. An image is worth one word: Personalizing text-to-image generation using textual inversion. arXiv preprint arXiv:2208.01618, 2022

    2022 arXiv

  4. [12]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022

    2022 arXiv

  5. [13]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33: 0 6840--6851, 2020

    2020

  6. [14]

    Smoothed energy guidance: Guiding diffusion models with reduced energy curvature of attention

    Susung Hong. Smoothed energy guidance: Guiding diffusion models with reduced energy curvature of attention. Advances in Neural Information Processing Systems, 37: 0 66743--66772, 2025

    2025

  7. [15]

    Elucidating the design space of diffusion-based generative models

    Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. Advances in neural information processing systems, 35: 0 26565--26577, 2022

    2022

  8. [16]

    Guiding a diffusion model with a bad version of itself

    Tero Karras, Miika Aittala, Tuomas Kynk \"a \"a nniemi, Jaakko Lehtinen, Timo Aila, and Samuli Laine. Guiding a diffusion model with a bad version of itself. arXiv preprint arXiv:2406.02507, 2024

    2024 arXiv

  9. [17]

    Repurposing diffusion-based image generators for monocular depth estimation

    Bingxin Ke, Anton Obukhov, Shengyu Huang, Nando Metzger, Rodrigo Caye Daudt, and Konrad Schindler. Repurposing diffusion-based image generators for monocular depth estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 9492--9502, 2024

    2024

  10. [18]

    Variational diffusion models

    Diederik Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. Variational diffusion models. Advances in neural information processing systems, 34: 0 21696--21707, 2021

    2021

  11. [19]

    Pick-a-pic: An open dataset of user preferences for text-to-image generation

    Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana, Joe Penna, and Omer Levy. Pick-a-pic: An open dataset of user preferences for text-to-image generation. Advances in Neural Information Processing Systems, 36: 0 36652--36663, 2023

    2023

  12. [20]

    Unleashing vecset diffusion model for fast shape generation

    Zeqiang Lai, Yunfei Zhao, Zibo Zhao, Haolin Liu, Fuyun Wang, Huiwen Shi, Xianghui Yang, Qingxiang Lin, Jingwei Huang, Yuhong Liu, et al. Unleashing vecset diffusion model for fast shape generation. arXiv preprint arXiv:2503.16302, 2025

    2025 arXiv

  13. [21]

    Rich human feedback for text-to-image generation

    Youwei Liang, Junfeng He, Gang Li, Peizhao Li, Arseniy Klimovskiy, Nicholas Carolan, Jiao Sun, Jordi Pont-Tuset, Sarah Young, Feng Yang, et al. Rich human feedback for text-to-image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognitio...

    2024

  14. [22]

    Step-aware preference optimization: Aligning preference with denoising performance at each step

    Zhanhao Liang, Yuhui Yuan, Shuyang Gu, Bohan Chen, Tiankai Hang, Ji Li, and Liang Zheng. Step-aware preference optimization: Aligning preference with denoising performance at each step. arXiv preprint arXiv:2406.04314, 2024 b

    2024 arXiv

  15. [23]

    Pseudo numerical methods for diffusion models on manifolds

    Luping Liu, Yi Ren, Zhijie Lin, and Zhou Zhao. Pseudo numerical methods for diffusion models on manifolds. arXiv preprint arXiv:2202.09778, 2022

    2022 arXiv

  16. [24]

    Osv: One step is enough for high-quality image to video generation

    Xiaofeng Mao, Zhengkai Jiang, Fu-Yun Wang, Wenbing Zhu, Jiangning Zhang, Hao Chen, Mingmin Chi, and Yabiao Wang. Osv: One step is enough for high-quality image to video generation. arXiv preprint arXiv:2409.11367, 2024

    2024 arXiv

  17. [25]

    badprompt, 2023

    Nerfgun3. badprompt, 2023. URL https://huggingface.co/datasets/Nerfgun3/bad_prompt

    2023

  18. [26]

    Training language models to follow instructions with human feedback

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35: 0 2...

    2022

  19. [27]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pp.\ 4195--4205, 2023

    2023

  20. [28]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M \"u ller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023

    2023 arXiv

  21. [29]

    Aligning text-to-image diffusion models with reward backpropagation

    Mihir Prabhudesai, Anirudh Goyal, Deepak Pathak, and Katerina Fragkiadaki. Aligning text-to-image diffusion models with reward backpropagation. arXiv preprint arXiv:2310.03739, 2023

    2023 arXiv

  22. [30]

    Video diffusion alignment via reward gradients

    Mihir Prabhudesai, Russell Mendonca, Zheyang Qin, Katerina Fragkiadaki, and Deepak Pathak. Video diffusion alignment via reward gradients. arXiv preprint arXiv:2407.08737, 2024

    2024 arXiv

  23. [31]

    Markov decision processes: discrete stochastic dynamic programming

    Martin L Puterman. Markov decision processes: discrete stochastic dynamic programming. John Wiley & Sons, 2014

    2014

  24. [32]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36, 2024

    2024

  25. [33]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj \"o rn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 10684--10695, 2022

    2022

  26. [34]

    Laion-aesthetics

    Christoph Schuhmann. Laion-aesthetics. https://laion.ai/blog/laion-aesthetics/, 2022. Accessed: 2023-11-10

    2022

  27. [35]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  28. [36]

    Rethinking the spatial inconsistency in classifier-free diffusion guidance

    Dazhong Shen, Guanglu Song, Zeyue Xue, Fu-Yun Wang, and Yu Liu. Rethinking the spatial inconsistency in classifier-free diffusion guidance. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 9370--9379, June 2024

    2024

  29. [37]

    Motion-i2v: Consistent and controllable image-to-video generation with explicit motion modeling

    Xiaoyu Shi, Zhaoyang Huang, Fu-Yun Wang, Weikang Bian, Dasong Li, Yi Zhang, Manyuan Zhang, Ka Chun Cheung, Simon See, Hongwei Qin, et al. Motion-i2v: Consistent and controllable image-to-video generation with explicit motion modeling. arXiv e-prints, pp.\ arXiv--2401, 2024

    2024

  30. [38]

    Make-a-video: Text-to-video generation without text-video data

    Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-a-video: Text-to-video generation without text-video data. arXiv preprint arXiv:2209.14792, 2022

    2022 arXiv

  31. [39]

    Journeydb: A benchmark for generative image understanding

    Keqiang Sun, Junting Pan, Yuying Ge, Hao Li, Haodong Duan, Xiaoshi Wu, Renrui Zhang, Aojun Zhou, Zipeng Qin, Yi Wang, et al. Journeydb: A benchmark for generative image understanding. Advances in Neural Information Processing Systems, 36, 2024

    2024

  32. [40]

    Reinforcement learning: An introduction

    Richard S Sutton. Reinforcement learning: An introduction. A Bradford Book, 2018

    2018

  33. [41]

    Diffusion model alignment using direct preference optimization

    Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. Diffusion model alignment using direct preference optimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and...

    2024

  34. [42]

    Phased consistency model

    Fu-Yun Wang, Zhaoyang Huang, Alexander William Bergman, Dazhong Shen, Peng Gao, Michael Lingelbach, Keqiang Sun, Weikang Bian, Guanglu Song, Yu Liu, et al. Phased consistency model. arXiv preprint arXiv:2405.18407, 2024 a

    2024 arXiv

  35. [43]

    Animatelcm: Computation-efficient personalized style video generation without personalized video data

    Fu-Yun Wang, Zhaoyang Huang, Weikang Bian, Xiaoyu Shi, Keqiang Sun, Guanglu Song, Yu Liu, and Hongsheng Li. Animatelcm: Computation-efficient personalized style video generation without personalized video data. In SIGGRAPH Asia 2024 Technical Communications, pp.\ 1--5. 2024 b

    2024

  36. [44]

    Zola: Zero-shot creative long animation generation with short video model

    Fu-Yun Wang, Zhaoyang Huang, Qiang Ma, Guanglu Song, Xudong Lu, Weikang Bian, Yijin Li, Yu Liu, and Hongsheng Li. Zola: Zero-shot creative long animation generation with short video model. In European Conference on Computer Vision, pp.\ 329--345. Springer, 2024 c

    2024

  37. [45]

    Animatelcm: Accelerating the animation of personalized diffusion models and adapters with decoupled consistency learning

    Fu-Yun Wang, Zhaoyang Huang, Xiaoyu Shi, Weikang Bian, Guanglu Song, Yu Liu, and Hongsheng Li. Animatelcm: Accelerating the animation of personalized diffusion models and adapters with decoupled consistency learning. arXiv preprint arXiv:2402.00769, 2024 d

    2024 arXiv

  38. [46]

    Be-your-outpainter: Mastering video outpainting through input-specific adaptation

    Fu-Yun Wang, Xiaoshi Wu, Zhaoyang Huang, Xiaoyu Shi, Dazhong Shen, Guanglu Song, Yu Liu, and Hongsheng Li. Be-your-outpainter: Mastering video outpainting through input-specific adaptation. In European Conference on Computer Vision, pp.\ 153--168. Springer, 2024 e

    2024

  39. [47]

    Rectified diffusion: Straightness is not your need in rectified flow

    Fu-Yun Wang, Ling Yang, Zhaoyang Huang, Mengdi Wang, and Hongsheng Li. Rectified diffusion: Straightness is not your need in rectified flow. arXiv preprint arXiv:2410.07303, 2024 f

    2024 arXiv

  40. [48]

    I made stable diffusion xl smarter by finetuning it on bad ai-generated images, 2023

    Max Woolf. I made stable diffusion xl smarter by finetuning it on bad ai-generated images, 2023. URL https://minimaxir.com/2023/08/stable-diffusion-xl-wrong/

    2023

  41. [49]

    Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis

    Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis. arXiv preprint arXiv:2306.09341, 2023

    2023 arXiv

  42. [50]

    Deep reward supervisions for tuning text-to-image diffusion models

    Xiaoshi Wu, Yiming Hao, Manyuan Zhang, Keqiang Sun, Zhaoyang Huang, Guanglu Song, Yu Liu, and Hongsheng Li. Deep reward supervisions for tuning text-to-image diffusion models. arXiv preprint arXiv:2405.00760, 2024

    2024 arXiv

  43. [51]

    Imagereward: Learning and evaluating human preferences for text-to-image generation

    Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagereward: Learning and evaluating human preferences for text-to-image generation. Advances in Neural Information Processing Systems, 36, 2024

    2024

  44. [52]

    Using human feedback to fine-tune diffusion models without any reward model

    Kai Yang, Jian Tao, Jiafei Lyu, Chunjiang Ge, Jiaxin Chen, Weihan Shen, Xiaolong Zhu, and Xiu Li. Using human feedback to fine-tune diffusion models without any reward model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 8941--8951, 2024

    2024

  45. [53]

    From slow bidirectional to fast causal video generators

    Tianwei Yin, Qiang Zhang, Richard Zhang, William T Freeman, Fredo Durand, Eli Shechtman, and Xun Huang. From slow bidirectional to fast causal video generators. arXiv preprint arXiv:2412.07772, 2024

    2024

  46. [54]

    Trans4d: Realistic geometry-aware transition for compositional text-to-4d synthesis

    Bohan Zeng, Ling Yang, Siyu Li, Jiaming Liu, Zixiang Zhang, Juanxi Tian, Kaixin Zhu, Yongzhen Guo, Fu-Yun Wang, Minkai Xu, et al. Trans4d: Realistic geometry-aware transition for compositional text-to-4d synthesis. arXiv preprint arXiv:2410.07155, 2024

    arXiv

  47. [55]

    Negative preference optimization: From catastrophic collapse to effective unlearning

    Ruiqi Zhang, Licong Lin, Yu Bai, and Song Mei. Negative preference optimization: From catastrophic collapse to effective unlearning. arXiv preprint arXiv:2404.05868, 2024 a

    2024 arXiv

  48. [56]

    Large-scale reinforcement learning for diffusion models

    Yinan Zhang, Eric Tzeng, Yilun Du, and Dmitry Kislyuk. Large-scale reinforcement learning for diffusion models. arXiv preprint arXiv:2401.12244, 2024 b

    2024 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.