Pith. sign in

REVIEW 2 major objections 5 minor 1 cited by

Preference Alignment on Diffusion Model: A Comprehensive Survey for Image Generation and Editing

T0 review · 2 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This survey claims to be the first comprehensive map of preference alignment for diffusion models, organizing methods by sampling, training strategy, reward feedback, and modality.

desk verdict Useful taxonomy and a broad method sweep, but the D3PO mischaracterization and sloppy citations keep it from being a reliable reference without revision. read the letter →

arxiv 2502.07829 v1 pith:GCSVOLWE submitted 2025-02-10 cs.CV cs.LG

Sihao Wu , Xiaonan Si , Chi Xing , Jianhong Wang , Gaojie Jin , Guangliang Cheng , Lijun Zhang , Xiaowei Huang This is my paper · ORCID
classification cs.CVcs.LG
keywords preferencealignmentdiffusionmodelsimagegenerationeditingreinforcementlearningRLHFDPOtaxonomy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to fill a gap: surveys of diffusion-based generation and editing exist, but none, in the authors' assessment, focuses specifically on preference alignment, the family of techniques that fine-tune diffusion models using human or AI judgments about what makes an output good. To make that family tractable, the paper casts all such methods as reinforcement-learning problems and sorts them into a taxonomy with four axes: sampling, training strategy, reward feedback, and modality. It reviews reinforcement learning with human feedback (RLHF), direct preference optimization (DPO) and its variants, and less common alternatives such as GFlowNets (generative flow networks, a class of reward-proportional samplers) and self-play, then maps applications in medical imaging, robotics, autonomous driving, and adjacent domains. It closes by naming computational cost, preference-data collection, multi-objective consistency, and reward subjectivity as the core challenges. A sympathetic reader would take the paper's contribution to be organizational: a structured entry point into a fast-moving intersection.

What carries the argument

The load-bearing machine is a taxonomy built on two moves. First, preference alignment is formulated as an RL problem in which the diffusion model is the policy, human or AI judgments supply a reward, and the denoising process is the environment. Second, every method is classified along four axes: sampling (online vs offline), training strategy (RLHF, DPO, or others, including GFlowNets, a class of reward-proportional samplers, and self-play), reward feedback (human vs AI annotation), and modality (textual, visual, or multimodal). The taxonomy is materialized in Table 1, where each surveyed method becomes a row and each axis a column, exposing which combinations of choices have been tried and which remain open.

What would settle it

A literature search for a previously published survey whose stated scope is preference alignment with diffusion models for image generation and editing would settle the priority claim: finding one falsifies it. To test the taxonomy, a reader could re-read each cited paper and check the Table 1 cells (online or offline, RLHF or DPO or other, human or AI feedback, text or visual or multimodal); systematic mismatches would falsify the organizing claim.

Watch

Extended reading notes

Core claim

The central claim is organizational: the many methods that fine-tune diffusion models with human or AI preferences share an RL-style skeleton—a policy, a reward, an action space, and an environment—and each method can be located by four design axes. The paper asserts that no prior survey has centered on this intersection, so its taxonomy gives the first systematic way to compare methods. If the organization holds, the field becomes easier to navigate for newcomers and easier to mine for open problems by experienced researchers.

Load-bearing premise

The survey's usefulness depends on each summarized method being described faithfully and assigned to the correct cell of its taxonomy.

Editorial extensions

If this is right

  • A newcomer can locate any given method in the taxonomy via Table 1, which splits methods by online versus offline sampling and by RLHF, DPO, or other training strategies.
  • The survey implies that pairwise preference data can replace a learned reward in many diffusion tasks, since DPO bypasses reward modeling and RL updates altogether.
  • The application chapters show the same alignment machinery moving across medical imaging, robotics, autonomous driving, biology, games, and content creation, suggesting a shared toolkit rather than domain-specific algorithms.
  • The challenges section turns computational cost, preference-data collection, multi-objective consistency, and reward subjectivity into concrete research targets.
  • The future-directions discussion identifies parameter-efficient tuning, multimodal preference interpretation, robust evaluation metrics, and safety and privacy as the next frontiers.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The four-axis taxonomy doubles as a design checklist: a new method is fully specified once it chooses online or offline sampling, RL or DPO or another training loop, human or AI feedback, and a text, visual, or multimodal target.
  • The survey does not quantitatively compare methods, so a natural next step is a shared benchmark that evaluates methods within each taxonomy cell to test whether the axes predict performance.
  • The same RL formulation should extend to video and 3D diffusion models, because the reward and preference signals transfer even though the denoising environment changes.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. This paper is a survey of preference alignment methods for diffusion models (DMs), focused on image generation and editing. It proposes a taxonomy based on sampling (online/offline), training strategy (RLHF, DPO, others), reward feedback (human vs. AI), and modality (textual/visual). The survey reviews representative methods in Sections 3.2–3.4, lists them in Table 1, discusses applications in medical imaging, robotics, and autonomous driving in Section 4, and closes with challenges and future directions. The authors claim this is the first survey specifically centered on preference alignment with DMs for image generation and editing.

Significance. If accurate, this survey would fill a genuine gap in the literature by organizing a rapidly growing body of work at the intersection of diffusion models and preference-based optimization. The proposed taxonomy is reasonable, the coverage of methods is broad (approximately 30 methods in Table 1), and the paper identifies relevant challenges such as computational cost, data scarcity, and multi-objective alignment. The survey also has practical value for newcomers entering this area. However, the manuscript's reliability as a reference depends critically on the faithfulness of each method summary; a demonstrable mischaracterization of a canonical method (D3PO) significantly weakens the survey's credibility. The paper also includes many works in Section 4 that do not obviously involve preference alignment, which undermines the claimed application coverage. The work is not a technical research contribution but a survey, so the core assessment is about accuracy and organization.

major comments (2)
  1. [Section 3.3, DPO and its variants; Table 1] The description of D3PO is factually incorrect. The manuscript states: "Using a pre-trained model to represent the action value function Q, the DPO framework is extended to allow direct parameter updates at each denoising step." D3PO (Yang et al., CVPR 2024) does not use any pre-trained action-value function Q; its central design is to extend DPO to the denoising process by treating it as a multi-step MDP and applying a step-wise DPO objective, thereby avoiding any reward model or critic. This error is load-bearing because the survey's value derives from accurate method summaries, and D3PO is presented as a core offline method in Table 1. The authors should correct the D3PO summary in Section 3.3 and audit the corresponding row in Table 1, and in light of this error, verify every other method description against its cited source.
  2. [Section 4, Applications] Many works described in Section 4 are not preference-aligned diffusion models, despite the abstract's claim that the paper "thoroughly explore[s] the applications of aligning preferences with DMs in autonomous driving, medical imaging, robotics, and more." For instance, Section 4.1 presents MedSegDiff and MedSegDiff-V2 as medical image segmentation methods with no mention of preference alignment or human feedback; Section 4.2 includes Janner et al. (2022) for trajectory planning, a work that does not involve preference alignment. If these are intended as examples of diffusion models used in the respective domains, the section's scope should be explicitly broadened; if they are intended as applications of preference-aligned DMs, the descriptions should clearly state how preference alignment is used. As written, the application coverage is not aligned with the paper's stated scope, and this is a load-bearing issue for the survey's completeness and coherence.
minor comments (5)
  1. [Section 3.2, RLHF] The text states that DMs can be described as a "Markov Chain Process (MDP)"; the correct term is "Markov Decision Process." The acronym MDP is standard and should be expanded correctly.
  2. [Section 2.1, DDIM equation] The DDIM backward-step equation is typeset in a manner that makes it hard to parse (e.g., "xt−1 = √ ¯αt−1 xt − √1 − αtϵθ(xt, t)√αt + p 1 − αt−1ϵθ(xt, t)"). The formula appears to have misplaced parentheses and missing division; please reformat it clearly, and consider referencing the original paper for the exact expression.
  3. [References] Many reference entries are incomplete: e.g., "Guan and et al.", "Gao and et al.", "Schulman and et al.", "Prabhudesai and et al.", and "Yang and et al." lack the first author's full name. This deviates from standard citation style and makes verification difficult. Full author lists (or at least first author names) should be provided.
  4. [Table 1] The submitted version of Table 1 does not visibly separate the column headers from the data rows, making it difficult to determine which method corresponds to which combination of training strategy, task, reward feedback, and modality. Please reformat the table so that column headers are clearly aligned with their entries.
  5. [Section 5, Challenges] The future-directions discussion is mostly qualitative and repeats common themes (efficiency, data, multimodality, safety). It would benefit from concrete pointers to the methods reviewed earlier (e.g., which of the surveyed approaches already address these issues) and from references to recent work on each direction.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the survey is descriptive and self-contained, with no fitted-parameter or self-citation chain; the D3PO description concern is an accuracy matter, not circularity.

full rationale

This is a literature survey rather than a derivation. Its central claims are the scope and taxonomy of preference-alignment methods for diffusion models (Sections 3.1-3.4) and the application/challenge summaries (Sections 4-5). These claims are descriptive and are not derived from a fitted parameter, an equation, or a self-citation chain. The 'first survey' assertion is a scope statement, not a mathematical consequence of the paper's own construction, and no cited uniqueness theorem or prior author result is used to exclude alternatives. The survey cites prior work only as external evidence for the existence and content of the reviewed methods; I found no place where an equation or method description reduces by construction to an input of the survey itself. The skeptical concern about the D3PO description in Section 3.3 (the claim that D3PO uses 'a pre-trained model to represent the action value function Q') is a potential factual accuracy issue in a secondary summary, not a circularity: even if the summary is wrong, the survey's taxonomy does not become equivalent to its input. Under the hard rules, accuracy disagreements are not circularity evidence. No circular steps are therefore reported.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

This is a review; it introduces no free parameters or invented entities. Its central claims rest on the correctness of the cited papers and on the validity of the survey's classification scheme.

assumptions (4)
  • domain assumption The survey's classification scheme (sampling, training strategy, reward feedback, modality) is a valid and complete partition of the preference alignment literature.
    The main contribution is the taxonomy in Sec. 3.1 and Table 1; if categories overlap or omit important methods, the survey's organizing value is reduced.
  • domain assumption The individual method summaries in Secs. 3.2 to 3.4 accurately represent the cited papers.
    A survey's soundness is inherited from the accuracy of its secondary descriptions; at least the D3PO summary appears inconsistent with the cited work.
  • domain assumption The 40 methods in Table 1 are representative of the field.
    The paper does not provide a search strategy or inclusion criteria, so representativeness is assumed rather than demonstrated.
  • standard math Standard background mathematics for DDPM, DDIM, PPO, and DPO taken from cited sources is correct and applicable.
    The survey uses these equations for exposition; no new derivations are provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Preference Alignment on Diffusion Model: A Comprehensive Survey for Image Generation and Editing." pith.science (2026). https://pith.science/paper/GCSVOLWE

@misc{pith2026250207829,
  author       = {Pith},
  title        = {Pith review of: Preference Alignment on Diffusion Model: A Comprehensive Survey for Image Generation and Editing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GCSVOLWE}},
  note         = {Machine review of arXiv:2502.07829}
}
read the original abstract

The integration of preference alignment with diffusion models (DMs) has emerged as a transformative approach to enhance image generation and editing capabilities. Although integrating diffusion models with preference alignment strategies poses significant challenges for novices at this intersection, comprehensive and systematic reviews of this subject are still notably lacking. To bridge this gap, this paper extensively surveys preference alignment with diffusion models in image generation and editing. First, we systematically review cutting-edge optimization techniques such as reinforcement learning with human feedback (RLHF), direct preference optimization (DPO), and others, highlighting their pivotal role in aligning preferences with DMs. Then, we thoroughly explore the applications of aligning preferences with DMs in autonomous driving, medical imaging, robotics, and more. Finally, we comprehensively discuss the challenges of preference alignment with DMs. To our knowledge, this is the first survey centered on preference alignment with DMs, providing insights to drive future innovation in this dynamic area.

Figures

Figures reproduced from arXiv: 2502.07829 by the authors.

Figure 1
Figure 1. Preference Alignment on DM Framework: The process begins with a prompt model encoding a prompt into an embedding, and a [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. General paradigm of preference alignment for various [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. $I^2G$: Generating Instructional Illustrations via Text-Conditioned Diffusion

    cs.CL 2025-05 reject novelty 5.0 of 10

    A pairwise-conditioned diffusion model generates instructional illustrations from procedural text and is finetuned with a text-image alignment reward.

Reference graph

Works this paper leans on

82 extracted references · 79 canonical work pages · cited by 1 Pith paper

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, and et al. Gpt-4 technical report. ArXiv , 2023

    2023

  2. [2]

    Gflownet foundations

    Yoshua Bengio, Salem Lahlou, and et al. Gflownet foundations. JMLR , 2023

    2023

  3. [3]

    Training diffusion models with reinforcement learning

    Kevin Black, Michael Janner, and et al. Training diffusion models with reinforcement learning. ArXiv , 2023

    work page 2023

  4. [4]

    Sheng, and et al

    Yuanjiang Cao, Quan Z. Sheng, and et al. Reinforcement learning for generative AI: A survey. Arxiv , 2023

    work page 2023

  5. [5]

    Drivinggpt: Unifying driving world modeling and planning with multi-modal autoregressive transformers

    Yuntao Chen, Yuqi Wang, and et al. Drivinggpt: Unifying driving world modeling and planning with multi-modal autoregressive transformers. ArXiv , 2024

    work page 2024

  6. [6]

    Deep reinforcement learning from human preferences

    Paul F Christiano, Jan Leike, and et al. Deep reinforcement learning from human preferences. Neurips , 2017

    work page 2017

  7. [7]

    Directly fine-tuning diffusion models on differentiable rewards

    Kevin Clark, Paul Vicol, and et al. Directly fine-tuning diffusion models on differentiable rewards. ArXiv , 2023

    work page 2023

  8. [8]

    On differentially private 3d medical image synthesis with controllable latent diffusion models

    Deniz Daum and et al. On differentially private 3d medical image synthesis with controllable latent diffusion models. MICCAI Workshop on Deep Generative Models , 2024

    work page 2024

Show all 82 references
  1. [9]

    Prdp: Proximal reward difference prediction for large-scale reward finetuning of diffusion models

    Fei Deng, Qifei Wang, and et al. Prdp: Proximal reward difference prediction for large-scale reward finetuning of diffusion models. CVPR , 2024

    2024

  2. [10]

    Aligndiff: Aligning diverse human preferences via behavior-customisable diffusion model

    Zibin Dong, Yifu Yuan, and et al. Aligndiff: Aligning diverse human preferences via behavior-customisable diffusion model. ArXiv , 2023

    2023

  3. [11]

    Beyond deep reinforcement learning: A tutorial on generative diffusion models in network optimization

    Hongyang Du, Ruichen Zhang, and et al. Beyond deep reinforcement learning: A tutorial on generative diffusion models in network optimization. ArXiv , 2023

    2023

  4. [12]

    Reinforcement learning for fine-tuning text-to-image diffusion models

    Ying Fan, Olivia Watkins, and et al. Reinforcement learning for fine-tuning text-to-image diffusion models. Neurips , 2024

    2024

  5. [13]

    A survey of world models for autonomous driving

    Tuo Feng, Wenguan Wang, and et al. A survey of world models for autonomous driving. ArXiv , 2025

    2025

  6. [14]

    Can pre-trained text-to-image models generate visual goals for reinforcement learning? Neurips , 2023

    Jialu Gao and et al. Can pre-trained text-to-image models generate visual goals for reinforcement learning? Neurips , 2023

    2023

  7. [15]

    World models for autonomous driving: An initial survey

    Yanchen Guan and et al. World models for autonomous driving: An initial survey. T-IV , 2024

    2024

  8. [16]

    Versat2i: Improving text-to-image models with versatile reward

    Jianshu Guo, Wenhao Chai, and et al. Versat2i: Improving text-to-image models with versatile reward. ArXiv , 2024

    2024

  9. [17]

    Advancing text-driven chest x-ray generation with policy-based reinforcement learning

    Woojung Han, Chanyoung Kim, and et al. Advancing text-driven chest x-ray generation with policy-based reinforcement learning. MICCAI , 2024

    2024

  10. [18]

    Learning profitable nft image diffusions via multiple visual-policy guided reinforcement learning

    Huiguo He, Tianfu Wang, and et al. Learning profitable nft image diffusions via multiple visual-policy guided reinforcement learning. ACMMM , 2023

    2023

  11. [19]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and et al. Denoising diffusion probabilistic models. Neurips , 2020

    2020

  12. [20]

    Diffusion model-based image editing: A survey

    Yi Huang, Jiancheng Huang, and et al. Diffusion model-based image editing: A survey. Arxiv , 2024

    2024

  13. [21]

    Gen-drive: Enhancing diffusion generative driving policies with reward modeling and reinforcement learning fine-tuning

    Zhiyu Huang, Xinshuo Weng, and et al. Gen-drive: Enhancing diffusion generative driving policies with reward modeling and reinforcement learning fine-tuning. ArXiv , 2024

    2024

  14. [22]

    Planning with diffusion for flexible behavior synthesis

    Michael Janner, Yilun Du, and et al. Planning with diffusion for flexible behavior synthesis. ArXiv , 2022

    2022

  15. [23]

    A theory of continuous generative flow networks

    Salem Lahlou, Tristan Deleu, and et al. A theory of continuous generative flow networks. ICML , 2023

    2023

  16. [24]

    Aligning text-to-image models using human feedback

    Kimin Lee, Hao Liu, and et al. Aligning text-to-image models using human feedback. ArXiv , 2023

    2023

  17. [25]

    Parrot: Pareto-optimal multi-reward reinforcement learning framework for text-to-image generation

    Seung Hyun Lee, Yinxiao Li, and et al. Parrot: Pareto-optimal multi-reward reinforcement learning framework for text-to-image generation. ECCV , 2025

    2025

  18. [26]

    Drivingdiffusion: Layout-guided multi-view driving scene video generation with latent diffusion model

    Xiaofan Li, Yifu Zhang, and et al. Drivingdiffusion: Layout-guided multi-view driving scene video generation with latent diffusion model. ArXiv , 2023

    2023

  19. [27]

    Aligning diffusion models by optimizing human utility

    Shufan Li, Konstantinos Kallidromitis, and et al. Aligning diffusion models by optimizing human utility. ArXiv , 2024

    2024

  20. [28]

    Instructrl4pix: Training diffusion for image editing by reinforcement learning

    Tiancheng Li, Jinxiu Liu, and et al. Instructrl4pix: Training diffusion for image editing by reinforcement learning. ArXiv , 2024

    2024

  21. [29]

    Textcraftor: Your text encoder can be image quality controller

    Yanyu Li, Xian Liu, and et al. Textcraftor: Your text encoder can be image quality controller. CVPR , 2024

    2024

  22. [30]

    Rich human feedback for text-to-image generation

    Youwei Liang, Junfeng He, and et al. Rich human feedback for text-to-image generation. CVPR , 2024

    2024

  23. [31]

    Diffusiondrive: Truncated diffusion model for end-to-end autonomous driving

    Bencheng Liao, Shaoyu Chen, and et al. Diffusiondrive: Truncated diffusion model for end-to-end autonomous driving. ArXiv , 2024

    2024

  24. [32]

    Magic3d: High-resolution text-to-3d content creation

    Chen-Hsuan Lin, Jun Gao, and et al. Magic3d: High-resolution text-to-3d content creation. CVPR , 2023

    2023

  25. [33]

    Ddm-lag: A diffusion-based decision-making model for autonomous vehicles with lagrangian safety enhancement

    Jiaqi Liu, Peng Hang, and et al. Ddm-lag: A diffusion-based decision-making model for autonomous vehicles with lagrangian safety enhancement. ArXiv , 2024

    2024

  26. [34]

    Rdt-1b: a diffusion foundation model for bimanual manipulation

    Songming Liu, Lingxuan Wu, and et al. Rdt-1b: a diffusion foundation model for bimanual manipulation. ArXiv , 2024

    2024

  27. [35]

    Improving text-to-image consistency via automatic prompt optimization

    Oscar Ma \ n as and et al. Improving text-to-image consistency via automatic prompt optimization. ArXiv , 2024

    2024

  28. [36]

    Promptable game models: Text-guided game simulation via masked diffusion models

    Willi Menapace, Aliaksandr Siarohin, and et al. Promptable game models: Text-guided game simulation via masked diffusion models. TOG , 2024

    2024

  29. [37]

    Training diffusion models towards diverse image generation with rl

    Zichen Miao and et al. Training diffusion models towards diverse image generation with rl. CVPR , 2024

    2024

  30. [38]

    Dynamic prompt optimizing for text-to-image generation

    Wenyi Mo, Tianyu Zhang, and et al. Dynamic prompt optimizing for text-to-image generation. CVPR , 2024

    2024

  31. [39]

    Rl for consistency models: Reward guided text-to-image generation with fast inference

    Owen Oertell, Jonathan Daniel Chang, and et al. Rl for consistency models: Reward guided text-to-image generation with fast inference. RLC , 2024

    2024

  32. [40]

    Optimizing negative prompts for enhanced aesthetics and fidelity in text-to-image generation

    Michael Ogezi and Ning Shi. Optimizing negative prompts for enhanced aesthetics and fidelity in text-to-image generation. ArXiv , 2024

    2024

  33. [41]

    Aligning text-to-image diffusion models with reward backpropagation, 2024

    Mihir Prabhudesai and et al. Aligning text-to-image diffusion models with reward backpropagation, 2024

    2024

  34. [42]

    Utilizing generative ai for vr exploration testing: A case study

    Xue Qin and Garrett Weaver. Utilizing generative ai for vr exploration testing: A case study. ASEW , 2024

    2024

  35. [43]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, and et al. Direct preference optimization: Your language model is secretly a reward model. Neurips , 2024

    2024

  36. [44]

    Ambiguous medical image segmentation using diffusion models

    Aimon Rahman, Jeya Maria Jose Valanarasu, and et al. Ambiguous medical image segmentation using diffusion models. CVPR , 2023

    2023

  37. [45]

    Hierarchical text-conditional image generation with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. ArXiv , 2022

    2022

  38. [46]

    Refining alignment framework for diffusion models with intermediate-step preference ranking

    Jie Ren, Yuhang Zhang, and et al. Refining alignment framework for diffusion models with intermediate-step preference ranking. ArXiv , 2025

    2025

  39. [47]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, and et al. Photorealistic text-to-image diffusion models with deep language understanding. Neurips , 2022

    2022

  40. [48]

    High-dimensional continuous control using generalized advantage estimation

    John Schulman and et al. High-dimensional continuous control using generalized advantage estimation. ArXiv , 2015

    2015

  41. [49]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, and et al. Proximal policy optimization algorithms. ArXiv , 2017

    2017

  42. [50]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and et al. Denoising diffusion implicit models. ArXiv , 2020

    2020

  43. [51]

    Fine-tuning of continuous-time diffusion models as entropy-regularized control

    Masatoshi Uehara, Yulai Zhao, and et al. Fine-tuning of continuous-time diffusion models as entropy-regularized control. ArXiv , 2024

    2024

  44. [52]

    Understanding reinforcement learning-based fine-tuning of diffusion models: A tutorial and review

    Masatoshi Uehara, Yulai Zhao, and et al. Understanding reinforcement learning-based fine-tuning of diffusion models: A tutorial and review. Arxiv , 2024

    2024

  45. [53]

    Diffusion model alignment using direct preference optimization

    Bram Wallace and et al. Diffusion model alignment using direct preference optimization. CVPR , 2024

    2024

  46. [54]

    Diffusebot: Breeding soft robots with physics-augmented generative diffusion models

    Tsun-Hsuan Johnson Wang, Juntian Zheng, and et al. Diffusebot: Breeding soft robots with physics-augmented generative diffusion models. Neurips , 2023

    2023

  47. [55]

    Diffchat: Learning to chat with text-to-image synthesis models for interactive image creation

    Jiapeng Wang, Chengyu Wang, and et al. Diffchat: Learning to chat with text-to-image synthesis models for interactive image creation. ArXiv , 2024

    2024

  48. [56]

    Sparse diffusion policy: A sparse, reusable, and flexible policy for robot learning

    Yixiao Wang, Yifei Zhang, and et al. Sparse diffusion policy: A sparse, reusable, and flexible policy for robot learning. ArXiv , 2024

    2024

  49. [57]

    Panacea: Panoramic and controllable video generation for autonomous driving

    Yuqing Wen and et al. Panacea: Panoramic and controllable video generation for autonomous driving. CVPR , 2024

    2024

  50. [58]

    Improving compositional text-to-image generation with large vision-language models

    Song Wen, Guian Fang, and et al. Improving compositional text-to-image generation with large vision-language models. ArXiv , 2023

    2023

  51. [59]

    Preference tuning with human feedback on language, speech, and vision tasks: A survey

    Genta Indra Winata, Hanyang Zhao, and et al. Preference tuning with human feedback on language, speech, and vision tasks: A survey. ArXiv , 2024

    2024

  52. [60]

    Medsegdiff: Medical image segmentation with diffusion probabilistic model

    Junde Wu and et al. Medsegdiff: Medical image segmentation with diffusion probabilistic model. ArXiv , 2022

    2022

  53. [61]

    Medsegdiff-v2: Diffusion-based medical image segmentation with transformer

    Junde Wu and et al. Medsegdiff-v2: Diffusion-based medical image segmentation with transformer. AAAI , 2024

    2024

  54. [62]

    Human preference score: Better aligning text-to-image models with human preference

    Xiaoshi Wu, Keqiang Sun, and et al. Human preference score: Better aligning text-to-image models with human preference. ICCV , 2023

    2023

  55. [63]

    Protein structure generation via folding diffusion

    Kevin E Wu, Kevin K Yang, and et al. Protein structure generation via folding diffusion. Nature communications , 2024

    2024

  56. [64]

    Deep reward supervisions for tuning text-to-image diffusion models

    Xiaoshi Wu, Yiming Hao, and et al. Deep reward supervisions for tuning text-to-image diffusion models. ECCV , 2025

    2025

  57. [65]

    Measurement-conditioned denoising diffusion probabilistic model for under-sampled medical image reconstruction

    Yutong Xie and Quanzheng Li. Measurement-conditioned denoising diffusion probabilistic model for under-sampled medical image reconstruction. MICCAI , 2022

    2022

  58. [66]

    Imagereward: Learning and evaluating human preferences for text-to-image generation

    Jiazheng Xu, Xiao Liu, and et al. Imagereward: Learning and evaluating human preferences for text-to-image generation. Neurips , 2024

    2024

  59. [67]

    Using human feedback to fine-tune diffusion models without any reward model

    Kai Yang and et al. Using human feedback to fine-tune diffusion models without any reward model. CVPR , 2024

    2024

  60. [68]

    Learning interactive real-world simulators

    Mengjiao Yang, Yilun Du, and et al. Learning interactive real-world simulators. ArXiv , 2023

    2023

  61. [69]

    Mastering text-to-image diffusion: Recaptioning, planning, and generating with multimodal llms

    Ling Yang, Zhaochen Yu, and et al. Mastering text-to-image diffusion: Recaptioning, planning, and generating with multimodal llms. ICML , 2024

    2024

  62. [70]

    A dense reward view on aligning text-to-image diffusion with preference

    Shentao Yang, Tianqi Chen, and Mingyuan Zhou. A dense reward view on aligning text-to-image diffusion with preference. ArXiv , 2024

    2024

  63. [71]

    Ipo: Iterative preference optimization for text-to-video generation

    Xiaomeng Yang, Zhiyu Tan, and et al. Ipo: Iterative preference optimization for text-to-video generation. ArXiv , 2025

    2025

  64. [72]

    Regularized conditional diffusion model for multi-task preference alignment

    Xudong Yu, Chenjia Bai, and et al. Regularized conditional diffusion model for multi-task preference alignment. ArXiv , 2024

    2024

  65. [73]

    Self-play fine-tuning of diffusion models for text-to-image generation

    Huizhuo Yuan, Zixiang Chen, and et al. Self-play fine-tuning of diffusion models for text-to-image generation. ArXiv , 2024

    2024

  66. [74]

    Preference aligned diffusion planner for quadrupedal locomotion control

    Xinyi Yuan, Zhiwei Shang, and et al. Preference aligned diffusion planner for quadrupedal locomotion control. ArXiv , 2024

    2024

  67. [75]

    Unifying generative models with gflownets and beyond

    Dinghuai Zhang, Ricky TQ Chen, and et al. Unifying generative models with gflownets and beyond. ArXiv , 2022

    2022

  68. [76]

    Text-to-image diffusion models in generative AI: A survey

    Chenshuang Zhang, Chaoning Zhang, and et al. Text-to-image diffusion models in generative AI: A survey. Arxiv , 2023

    2023

  69. [77]

    Improving gflownets for text-to-image diffusion alignment

    Dinghuai Zhang, Yizhe Zhang, and et al. Improving gflownets for text-to-image diffusion alignment. ArXiv , 2024

    2024

  70. [78]

    Onlinevpo: Align video diffusion model with online video-centric preference optimization

    Jiacheng Zhang, Jie Wu, and et al. Onlinevpo: Align video diffusion model with online video-centric preference optimization. ArXiv , 2024

    2024

  71. [79]

    Hive: Harnessing human feedback for instructional visual editing

    Shu Zhang, Xinyi Yang, and et al. Hive: Harnessing human feedback for instructional visual editing. CVPR , 2024

    2024

  72. [80]

    Large-scale reinforcement learning for diffusion models

    Yinan Zhang, Eric Tzeng, and et al. Large-scale reinforcement learning for diffusion models. ECCV , 2025

    2025

  73. [81]

    A survey on generative ai and llm for video generation, understanding, and streaming

    Pengyuan Zhou, Lin Wang, and et al. A survey on generative ai and llm for video generation, understanding, and streaming. arXiv , 2024

    2024

  74. [82]

    write newline

    " write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence '...

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.