Pith. sign in

REVIEW 3 major objections 7 minor 71 references

Towards Self-Improvement of Diffusion Models via Group Preference Optimization

T0 review · 3 major / 7 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that Group Preference Optimization, which trains on a model's own generated images with reward-standardized group scores, lifts Stable Diffusion 3.5 Medium's accurate counting and text rendering by roughly 20 percentage…

desk verdict A plausible self-improvement method with good ablations, but the headline gains are measured with the same evaluator family used as reward and the Appendix A.1 derivation is wrong; needs independent evaluation before the 20-point claim is taken seriously. read the letter →

arxiv 2505.11070 v1 pith:6MR7RJEM submitted 2025-05-16 cs.CV cs.AI

Renjie Chen , Wenfeng Lin , Yichen Zhang , Jiangchuan Wei , Boyuan Liu , Chao Feng , Jiao Ran , Mingyu Guo This is my paper · ORCID
classification cs.CVcs.AI
keywords grouppreferenceoptimizationdirecttext-to-imagediffusionmodelsself-improvementrewardstandardizationaccuratecountingtextrenderingonlineself-training
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Pairwise direct preference optimization (DPO) for text-to-image diffusion models is fragile: when the two images in a pair are nearly equal in quality, the ranking signal is noise, and training on such pairs can hurt. This paper argues that replacing pairwise comparisons with groupwise ones, and reweighting each image by its standardized reward score within the group, removes that fragility without any data filtering. The resulting method, Group Preference Optimization (GPO), uses the model's own online-generated images as training data and an off-the-shelf vision model (YOLO for counting, OCR for text) as the reward source. The central empirical claim is that this self-improvement loop raises Stable Diffusion 3.5 Medium's counting accuracy from 41.8% to 61.1% and its text-rendering IoU from 0.258 to 0.485, with no extra cost at inference. If true, targeted capabilities of existing T2I models can be improved without human preference annotations.

What carries the argument

The carrying object is the GPO objective and its standardization coefficient. Starting from pairwise DPO, the paper derives a groupwise loss over all $\binom{G}{2}$ pairs that collapses to $\sum_{i=0}^{G-1}(G-1-2i)\,s(x_i,t,\epsilon)$, reducing computation from $O(G^2)$ to $O(G)$; replacing the linear rank weights with $A_i=(r_i-\bar r)/\sigma_r$ makes the preference signal scale-free and stabilizes training. The coefficient is the entire mechanism: it marks images above the group mean as wins and below as losses, with magnitude proportional to how far they stand out, and it is what lets the method work on self-generated data without pair filtering.

What would settle it

A decisive test is to run GPO on a task where the base model's best-of-32 oracle accuracy is exactly zero (no generated image satisfies the reward). The method should show no improvement, whereas on a task with nonzero best-of-32 accuracy it should improve; observing improvement in the zero-capability case, or no improvement in the nonzero case, would refute the claim.

Watch

Extended reading notes

Core claim

GPO's discovery is that preference margin, not just preference order, is the key signal DPO discards. The paper shows experimentally that training DPO on only the largest-margin pairs converges faster and reaches higher final reward than training on all pairs or on smallest-margin pairs, and that a groupwise loss with standardized scores outperforms even the max-margin selection. The objective is $L_{\mathrm{GPO}} = \mathbb{E}_{t,\epsilon} \sum_i A_i (\|\epsilon-\epsilon_\theta(x_i^t,t)\|_2^2 - \|\epsilon-\epsilon_{\mathrm{ref}}(x_i^t,t)\|_2^2)$ with $A_i=(r_i-\mathrm{mean}(r))/\mathrm{std}(r)$; this simultaneously provides relative preference within a group of $G$ images and normalizes gradient scale. Because the model generates its own training groups, the method requires no external preference data; the paper reports consistent gains across SD1.5, SDXL, SD3.5 Medium, and Wan2.1 on counting, text rendering, and compositional alignment, while noting that a base model that systematically fails a task (Wan text rendering) provides no useful self-improvement signal.

Load-bearing premise

The load-bearing premise is that the model being trained can already, by chance, generate at least some samples that score above average on the target reward; if its stochastic sampling never produces a 'good' image for a task, GPO has no positive signal to amplify, so self-improvement stalls.

Editorial extensions

If this is right

  • For a model that can already generate at least some correct images per prompt, GPO converts that latent ability into a consistent behavior: the paper reports Wan2.1 counting accuracy rising from 29.3% to 52.2% and SD3.5M counting from 41.8% to 61.1%.
  • Because the reward comes from off-the-shelf models, no human preference labels are needed to improve a specific capability; the same training loop could be pointed at any measurable objective.
  • GPO is plug-and-play for inference: all changes happen during training, so aligned models incur no extra latency or memory at generation time.
  • The reported limitation is direct: if the base model's self-generated samples never satisfy the reward, GPO has nothing to reinforce, as shown by Wan's weak text-rendering gains (IoU 0.024 to 0.050).
  • On generic human-preference metrics, GPO matches or exceeds pairwise DPO variants (Diff-DPO, SPO, LPO) on SD1.5 and SDXL, suggesting the groupwise signal generalizes beyond the specific tasks used to define rewards.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the standardization coefficient is a z-score over a group, GPO is a generic recipe: any stochastic generator with a scalar evaluator could use the same loop, including video diffusion or LLM sample generation with automated scoring.
  • The method's dependence on the evaluator's blind spots is untested: if YOLO misdetects a class, GPO may amplify images that fool the detector; a natural check is to train with one detector and evaluate with another.
  • The self-improvement framing suggests a bootstrapping schedule: a small supervised fine-tuning step to make a failing ability occasionally succeed, followed by GPO, could extend the method to capabilities the base model lacks entirely; the paper names this direction but does not test it.
  • One could test the margin hypothesis directly in other alignment settings: filter an existing pairwise preference dataset by margin size and measure whether DPO's performance tracks the filter threshold, as the paper's Figure 2 suggests for ImageReward.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper argues that pairwise DPO for text-to-image diffusion models is sensitive to the margin between preferred and dispreferred samples, and proposes Group Preference Optimization (GPO), which (i) generalizes DPO to groups of G images by summing pairwise losses and then replacing the group ranking coefficients with standardized reward scores, and (ii) trains on images generated online by the model itself, scored by external reward models such as YOLO, PPOCR, and BLIP-VQA. The reported results show large gains in object counting and text rendering for SD3.5M (e.g., +19.3 counting accuracy and +0.227 text IoU in Table 2), smaller gains for Wan2.1-1.3B counting, and modest or mixed gains on T2I-CompBench++ and DPG-Bench. The paper also reports that Wan text rendering is not improved.

Significance. If the headline numbers survive independent evaluation, GPO would be a valuable practical contribution: it adds no inference-time overhead, does not require human preference annotation for the target capability, and the ablations in Figure 6 support the core design choices of group size, standardized reweighting, and online data. The paper covers four diverse model architectures and reports detailed training settings. However, the mathematical derivation of the groupwise form (Appendix A.1) is incorrect, and the evaluation of the two headline tasks uses the same detector/OCR families as the training rewards. These issues currently prevent the central claims from being accepted as stated.

major comments (3)
  1. [Appendix A.1, Eq. (6)] The claimed equivalence between the groupwise DPO sum and the linear weighting is mathematically incorrect. Since -log sigma(-beta Delta) = log(1 + e^{beta Delta}), this term is not proportional to Delta, and monotonicity of log sigma does not justify the linear reduction. Consequently, the GPO objective in Eq. (5), which replaces the coefficients with standardized rewards, is not a DPO-derived loss but a new, ad hoc reward-weighted objective. This invalidates the theoretical motivation in Section 4.2 and needs to be fixed, either by a correct derivation or by explicitly presenting GPO as an empirically motivated objective.
  2. [Section 5.1, Table 2] The headline improvements are measured with the same evaluator families used to compute the training rewards: YOLO-family detectors for counting (training on YOLO N/S/L, evaluation on YOLO-X) and PPOCR for text rendering (training and evaluation). A scale change within one detector family is not an independent test. The paper should provide cross-family evaluations (e.g., DETR or Faster R-CNN for counting; a different OCR engine or human reading tests for text) on the held-out prompts; otherwise the 20-percentage-point claim remains consistent with reward overfitting rather than with improved capability.
  3. [Algorithm 1, line 10] Updating the reference model epsilon_ref <- epsilon_theta at each iteration is nonstandard for DPO, whose derivation assumes a fixed reference policy, and it turns the objective in Eq. (5) into a moving-baseline regularizer rather than a preference optimization step. The paper neither motivates nor ablates this choice; the authors should clarify whether this is intentional and how it affects the preference interpretation of the loss.
minor comments (7)
  1. [Appendix A title] The appendix title contains a typo: 'Group Preferecne Optimization' should read 'Group Preference Optimization'.
  2. [Figures 8 and 9 captions] The captions in Appendix C say 'SD3.5M+DPO' but the method presented is GPO; the captions should say '+GPO'.
  3. [Section 5.3 and Section 5.1 headings/text] Section 5.3 heading has 'Comparsion' (should be 'Comparison') and Section 5.1 text has 'acurate' (should be 'accurate').
  4. [Eq. (4)] The symbol epsilon is used both for the noise variable and for the denoising network (epsilon_theta); using a distinct symbol for the noise, such as z or n, would improve clarity.
  5. [Section 4.2] The statement that 'the mean of the group coefficient is zero, and the variance is fixed' applies to the ranking coefficients (G-1-2i), but the standardized rewards A_i are random variables; the text should clarify which object the statement refers to.
  6. [Table 3] Several cells show decreases (e.g., SD3.5M 2D-Spatial -0.41, SDXL DPG Overall +0.54, SD3.5M DPG Attribute -0.25); the current text attributes these to benchmark properties, but a short discussion of the patterns would help readers assess the alignment results.
  7. [Section 5.1 and Section 5.5] The Wan text-rendering result (IoU 0.024 to 0.050) is a clear boundary condition for the self-improvement framework; stating this limitation earlier and perhaps providing a criterion for when self-generated data gives a usable signal would strengthen the paper.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: GPO is a reward-weighted preference optimization with external rewards; the headline counting/text metrics overlap with the training evaluators, a validity caveat rather than a by-construction derivation.

full rationale

GPO's training signal is an external reward (YOLO, PPOCR, BLIP-VQA, ImageReward/MPS) applied to online self-generated images; the GPO loss (Eq. 5) reweights the denoising objective by standardized rewards. This is a legitimate reward-weighted preference update, not a derivation whose conclusion is assumed. The central self-improvement premise is explicitly conditional and empirically falsifiable: the paper reports Wan text rendering barely improves (IoU 0.024 to 0.050), which would not happen if the result were forced by construction. The only self-referential aspect is that the headline counting/text metrics in Table 2 are computed with the same evaluator families (YOLO, PPOCR) used to define the rewards in Table 1; switching YOLO-N/S/L to YOLO-X is not an independent evaluation. This is a reward-hacking/validity concern, not circularity of the derivation. The aesthetic-preference results (Sec. 5.3) train on MPS and evaluate on ImageReward, PickScore, HPS, and Aesthetic, which are not the training reward, providing an external check. No load-bearing self-citations or imported uniqueness theorems appear. The derivation is therefore self-contained and non-circular, with only a minor evaluator-overlap caveat.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests on a small number of unverified but stated premises: the base model can generate at least some reward-positive samples, the automatic evaluators provide trustworthy scores, and the DPO and ELBO framework imported from prior work is valid. The GPO objective itself introduces no learned free parameters beyond standard hyperparameters; the listed hyperparameters are hand-chosen and affect the reported results.

free parameters (5)
  • group size G = 32
    Chosen by hand as a trade-off; ablations in Fig. 6a show larger groups improve performance, so the default affects final results.
  • k (timesteps sampled per update) = 5
    Empirically set without extensive tuning; controls data utilization and gradient update frequency.
  • tau (inner update iterations) = 3
    Empirically set; trades off training cost against utilization of generated data.
  • learning rate = 2e-8 (SD1.5/SDXL), 4e-8 (SD3.5M/Wan)
    Tuned iteratively; authors report 1e-5 caused overfitting and collapse, so the small learning rate is load-bearing for stability.
  • training epochs = 2
    Set empirically to balance performance improvement and overfitting prevention.
assumptions (4)
  • domain assumption The base model inherently can generate reward-positive samples for the target ability, though unstably.
    Stated in Section 4.2 citing references [56, 2, 46]; the self-improvement loop depends on it. Section 5.1's failure to improve Wan text rendering confirms the boundary.
  • domain assumption Automatic evaluator scores (YOLO, PPOCR, BLIP-VQA, ImageReward) are reliable proxies for the desired capability and provide a usable learning signal.
    Used in Sections 4.3 and 5; if the evaluator is noisy or gameable, GPO optimizes the wrong objective.
  • standard math The Diff-DPO objective, Eq. (4), is a valid ELBO simplification of the DPO and RLHF objectives in Eq. (3).
    Imported from prior work [44] without proof; the GPO objective is built on the same s(x, t, epsilon) terms.
  • domain assumption The group mean reward partitions samples into winning and losing subsets, and standardization gives stable gradient updates.
    Section 4.2; the ablation in Fig. 6b supports it empirically, but it is assumed for all tasks.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards Self-Improvement of Diffusion Models via Group Preference Optimization." pith.science (2026). https://pith.science/paper/6MR7RJEM

@misc{pith2026250511070,
  author       = {Pith},
  title        = {Pith review of: Towards Self-Improvement of Diffusion Models via Group Preference Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6MR7RJEM}},
  note         = {Machine review of arXiv:2505.11070}
}
read the original abstract

Aligning text-to-image (T2I) diffusion models with Direct Preference Optimization (DPO) has shown notable improvements in generation quality. However, applying DPO to T2I faces two challenges: the sensitivity of DPO to preference pairs and the labor-intensive process of collecting and annotating high-quality data. In this work, we demonstrate that preference pairs with marginal differences can degrade DPO performance. Since DPO relies exclusively on relative ranking while disregarding the absolute difference of pairs, it may misclassify losing samples as wins, or vice versa. We empirically show that extending the DPO from pairwise to groupwise and incorporating reward standardization for reweighting leads to performance gains without explicit data selection. Furthermore, we propose Group Preference Optimization (GPO), an effective self-improvement method that enhances performance by leveraging the model's own capabilities without requiring external data. Extensive experiments demonstrate that GPO is effective across various diffusion models and tasks. Specifically, combining with widely used computer vision models, such as YOLO and OCR, the GPO improves the accurate counting and text rendering capabilities of the Stable Diffusion 3.5 Medium by 20 percentage points. Notably, as a plug-and-play method, no extra overhead is introduced during inference.

Figures

Figures reproduced from arXiv: 2505.11070 by the authors.

Figure 1
Figure 1. Overview of Group Preference Optimization. Combined with YOLO v11, our approach [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Pair Margin Influence Problem Hypothesis. Suppose we have a reward model R whose output scores align with human preferences, that is, whenever R(x) > R(y), humans prefer x over y. While DPO training uses preference pairs (x w, xl ) that only provide ordinal information (R(x w) > R(x l )), it discards the pair margin ∆(x w, xl ) = |R(x w) − R(x l )|. We hypothesize that ignoring this pair margin ∆ leads to suboptimal… view at source ↗
Figure 3
Figure 3. GPO Visualization. Prompt: There are three adorable puppies playfully running across a lush, sunlit green meadow, their fur glistening in the warm sunlight Qualitative result. Through an empirical analysis of samples generated during GPO training, we demonstrate its effectiveness. As shown in [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Qualitative comparisons between SD3.5M and SD3.5M+GPO. All pairs are generated with [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Qualitative comparisons between SD3.5M and SD3.5M+GPO on text-image alignment. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Alblation Studies of GPO. All experiments are performed on the accurate counting task of [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Performance metric of YOLO v11. B.3 Hyperparameter Choosen Group Size. As discussed earlier, we default to using a group size of 32, which achieves a better trade off in terms of performance improvement and training time. Learning Rate. In our initial verification expe…
Figure 8
Figure 8. Figure 8: More Comparisons between SD3.5M and SD3.5M+DPO on accurate counting task. All [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]
Figure 9
Figure 9. Figure 9: More Comparisons between SD3.5M and SD3.5M+DPO on text rendering task. All pairs [PITH_FULL_IMAGE:figures/full_fig_p019_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

71 extracted references · 25 canonical work pages

  1. [20]

    Scalable ranked preference optimization for text-to-image generation.arXiv preprint arXiv:2410.18013, 2024

    Shyamgopal Karthik, Huseyin Coskun, Zeynep Akata, Sergey Tulyakov, Jian Ren, and Anil Kag. Scalable ranked preference optimization for text-to-image generation.arXiv preprint arXiv:2410.18013, 2024

    arXiv 2024

  2. [1]

    Back to basics: Revisiting reinforce style optimization for learning from human feedback in llms.arXiv preprint arXiv:2402.14740, 2024

    Arash Ahmadian, Chris Cremer, Matthias Gallé, Marzieh Fadaee, Julia Kreutzer, Olivier Pietquin, Ahmet Üstün, and Sara Hooker. Back to basics: Revisiting reinforce style optimization for learning from human feedback in llms.arXiv preprint arXiv:2402.14740, 2024

    arXiv 2024

  3. [2]

    A noise is worth diffusion guidance

    Donghoon Ahn, Jiwon Kang, Sanghyun Lee, Jaewon Min, Minjae Kim, Wooseok Jang, Hyoung- won Cho, Sayak Paul, SeonHwa Kim, Eunju Cha, et al. A noise is worth diffusion guidance. arXiv preprint arXiv:2412.03895, 2024

    arXiv 2024

  4. [3]

    Training a helpful and harmless assistant with reinforcement learning from human feedback.arXiv preprint arXiv:2204.05862, 2022

    Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Training a helpful and harmless assistant with reinforcement learning from human feedback.arXiv preprint arXiv:2204.05862, 2022

    arXiv 2022

  5. [4]

    Improving image generation with better captions

    James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al. Improving image generation with better captions. Computer Science. https://cdn. openai. com/papers/dall-e-3. pdf, 2(3):8, 2023

    2023

  6. [5]

    Make it count: Text-to-image generation with an accurate number of objects.arXiv preprint arXiv:2406.10210, 2024

    Lital Binyamin, Yoad Tewel, Hilit Segev, Eran Hirsch, Royi Rassin, and Gal Chechik. Make it count: Text-to-image generation with an accurate number of objects.arXiv preprint arXiv:2406.10210, 2024

    arXiv 2024

  7. [6]

    Training diffusion models with reinforcement learning

    Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine. Training diffusion models with reinforcement learning. InThe Twelfth International Conference on Learning Representations, 2023

    work page 2023

  8. [7]

    Text-to-image diffusion models cannot count, and prompt refinement cannot help.arXiv preprint arXiv:2503.06884, 2025

    Yuefan Cao, Xuyang Guo, Jiayan Huo, Yingyu Liang, Zhenmei Shi, Zhao Song, Jiahao Zhang, and Zhen Zhuang. Text-to-image diffusion models cannot count, and prompt refinement cannot help.arXiv preprint arXiv:2503.06884, 2025

    arXiv 2025

Show all 71 references
  1. [8]

    Getting it right: Improving spatial consistency in text-to-image models

    Agneet Chatterjee, Gabriela Ben Melech Stan, Estelle Aflalo, Sayak Paul, Dhruba Ghosh, Tejas Gokhale, Ludwig Schmidt, Hannaneh Hajishirzi, Vasudev Lal, Chitta Baral, et al. Getting it right: Improving spatial consistency in text-to-image models. InEuropean Conference on Comput...

    2024

  2. [9]

    Textdiffuser: Diffusion models as text painters.Advances in Neural Information Processing Systems, 36: 9353–9387, 2023

    Jingye Chen, Yupan Huang, Tengchao Lv, Lei Cui, Qifeng Chen, and Furu Wei. Textdiffuser: Diffusion models as text painters.Advances in Neural Information Processing Systems, 36: 9353–9387, 2023

    2023

  3. [10]

    Deep reinforcement learning from human preferences.Advances in neural information processing systems, 30, 2017

    Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences.Advances in neural information processing systems, 30, 2017

    2017

  4. [11]

    Directly fine-tuning diffusion models on differentiable rewards

    Kevin Clark, Paul Vicol, Kevin Swersky, and David J Fleet. Directly fine-tuning diffusion models on differentiable rewards. InThe Twelfth International Conference on Learning Representations, 2023

    2023

  5. [12]

    Less is more: Improving llm alignment via preference data selection.arXiv preprint arXiv:2502.14560, 2025

    Xun Deng, Han Zhong, Rui Ai, Fuli Feng, Zheng Wang, and Xiangnan He. Less is more: Improving llm alignment via preference data selection.arXiv preprint arXiv:2502.14560, 2025

    2025

  6. [13]

    Scaling rectified flow trans- formers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow trans- formers for high-resolution image synthesis. InForty-first international conference on machine...

    2024

  7. [14]

    Kto: Model alignment as prospect theoretic optimization.arXiv preprint arXiv:2402.01306, 2024

    Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Jurafsky, and Douwe Kiela. Kto: Model alignment as prospect theoretic optimization.arXiv preprint arXiv:2402.01306, 2024

    2024 arXiv

  8. [15]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Proceedings of the 34th International Conference on Neural Information Processing Systems, pages 6840–6851, 2020. 11

    2020

  9. [16]

    Reference-free monolithic preference optimization with odds ratio.arXiv e-prints, pages arXiv–2403, 2024

    Jiwoo Hong, Noah Lee, and James Thorne. Reference-free monolithic preference optimization with odds ratio.arXiv e-prints, pages arXiv–2403, 2024

    2024

  10. [17]

    Ella: Equip diffusion models with llm for enhanced semantic alignment.arXiv preprint arXiv:2403.05135, 2024

    Xiwei Hu, Rui Wang, Yixiao Fang, Bin Fu, Pei Cheng, and Gang Yu. Ella: Equip diffusion models with llm for enhanced semantic alignment.arXiv preprint arXiv:2403.05135, 2024

    2024 arXiv

  11. [18]

    T2i-compbench: A compre- hensive benchmark for open-world compositional text-to-image generation.Advances in Neural Information Processing Systems, 36:78723–78747, 2023

    Kaiyi Huang, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. T2i-compbench: A compre- hensive benchmark for open-world compositional text-to-image generation.Advances in Neural Information Processing Systems, 36:78723–78747, 2023

    2023

  12. [19]

    Ultralytics YOLO, 2023

    Glenn Jocher, Jing Qiu, and Ayush Chaurasia. Ultralytics YOLO, 2023. URL https:// github.com/ultralytics/ultralytics

    2023

  13. [21]

    Pick-a-pic: An open dataset of user preferences for text-to-image generation.Advances in Neural Information Processing Systems, 36:36652–36663, 2023

    Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana, Joe Penna, and Omer Levy. Pick-a-pic: An open dataset of user preferences for text-to-image generation.Advances in Neural Information Processing Systems, 36:36652–36663, 2023

    2023

  14. [22]

    Flux.https://github.com/black-forest-labs/flux, 2024

    Black Forest Labs. Flux.https://github.com/black-forest-labs/flux, 2024

    2024

  15. [23]

    Aligning text-to-image models using human feedback.arXiv preprint arXiv:2302.12192, 2023

    Kimin Lee, Hao Liu, Moonkyung Ryu, Olivia Watkins, Yuqing Du, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, and Shixiang Shane Gu. Aligning text-to-image models using human feedback.arXiv preprint arXiv:2302.12192, 2023

    2023 arXiv

  16. [24]

    Calibrated multi-preference optimization for aligning diffusion models.arXiv preprint arXiv:2502.02588, 2025

    Kyungmin Lee, Xiaohang Li, Qifei Wang, Junfeng He, Junjie Ke, Ming-Hsuan Yang, Irfan Essa, Jinwoo Shin, Feng Yang, and Yinxiao Li. Calibrated multi-preference optimization for aligning diffusion models.arXiv preprint arXiv:2502.02588, 2025

    2025

  17. [25]

    Align- ing diffusion models by optimizing human utility

    Shufan Li, Konstantinos Kallidromitis, Akash Gokul, Yusuke Kato, and Kazuki Kozuka. Align- ing diffusion models by optimizing human utility. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    2024

  18. [26]

    Policy optimization in rlhf: The impact of out-of-preference data.arXiv preprint arXiv:2312.10584, 2023

    Ziniu Li, Tian Xu, and Yang Yu. Policy optimization in rlhf: The impact of out-of-preference data.arXiv preprint arXiv:2312.10584, 2023

    2023 arXiv

  19. [27]

    Aesthetic post-training diffusion models from generic preferences with step-by-step preference optimization, 2025

    Zhanhao Liang, Yuhui Yuan, Shuyang Gu, Bohan Chen, Tiankai Hang, Mingxi Cheng, Ji Li, and Liang Zheng. Aesthetic post-training diffusion models from generic preferences with step-by-step preference optimization, 2025. URLhttps://arxiv.org/abs/2406.04314

    2025 arXiv

  20. [28]

    Flow matching for generative modeling

    Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modeling. InThe Eleventh International Conference on Learning Representations, 2022

    2022

  21. [29]

    Alignment of diffusion models: Fundamentals, challenges, and future

    Buhua Liu, Shitong Shao, Bao Li, Lichen Bai, Zhiqiang Xu, Haoyi Xiong, James Kwok, Sumi Helal, and Zeke Xie. Alignment of diffusion models: Fundamentals, challenges, and future. arXiv preprint arXiv:2409.07253, 2024

    2024

  22. [30]

    Character-aware models improve visual text rendering.arXiv preprint arXiv:2212.10562, 2022

    Rosanne Liu, Dan Garrette, Chitwan Saharia, William Chan, Adam Roberts, Sharan Narang, Irina Blok, RJ Mical, Mohammad Norouzi, and Noah Constant. Character-aware models improve visual text rendering.arXiv preprint arXiv:2212.10562, 2022

    2022 arXiv

  23. [31]

    Videodpo: Omni-preference alignment for video diffusion generation.arXiv preprint arXiv:2412.14167, 2024

    Runtao Liu, Haoyu Wu, Zheng Ziqiang, Chen Wei, Yingqing He, Renjie Pi, and Qifeng Chen. Videodpo: Omni-preference alignment for video diffusion generation.arXiv preprint arXiv:2412.14167, 2024

    2024 arXiv

  24. [32]

    Flow straight and fast: Learning to generate and transfer data with rectified flow

    Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. InThe Eleventh International Conference on Learning Representations (ICLR), 2023

    2023

  25. [33]

    Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017. 12

    2017 arXiv

  26. [34]

    Exploring the role of large language models in prompt encoding for diffusion models

    Bingqi Ma, Zhuofan Zong, Guanglu Song, Hongsheng Li, and Yu Liu. Exploring the role of large language models in prompt encoding for diffusion models. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    2024

  27. [35]

    Simpo: Simple preference optimization with a reference-free reward.Advances in Neural Information Processing Systems, 37:124198–124235, 2024

    Yu Meng, Mengzhou Xia, and Danqi Chen. Simpo: Simple preference optimization with a reference-free reward.Advances in Neural Information Processing Systems, 37:124198–124235, 2024

    2024

  28. [36]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. InThe Twelfth International Conference on Learning Representations, 2023

    2023

  29. [37]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36:53728–53741, 2023

    2023

  30. [38]

    High- resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022

    2022

  31. [39]

    Laion-400m: Open dataset of clip-filtered 400 million image-text pairs.arXiv preprint arXiv:2111.02114, 2021

    Christoph Schuhmann, Richard Vencu, Romain Beaumont, Robert Kaczmarczyk, Clayton Mullis, Aarush Katta, Theo Coombes, Jenia Jitsev, and Aran Komatsuzaki. Laion-400m: Open dataset of clip-filtered 400 million image-text pairs.arXiv preprint arXiv:2111.02114, 2021

    2021 arXiv

  32. [40]

    Laion- 5b: An open large-scale dataset for training next generation image-text models.Advances in neural information processing systems, 35:25278–25294, 2022

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion- 5b: An open large-scale dataset for training next generation image-text models.Advances in neural informa...

    2022

  33. [41]

    Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347, 2017

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  34. [42]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    2024 arXiv

  35. [43]

    Anytext: Multilingual visual text generation and editing

    Yuxiang Tuo, Wangmeng Xiang, Jun-Yan He, Yifeng Geng, and Xuansong Xie. Anytext: Multilingual visual text generation and editing. InThe Twelfth International Conference on Learning Representations, 2023

    2023

  36. [44]

    Diffusion model alignment using direct preference optimization

    Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. Diffusion model alignment using direct preference optimization. InProceedings of the IEEE/CVF Conference on Computer Vision and ...

    2024

  37. [45]

    Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314, 2025

    Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, Jianyuan Zeng, et al. Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314, 2025

    2025 arXiv

  38. [46]

    The silent prompt: Initial noise as implicit guidance for goal-driven image generation.arXiv preprint arXiv:2412.05101, 2024

    Ruoyu Wang, Huayang Huang, Ye Zhu, Olga Russakovsky, and Yu Wu. The silent prompt: Initial noise as implicit guidance for goal-driven image generation.arXiv preprint arXiv:2412.05101, 2024

    2024 arXiv

  39. [47]

    Diffusiondb: A large-scale prompt gallery dataset for text-to-image genera- tive models.arXiv preprint arXiv:2210.14896, 2022

    Zijie J Wang, Evan Montoya, David Munechika, Haoyang Yang, Benjamin Hoover, and Duen Horng Chau. Diffusiondb: A large-scale prompt gallery dataset for text-to-image genera- tive models.arXiv preprint arXiv:2210.14896, 2022

    2022 arXiv

  40. [48]

    Human preference score: Better aligning text-to-image models with human preference

    Xiaoshi Wu, Keqiang Sun, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score: Better aligning text-to-image models with human preference. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 2096–2105, 2023

    2023

  41. [49]

    Deep reward supervisions for tuning text-to-image diffusion models

    Xiaoshi Wu, Yiming Hao, Manyuan Zhang, Keqiang Sun, Zhaoyang Huang, Guanglu Song, Yu Liu, and Hongsheng Li. Deep reward supervisions for tuning text-to-image diffusion models. InEuropean Conference on Computer Vision, pages 108–124. Springer, 2024. 13

    2024

  42. [50]

    Imagereward: Learning and evaluating human preferences for text-to-image generation

    Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagereward: Learning and evaluating human preferences for text-to-image generation. Advances in Neural Information Processing Systems, 36:15903–15935, 2023

    2023

  43. [51]

    Using human feedback to fine-tune diffusion models without any reward model

    Kai Yang, Jian Tao, Jiafei Lyu, Chunjiang Ge, Jiaxin Chen, Weihan Shen, Xiaolong Zhu, and Xiu Li. Using human feedback to fine-tune diffusion models without any reward model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8941–8951, 2024

    2024

  44. [52]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. InProceedings of the IEEE/CVF international conference on computer vision, pages 3836–3847, 2023

    2023

  45. [53]

    Learning multi-dimensional human preference for text-to-image generation

    Sixian Zhang, Bohan Wang, Junqiang Wu, Yan Li, Tingting Gao, Di Zhang, and Zhongyuan Wang. Learning multi-dimensional human preference for text-to-image generation. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8018–8027, 2024

    2024

  46. [54]

    Diffusion model as a noise-aware latent reward model for step-level preference optimization.arXiv preprint arXiv:2502.01051, 2025

    Tao Zhang, Cheng Da, Kun Ding, Kun Jin, Yan Li, Tingting Gao, Di Zhang, Shiming Xiang, and Chunhong Pan. Diffusion model as a noise-aware latent reward model for step-level preference optimization.arXiv preprint arXiv:2502.01051, 2025

    2025

  47. [55]

    Cogview3: Finer and faster text-to-image generation via relay diffusion

    Wendi Zheng, Jiayan Teng, Zhuoyi Yang, Weihan Wang, Jidong Chen, Xiaotao Gu, Yuxiao Dong, Ming Ding, and Jie Tang. Cogview3: Finer and faster text-to-image generation via relay diffusion. InEuropean Conference on Computer Vision, pages 1–22. Springer, 2024

    2024

  48. [56]

    Home Sweet Home

    Zikai Zhou, Shitong Shao, Lichen Bai, Zhiqiang Xu, Bo Han, and Zeke Xie. Golden noise for diffusion models: A learning framework.arXiv preprint arXiv:2411.09502, 2024. 14 A Group Preferecne Optimization A.1 Group DPO Objective Given a group of G images{xi}G−1 i=0 ranked by pre...

    2024 arXiv

  49. [57]

    Subject category (e.g., animal/person/scene/object)

  50. [58]

    Subject quantity (e.g., single/specific number/plural)

  51. [59]

    Optional details (style/action/environment etc.) [Output Requirements] Generate prompts with this structure:

  52. [60]

    Core subject: Precise noun phrase

  53. [62]

    Environment: Describe setting/lighting/weather

  54. [63]

    [Example Template] Input: 3 cat Output: Three cats curled up together on a sunny windowsill

    Art style: Specify photography/painting/digital art etc. [Example Template] Input: 3 cat Output: Three cats curled up together on a sunny windowsill. Input: 4 apple Output: A close-up of 4 fresh green apples with dewdrops, resting on a marble counter. Input: 1 dog, 2 cat Outpu...

  55. [67]

    Follow these guidelines: [Output Requirements] Generate prompts with this structure:

    Keep under 50 words Generate a prompt for this input: Input: <INPUTS> 20 Prompt for Text Render Dataset [System Instruction] You are a professional prompt engineer specialized in generating high-quality text-to-image captions. Follow these guidelines: [Output Requirements] Gen...

  56. [68]

    it muse contain text to render wrapped by ""

  57. [69]

    Visual details: Include color/material/texture

  58. [70]

    optional Environment: Describe setting/lighting/weather

  59. [71]

    [Optimization Principles]

    optional Art style: Specify photography/painting/digital art etc. [Optimization Principles]

  60. [72]

    Avoid abstract concepts - use concrete visual elements

  61. [73]

    Reduce redundant descriptions

  62. [74]

    Separate different dimensions with commas

  63. [75]

    The prompt start should various [Examples] <EXAMPLES> Generate 3 prompts without serial number 21

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.