Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

ImageReFL: Balancing Quality and Diversity in Human-Aligned Diffusion Models

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Reward-tuned diffusion models can keep image diversity when the base model handles early denoising steps and the tuned model only refines the final steps, trained on real images.

desk verdict The combined generation idea is simple and sensible, but the head-to-head evidence for ImageReFL rests on per-method test-set-selected handover points, so the main claim is not yet established. read the letter →

arxiv 2505.22569 v1 pith:V6MLAKHA submitted 2025-05-28 cs.CV

Dmitrii Sorokin , Maksim Nakhodnov , Andrey Kuznetsov , Aibek Alanov This is my paper · ORCID
classification cs.CV
keywords diffusionmodelstext-to-imagegenerationhumanpreferencealignmentrewardfine-tuningdiversitycollapsehackingcombinedImageReFL
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Text-to-image diffusion models can be aligned with human preferences through reward-based fine-tuning, but the standard ReFL recipe collapses output diversity and introduces artifacts. This paper tries to establish that the collapse is avoidable if the reward-tuned model is restricted to the final denoising steps while the original base model keeps control of the early steps, and if the tuned model is trained to refine real images rather than generated from scratch. The authors propose combined generation, a sampling-time switch between the two models, and ImageReFL, a training objective combining the reward loss with standard diffusion loss and periodic ReFL steps. The reported results, including a user study, indicate that ImageReFL keeps most of the quality gain of ReFL while restoring diversity and text alignment across SD1.5 and SDXL with HPSv2.1 and PickScore rewards.

What carries the argument

The machinery is a piecewise noise-prediction switch, $\epsilon_{\text{combined}}(x_t,t,c) = \epsilon_{\theta_0}(x_t,t,c)$ for $t>T'$ and $\epsilon_\theta(x_t,t,c)$ for $t \le T'$, which lets the original model set global structure before the reward-tuned model takes over. ImageReFL is the matching training objective: start from a noised real image $x_{t'} = \sqrt{\alpha_{t'}}x^{\text{real}}_0 + \sqrt{1-\alpha_{t'}}\epsilon$, denoise with the trainable model, and backpropagate the reward through the final step as $\mathcal{L}'_{\text{reward}}=-R(F_\theta(x_{t'},t',c),c)$, regularized by the standard diffusion loss $\|\epsilon-\epsilon_\theta(x_{t'},t',c)\|_2^2$ and by periodic full ReFL steps. ReFL is a reward fine-tuning scheme that backpropagates through only the final denoising step. The combination keeps early steps untouched and forces the fine-tuned model to make its quality changes late.

What would settle it

Generate the same prompt from the same initial noise with the base model alone and with combined generation switching at $T'$, then compare the final images' global structure, for example by DINO embedding distance; if composition or pose changes substantially, the early-step invariance premise fails.

Watch

Extended reading notes

Core claim

The central claim is a division of labor across the denoising trajectory: early steps decide global structure, so a reward-tuned model should only be trusted later. The paper implements this as combined generation, which applies the base model for steps $t > T'$ and the fine-tuned model for $t \le T'$, and as ImageReFL, which starts training from a noised real image $x_{t'}$ and backpropagates the reward through the final step while adding a diffusion-loss regularizer and periodic ReFL updates. On quality-diversity trade-off curves, ImageReFL sits at a more favorable point than ReFL and ReFL AIG, and the user study gives it higher diversity and alignment votes.

Load-bearing premise

The load-bearing premise is that global features such as composition and pose are fixed during early denoising steps and stay stable, so a model switched in later can refine local quality without disturbing them.

Editorial extensions

If this is right

  • Fine-tuned diffusion models can be deployed with a tunable quality-diversity knob: shifting the switch step $T'$ trades reward score against sample variety.
  • Training directly on real images with the diffusion-loss regularizer reduces reward hacking and lets the model make its main changes in the final denoising steps.
  • The two-model sampler works across base models (SD1.5, SDXL) and rewards (HPSv2.1, PickScore), so it is not tied to one architecture.
  • The regularization recipe can be grafted onto other reward-tuning methods such as DRTune and DraftK, as the paper suggests, potentially improving their diversity as well.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If early-step invariance is the mechanism, the optimal switch step $T'$ should depend on the prompt: scenes with complex geometry may need more late-stage refinement than simple ones, and the paper only sweeps a global $T'$.
  • A natural extension is to apply combined generation as a wrapper at inference time around any reward-tuned diffusion model, even one not trained with ImageReFL, since the paper's evidence suggests the switch alone restores much of the diversity.
  • The reported DinoDiversity metric captures intra-prompt variety only; a fuller test of the method would track cross-prompt and distribution-level diversity together, which FID and covariance distances partially cover.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes two mechanisms to mitigate diversity collapse in reward-fine-tuned text-to-image diffusion models: (i) combined generation, which runs the base model for early denoising steps and switches to the reward-tuned model for later steps; and (ii) ImageReFL, a training procedure that starts from noised real images, applies the reward loss only through the last denoising step, and regularizes with the standard diffusion loss and intermittent ReFL steps. The authors evaluate on SD1.5 and SDXL with HPSv2.1 and PickScore rewards, reporting quality metrics (HPSv2.1, MPS, PickScore, ImageReward), diversity metrics (FID, CovDist, LogCovDist, DinoDiversity), and a user study. They report improvements over ReFL and a ReFL AIG baseline.

Significance. If the claimed trade-off improvements were established on a fair comparison, the paper would make a useful practical contribution: it offers a simple inference-time knob (combined generation) and a training regularizer (ImageReFL) that could help practitioners retain diversity in reward-aligned diffusion models. The idea of training a model to refine images from intermediate latents of real images, rather than from pure noise, is interesting and plausibly complementary to existing reward fine-tuning. The paper also ships source code and evaluates across two base models and two reward models, which is a strength. However, the central quantitative claim—that ImageReFL improves the quality–diversity frontier relative to ReFL-based baselines—is currently not supported with adequate experimental controls, for the reasons detailed below.

major comments (4)
  1. [§5.4 and Appendix A.2 (Table 3)] The comparison is made at different handover points for ImageReFL and the ReFL baselines: for SD1.5, ReFL uses 10 base + 30 fine-tuned steps while ImageReFL uses 30 base + 10 fine-tuned steps; for SDXL the numbers are 20/20 versus 35/5. Since the handover point is precisely the knob that trades diversity against quality in combined generation, ImageReFL’s larger FID and DinoDiversity gains may simply reflect that fewer fine-tuned steps are being used, rather than a property of the training method. Table 3’s caption states these are the “best-performing configurations,” selected on the same test used for evaluation, which compounds the issue. The authors should report results at matched handover points (e.g., equal numbers of fine-tuned steps across methods) and, if different points are chosen per method, they should be selected on a separate validation split with the selection rule stated in advance. Without this, the central claim that ImageReFL “outperforms conventional reward tuning methods on the quality–diversity balance” is not established.
  2. [§5.5 and Table 1] The baseline “ReFL AIG” is used as a key comparison throughout the experiments, but it is never defined in the main text, the appendix, or the references. The abbreviation “AIG” (“Artificially-Inspired Guidance”) appears only in Table 1 and Appendix A.2, where a parameter λ is set to 1 without explanation. The paper must either provide a full definition of this baseline (including the guiding signal and the source paper, or specify that this is a new baseline introduced here) or replace it with a properly documented baseline. As written, the reader cannot verify what ReFL AIG is or why its parameter is set to that value.
  3. [§5.5 and Table 1] The paper states results “consistently and significantly outperform” the baselines, but no statistical significance tests are reported. The ± values in the table headers appear to be metric-level uncertainties (possibly bootstrap confidence intervals over sampled images) rather than variability due to training seeds or model initialization. With only one training run per configuration, the reported differences (e.g., ImageReFL versus ReFL Combined on DinoDiversity: 0.220 vs 0.220 in the SD1.5 HPS row, or 0.2116 vs 0.2317 on SDXL, where the latter actually favors ReFL Combined) may not be reproducible. The authors should report multiple seeds or otherwise quantify the variance of the training procedure and use an appropriate significance test, especially since some of the apparent gaps are small relative to the reported error bars.
  4. [§5.6 and Table 2] The user study compares ReFL and ImageReFL, but it does not appear to control the handover point. Given that the quantitative comparison at different handover points casts doubt on the fairness of the comparison, the user study inherits the same problem. The study also compresses three annotators per task into a single aggregated vote; the number of pairwise comparisons (4,800 votes over 20 participants) should be reported with per-criterion counts and a measure of inter-annotator agreement. As it stands, the user study does not provide independent evidence for the method-specific claim.
minor comments (5)
  1. [§5.4 and Table 1] The abbreviation “ReFL AIG” is not expanded at its first use in Table 1; consider spelling it out and citing the origin or explicitly introducing it as a new baseline.
  2. [§6 (Conclusion)] “DraftK” appears to be a typo for “DRAFT” (the method of Clark et al., reference [5]); the sentence mentions DRTune and DraftK with no citations or definitions and should be corrected.
  3. [§5.1 and References] Reference [25] is cited for DINO embeddings, but the cited paper is “DETR with Improved DeNoising anchor boxes,” an object detection model; the actual DINOv2 paper is reference [14], and the metric DinoDiversity likely uses DINOv2. The reference list should be reconciled with the model actually used.
  4. [Figure 5 caption] The caption says “using combined generation,” but the figure appears to show curves for both ReFL and ImageReFL under combined generation; please clarify which methods and which switch points are included.
  5. [Equation (3)] The condition “t=T,...,0” is redundant and should be simply “t≥T′”; also note that in standard DDIM sampling the last step index is often 0, but the set description should be consistent.

Circularity Check

1 steps flagged · score 4.0 of 10

Reported ImageReFL-vs-ReFL comparison uses per-method, test-selected handover steps, so the diversity/quality advantage is partly an artifact of the chosen operating point rather than the training method alone.

  1. fitted input called prediction [Appendix A.2, Table 3 caption (cf. Section 5.4)]
    "Table 3 presents the evaluation metrics for the best-performing configuration of each method. For SD1.5, we use 10 steps with the base model and 30 steps with the fine-tuned model in the ReFL setup, and 30 steps with the base model and 10 with the fine-tuned model in ImageReFL. For SDXL, the corresponding step configurations are 20/20 for ReFL and 35/5 for ImageReFL."

    The handover step T' is the paper's own quality-diversity control knob: Figure 5 shows that increasing fine-tuned steps improves quality but reduces diversity. The headline comparisons in Table 3 use different, 'best-performing' T' values per method, selected on the same HPDv2 test split from which the metrics are computed. ReFL is measured at 30 (or 20) fine-tuned steps while ImageReFL is measured at 10 (or 5), i.e., at the diversity-favorable end of the curve. Thus the larger DinoDiv/FID gains and comparable HPSv2.1 partly reflect the chosen operating point rather than a property of the training loss; the claimed superiority is not tested at matched step counts in the main tables.

full rationale

Apart from the unequal, test-selected operating points, the method derivation is self-contained: combined generation is defined by Eq. 3 and ImageReFL by Eqs. 4-5, with the early-stability premise cited to external works [6,3]. The DinoDiversity metric is introduced by the authors and the CLIPDiversity reference [1] includes two of the same authors, but the main diversity conclusions also hold for external FID, CovDist, and LogCovDist, so the self-citation is not load-bearing. The ReFL training loss and reward models are external. The reported advantage of ImageReFL, however, is not independent of the evaluation data because the switch points are selected per method on the same test split; this is an evaluation-leakage/confounded-comparison issue rather than a fully circular derivation, hence score 4.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The method depends on several hand-chosen hyperparameters, most importantly the inference-time handover step which is tuned on the evaluation set. The core axioms are standard domain assumptions about diffusion dynamics and reward validity. No new physical or conceptual entities are introduced beyond a new evaluation metric (DinoDiversity), which is a simple embedding distance.

free parameters (5)
  • Handover step T' for combined generation = 10/30, 20/20, 30/10, 35/5 per setup
    The switch point between base and fine-tuned models is varied in Figure 5 and the 'best-performing configuration' is selected per method on the evaluation set.
  • Noise level t' for ImageReFL training = not reported explicitly
    The starting corruption step for real images is a hyperparameter; the paper says t' but does not specify its distribution or value.
  • Diffusion loss scale = 1e-5
    Weight of L_diffusion regularizer chosen by hand across all setups.
  • Reward scaling = 1e-3
    Reward models rescaled to [0,1] and multiplied by 1e-3.
  • ReFL interleave frequency = one ReFL step per 4 training steps
    Every 4 steps the classical ReFL loss is applied, chosen without reported tuning.
assumptions (3)
  • domain assumption Global image features are formed mainly in early diffusion steps and remain stable in later steps.
    Invoked in Section 4.1 to justify combined generation; cited to [6,3].
  • domain assumption Reward models HPSv2.1, PickScore, ImageReward, MPS are valid proxies for human preference.
    Used as both training rewards and evaluation metrics; no independent validation offered.
  • domain assumption COCO is an appropriate reference distribution for FID and covariance distance metrics.
    Section 5.4 uses COCO test/validation splits as reference for diversity metrics, even though evaluation prompts are from HPDv2.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ImageReFL: Balancing Quality and Diversity in Human-Aligned Diffusion Models." pith.science (2026). https://pith.science/paper/V6MLAKHA

@misc{pith2026250522569,
  author       = {Pith},
  title        = {Pith review of: ImageReFL: Balancing Quality and Diversity in Human-Aligned Diffusion Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/V6MLAKHA}},
  note         = {Machine review of arXiv:2505.22569}
}
read the original abstract

Recent advances in diffusion models have led to impressive image generation capabilities, but aligning these models with human preferences remains challenging. Reward-based fine-tuning using models trained on human feedback improves alignment but often harms diversity, producing less varied outputs. In this work, we address this trade-off with two contributions. First, we introduce \textit{combined generation}, a novel sampling strategy that applies a reward-tuned diffusion model only in the later stages of the generation process, while preserving the base model for earlier steps. This approach mitigates early-stage overfitting and helps retain global structure and diversity. Second, we propose \textit{ImageReFL}, a fine-tuning method that improves image diversity with minimal loss in quality by training on real images and incorporating multiple regularizers, including diffusion and ReFL losses. Our approach outperforms conventional reward tuning methods on standard quality and diversity metrics. A user study further confirms that our method better balances human preference alignment and visual diversity. The source code can be found at https://github.com/ControlGenAI/ImageReFL .

Figures

Figures reproduced from arXiv: 2505.22569 by the authors.

Figure 1
Figure 1. Diversity degradation during ReFL training on HPSv2.1 using SD1.5. The first row shows [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Image quality and diversity metrics during ReFL training on SD1.5 with HPSv2.1 as the [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Comparison of generations by ReFL, ReFL Combined, and ImageReFL. ReFL achieves [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (16 more)
Figure 4
Figure 4. Figure 4: Demonstration of the combined generation method applied to different base models (Stable [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Image quality and diversity trade-off for SD1.5 trained on the HPSv2.1 score using [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Comparison of ImageReFL with the standard approach. Unlike classical ReFL, ImageReFL [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Comparison between ReFL and ImageReFL with combined generation. The X-axis [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: Comparison of ImageReFL with ReFL and ReFL AIG baselines using SDXL and HPSv2.1 [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: Quality/diversity trade-off for SD1.5 trained on HPSv2.1. ImageReFL outperforms both [PITH_FULL_IMAGE:figures/full_fig_p009_9.png]
Figure 10
Figure 10. Figure 10: Quality/diversity trade-off for SDXL trained on HPSv2.1. ImageReFL outperforms both [PITH_FULL_IMAGE:figures/full_fig_p009_10.png]
Figure 11
Figure 11. Figure 11: shows an example task from the study. For each question, annotators could choose between three options: preference for the first row, preference for the second row, or no preference. We generated examples for the user study using the test split of the Human Preference…
Figure 12
Figure 12. Figure 12: Quality/diversity trade-off for SD1.5 trained on HPSv2.1. ImageReFL outperforms both [PITH_FULL_IMAGE:figures/full_fig_p014_12.png]
Figure 13
Figure 13. Figure 13: Quality/diversity trade-off for SDXL trained on HPSv2.1. ImageReFL outperforms both [PITH_FULL_IMAGE:figures/full_fig_p015_13.png]
Figure 14
Figure 14. Figure 14: Quality/diversity trade off for SD1.5 trained on PickScore. ImageReFL outperforms both [PITH_FULL_IMAGE:figures/full_fig_p016_14.png]
Figure 15
Figure 15. Figure 15: Quality/diversity trade off for SDXL trained on PickScore. ImageReFL outperforms both [PITH_FULL_IMAGE:figures/full_fig_p017_15.png]
Figure 16
Figure 16. Figure 16: Comparison of ImageReFL with ReFL and ReFL AIG baselines using SD1.5 trained on [PITH_FULL_IMAGE:figures/full_fig_p018_16.png]
Figure 17
Figure 17. Figure 17: Comparison of ImageReFL with ReFL and ReFL AIG baselines using SDXL trained on [PITH_FULL_IMAGE:figures/full_fig_p018_17.png]
Figure 18
Figure 18. Figure 18: Comparison of ImageReFL with ReFL and ReFL AIG baselines using SD1.5 trained on [PITH_FULL_IMAGE:figures/full_fig_p019_18.png]
Figure 19
Figure 19. Figure 19: Comparison of ImageReFL with ReFL and ReFL AIG baselines using SDXL trained on [PITH_FULL_IMAGE:figures/full_fig_p019_19.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Calibri: Enhancing Diffusion Transformers via Parameter-Efficient Calibration

    cs.CV 2026-03 conditional novelty 6.0 of 10

    Post-hoc scalar scaling of individual DiT block outputs, found by CMA-ES, improves text-to-image quality and cuts inference steps from 30-100 to 10-15.

  2. When Models Learn to Ask Why: Adaptive Causal Reasoning for Trustworthy Medical Vision-Language Models

    cs.AI 2026-03 conditional novelty 5.0 of 10

    MedCausalX adds adaptive causal reflection tokens and trajectory-level RL on a new CRMed dataset to cut medical VLM hallucinations and raise diagnostic consistency.

Reference graph

Works this paper leans on

28 extracted references · 6 canonical work pages · cited by 2 Pith papers

  1. [1]

    StyleDomain: Efficient and Lightweight Parameterizations of StyleGAN for One-shot and Few-shot Domain Adaptation

    Aibek Alanov, Vadim Titov, Maksim Nakhodnov, and Dmitry Vetrov. Styledomain: Efficient and lightweight parameterizations of stylegan for one-shot and few-shot domain adaptation. https://arxiv.org/abs/2212.10229, 2022

    work page Pith review arXiv 2022

  2. [2]

    Training diffusion models with reinforcement learning.arXiv preprint arXiv:2305.13301, 2023

    Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine. Training diffusion models with reinforcement learning.arXiv preprint arXiv:2305.13301, 2023

    arXiv 2023

  3. [3]

    Perception prioritized training of diffusion models.https://arxiv.org/abs/2204.00227, 2022

    Jooyoung Choi, Jungbeom Lee, Chaehun Shin, Sungwon Kim, Hyunwoo Kim, and Sungroh Yoon. Perception prioritized training of diffusion models.https://arxiv.org/abs/2204.00227, 2022

    arXiv 2022

  4. [4]

    Brown, Miljan Martic, Shane Legg, and Dario Amodei

    Paul Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences.arXiv preprint arXiv:1706.03741, 2017

    arXiv 2017

  5. [5]

    Draft: Diffusion reward fine-tuning

    Kevin Clark, Paul Vicol, Kevin Swersky, and David J Fleet. Draft: Diffusion reward fine-tuning. arXiv preprint arXiv:2309.17400, 2023

    arXiv 2023

  6. [6]

    Kamil Deja, Anna Kuzina, Tomasz Trzci ´nski, and Jakub M. Tomczak. On analyz- ing generative and denoising capabilities of diffusion-based deep generative models. https://arxiv.org/abs/2206.00070, 2022

    work page Pith review arXiv 2022

  7. [7]

    Dpok: Reinforcement learning for fine-tuning text-to-image diffusion models.arXiv preprint arXiv:2305.16381, 2023

    Ying Fan, Olivia Watkins, Yuqing Du, Hao Liu, Moonkyung Ryu, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, Kangwook Lee, and Kimin Lee. Dpok: Reinforcement learning for fine-tuning text-to-image diffusion models.arXiv preprint arXiv:2305.16381, 2023

    arXiv 2023

  8. [8]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochre- iter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. https://arxiv.org/abs/1706.08500, 2017

    arXiv 2017

Show all 28 references
  1. [9]

    Denoising diffusion probabilistic models.arXiv preprint arXiv:2006.11239, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models.arXiv preprint arXiv:2006.11239, 2020

    2006 arXiv

  2. [10]

    Elucidating optimal reward-diversity tradeoffs in text-to-image diffusion models

    Rohit Jena, Ali Taghibakhshi, Sahil Jain, Gerald Shen, Nima Tajbakhsh, and Arash Vah- dat. Elucidating optimal reward-diversity tradeoffs in text-to-image diffusion models. https://arxiv.org/abs/2409.06493, 2024

    2024 arXiv

  3. [11]

    Pick-a-pic: An open dataset of user preferences for text-to-image generation.arXiv preprint arXiv:2305.01569, 2023

    Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana andJoe Penna, and Omer Levy. Pick-a-pic: An open dataset of user preferences for text-to-image generation.arXiv preprint arXiv:2305.01569, 2023

    2023 arXiv

  4. [12]

    Aesthetic post-training diffusion models from generic preferences with step-by-step preference optimization.arXiv preprint arXiv:2406.04314, 2024

    Zhanhao Liang, Yuhui Yuan, Shuyang Gu, Bohan Chen, Tiankai Hang, Mingxi Cheng, Ji Li, and Liang Zheng. Aesthetic post-training diffusion models from generic preferences with step-by-step preference optimization.arXiv preprint arXiv:2406.04314, 2024

    2024 arXiv

  5. [13]

    Lawrence Zitnick, and Piotr Dollár

    Tsung-Yi Lin, Michael Maire, Serge Belongie, Lubomir Bourdev, Ross Girshick, James Hays, Pietro Perona, Deva Ramanan, C. Lawrence Zitnick, and Piotr Dollár. Microsoft coco: Common objects in context.https://arxiv.org/abs/1405.0312, 2014

    2014 arXiv

  6. [15]

    Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and...

    2022 arXiv

  7. [16]

    Aligning text- to-image diffusion models with reward backpropagation.arXiv preprint arXiv:2310.03739, 2023

    Mihir Prabhudesai, Anirudh Goyal, Deepak Pathak, and Katerina Fragkiadaki. Aligning text- to-image diffusion models with reward backpropagation.arXiv preprint arXiv:2310.03739, 2023

    2023 arXiv

  8. [17]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. InInternational conference on machine learning, pag...

    2021

  9. [18]

    Manning, and Chelsea Finn

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. arXiv preprint arXiv:2305.18290, 2023

    2023 arXiv

  10. [19]

    High- resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10684–10695, June 2022

    2022

  11. [20]

    High- resolution image synthesis with latent diffusion models.arXiv preprint arXiv:2112.10752, 2021

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models.arXiv preprint arXiv:2112.10752, 2021

    2021 arXiv

  12. [21]

    Diffusion model alignment using direct preference optimization.arXiv preprint arXiv:2311.12908, 2023

    Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. Diffusion model alignment using direct preference optimization.arXiv preprint arXiv:2311.12908, 2023

    2023 arXiv

  13. [22]

    Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis.arXiv preprint arXiv:2306.09341, 2023

    Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis.arXiv preprint arXiv:2306.09341, 2023

    2023 arXiv

  14. [23]

    Deep reward supervisions for tuning text-to-image diffusion models

    Xiaoshi Wu, Yiming Hao, Manyuan Zhang, Keqiang Sun, Zhaoyang Huang, Guanglu Song, Yu Liu, and Hongsheng Li. Deep reward supervisions for tuning text-to-image diffusion models. arXiv preprint arXiv:2405.00760, 2024

    2024 arXiv

  15. [24]

    Imagereward: Learning and evaluating human preferences for text-to-image generation

    Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagereward: Learning and evaluating human preferences for text-to-image generation. arXiv preprint arXiv:2304.05977, 2023

    2023 arXiv

  16. [25]

    Ni, and Heung- Yeung Shum

    Hao Zhang, Feng Li, Shilong Liu, Lei Zhang, Hang Su, Jun Zhu, Lionel M. Ni, and Heung- Yeung Shum. Dino: Detr with improved denoising anchor boxes for end-to-end object detection. https://arxiv.org/abs/2203.03605, 2022

    2022 arXiv

  17. [26]

    Learning multi-dimensional human preference for text-to-image gener- ation.https://arxiv.org/abs/2405.14705, 2024

    Sixian Zhang, Bohan Wang, Junqiang Wu, Yan Li, Tingting Gao, Di Zhang, and Zhongyuan Wang. Learning multi-dimensional human preference for text-to-image gener- ation.https://arxiv.org/abs/2405.14705, 2024. 11 A Appendix A.1 User study settings In the user study, annotators wer...

    2024 arXiv

  18. [27]

    Which row of images is more diverse? To what extent do the images within a single row differ from each other in terms of content, colors, composition, and other visual elements?

  19. [28]

    Which row of images corresponds better to the textual description? To what degree do the images reflect the content described in the textual prompt above?

  20. [29]

    For each question, annotators could choose between three options: preference for the first row, preference for the second row, or no preference

    Which row of images is more visually appealing? Which images are more aesthetically pleasing in terms of visual quality, design, and overall presentation? Figure 11 shows an example task from the study. For each question, annotators could choose between three options: preferen...

    1923

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.