Pith. sign in

REVIEW 3 major objections 6 minor 72 references

PAST: Prompt-Adaptive Sampling Termination for Efficient Diffusion Model

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read PAST, a plug-in for RL fine-tuning of diffusion models, claims up to 66.7% savings in training time and up to 29.5% better preference optimization by terminating episodes adaptively.

desk verdict Plausible efficiency idea for RL fine-tuning of diffusion models undermined by a wrong termination rule (min vs max) and an overstated theorem. read the letter →

arxiv 2608.06794 v1 pith:IFACVJVR submitted 2026-08-07 cs.CV

Renye Yan , Jikang Cheng , You Wu , Wei Peng , Zongwei Wang , Ling Liang , Yimao Cai This is my paper · ORCID
classification cs.CV
keywords TexttoImageGenerationComputationalEfficiencyOptimizationDiffusionModelsReinforcementLearningFine-tuningIntrinsicRewardAdaptiveEpisodeTerminationCross-AttentionAlignmentSparsity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

PAST is a plugin for reinforcement-learning fine-tuning of text-to-image diffusion models. It claims that current RL methods waste most of their training compute on denoising steps whose marginal quality gain has already saturated, and that this waste can be avoided without hurting the final reward. The paper proposes an intrinsic reward based on distance from initial noise to relieve sparse feedback, and a prompt-adaptive termination rule that stops an episode as soon as a denoise-completion classifier and a cross-attention semantic-alignment signal both indicate the image is ready. In experiments, adding PAST to existing RL fine-tuning methods reduces time-to-target reward by up to 66.7% and improves preference scores by up to 29.5%.

What carries the argument

The Denoise-Aware Model (DAM), a small binary classifier trained on endpoint labels with a tunable neighborhood $\nu$, produces the real-time cleanliness estimate $\alpha_t^d$. The semantic-alignment evaluator computes pairwise cosine distances between cross-attention maps inside noun-attribute groups, producing $\alpha_t^s$. The termination rule takes the earlier of the denoising-ready and alignment-ready steps: $t_{\text{len}} = \min\{t^d_{\text{len}}, t^s_{\text{len}}\}$. Intrinsic reward $R_{\text{int}}(s_t,a_t)=\|x_t-x_T\|_2^2$ provides dense mid-trajectory feedback, while the adaptive factor $(1-\alpha_t)$ orchestrates the extrinsic-intrinsic and exploration-convergence trade-offs. Theorem 2.1 gives the optimal policy in closed form and is used to justify stable convergence of the training objective.

What would settle it

Run PAST on a prompt set where the decisive visual content, such as object identity, count, or spatial relation, first appears in the last few denoising steps. If the denoise or alignment proxies fire early on such prompts, adaptive episodes will terminate before that content is fixed, and quality metrics will measurably fall relative to full-length episodes.

Watch

Extended reading notes

Core claim

The paper's central claim is that episode length in RL fine-tuning does not need to be a fixed full denoising schedule. A Denoise-Aware Model learns to classify intermediate latents as noise-side or clean-side, giving a real-time cleanliness estimate; a second signal measures whether the attention maps of a prompt's noun and its attributes focus on the same regions, which the paper treats as semantic alignment. PAST terminates an episode at the earlier of the two 'ready' steps, and uses the adaptive factor $(1-\alpha_t)$ to shift from intrinsic exploration reward to extrinsic task reward and to strengthen KL regularization as denoising proceeds. The paper reports comparable or better image quality on the same or shorter episodes, with computational savings up to 66.7% and preference optimization gains up to 29.5%.

Load-bearing premise

The whole gain rests on the Denoise-Aware Model's binary cleanliness estimate and the noun-attribute attention-map similarity being accurate proxies for whether the image is actually ready; if either lags reality, the min-rule ends episodes too early and quality drops.

Editorial extensions

If this is right

  • If the two readiness proxies are reliable, every RL fine-tuning run can skip steps beyond the earlier of denoising completion and semantic alignment, so compute savings multiply with prompt difficulty.
  • Adding PAST to reward-based methods preserves the base method's gains while substantially shortening time-to-target, as reported for Aesthetic Score and PickScore objectives.
  • The intrinsic reward shifts the policy toward reaching clean latents earlier, so per-episode step budgets can be set dynamically rather than fixed for all prompts.
  • The adaptive termination mechanism transfers across backbone architectures, including SD1.4, SD2.1, SDXL, and the flow-matching SD3.5, and across heterogeneous reward objectives.
  • By reducing over-optimization of late-step rewards, PAST also reduces reward-hacking artifacts such as style or background collapse, improving fidelity and diversity metrics.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension of the paper's logic is to reuse the same real-time denoising and attention-alignment signals as inference-time stopping rules, which would make every generation adaptive rather than only the RL training episodes.
  • Because the termination rule takes a minimum of two imperfect monitors, a conservative variant that requires both signals to persist for several consecutive steps could avoid premature termination at a small compute cost.
  • If the cross-attention alignment proxy truly measures semantic binding, the same signal could serve as an auxiliary loss or reward for non-RL fine-tuning methods, a direction the paper does not test.
  • The reported savings may grow when reward models become more expensive to query, since adaptive termination prevents paying those query costs on steps whose marginal value has saturated.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes PAST, a plugin for reinforcement-learning fine-tuning of text-to-image diffusion models. PAST adds an intrinsic reward based on the squared distance between the current denoised state and the initial noise, which is claimed to accelerate denoising and improve exploration, and it introduces an adaptive episode-termination mechanism that stops training episodes early based on a Denoise-Aware Model (DAM) and a cross-attention semantic-alignment metric. The authors report up to 66.7% computational savings and up to 29.5% preference-optimization improvement when PAST is added to DDPO and DPOK, and they provide a theorem intended to justify the intrinsic reward. Experiments span multiple backbones (SDv1.4, SDv1.5, SDv2.1, SDXL, SD3.5), reward functions (Aesthetic Score, PickScore, JPEG compressibility, incompressibility), and evaluation metrics including human and large-vision-model scoring.

Significance. If the empirical claims are correct, PAST addresses a practically important bottleneck: the high computational cost of RL fine-tuning for text-to-image diffusion models. The paper's strengths include a model-agnostic design, a broad experimental evaluation across backbones, reward functions, and datasets, and an explicit effort to measure diversity (LPIPS, TCE) in addition to preference scores. The reported wall-clock improvements (Tab. 1) are substantial and, if reproducible, would make the method useful to practitioners. However, the theoretical justification (Theorem 2.1) is not a convergence guarantee in its current form, and the termination rule in Eq. (11) is inconsistent with the stated requirement that both denoising and semantic criteria be satisfied. These issues are load-bearing for the central claims of efficiency and quality preservation, so the manuscript requires substantive revision before the result can be accepted.

major comments (3)
  1. [Sec. 2.3.3, Eq. (11)] The termination rule is inconsistent with the stated requirement. The text says 'the final episode length needs to satisfy both noise and semantic requirements,' and the abstract says termination occurs when 'both metrics satisfy generation requirements.' However, Eq. (11) defines t_len = min(t_s_len, t_d_len), which stops the episode as soon as either signal fires. If semantic alignment saturates before full denoising (or vice versa), the min rule returns an image from an incomplete denoising step. The correct joint stopping time for 'both' is t_len = max(t_s_len, t_d_len). Table 2 reports only the average adaptive length (31) and aggregate metrics; it does not report the joint distribution of t_s_len and t_d_len, nor an ablation comparing min versus max. Since the efficiency claim depends on early termination and the quality claim depends on not truncating under-denoised episodes, the authors must either justify min as a deliberate early-stop heuristic with evidence that quality is preserved in the cases where one signal lags, or change the rule to max and re-run the experiments.
  2. [Sec. 2.2, Theorem 2.1 and proof] Theorem 2.1 is stated as an expression for the optimal policy, but the text immediately after the theorem says 'The theorem guarantees the convergence of Eq. 5.' This is a category error: a closed-form expression for the optimal policy, even if correct, does not by itself guarantee convergence of any training algorithm to that policy. In the proof, the alpha indices are inconsistent across lines: alpha_{t-1} appears in the exponent of Eq. (6), alpha_{t-2} in the soft-Q recursions, and alpha_{t-3} in the expansion. The recursion also drops the extrinsic reward R_ext: after the second line, the term exp{R_int(x_{t-1})/alpha_{t-2}} is factored out, but R_ext(x_{t-1}) disappears entirely, and the final factorization into Q_ext^* and Q_int^* is not derived from the preceding algebra. As written, the proof does not establish the theorem, and the theorem does not establish convergence. The authors should either repair the statement and proof with consistent indexing and a complete derivation, or remove the convergence claim and present the intrinsic reward as a heuristic whose effectiveness is demonstrated empirically.
  3. [Sec. 2.1.3, Eq. (4) and Sec. 2.3.1, Eqs. (7)-(8)] The adaptive termination mechanism relies on two proxies whose validity is not demonstrated. First, the DAM classifier is trained with labels derived from the endpoints x_T and x_0 of the same denoising trajectories that define the intrinsic reward, and the neighborhood range nu and endpoint sample count n are tunable hyperparameters. The paper does not report DAM accuracy, calibration, or any comparison of its output against a ground-truth noise level on held-out trajectories, so it is unclear whether alpha_t^d reliably indicates denoising completion for intermediate states. Second, the semantic alignment metric alpha_s^t is defined as the average pairwise cosine distance among cross-attention maps in noun-attribute groups. This is a plausible proxy, but no evidence is given that this distance correlates with actual prompt-image semantic alignment (e.g., against CLIP score or human ratings on intermediate steps). Given that the termination rule uses both signals to decide episode length, the paper should include a validation study of both DAM and the attention-based alignment metric, for example by reporting the correlation of alpha_s^t with a reference alignment score across denoising steps.
minor comments (6)
  1. [Section 2] There are several typos: 'seciton' should be 'section,' 'epsoide' should be 'episode,' and the section heading 'Sementic alignment evaluator' should be 'Semantic alignment evaluator.'
  2. [Figure 1] The label 'None-AttrGroups' in Figure 1 should be 'Noun-AttrGroups' to match the terminology used in the text.
  3. [Sec. 2.2, Eq. (5)] Equation (5) contains a formatting error in the KL term: 'KL(p_theta(·))||p_pre(·))' has an extra closing parenthesis and is missing the second argument of the KL divergence. Please fix the notation.
  4. [Table 3] Table 3 lists hyperparameters but omits the DAM neighborhood range nu and the endpoint sample count n, both described as tunable in Sec. 2.1.3. Please report the values used in the experiments, since the termination behavior is sensitive to them.
  5. [Sec. 3.2] The statement that DeepSeek-V3 and Qwen2.5-32B-Instruct 'achieved 100% consistency' in noun-attribute decomposition is vague. Please specify what the 100% consistency is measured against (e.g., human annotations, agreement between the two models) and how many prompts were tested.
  6. [Section 2.3.1] The claim that CLIP 'struggle[s] to provide accurate real-time assessments of semantic alignment during denoising' is supported by citation [7], but [7] is 'Diffusion models beat GANs on image synthesis' and does not address CLIP-based alignment evaluation. Please cite an appropriate reference for this claim.

Circularity Check

1 steps flagged · score 4.0 of 10

Theoretical justification of the intrinsic reward is self-referential; empirical efficiency/quality claims remain externally benchmarked.

  1. self definitional [Sec. 2.1.2 (Intrinsic Reward Rationality) and Sec. 2.2, Eq. (5) and Theorem 2.1]
    "Intrinsic Reward Rationality: • Theoretical guarantees: the intrinsic reward is theoretically justified in Theorem. 2.1. ... Theorem 2.1. Given the objective of Eq. (5), the optimal policy pθ∗(x_{t−1}|x_t,c) ∝ exp{...} ... [Eq. (5):] J=E_{p(c)}E_{pθ(τ|c)}[r(x0,c))−Σ_{t=1}^{T}α_t KL(pθ(·)||p_pre(·))]+E_{p(c)}E_{pθ(τ|c)}Σ_{t=0}^{T}(1−α_t)R_int(s_t,a_t)(x_t)."

    Eq. (5) is the training objective that already contains the intrinsic-reward term (1−α_t)R_int(s_t,a_t)(x_t) before Theorem 2.1 is stated. Theorem 2.1 then only computes the soft-Q form of the optimal policy for this pre-augmented objective, using the SBEED recursion [6]; it does not derive that adding R_int improves optimization, avoids reward hacking, or accelerates denoising. Calling this theorem a 'theoretical guarantee' for the intrinsic reward makes the justification identical to the assumption: the benefit asserted for R_int is an input to Eq. (5), not an output of the theorem. The separate efficiency/quality results (Tabs. 1–2, Fig. 3) are externally benchmarked, so the circularity affects the stated theoretical rationale rather than all empirical claims.

full rationale

The main numerical claims—66.7% training-time reduction and up to 29.5% preference improvement—are measured against external reward thresholds and baseline methods (DDPO, DPOK) on AES and PickScore, and the adaptive-termination comparison in Tab. 2 uses fixed/complete episode controls with external metrics (AES, PS, IR, CLIP, IS, TCE, LPIPS). Those comparisons do not reduce to the fitted DAM or to R_int by construction. The DAM is admittedly trained from the same trajectory endpoints (x_T labeled 0, x_0 labeled 1), so alpha^d_t is a self-referential control signal, but the paper does not use alpha alone as the quality benchmark, which limits the circularity. The one concrete circular step is the 'theoretical justification' of the intrinsic reward: Theorem 2.1 assumes R_int inside Eq. (5) and derives the optimal-policy form, so it cannot independently establish R_int's benefit. The self-citation [57] (intrinsic rewards usually do not alter the optimal extrinsic policy) is related but not load-bearing, since the paper's main derivation is self-contained. The min-based Eq. (11) vs. the 'both metrics' wording is an internal consistency/correctness risk, not a circularity, and does not drive this score.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The method introduces a learned Denoise-Aware Model and a semantic alignment metric, both fitted to the training process itself. The central hyperparameters (nu, termination thresholds) are not fully specified. The theoretical justification relies on an existing result but does not clearly verify its conditions.

free parameters (3)
  • nu (neighborhood range for DAM labels) = not reported in visible text
    Used in Sec. 2.1.3 to set clean-side and noise-side neighborhoods for training the Denoise-Aware Model; claimed to be in the appendix but not found in the provided hyperparameter table.
  • n (number of endpoint samples for DAM training) = 10
    Appendix B.5: take the first n images near x_T as negatives and last n near x_0 as positives. This choice affects the learned cleanliness measure.
  • Termination thresholds for alpha^d -> 1 and alpha^s -> 0 = not reported
    The adaptive episode rule in Eqs. 9-10 requires concrete thresholds to trigger episode termination; these are not specified in the main text or hyperparameter table.
assumptions (5)
  • domain assumption The diffusion denoising process can be modeled as the MDP in Eq. 1 with state s_t=(c,x_{T-t}) and action a_t=x_{T-t-1}.
    Standard for RL fine-tuning of diffusion models and stated in Sec. 2.1.1.
  • domain assumption Attention maps of tokens in a Noun-Attr group focus on similar visual regions, so pairwise cosine distance between attention maps measures semantic alignment.
    Introduced in Sec. 2.3.1 without empirical validation on a held-out alignment benchmark.
  • domain assumption The Denoise-Aware Model trained on endpoint labels (x_T as 0, x_0 as 1) generalizes to intermediate states and produces a monotone cleanliness measure alpha_t.
    Sec. 2.1.3 relies on this generalization for reward weighting and termination.
  • standard math The SBEED result (ref [6]) applies to the diffusion MDP with the modified intrinsic reward and alpha_t weighting.
    Invoked in the proof of Theorem 2.1; the proof does not verify all conditions required by SBEED.
  • ad hoc to paper The intrinsic reward R_int = ||x_t - x_T||^2 encourages useful exploration and faster denoising without harming diversity.
    Asserted in Sec. 2.1.2 and supported only by internal visualizations, not by an independent theoretical or empirical test.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PAST: Prompt-Adaptive Sampling Termination for Efficient Diffusion Model." pith.science (2026). https://pith.science/paper/IFACVJVR

@misc{pith2026260806794,
  author       = {Pith},
  title        = {Pith review of: PAST: Prompt-Adaptive Sampling Termination for Efficient Diffusion Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IFACVJVR}},
  note         = {Machine review of arXiv:2608.06794}
}
read the original abstract

While diffusion models have made significant progress in text-to-image tasks, they still exhibit limitations when directly optimizing downstream objectives. Although Reinforcement Learning (RL) enables targeted optimization, existing methods are generally constrained by low-efficiency fine-tuning and sparse rewards. To address these challenges, we propose PAST, which provides differentiated rewards while adaptively regulating training episode length by jointly perceiving denoising progress and prompt difficulty. Specifically, we design an intrinsic reward paradigm to compensate for sparse extrinsic rewards and guide the model to explore paths that diverge more efficiently from noise patterns. We further provide theoretical justification for intrinsic rewards. Then, PAST dynamically monitors denoising completion and semantic alignment between image structures and prompt semantics. When both metrics satisfy generation requirements, the system adaptively terminates training. This enables appropriate allocation of episode lengths based on prompt difficulty and the current generation process. Finally, based on the predicted residual noise level, we establish a dual adaptive coordination mechanism. Specifically, it not only balances the extrinsic and intrinsic rewards but also balances the exploration and convergence. Experimental results demonstrate that PAST enhances computational efficiency of existing RL fine-tuning methods by up to 66.7%, while improving preference optimization quality by up to 29.5% through its dual adaptive regulation mechanism.

Figures

Figures reproduced from arXiv: 2608.06794 by the authors.

Figure 1
Figure 1. Overview of PAST. Tackling the challenges of high computational cost and sparse rewards of existing RL fine-tuning methods, we introduce a) adaptive episode control based on prompt complexity and denoising progress, enabling a broader search range for high reward policy preserving real distribution. b) We design an intrinsic motivation based on fine-grained text￾image alignment to alleviate reward sparsity. c) Both … view at source ↗
Figure 2
Figure 2. Image quality during denoising. The quality improvement becomes marginal in the last segment of denoising steps [ [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Reward optimization results with two datasets ( [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (14 more)
Figure 4
Figure 4. Figure 4: Computational cost to reach specific reward scores [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Ablation Study. Left: Computational cost per epoch. [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 8
Figure 8. Figure 8: Transferring to different diffusion backbones. [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]
Figure 9
Figure 9. Figure 9: Performance on a flow-matching model (SD3.5). Results on OCR, GenEval, and Aesthetic objectives show that PAST remains consistently effective under diverse evaluation targets. Combination Count Color Location A fork and a knife Four vases A A dog right of a teddy purpl…
Figure 10
Figure 10. Figure 10: Conditional Generation on the flow-matching Diffusion Model (SD3.5). Compared to baseline methods, PAST better follows instructions when prompts explicitly define combinations, counts, colors, and locations. Count & Semantic: Four roses Color: A green colored rabbit C…
Figure 12
Figure 12. Figure 12: Subjective Evaluation on Human and VLM. 4 CONCLUSION In this paper, we propose PAST, an efficient plugin for RL fine￾tuning of diffusion models, which involves an adaptive episode control to enable a broader search range for high reward policy preserving real distribu…
Figure 13
Figure 13. Figure 13: Transferring to different reward functions. [PITH_FULL_IMAGE:figures/full_fig_p013_13.png]
Figure 15
Figure 15. Figure 15: Aesthetic quality of generated images. Name Description Value lr learning rate of PAST 3e-4 optimizer type of optimizer Adam [23] 𝜉 weight decay of optimizer 1e-4 𝜖 Gradient clip norm 1.0 𝛽1 𝛽1 of Adam 0.9 𝛽2 𝛽2 of Adam 0.999 𝑇 total timesteps of inference 50 𝑏𝑠 train…
Figure 21
Figure 21. Figure 21: 𝑅𝑖𝑛𝑡 effect on training [PITH_FULL_IMAGE:figures/full_fig_p014_21.png]
Figure 16
Figure 16. Figure 16: Overall visual quality on unseen prompts. [PITH_FULL_IMAGE:figures/full_fig_p015_16.png]
Figure 17
Figure 17. Figure 17: Diversity Experiments. Results demonstrate that [PITH_FULL_IMAGE:figures/full_fig_p016_17.png]
Figure 18
Figure 18. Figure 18: Interface for Subjective Evaluation [PITH_FULL_IMAGE:figures/full_fig_p017_18.png]
Figure 19
Figure 19. Figure 19: Intrinsic reward alleviates reward hacking: Using only intrinsic reward without other modules, we observe a [PITH_FULL_IMAGE:figures/full_fig_p018_19.png]
Figure 20
Figure 20. Figure 20: Please zoom in. 𝑅int (s𝑡 , a𝑡 ) effect on one denoising trajectory [PITH_FULL_IMAGE:figures/full_fig_p019_20.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

72 extracted references · 18 canonical work pages

  1. [55]

    Renye Yan, Jikang Cheng, Shikun Sun, Yi Sun, You Wu, Wei Peng, Zongwei Wang, Ling Liang, Junliang Xing, and Yimao Cai. 2026. Do Less, Achieve More: Do We Need Every-Step Optimization for RL Fine-tuning of Diffusion Models?arXiv preprint arXiv:2605.15855(2026)

    arXiv 2026

  2. [1]

    James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al . 2023. Improving im- age generation with better captions.Computer Science. https://cdn. openai. com/papers/dall-e-3. pdf2, 3 (2023), 8

    2023

  3. [2]

    Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine

  4. [3]

    Yuri Burda, Harrison Edwards, Amos Storkey, and Oleg Klimov. 2018. Exploration by random network distillation.arXiv preprint arXiv:1810.12894(2018)

    arXiv 2018

  5. [4]

    Kevin Clark, Paul Vicol, Kevin Swersky, and David J Fleet. 2023. Directly fine- tuning diffusion models on differentiable rewards.arXiv preprint arXiv:2309.17400 (2023)

    arXiv 2023

  6. [5]

    Will Dabney, Georg Ostrovski, and André Barreto. 2020. Temporally-extended {\epsilon}-greedy exploration.arXiv preprint arXiv:2006.01782(2020)

    arXiv 2020

  7. [6]

    Shaw, Lihong Li, Lin Xiao, Niao He, Zhen Liu, Jianshu Chen, and Le Song

    Bo Dai, Albert E. Shaw, Lihong Li, Lin Xiao, Niao He, Zhen Liu, Jianshu Chen, and Le Song. 2018. SBEED: Convergent Reinforcement Learning with Nonlinear Function Approximation. InProceedings of International Conference on Machine Learning,

    work page 2018

  8. [7]

    Prafulla Dhariwal and Alexander Nichol. 2021. Diffusion models beat gans on image synthesis.Advances in neural information processing systems34 (2021), 8780–8794

    2021

Show all 72 references
  1. [8]

    Xin Ding, Lei Yu, Xin Li, Zhijun Tu, Hanting Chen, Jie Hu, and Zhibo Chen

  2. [9]

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. 2024. Scaling rectified flow transformers for high-resolution image synthesis. InForty- first international conference on ma...

    2024

  3. [10]

    Ying Fan, Olivia Watkins, Yuqing Du, Hao Liu, Moonkyung Ryu, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, Kangwook Lee, and Kimin Lee. 2023. Dpok: Reinforcement learning for fine-tuning text-to-image diffusion models. Advances in Neural Information Processing Systems3...

    2023

  4. [11]

    Ying Fan, Olivia Watkins, Yuqing Du, Hao Liu, Moonkyung Ryu, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, Kangwook Lee, and Kimin Lee. 2024. Reinforcement learning for fine-tuning text-to-image diffusion models.Advances in Neural Information Processing Systems36 (2024)

    2024

  5. [12]

    Giorgio Franceschelli and Mirco Musolesi. 2024. Reinforcement learning for generative ai: State of the art, opportunities and open research challenges.Journal of Artificial Intelligence Research79 (2024), 417–446

    2024

  6. [13]

    Yaozhong Gan, Renye Yan, Xiaoyang Tan, Zhe Wu, and Junliang Xing. 2024. Trans- ductive off-policy proximal policy optimization.arXiv preprint arXiv:2406.03894 (2024)

    2024 arXiv

  7. [14]

    Yaozhong Gan, Renye Yan, Zhe Wu, and Junliang Xing. 2024. Reflective Policy Optimization.arXiv preprint arXiv:2406.03678(2024)

    2024 arXiv

  8. [15]

    QIANYUE HAO, XIAOQIAN QI, YUAN YUAN, ZEFANG ZONG, HONGYI CHEN, KEYU ZHAO, SHENGYUAN WANG, YUNKE ZHANG, JIAN YUAN, and YONG LI. 2024. Reinforcement learning in the era of large language models: Challenges and opportunities. (2024)

    2024

  9. [16]

    Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P Kingma, Ben Poole, Mohammad Norouzi, David J Fleet, et al. 2022. Imagen video: High definition video generation with diffusion models. arXiv preprint arXiv:2210.02303(2022)

    2022 arXiv

  10. [17]

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising diffusion probabilistic models.Advances in neural information processing systems33 (2020), 6840–6851

    2020

  11. [18]

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. 2022. Video diffusion models.Advances in Neu- ral Information Processing Systems35 (2022), 8633–8646

    2022

  12. [19]

    Francisco Ibarrola and Kazjon Grace. 2024. Measuring diversity in co-creative image generation.arXiv preprint arXiv:2403.13826(2024)

    2024 arXiv

  13. [20]

    Rohit Jena, Ali Taghibakhshi, Sahil Jain, Gerald Shen, Nima Tajbakhsh, and Arash Vahdat. 2024. Elucidating optimal reward-diversity tradeoffs in text-to-image diffusion models.arXiv preprint arXiv:2409.06493(2024)

    2024 arXiv

  14. [21]

    Zhiyu Jin, Xuli Shen, Bin Li, and Xiangyang Xue. 2023. Training-free diffusion model adaptation for variable-sized text-to-image synthesis.Advances in Neural Information Processing Systems36 (2023), 70847–70860

    2023

  15. [22]

    Sunwoo Kim, Minkyu Kim, and Dongmin Park. 2025. Test-time alignment of dif- fusion models without reward over-optimization.arXiv preprint arXiv:2501.05803 (2025)

    2025 arXiv

  16. [23]

    Diederik P Kingma. 2014. Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980(2014)

    2014 arXiv

  17. [24]

    Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana, Joe Penna, and Omer Levy. 2023. Pick-a-pic: An open dataset of user preferences for text-to- image generation.Advances in Neural Information Processing Systems(2023)

    2023

  18. [25]

    Kyungmin Lee, Sangkyung Kwak, Kihyuk Sohn, and Jinwoo Shin. 2024. Direct consistency optimization for robust customization of text-to-image diffusion models.Advances in neural information processing systems37 (2024), 103269– 103304

    2024

  19. [26]

    Haodong Li, Tianfei Ren, Xiaoxiao Ma, Chunmei Qing, Zhen Fang, Sipeng He, Ziyu Guo, Haoyu Wu, Juanxi Tian, Yihang Zou, et al. 2026. VideoCoCo: Code- as-CoT for Physically-Consistent Video Generation via an Agentic Dual-Engine System.arXiv preprint arXiv:2607.27380(2026)

    2026 arXiv

  20. [27]

    Shufan Li, Konstantinos Kallidromitis, Akash Gokul, Yusuke Kato, and Kazuki Kozuka. 2024. Aligning diffusion models by optimizing human utility.Advances in Neural Information Processing Systems37 (2024), 24897–24925

    2024

  21. [28]

    Yanzeng Li, Cheng Zeng, Jialun Zhong, Ruoyu Zhang, Minhao Zhang, and Lei Zou. 2024. Leveraging large language model as simulated patients for clinical education.arXiv preprint arXiv:2404.13066(2024)

    2024 arXiv

  22. [29]

    Yuqi Li, Zijie Zhou, Zhiyuan Peng, Junhao Dong, Haochen You, Renye Yan, Ship- ing Wen, Yingli Tian, and Tingwen Huang. 2025. A preference-driven method- ology for efficient code generation.IEEE Transactions on Artificial Intelligence (2025)

    2025

  23. [30]

    Zhiqiu Lin, Deepak Pathak, Baiqi Li, Jiayao Li, Xide Xia, Graham Neubig, Pengchuan Zhang, and Deva Ramanan. 2024. Evaluating text-to-visual gen- eration with image-to-text generation. InEuropean Conference on Computer Vision. Springer, 366–384

    2024

  24. [31]

    Qichao Ma, Rui-Jie Zhu, Peiye Liu, Renye Yan, Fahong Zhang, Ling Liang, Meng Li, Zhaofei Yu, Zongwei Wang, Yimao Cai, et al. 2025. Inner-Probe: Discovering Copyright-related Data Generation in LLM Architecture.IEEE Transactions on Artificial Intelligence(2025)

    2025

  25. [32]

    Xinyin Ma, Gongfan Fang, and Xinchao Wang. 2024. Deepcache: Accelerating diffusion models for free. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition. 15762–15772

    2024

  26. [33]

    Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. 2021. Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741(2021)

    2021 arXiv

  27. [34]

    Deepak Pathak, Pulkit Agrawal, Alexei A Efros, and Trevor Darrell. 2017. Curiosity-driven exploration by self-supervised prediction. InInternational con- ference on machine learning. PMLR, 2778–2787

    2017

  28. [35]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sand- hini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. [n. d.]. Learning transferable visual models from natural language supervision. InInternational conference on machine le...

  29. [36]

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model.Advances in neural information processing systems36 (2023), 53728–53741

    2023

  30. [37]

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen

  31. [38]

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 10684–10695

    2022

  32. [39]

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. 2022. Photorealistic text-to-image diffusion models with deep language understanding.Advances in neural inform...

    2022

  33. [40]

    Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. 2016. Improved techniques for training gans.Advances in neural information processing systems29 (2016)

    2016

  34. [41]

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. 2022. Laion-5b: An open large-scale dataset for training next generation image-text models.Advances in neural in...

    2022

  35. [42]

    Joar Skalse, Nikolaus Howe, Dmitrii Krasheninnikov, and David Krueger. 2022. Defining and characterizing reward gaming.Advances in Neural Information Processing Systems35 (2022), 9460–9471

    2022

  36. [43]

    Richard S Sutton. 2018. Reinforcement learning: An introduction.A Bradford Book(2018)

    2018

  37. [44]

    Adrien Ali Taiga, William Fedus, Marlos C Machado, Aaron Courville, and Marc G Bellemare. 2021. On bonus-based exploration methods in the arcade learning environment.arXiv preprint arXiv:2109.11052(2021)

    2021 arXiv

  38. [45]

    Masatoshi Uehara, Yulai Zhao, Tommaso Biancalani, and Sergey Levine. 2024. Understanding reinforcement learning-based fine-tuning of diffusion models: A tutorial and review.arXiv preprint arXiv:2407.13734(2024)

    2024 arXiv

  39. [46]

    Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. 2024. Diffusion model alignment using direct preference optimization. InProceedings of the IEEE/CVF Conference on Computer Visio...

    2024

  40. [47]

    Zhengren Wang, Jiayang Yu, Dongsheng Ma, Zhe Chen, Yu Wang, Zhiyu Li, Feiyu Xiong, Yanfeng Wang, Linpeng Tang, Wentao Zhang, et al. 2025. Rare: Retrieval-augmented reasoning modeling.arXiv preprint arXiv:2503.23513(2025)

    2025 arXiv

  41. [48]

    Linye Wei, Zixiang Luo, Pingzhi Tang, and Meng Li. 2026. Team: Temporal- spatial consistency guided expert activation for moe diffusion language model acceleration.arXiv preprint arXiv:2602.08404(2026)

    2026 arXiv

  42. [49]

    Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. 2023. Human preference score v2: A solid benchmark for evaluat- ing human preferences of text-to-image synthesis.arXiv preprint arXiv:2306.09341 (2023)

    2023 arXiv

  43. [50]

    Xiaoshi Wu, Keqiang Sun, Feng Zhu, Rui Zhao, and Hongsheng Li. 2023. Human preference score: Better aligning text-to-image models with human preference. InProceedings of the IEEE/CVF International Conference on Computer Vision. 2096– 2105

    2023

  44. [51]

    Xin Xie and Dong Gong. 2025. DyMO: Training-Free Diffusion Model Alignment with Dynamic Multi-Objective Scheduling. InProceedings of the Computer Vision and Pattern Recognition Conference. 13220–13230

    2025

  45. [52]

    Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. 2023. Imagereward: Learning and evaluating human prefer- ences for text-to-image generation.Advances in Neural Information Processing Systems(2023)

    2023

  46. [53]

    Jiale Xu, Xintao Wang, Weihao Cheng, Yan-Pei Cao, Ying Shan, Xiaohu Qie, and Shenghua Gao. 2023. Dream3d: Zero-shot text-to-3d synthesis using 3d shape prior and text-to-image diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition...

    2023

  47. [54]

    Renye Yan, Jikang Cheng, Yaozhong Gan, Shikun Sun, You Wu, Yunfan Yang, Liang Ling, Jinlong Lin, Yeshuang Zhu, Jie Zhou, et al. 2025. Entropy-adaptive diffusion policy optimization with dynamic step alignment. InProceedings of the IEEE/CVF International Conference on Computer ...

    2025

  48. [56]

    Renye Yan, Jikang Cheng, You Wu, Ling Liang, Wei Peng, Athanasios V Vasilakos, Qingyu Zhao, Yu Zhang, Ehsan Adeli, Kilian M Pohl, et al . 2026. Pixel-Space Diffusion Transformers.arXiv preprint arXiv:2607.17585(2026)

    2026 arXiv

  49. [57]

    Renye Yan, Yaozhong Gan, You Wu, Ling Liang, Junliang Xing, Yimao Cai, and Ru Huang. 2024. The Exploration-Exploitation Dilemma Revisited: An Entropy Perspective.arXiv preprint arXiv:2408.09974(2024)

    2024 arXiv

  50. [58]

    Kai Yang, Jian Tao, Jiafei Lyu, Chunjiang Ge, Jiaxin Chen, Weihan Shen, Xiaolong Zhu, and Xiu Li. 2024. Using human feedback to fine-tune diffusion models without any reward model. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 8941–8951

    2024

  51. [59]

    Jiwen Yu, Yinhuai Wang, Chen Zhao, Bernard Ghanem, and Jian Zhang. 2023. Freedom: Training-free energy-guided conditional diffusion model. InProceedings of the IEEE/CVF International Conference on Computer Vision. 23174–23184

    2023

  52. [60]

    Huizhuo Yuan, Zixiang Chen, Kaixuan Ji, and Quanquan Gu. 2024. Self-play fine-tuning of diffusion models for text-to-image generation.Advances in Neural Information Processing Systems37 (2024), 73366–73398

    2024

  53. [61]

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang

  54. [62]

    Xueyi Zhang, Chengwei Zhang, Zheng Li, Xiyu Wang, Siqi Cai, Mingrui Lao, Yan- ming Guo, and Huiping Zhuang. 2026. Rep deep & machine learning: Exemplar- free continual video action recognition via slow-fast collaborative learning. In Proceedings of the AAAI Conference on Artif...

    2026

  55. [63]

    Xueyi Zhang, Peiyin Zhu, Yuan Liao, Xiyu Wang, Mingrui Lao, Siqi Cai, Yanming Guo, and Haizhou Li. 2025. Trustclip: Learning from noisy labels via semantic label verification and trust-aligned gradient projection. InProceedings of the 33rd ACM International Conference on Multi...

    2025

  56. [64]

    Xueyi Zhang, Peiyin Zhu, Chengwei Zhang, Zhiyuan Yan, Jikang Cheng, Mingrui Lao, Siqi Cai, and Yanming Guo. 2025. Generalization-preserved learning: Closing the backdoor to catastrophic forgetting in continual deepfake detection. In2025 IEEE/CVF International Conference on Com...

    2025

  57. [65]

    Ziyi Zhang, Sen Zhang, Yibing Zhan, Yong Luo, Yonggang Wen, and Dacheng Tao

  58. [66]

    Jialun Zhong, Wei Shen, Yanzeng Li, Songyang Gao, Hua Lu, Yicheng Chen, Yang Zhang, Wei Zhou, Jinjie Gu, and Lei Zou. 2025. A comprehensive survey of reward models: Taxonomy, applications, challenges, and future.arXiv preprint arXiv:2504.12328(2025)

    2025 arXiv

  59. [67]

    Linqi Zhou, Yilun Du, and Jiajun Wu. 2021. 3d shape generation and comple- tion through point-voxel diffusion. InProceedings of the IEEE/CVF international conference on computer vision. 5826–5835. PAST: Prompt-Adaptive Sampling Termination for Efficient Diffusion Model Confere...

    2021

  60. [2018]

    InProceedings of the IEEE conference on computer vision and pattern recognition

    The unreasonable effectiveness of deep features as a perceptual metric. InProceedings of the IEEE conference on computer vision and pattern recognition. 586–595

  61. [2022]

    Hierarchical text-conditional image generation with clip latents.arXiv preprint arXiv:2204.061251, 2 (2022), 3

    2022 arXiv

  62. [2023]

    Training diffusion models with reinforcement learning.arXiv preprint arXiv:2305.13301(2023)

    2023 arXiv

  63. [2024]

    InInternational Conference on Machine Learning

    Confronting Reward Overoptimization for Diffusion Models: A Perspective of Inductive and Primacy Biases. InInternational Conference on Machine Learning. PMLR, 60396–60413

  64. [2025]

    InProceedings of the Computer Vision and Pattern Recognition Conference

    RaSS: Improving Denoising Diffusion Samplers with Reinforced Active Sampling Scheduler. InProceedings of the Computer Vision and Pattern Recognition Conference. 12923–12933

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.