Pith. sign in

REVIEW 3 major objections 4 minor 36 references

Optimizing the null-text embedding aligns diffusion models to new rewards without retraining and without reward hacking.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review

2026-08-03 20:06 UTC pith:Z2WEBEFO

load-bearing objection Useful empirical result with an overblown mechanism story; deserves review, not desk reject. the 3 major comments →

arxiv 2511.20889 v2 pith:Z2WEBEFO submitted 2025-11-25 cs.CV cs.AIcs.LG

Test-Time Alignment of Text-to-Image Diffusion Models via Null-Text Embedding Optimisation

Taehoon Kim , Henry Gouk , Timothy Hospedales This is my paper
classification cs.CV cs.AIcs.LG
keywords test-time alignmentdiffusion modelstext-to-imageclassifier-free guidancenull-text embeddingreward hackingreward optimizationinference-time adaptation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces Null-TTA, a test-time alignment method that optimizes the null-text (unconditional) embedding used in classifier-free guidance instead of latents or noise. The claim is that this embedding lives on a structured semantic manifold, so reward optimization moves the generative distribution in coherent semantic directions and prevents reward hacking—exploiting non-semantic noise to game the reward. Since the null embedding anchors the conditional generation, steering it reshapes the model's distribution toward the reward even though no parameters are updated. The paper reports state-of-the-art target reward scores with strong generalization to held-out rewards, efficient memory use, and an extension to non-differentiable rewards via zeroth-order gradient estimation. A sympathetic reader would care because it suggests a simple, training-free route to aligning deployed text-to-image models.

Core claim

Null-TTA claims that test-time alignment of text-to-image diffusion models can be achieved by optimizing only the null-text embedding in the classifier-free guidance formula, rather than latent or noise variables. Because the text-embedding space is semantically structured, updates are constrained to meaningful directions, which the paper argues prevents reward hacking. Because the null embedding anchors the model's generative distribution, optimizing it steers the whole distribution toward the reward-conditioned target, even with frozen weights. The paper derives a KL-regularized objective (reward minus trajectory-level KL between denoising processes and embedding KL) that has a closed-form

What carries the argument

The central object is the null-text embedding, the unconditional embedding in classifier-free guidance, which serves as the geometric anchor of the conditional generative distribution. Null-TTA promotes the null-text embedding as an optimizable variable and maximizes a Monte-Carlo estimate of the reward on Tweedie-estimated clean images, penalized by (i) a closed-form KL divergence between the original and optimized denoising trajectories and (ii) a Gaussian KL on the embedding itself. The structured, low-dimensional semantic space of the text encoder is the mechanism claimed to deliver implicit manifold regularization; a greedy particle filter over latent candidates adds search without movi

Load-bearing premise

The central assumption is that the null-text embedding space is semantically structured in a way that inherently prevents reward hacking, rather than the KL regularization strength being the true cause of the observed generalization.

What would settle it

Run the same KL-regularized optimization on a random projection of the null-text embedding (or on a noise-space variable) with the same annealing schedule; if cross-reward scores remain as high as with the true null embedding, the semantic-manifold claim is not the operative mechanism.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • If correct, any text-to-image diffusion model using classifier-free guidance can be aligned to a new reward at inference time with no parameter updates.
  • Alignment improves target reward while held-out quality metrics degrade much less than with noise-space optimization, so reward hacking is mitigated in practice.
  • Because only the null embedding is optimized, backpropagation passes through cross-attention rather than the full denoising network, lowering memory and enabling efficient scaling of optimization intensity.
  • Zeroth-order gradient estimation allows the same procedure to align non-differentiable rewards (e.g., compressibility), broadening applicability to scientific objectives.
  • The method can trade off target reward against generalization by adjusting optimization intensity, giving a controllable Pareto frontier.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the semantic-manifold explanation is the mechanism, the benefit should increase with the semantic richness of the embedding space; a testable extension is to compare null-embedding optimization in text, image, or combined embedding spaces.
  • The same principle may transfer to other conditioning signals (e.g., audio or visual prompts), suggesting a general recipe for inference-time alignment of conditional generative models.
  • A critical unresolved question is whether the observed cross-reward preservation comes from the structure of the embedding space or simply from strong KL regularization; optimizing a random projection of the null embedding under the same KL objective would settle this.
  • From a safety standpoint, this opens an inference-time mitigation for deployed models, but also implies the null embedding is an attack surface: small semantic-space perturbations could steer outputs toward unwanted rewards without touching weights.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes Null-Text Test-Time Alignment (Null-TTA), a training-free method that aligns text-to-image diffusion models to a reward by optimizing the null-text embedding in classifier-free guidance rather than latent or noise variables. The objective combines the target reward with a per-step KL divergence and an L2 penalty to the original embedding, and is optimized online during denoising; a greedy particle filter selects among K candidates. Experiments on SD-v1.5 and SDXL compare against DNO, DAS, DPS, TDPO, and DSearch (the latter only in related work), reporting target and held-out cross-rewards. The paper claims that optimization in the structured semantic space of text embeddings prevents reward hacking, and reports state-of-the-art target reward with superior cross-reward generalization.

Significance. If validated, the central idea is significant: it offers a new axis for test-time alignment—the text-conditioning space—and a training-free method that avoids the typical reward-hacking trade-off. The evaluation design is a real strength: measuring generalization to held-out rewards directly targets over-optimization, and the use of multiple rewards, SDXL, and a non-differentiable reward adds robustness. The paper ships a clear algorithm and reproduces competitors from official implementations. However, the causal claim about the semantic manifold is not yet isolated from the strong KL/L2 regularization, which is the main risk to the paper's novelty.

major comments (3)
  1. [Section 3.3, Eq. (22)] The closed-form KL is inconsistent with the preceding definitions. From Eqs. (18)-(20), the conditional mean difference is μ_i(φ')-μ_i(φ) = -[(1-α_i)/√(α_i(1-ᾱ_i))] (ε̃(x_i,φ')-ε̃(x_i,φ)). With p(x_{i-1}|x_i,φ')=N(μ_i,σ_i²I), the KL is (1-α_i)²/(2σ_i²α_i(1-ᾱ_i)) ||ε̃'-ε̃||². Eq. (22) instead writes (1-α_i)/(2α_i(1-ᾱ_i)) ||...||², which only holds if σ_i²=1-α_i. The paper never defines σ_i² in Eq. (18). Please state the variance assumption and, if σ_i² ≠ 1-α_i, correct Eq. (22) and Eqs. (24)-(25). If σ_i² is intended to be 1-α_i, say so explicitly.
  2. [Section 3.3 and Section 4.5, Table 5] The paper's headline claim is that optimizing the null-text embedding prevents reward hacking because the text-embedding space is a structured semantic manifold (Abstract, Sec. 3.3). However, the objective (Eq. 25) also contains a per-step trajectory KL and an L2 penalty ∥φ'-φ∥²/(2σ_φ²). Table 5 shows that increasing either regularizer (larger λ2 or smaller σ_φ²) improves cross-reward generalization while reducing target reward—the same trade-off attributed to the semantic manifold. The paper never varies the optimization variable while holding the regularizers fixed, e.g., optimizing a random linear projection of the null-text embedding or a noise-space vector under the same Eq. (25) loss. Without this counterfactual, the evidence is equally consistent with the hypothesis that the KL/L2 regularization alone, rather than the 'semantic manifold,' is responsible for avoiding reward hacking
  3. [Sections 4.1-4.2 and Table 1] The claim of state-of-the-art is not fully supported by the comparison set. DSearch [18] is described in Related Work (Sec. 2.1) but is absent from the main quantitative results (Tables 1-2, Fig. 1). Without DSearch, 'state-of-the-art' is only relative to DNO and DAS. Please include DSearch or justify its exclusion. Also, Sec. 4.1 gives per-reward hyperparameters (λ1, σ_φ²) but no description of how they were selected; if they were tuned on the HPDv2 prompts used for evaluation, the target-reward results may be optimistically biased. State the selection procedure or provide a validation split.
minor comments (4)
  1. [Eq. (25) and Algorithm 1] Eq. (25) uses λ2 for both regularizers, while Algorithm 1 uses the annealed weight \tilde{λ}_{2,t} for the L2 term as well. Clarify whether the embedding prior is also annealed and fix the parenthesis mismatch in Eq. (25).
  2. [Sec. 4.2, Sec. 3.2, Sec. 4.1] Typographical errors: 'cosntrains' should be 'constrains', 'This them prone' should be 'This makes them prone', 'Tweeidie's' should be 'Tweedie's'. Also, Fig. 1 does not identify which star corresponds to TDPO/DPS; add a legend.
  3. [Eq. (13)] In Eq. (13), p(x_T|φ') is replaced by p(x_T). This is true because x_T is independent of the text embedding, but the assumption should be stated explicitly, as it is load-bearing for the factorisation.
  4. [Supplementary, Table 6] The SDXL results are reported only for the PickScore target, with no SDXL baselines for other targets. If space allows, a brief statement on SDXL across targets would strengthen the generality claim.

Circularity Check

0 steps flagged

No significant circularity: the empirical claims are evaluated on held-out rewards and the objective follows a standard KL-regularized derivation.

full rationale

The paper's central empirical claim—that Null-TTA achieves strong target-reward optimization while preserving cross-reward generalization—is tested against held-out metrics (PickScore, HPSv2, Aesthetic, ImageReward) that are not used in the optimization objective. The derivation from the KL-regularized reward maximization in Eq. (8) to the practical objective in Eq. (25) is a standard closed-form computation for Gaussian diffusion transitions plus an L2 penalty on the embedding; no fitted parameter is renamed as a prediction. The null-text embedding itself is inherited from external prior work (Null-Text Inversion, [19]), and no load-bearing argument reduces to a self-citation by the present authors. The 'structured semantic manifold' language in the Abstract and Sec. 3.3 is an interpretive mechanism claim rather than a derived result; the concern that the same behavior could be obtained by equally strong regularization in noise space is an underdetermination/correctness issue, not a circularity, because the paper does not define the semantic-space property in terms of the outcome it claims to explain. Therefore no step in the claimed derivation chain is equivalent to its inputs by construction.

Axiom & Free-Parameter Ledger

8 free parameters · 5 axioms · 0 invented entities

The paper does not introduce new particles, forces, or physical entities. Its free parameters are the usual reward-regularization hyperparameters, plus the annealing schedule and particle count. The main unstated assumption is the semantic-manifold property of the null-text space, which is the load-bearing premise of the 'new paradigm' claim but is never independently validated.

free parameters (8)
  • λ₁ (reward weight) = 100 (HPSv2/PickScore), 2 (Aesthetic), 50 (JPEG)
    Set per target reward; directly controls reward optimization magnitude and is tuned for best reported scores.
  • λ₂ (KL regularizer weight) = 0.002
    Annealed as max(2^{-(1+γ)^{T-t}},0); chosen by hand to balance target reward vs cross-reward; ablation in Table 5 shows sensitivity.
  • σ_φ² (embedding prior variance) = 0.01
    Sets scale of the embedding KL penalty; varied in Table 5 (0.002–0.05) to change regularization strength.
  • γ (annealing growth rate) = 0.008
    Controls both optimization-step growth and regularizer decay; ablation in Table 4.
  • n_min, n_max (optimization steps bounds) = n_min=5; n_max swept {25,...,115}
    n_max determines the Pareto trade-off curve; sweeping it is how 'state-of-the-art' Pareto front is demonstrated.
  • K (particle count) = 3 (default)
    Number of latent candidates per step; ablation in Table 3 shows performance relatively stable across K.
  • learning rate = 0.01
    Adam optimizer step size for null-text embedding updates.
  • Zeroth-order gradient parameters μ, K for non-differentiable rewards = μ=0.02, K=4
    Perturbation scale and sample count for JPEG compressibility target; not used in the main differentiable-reward experiments.
axioms (5)
  • domain assumption The optimal aligned distribution takes the form p_tar(x) ∝ p_pre(x) exp(r(x)/α), an entropy/KL-regularized reward maximization.
    Invoked in Sec. 3.2, Eq. (9), following [23]. Standard in reward alignment, but assumes the KL-regularized objective is the right alignment target.
  • ad hoc to paper The null-text embedding space is a 'semantic manifold' whose directions correspond to coherent semantic changes, and optimizing there prevents non-semantic reward hacking.
    Central to the paper's contribution; never formally defined or measured. No independent evidence is provided that this property holds in CLIP text-embedding space.
  • standard math The initial noise x_T is independent of the conditioning embedding φ', so p(x_T|φ')=p(x_T) in the KL decomposition.
    Used in Eq. (13). True for the prior, but the method optimizes φ' online while holding x_T fixed, so the per-trajectory measure is not exactly the one decomposed.
  • domain assumption Denoising transition distributions p(x_{t-1}|x_t, φ) are Gaussian with identical variance σ_t² for both φ and φ', enabling the simplified KL of Eq. (22).
    Assumed in Eq. (18)-(22). Standard for DDPM but the coefficient in Eq. (22) does not match the standard 1/(2σ²) scaling with the stated σ_t.
  • domain assumption Null-text embedding priors are modeled as Gaussians p(φ)=N(φ, σ_φ²I) and p(φ')=N(φ', σ_φ²I), yielding the closed-form KL of Eq. (23).
    Stated as 'for intuitive interpretation' in Sec. 3.3; the paper optimizes the mean directly and does not sample.

reviewed 2026-08-03 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Test-Time Alignment of Text-to-Image Diffusion Models via Null-Text Embedding Optimisation." pith.science (2026). https://pith.science/paper/Z2WEBEFO

@misc{pith2026251120889,
  author       = {Pith},
  title        = {Pith review of: Test-Time Alignment of Text-to-Image Diffusion Models via Null-Text Embedding Optimisation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Z2WEBEFO}},
  note         = {Machine review of arXiv:2511.20889}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Test-time alignment (TTA) aims to adapt models to specific rewards during inference. However, existing methods tend to either under-optimise or over-optimise (reward hack) the target reward function. We propose Null-Text Test-Time Alignment (Null-TTA), which aligns diffusion models by optimising the unconditional embedding in classifier-free guidance, rather than manipulating latent or noise variables. Due to the structured semantic nature of the text embedding space, this ensures alignment occurs on a semantically coherent manifold and prevents reward hacking (exploiting non-semantic noise patterns to improve the reward). Since the unconditional embedding in classifier-free guidance serves as the anchor for the model's generative distribution, Null-TTA directly steers model's generative distribution towards the target reward rather than just adjusting the samples, even without updating model parameters. Thanks to these desirable properties, we show that Null-TTA achieves state-of-the-art target test-time alignment while maintaining strong cross-reward generalisation. This establishes semantic-space optimisation as an effective and principled novel paradigm for TTA.

Figures

Figures reproduced from arXiv: 2511.20889 by Henry Gouk, Taehoon Kim, Timothy Hospedales.

Figure 1
Figure 1. Figure 1: Evaluation of reward-optimisation (x-axes) vs over-optimisation (generalisation to held-out rewards on y-axes). Top/Bottom rows [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Qualitative comparison on six challenging categories—counting ( [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Multi-objective optimisation using Rmulti = w ⋅ PickScore + (1 − w) ⋅ HPSv2. Each marker corresponds to a dif￾ferent weight w. Null-TTA consistently achieves a superior trade￾off curve: when increasing PickScore, its HPSv2 degradation is smaller than DAS, and when increasing HPSv2, its PickScore degradation is also milder. This bidirectional advantage demon￾strates that Null-TTA provides more efficient and… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

36 extracted references · 10 linked inside Pith

  1. [1]

    Training diffusion models with reinforce- ment learning

    Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine. Training diffusion models with reinforce- ment learning. InInternational Conference on Learning Representations (ICLR), 2024. 1, 2

    2024

  2. [2]

    McCann, Marc Klasky, and Jong Chul Ye

    Hyungjin Chung, Jeongsol Kim, Michael T. McCann, Marc Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. InInternational Conference on Learning Representations (ICLR), 2023. 1, 2, 3, 5

    2023

  3. [3]

    Directly fine-tuning diffusion models on differentiable re- wards.arXiv preprint arXiv:2309.17400, 2023

    Kevin Clark, Paul Vicol, Kevin Swersky, and David J Fleet. Directly fine-tuning diffusion models on differentiable re- wards.arXiv preprint arXiv:2309.17400, 2023. 1, 2

    Pith/arXiv arXiv 2023

  4. [4]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. InAdvances in Neural Infor- mation Processing Systems (NeurIPS), 2021. 1

    2021

  5. [5]

    Raft: Reward ranked finetuning for generative foundation model alignment.arXiv preprint arXiv:2304.06767, 2023

    Hanze Dong, Wei Xiong, Deepanshu Goyal, Yihan Zhang, Winnie Chow, Rui Pan, Shizhe Diao, Jipeng Zhang, Kashun Shum, and Tong Zhang. Raft: Reward ranked finetuning for generative foundation model alignment.arXiv preprint arXiv:2304.06767, 2023. 1, 2

    Pith/arXiv arXiv 2023

  6. [6]

    Diffusion posterior sampling for linear inverse problem solving: A filtering perspective

    Zehao Dou and Yang Song. Diffusion posterior sampling for linear inverse problem solving: A filtering perspective. InThe Twelfth International Conference on Learning Repre- sentations, 2024. 1, 2, 3

    2024

  7. [7]

    Dpok: Reinforcement learning for fine-tuning text-to-image diffu- sion models.Advances in Neural Information Processing Systems, 36:79858–79885, 2023

    Ying Fan, Olivia Watkins, Yuqing Du, Hao Liu, Moonkyung Ryu, Craig Boutilier, Pieter Abbeel, Moham- mad Ghavamzadeh, Kangwook Lee, and Kimin Lee. Dpok: Reinforcement learning for fine-tuning text-to-image diffu- sion models.Advances in Neural Information Processing Systems, 36:79858–79885, 2023. 1, 2

    2023

  8. [8]

    Scaling laws for reward model overoptimization

    Leo Gao, John Schulman, and Jacob Hilton. Scaling laws for reward model overoptimization. InInternational Conference on Machine Learning, pages 10835–10866. PMLR, 2023. 3

    2023

  9. [9]

    Initno: Boosting text-to-image diffu- sion models via initial noise optimization

    Xiefan Guo, Jinlin Liu, Miaomiao Cui, Jiankai Li, Hongyu Yang, and Di Huang. Initno: Boosting text-to-image diffu- sion models via initial noise optimization. InCVPR, 2024. 1, 2, 3

    2024

  10. [10]

    Mani- fold preserving guided diffusion

    Yutong He, Naoki Murata, Chieh-Hsin Lai, Yuhta Takida, Toshimitsu Uesaka, Dongjun Kim, Wei-Hsiang Liao, Yuki Mitsufuji, J Zico Kolter, Ruslan Salakhutdinov, et al. Mani- fold preserving guided diffusion. InICLR, 2024. 1, 2, 3

    2024

  11. [11]

    Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598, 2022

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598, 2022. 1, 3

    Pith/arXiv arXiv 2022

  12. [12]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. InAdvances in Neural Informa- tion Processing Systems (NeurIPS), 2020. 1, 2

    2020

  13. [13]

    Test- time alignment of diffusion models without reward over- optimisation

    Sunwoo Kim, Minkyu Kim, and Dongmin Park. Test- time alignment of diffusion models without reward over- optimisation. InInternational Conference on Learning Rep- resentations (ICLR), 2025. 1, 2, 3, 5, 6, 7, 11

    2025

  14. [14]

    Adam: A method for stochastic opti- mization.arXiv preprint arXiv:1412.6980, 2014

    Diederik P Kingma. Adam: A method for stochastic opti- mization.arXiv preprint arXiv:1412.6980, 2014. 5

    Pith/arXiv arXiv 2014

  15. [15]

    Pick-a-pic: An open dataset of user preferences for text-to-image generation.Ad- vances in neural information processing systems, 36:36652– 36663, 2023

    Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Ma- tiana, Joe Penna, and Omer Levy. Pick-a-pic: An open dataset of user preferences for text-to-image generation.Ad- vances in neural information processing systems, 36:36652– 36663, 2023. 5, 6, 7, 11

    2023

  16. [16]

    Aligning text- to-image models using human feedback.arXiv preprint arXiv:2302.12192, 2023

    Kimin Lee, Hao Liu, Moonkyung Ryu, Olivia Watkins, Yuqing Du, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, and Shixiang Shane Gu. Aligning text- to-image models using human feedback.arXiv preprint arXiv:2302.12192, 2023. 1, 2

    Pith/arXiv arXiv 2023

  17. [17]

    Diffusion-lm improves control- lable text generation.Advances in neural information pro- cessing systems, 35:4328–4343, 2022

    Xiang Li, John Thickstun, Ishaan Gulrajani, Percy S Liang, and Tatsunori B Hashimoto. Diffusion-lm improves control- lable text generation.Advances in neural information pro- cessing systems, 35:4328–4343, 2022. 1

    2022

  18. [18]

    Dynamic search for inference-time alignment in diffusion models.arXiv preprint arXiv:2503.02039, 2025

    Xiner Li, Masatoshi Uehara, Xingyu Su, Gabriele Scalia, Tommaso Biancalani, Aviv Regev, Sergey Levine, and Shui- wang Ji. Dynamic search for inference-time alignment in diffusion models.arXiv preprint arXiv:2503.02039, 2025. 1, 2, 3, 5

    Pith/arXiv arXiv 2025

  19. [19]

    Null-text inversion for editing real im- ages using guided diffusion models

    Ron Mokady, Amir Hertz, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Null-text inversion for editing real im- ages using guided diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6038–6047, 2023. 1

    2023

  20. [20]

    Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis.arXiv preprint arXiv:2307.01952, 2023

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis.arXiv preprint arXiv:2307.01952, 2023. 11

    Pith/arXiv arXiv 2023

  21. [21]

    Aligning text-to-image diffusion models with reward backpropagation.arXiv preprint arXiv:2310.03739, 2023

    Mihir Prabhudesai, Anirudh Goyal, Deepak Pathak, and Katerina Fragkiadaki. Aligning text-to-image diffusion models with reward backpropagation.arXiv preprint arXiv:2310.03739, 2023. 1, 2

    Pith/arXiv arXiv 2023

  22. [22]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. InInternational conference on machine learning, pages 8748–8763. PmLR, 2021. 3

    2021

  23. [23]

    Manning, and Chelsea Finn

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christo- pher D. Manning, and Chelsea Finn. Direct preference optimisation: Your language model is secretly a reward model. InAdvances in Neural Information Processing Sys- tems (NeurIPS), 2024. 3

    2024

  24. [24]

    Test-time scaling of diffusion models via noise trajectory search.arXiv preprint arXiv:2506.03164, 2025

    Vignav Ramesh and Morteza Mardani. Test-time scaling of diffusion models via noise trajectory search.arXiv preprint arXiv:2506.03164, 2025. 1, 2, 3 9

    Pith/arXiv arXiv 2025

  25. [25]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022. 1, 3, 5, 11

    2022

  26. [26]

    Photorealistic text-to-image diffusion models with deep language understanding.Advances in neural information processing systems, 35:36479–36494, 2022

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding.Advances in neural information processing systems, 35:36479–36494, 2022. 1

    2022

  27. [27]

    Laion-ai aesthetic predictor (v1): A linear estimator on top of clip to predict the aesthetic quality of pictures.https://github.com/LAION- AI/aesthetic-predictor, 2022

    Christoph Schuhmann. Laion-ai aesthetic predictor (v1): A linear estimator on top of clip to predict the aesthetic quality of pictures.https://github.com/LAION- AI/aesthetic-predictor, 2022. MIT License. Com- mit 6d122ad (2022-08-15). Accessed: 2025-11-13. 5, 6, 11

    2022

  28. [28]

    Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole

    Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equa- tions. InInternational Conference on Learning Represen- tations (ICLR), 2021. 1, 2

    2021

  29. [29]

    Inference-time alignment of diffusion models with direct noise optimization

    Zhiwei Tang, Jiangweizhi Peng, Jiasheng Tang, Mingyi Hong, Fan Wang, and Tsung-Hui Chang. Inference-time alignment of diffusion models with direct noise optimization. ICML, 2025. 1, 2, 3, 5, 7, 11

    2025

  30. [30]

    Fine- tuning of continuous-time diffusion models as entropy- regularized control.arXiv preprint arXiv:2402.15194, 2024

    Masatoshi Uehara, Yulai Zhao, Kevin Black, Ehsan Haji- ramezanali, Gabriele Scalia, Nathaniel Lee Diamant, Alex M Tseng, Tommaso Biancalani, and Sergey Levine. Fine- tuning of continuous-time diffusion models as entropy- regularized control.arXiv preprint arXiv:2402.15194, 2024. 3

    Pith/arXiv arXiv 2024

  31. [31]

    Diffusion model align- ment using direct preference optimization

    Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. Diffusion model align- ment using direct preference optimization. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8228–8238, 2024. 1, 2

    2024

  32. [32]

    Practical and asymptotically exact conditional sampling in diffusion models.Advances in Neu- ral Information Processing Systems, 36:31372–31403, 2023

    Luhuan Wu, Brian Trippe, Christian Naesseth, David Blei, and John P Cunningham. Practical and asymptotically exact conditional sampling in diffusion models.Advances in Neu- ral Information Processing Systems, 36:31372–31403, 2023. 1, 2, 3

    2023

  33. [33]

    Human preference score: Better aligning text- to-image models with human preference

    Xiaoshi Wu, Keqiang Sun, Feng Zhu, Rui Zhao, and Hong- sheng Li. Human preference score: Better aligning text- to-image models with human preference. InProceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 2096–2105, 2023. 5, 6, 7, 11

    2096

  34. [34]

    Dymo: Training-free diffusion model alignment with dynamic multi-objective scheduling

    Xin Xie and Dong Gong. Dymo: Training-free diffusion model alignment with dynamic multi-objective scheduling. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025. 1, 3

    2025

  35. [35]

    Imagere- ward: Learning and evaluating human preferences for text- to-image generation.Advances in Neural Information Pro- cessing Systems, 36:15903–15935, 2023

    Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagere- ward: Learning and evaluating human preferences for text- to-image generation.Advances in Neural Information Pro- cessing Systems, 36:15903–15935, 2023. 5, 6, 11

    2023

  36. [36]

    Confronting reward overoptimiza- tion for diffusion models: A perspective of inductive and pri- macy biases

    Ziyi Zhang, Sen Zhang, Yibing Zhan, Yong Luo, Yonggang Wen, and Dacheng Tao. Confronting reward overoptimiza- tion for diffusion models: A perspective of inductive and pri- macy biases. InInternational Conference on Machine Learn- ing, pages 60396–60413. PMLR, 2024. 1, 2, 5 10 Test-Time Alignment of Text-to-Image Diffusion Models via Null-Text Embedding O...

    arXiv 2024

This paper was first reviewed by deepseek-v4-flash on August 3, 2026.