Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Test-time Prompt Refinement for Text-to-Image Models

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Test-time prompt refinement—an iterative loop in which a multimodal language model critiques each generated image and rewrites the prompt—improves text-to-image alignment without retraining the generator.

desk verdict Useful MLLM-driven prompt refinement loop with real aggregate gains, but the paper overclaims consistency and its fixed-K algorithm lacks the best-image selection its own figure promises. read the letter →

arxiv 2507.22076 v1 pith:M56ADTKW submitted 2025-07-22 cs.LG

Mohammad Abdul Hafeez Khan , Yash Jain , Siddhartha Bhattacharyya , Vibhav Vineet This is my paper · ORCID
classification cs.LG
keywords test-timerefinementtext-to-imagegenerationmultimodallargelanguagemodelpromptalignmenttraining-freeclosed-loopfeedbackcompositionalreasoningdiffusionmodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Text-to-image models are sensitive to prompt wording, so a small rewording can silently change the output. This paper claims that a closed-loop, training-free process can repair that sensitivity: a multimodal large language model (MLLM) inspects the generated image, compares it with the original prompt, and writes a more explicit prompt aimed at the observed failures; then the image generator tries again. The authors show that a few iterations of this loop improve semantic alignment across the GENEVAL, LLM-Grounded Diffusion, and DrawBench benchmarks, on open and closed image models alike. The idea matters because it offers a path to correcting compositional errors—spatial relations, counting, negation—at inference time, without fine-tuning or model introspection.

What carries the argument

The mechanism is Algorithm 1's fixed-max-iteration closed loop. At each step, the refinement function $F_{\mathrm{MLLM}}(x_{i-1}, \{c_{i-1},\dots,c_0\}, \{f_{i-1},\dots,f_1\})$ performs prompt-image consistency analysis and emits a corrected prompt $c_i$ together with feedback $f_i$; the T2I model then regenerates $x_i = G_\theta(c_i)$. The MLLM's instruction is to focus on what differs from the original prompt, preserve correct elements from earlier refinements, and produce a detailed prompt that makes implicit semantics explicit. This loop converts the image model's prompt sensitivity into a steering mechanism, and the full prompt-and-feedback history is what prevents the corrections from oscillating.

What would settle it

A concrete test would be to run the loop with an MLLM whose visual feedback is deliberately corrupted (for example, by relabeling the image-prompt mismatches), and show that the final image alignment degrades relative to the no-refinement baseline; alternately, one could measure whether the MLLM's consistency judgment at each iteration correlates with human judgments across a diverse prompt set.

Watch

Extended reading notes

Core claim

TIR (Test-time Image Refinement) is an iterative procedure that replaces one-shot text-to-image generation with a feedback loop. Starting from the user's prompt, a T2I model produces an image; an MLLM then evaluates how well the image matches the original prompt, identifies concrete misalignments, and outputs a refined, physically grounded prompt that incorporates the correction history. The image model regenerates from the refined prompt, and the cycle repeats for a fixed number of iterations (three in the main experiments). The central claim is that this history-aware, closed-loop refinement steadily improves prompt-image alignment without any modification of the image generator, and the paper's experiments support that claim with consistent gains across models and benchmarks.

Load-bearing premise

The loop depends on the multimodal language model's visual judgment being accurate enough; if it misreads what is wrong in the image, the rewritten prompt can carry the model farther from the user's intent instead of converging to it.

Editorial extensions

If this is right

  • TIR can be applied to any black-box text-to-image model, including closed APIs, without accessing weights or internal activations.
  • The largest reported gains are in spatial reasoning, counting, and negation tasks, suggesting that these are the failure modes most easily repaired at the prompt level.
  • Stronger language backbones in the base image model amplify the gains, so the method should become more useful as text-to-image models improve.
  • Different MLLMs both improve alignment, with the stronger MLLM giving larger gains, which points to feedback quality as a key resource.
  • A fixed budget of three iterations captures most of the benefit, so the loop can be run at a modest multiple of the base inference cost.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The loop could double as a diagnostic: if a failure mode (e.g., color attribution) does not improve across refinements, that plateau likely reflects a bias of the base generator rather than a prompt-level issue.
  • An adaptive stopping rule that halts when the MLLM's consistency judgment stops improving could replace the fixed three-iteration budget and cut inference cost.
  • Tracking the semantic drift of the rewritten prompts across iterations would give a direct measurement of whether the loop is converging or wandering, and could be used to validate the MLLM's feedback quality.
  • The same history-conditioned refinement idea should transfer to video generation or multi-step image editing, where the consistency check is over time rather than over a single prompt.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes TIR, a closed-loop test-time prompt refinement framework for text-to-image models. Starting from the user prompt, a frozen T2I model generates an image; a pretrained MLLM then analyzes the image against the original prompt, produces a refined prompt, and the T2I model regenerates. This cycle is repeated for a fixed number of iterations K=3 (Algorithm 1). The authors evaluate TIR on the LLM-Grounded Diffusion Benchmark, GENEVAL, and DrawBench using GPT-4o and Qwen-2.5-7B as refiners and Stable Diffusion, Flux, and DALL-E 3 as generators, reporting improved average scores in most settings, with per-category regressions in some attribute and spatial columns. The central claim is that iterative MLLM-guided prompt refinement improves prompt-image alignment without training or modifying the T2I model.

Significance. If the average gains hold up, the contribution is useful: a training-free, plug-and-play method that can improve prompt adherence of black-box T2I models would be a practical tool, and the use of external benchmarks (Mask2Former, OWL-ViT, CLIP, ImageReward, human annotation) is a strength because the headline numbers are not computed by the refinement MLLM itself. The paper also tests multiple T2I backbones and two MLLMs, and the use of fixed seeds is a reproducibility plus. However, the reported per-category regressions and the mismatch between the pipeline diagram and Algorithm 1 mean the claim of 'consistent improvements' is currently stronger than the evidence.

major comments (3)
  1. [Algorithm 1 and Fig. 2] Algorithm 1 (lines 4–9) unconditionally returns x_K, while Fig. 2 includes an 'Image-prompt consistency check' branch with an 'Output: Best Image' path that is not implemented in the algorithm. Section 7 concedes that 'the MLLM itself may sometimes introduce flawed refinements' and that 'these errors can persist over iterations' (Fig. 10c). With no early exit and no best-image selection, the algorithm as written can return an image that is worse than an earlier iteration, so the claim that TIR 'corrects errors' is not guaranteed by the reported pipeline. Please either implement the selection/early-stop mechanism shown in Fig. 2 or report results for the algorithm exactly as written.
  2. [Table 1] The text in Section 6 says TIR yields 'consistent improvements,' but Table 1 shows clear regressions: Attribute accuracy drops from 42.0 to 30.0 on SD-1.5, stays flat at 18.0 on SD-2.1, and drops from 83.0 to 81.8 on Flux and from 73.0 to 71.0 on DALL-E 3; Spatial accuracy drops from 44.5 to 43.0 on SD-2.1 and from 81.5 to 77.0 on Flux. Table 2 also shows Flux Color Attr dropping from 48.00 to 47.00. The aggregate gains do not establish consistent improvement, and the paper should either report per-prompt/per-category error statistics and discuss these regressions explicitly, or substantially weaken the wording of the central claim.
  3. [Section 6, Tables 1–2] No statistical significance tests, confidence intervals, or variance estimates are reported for the main quantitative comparisons. Several changes are small (e.g., Flux Color Attr 48.00 to 47.00, SD-2.1 Attribute 18.0 to 18.0), and without error bars or a paired analysis it is unclear which differences are reliable. Reporting per-prompt paired differences or bootstrap confidence intervals would materially strengthen the evidence for the claimed improvements.
minor comments (5)
  1. [Throughout] The method is called 'Test-time Image Refinement' in Fig. 1 and the algorithm name, but the title and abstract call it 'Test-time Prompt Refinement'; please unify the terminology.
  2. [Section 5.1] 'GENEV AL' is typeset with a space throughout the paper; the standard name is GENEVAL.
  3. [Section 9.1.3] The phrase 'It helps measures perceptual plausibility' should be 'It helps measure perceptual plausibility.'
  4. [References] Several references are incomplete or informal, including [22] (Flux, no author or venue) and [42] (a GitHub repository), which makes it difficult for readers to verify the setup.
  5. [Fig. 10 caption] The caption describes failure cases with Qwen-2.5-7B, but the example in (c) shows an attribute regression that is also relevant to the main text's discussion of 'style dominance'; a pointer from Section 7 to this figure would improve readability.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: TIR's improvements are measured by independent external metrics, and the refinement loop's output is not equated to its input by construction.

full rationale

The paper's load-bearing claim is empirical: a fixed-K loop in which an MLLM rewrites the prompt after inspecting the generated image improves prompt-image alignment. Nothing in Algorithm 1 or Eq. (3) makes that improvement true by definition. The reported gains are evaluated with independent external instruments — Mask2Former (object presence/counting), CLIP ViT-L/14 (attributes), OWL-ViT (LLM-Grounded benchmark), ImageReward and human annotation (DrawBench) — none of which is the refinement MLLM or a function fitted by the paper. The MLLM instruction in Fig. 5 asks the refiner to bring the image closer to the original prompt, but whether the fixed T2I model complies is contingent; the paper's own Sec. 7 concedes that flawed refinements can persist and cause regressions (Fig. 10c). That concession is a limitation on efficacy, not evidence of circularity. The self-citations ([17]-[19], [54]) appear only in related-work enumerations and are not load-bearing; there is no invoked uniqueness theorem and no fitted parameter renamed as a prediction. The inconsistency between Fig. 2's 'Output: Best Image' branch and Algorithm 1's unconditional return of x_K is a correctness concern (the reported aggregate gains may conceal per-prompt regressions), but it does not make the derivation equivalent to its inputs. Accordingly, the circularity score is 0.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

No invented entities are introduced. Two domain assumptions carry the method and one measurement assumption carries the evaluation; K is a hand-tuned free parameter that all headline numbers depend on.

free parameters (1)
  • K, max refinement iterations = 3
    Chosen by 'empirical analysis' on DrawBench human evaluation (Sec. 5.2, Fig. 4) and reused for all models and benchmarks; all reported results depend on this hand-chosen stopping rule.
assumptions (3)
  • domain assumption A multimodal LLM can reliably detect prompt-image misalignments and produce corrections that move the T2I output closer to the original prompt.
    Central to Eq. (3) and Fig. 5. The paper concedes the MLLM can introduce flawed refinements and errors can persist (Sec. 7, Fig. 10).
  • domain assumption The T2I model is prompt-sensitive, so rewritten prompts change the output meaningfully.
    Stated in Sec. 3 and used by Eq. (4); if a model ignores wording, prompt rewriting cannot converge.
  • domain assumption Automatic metrics (Mask2Former, OWL-ViT, CLIP, ImageReward) and binary human scores measure the alignment that TIR targets.
    Evaluation protocol in Sec. 5.1; no validation that these metrics correlate with the MLLM's internal alignment judgments.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Test-time Prompt Refinement for Text-to-Image Models." pith.science (2026). https://pith.science/paper/M56ADTKW

@misc{pith2026250722076,
  author       = {Pith},
  title        = {Pith review of: Test-time Prompt Refinement for Text-to-Image Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/M56ADTKW}},
  note         = {Machine review of arXiv:2507.22076}
}
read the original abstract

Text-to-image (T2I) generation models have made significant strides but still struggle with prompt sensitivity: even minor changes in prompt wording can yield inconsistent or inaccurate outputs. To address this challenge, we introduce a closed-loop, test-time prompt refinement framework that requires no additional training of the underlying T2I model, termed TIR. In our approach, each generation step is followed by a refinement step, where a pretrained multimodal large language model (MLLM) analyzes the output image and the user's prompt. The MLLM detects misalignments (e.g., missing objects, incorrect attributes) and produces a refined and physically grounded prompt for the next round of image generation. By iteratively refining the prompt and verifying alignment between the prompt and the image, TIR corrects errors, mirroring the iterative refinement process of human artists. We demonstrate that this closed-loop strategy improves alignment and visual coherence across multiple benchmark datasets, all while maintaining plug-and-play integration with black-box T2I models.

Figures

Figures reproduced from arXiv: 2507.22076 by the authors.

Figure 1
Figure 1. Effectiveness of the proposed Test-time Image Refinement (TIR) approach in enhancing image generation quality. TIR iteratively refines input prompts based on the generated images to more accurately reflect the given text instructions. For each prompt, the left image shows the original DALL-E 3 output, while the right image presents the refined result produced by our approach (TIR), demonstrating improved content con… view at source ↗
Figure 2
Figure 2. Test-time prompt refinement for Text-to-Image Models [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Illustrative example of TIR. Each row represents a dif￾ferent iteration of our approach. The first row shows the initial in￾put prompt alongside its generated image. In subsequent rows, the left column presents progressively refined prompts, while the right column shows the corresponding images generated from these re￾fined prompts. Notice how the condition of the umbrella evolves with each refinement. For instance,… view at source ↗
Figures from the paper (9 more)
Figure 5
Figure 5. Figure 5: Refinement prompt provided to the MLLM. breaking down complex compositional attributes into struc￾tured semantic components, allowing the model to better capture fine-grained details and spatial relationships. As a result, TIR generates images with higher fidelity, imp…
Figure 4
Figure 4. Figure 4: Human evaluation on DrawBench for K = 1 to 6 shows K = 3 balances quality and cost. Best viewed in zoom. 6. Results Qualitative results [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 6
Figure 6. Figure 6: Qualitative comparison of TIR against the LMD and LMD+ baselines on prompts from the LLM-Grounded Diffusion Benchmark. Each column corresponds to one method’s outputs for the same input prompt. Although LMD and LMD+ are tuned for this benchmark, TIR consistently produc…
Figure 7
Figure 7. Figure 7: Qualitative results of TIR on open-ended prompts from DrawBench. Images on the left are from base model while the right image is after TIR refinement. Even stronger T2I models struggle to disentangle semantic components in complex prompts, TIR extracts capabilities of …
Figure 8
Figure 8. Figure 8: Evaluation on Drawbench dataset. We compare nor￾malized ImageReward [48] score on DALL-E 3 model and observe consistent improvement over multiple tries. The error bars show the std dev. over five runs. Further, we also performed human evaluation for prompt-image correc…
Figure 9
Figure 9. Figure 9: Qualitative comparison with different MLLMs. Input prompt - a photo of a train above a potted plant. While Qwen-2.5- 7B produces a semantically correct output, GPT-4o optimizes the prompt to produce a more natural image. Specifically, we use the Qwen-2.5-7B MLLM [44, 4…
Figure 10
Figure 10. Figure 10: Failure cases using DALL-E 3 as the T2I model and [PITH_FULL_IMAGE:figures/full_fig_p008_10.png]
Figure 11
Figure 11. Figure 11: Qualitative results on Drawbench using DALL-E 3 as the T2I model and GPT-4o as the MLLM. Each row shows the prompt refinement trajectory, beginning with the initial prompt and generation on the left, followed by three rounds of GPT-4o-guided refinements. The results d…
Figure 12
Figure 12. Figure 12: Qualitative results on Drawbench using DALL-E 3 as the T2I model and GPT-4o as the MLLM. Each row shows the prompt refinement trajectory, beginning with the initial prompt and generation on the left, followed by three rounds of GPT-4o-guided refinements. The results d…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. DuET: Dual Expert Trajectories for Diffusion Image Editing

    cs.CV 2026-06 unverdicted novelty 6.0 of 10

    Switching a diffusion editor from image-conditioned to caption-only mode for a mid-trajectory interval and back improves edit fidelity and naturalness on FLUX2-Klein and BAGEL, while predictably reducing source-image ...

Reference graph

Works this paper leans on

59 extracted references · 35 canonical work pages · cited by 1 Pith paper

  1. [1]

    Flamingo: a visual language model for few-shot learning

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Men- sch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems, 35:23716–23736,

  2. [2]

    Blended latent diffusion

    Omri Avrahami, Ohad Fried, and Dani Lischinski. Blended latent diffusion. ACM Transactions on Graphics (Proc. SIG- GRAPH), 42(4), 2023. 1

    work page 2023

  3. [3]

    ediff-i: Text-to-image dif- fusion models with an ensemble of expert denoisers

    Yogesh Balaji, Seungjun Nah, Xun Huang, Arash Vahdat, Ji- aming Song, Qinsheng Zhang, Karsten Kreis, Miika Aittala, Timo Aila, Samuli Laine, et al. ediff-i: Text-to-image dif- fusion models with an ensemble of expert denoisers. arXiv preprint arXiv:2211.01324, 2022. 2

    arXiv 2022

  4. [4]

    Multidiffusion: Fusing diffusion paths for controlled image generation

    Omer Bar-Tal, Lior Yariv, Yaron Lipman, and Tali Dekel. Multidiffusion: Fusing diffusion paths for controlled image generation. 2023. 6, 7

    work page 2023

  5. [5]

    Improving image generation with better captions

    James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al. Improving image generation with better captions. Computer Science. https://cdn. openai. com/papers/dall-e-3. pdf, 2(3):8, 2023. 1, 2, 5, 6, 7

    work page 2023

  6. [6]

    Training-free layout control with cross-attention guidance

    Minghao Chen, Iro Laina, and Andrea Vedaldi. Training-free layout control with cross-attention guidance. In Proceedings of the IEEE/CVF winter conference on applications of com- puter vision, pages 5343–5353, 2024. 2, 6, 7

    work page 2024

  7. [7]

    Masked-attention mask transformer for universal image segmentation

    Bowen Cheng, Ishan Misra, Alexander G Schwing, Alexan- der Kirillov, and Rohit Girdhar. Masked-attention mask transformer for universal image segmentation. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1290–1299, 2022. 4

    work page 2022

  8. [8]

    Cogview: Mastering text-to-image generation via transformers

    Ming Ding, Zhuoyi Yang, Wenyi Hong, Wendi Zheng, Chang Zhou, Da Yin, Junyang Lin, Xu Zou, Zhou Shao, Hongxia Yang, et al. Cogview: Mastering text-to-image generation via transformers. Advances in neural information processing systems, 34:19822–19835, 2021. 2

    work page 2021

Show all 59 references
  1. [9]

    Taming transformers for high-resolution image synthesis

    Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12873–12883, 2021. 2

    2021

  2. [10]

    DPOK: Reinforcement learning for fine-tuning text-to-image diffu- sion models

    Ying Fan, Olivia Watkins, Yuqing Du, Hao Liu, Moonkyung Ryu, Craig Boutilier, Pieter Abbeel, Moham- mad Ghavamzadeh, Kangwook Lee, and Kimin Lee. DPOK: Reinforcement learning for fine-tuning text-to-image diffu- sion models. In Advances in Neural Information Processing Systems...

    2023

  3. [11]

    Layoutgpt: Compositional visual plan- ning and generation with large language models

    Weixi Feng, Wanrong Zhu, Tsu-jui Fu, Varun Jampani, Ar- jun Akula, Xuehai He, Sugato Basu, Xin Eric Wang, and William Yang Wang. Layoutgpt: Compositional visual plan- ning and generation with large language models. Advances in Neural Information Processing Systems, 36:18225–18250,

  4. [12]

    Layoutgpt: Compositional visual plan- ning and generation with large language models

    Weixi Feng, Wanrong Zhu, Tsu-jui Fu, Varun Jampani, Ar- jun Akula, Xuehai He, Sugato Basu, Xin Eric Wang, and William Yang Wang. Layoutgpt: Compositional visual plan- ning and generation with large language models. Advances in Neural Information Processing Systems, 36, 2024. 2, 3

    2024

  5. [13]

    Llm blueprint: Enabling text-to-image generation with complex and detailed prompts

    Hanan Gani, Shariq Farooq Bhat, Muzammal Naseer, Salman Khan, and Peter Wonka. Llm blueprint: Enabling text-to-image generation with complex and detailed prompts. arXiv preprint arXiv:2310.10640, 2023. 3

    2023 arXiv

  6. [14]

    Geneval: An object-focused framework for evaluating text- to-image alignment

    Dhruba Ghosh, Hannaneh Hajishirzi, and Ludwig Schmidt. Geneval: An object-focused framework for evaluating text- to-image alignment. Advances in Neural Information Pro- cessing Systems, 36:52132–52152, 2023. 4, 8, 1

    2023

  7. [15]

    Prompt-to-prompt im- age editing with cross attention control

    Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-prompt im- age editing with cross attention control. arXiv preprint arXiv:2208.01626, 2022. 1

    2022 arXiv

  8. [16]

    Gpt-4o system card

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perel- man, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Weli- hinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024. 2

    2024 arXiv

  9. [17]

    Few-shot classification and anatomical localization of tissues in spect imaging

    MAH Khan, SM Boddepalli, S Bhattacharyya, and D Mitra. Few-shot classification and anatomical localization of tissues in spect imaging. In 2024 IEEE Nuclear Science Symposium (NSS), Medical Imaging Conference (MIC) and Room Tem- perature Semiconductor Detector Conference (RTSD...

    2024

  10. [18]

    Clas- sification of microstructure images of metals using transfer learning

    Mohammed Abdul Hafeez Khan, Hrishikesh Sabnis, J An- gel Arul Jothi, J Kanishkha, and AM Deva Prasad. Clas- sification of microstructure images of metals using transfer learning. In International Conference on Modelling and De- velopment of Intelligent Systems , pages 136–147....

    2022

  11. [19]

    Alina: Advanced line identification and notation algorithm

    Mohammed Abdul Hafeez Khan, Parth Ganeriwala, Sid- dhartha Bhattacharyya, Natasha Neogi, and Raja Muthalagu. Alina: Advanced line identification and notation algorithm. In 2024 IEEE/CVF Conference on Computer Vision and Pat- tern Recognition Workshops (CVPRW) , pages 7293–7302...

    2024

  12. [20]

    Gen- erating images with multimodal language models

    Jing Yu Koh, Daniel Fried, and Russ R Salakhutdinov. Gen- erating images with multimodal language models. Advances in Neural Information Processing Systems, 36, 2024. 3

    2024

  13. [21]

    Zero- shot text-guided infinite image synthesis with llm guidance

    Soyeong Kwon, Taegyeong Lee, and Taehwan Kim. Zero- shot text-guided infinite image synthesis with llm guidance. arXiv preprint arXiv:2407.12642, 2024. 3

    2024 arXiv

  14. [22]

    Black Forest Labs. Flux. https://github.com/ black-forest-labs/flux, 2024. 1, 2, 5, 6, 7, 8

    2024

  15. [23]

    Aligning text- to-image models using human feedback

    Kimin Lee, Hao Liu, Moonkyung Ryu, Olivia Watkins, Yuqing Du, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, and Shixiang Shane Gu. Aligning text- to-image models using human feedback. arXiv preprint arXiv:2302.12192, 2023. 1

    2023 arXiv

  16. [24]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In In- ternational conference on machine learning , pages 19730– 19742. PMLR, 2023. 3

    2023

  17. [25]

    Gligen: Open-set grounded text-to-image generation

    Yuheng Li, Haotian Liu, Qingyang Wu, Fangzhou Mu, Jian- wei Yang, Jianfeng Gao, Chunyuan Li, and Yong Jae Lee. Gligen: Open-set grounded text-to-image generation. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22511–22521, 2023. 2, 6

    2023

  18. [26]

    Llm- grounded diffusion: Enhancing prompt understanding of text-to-image diffusion models with large language models

    Long Lian, Boyi Li, Adam Yala, and Trevor Darrell. Llm- grounded diffusion: Enhancing prompt understanding of text-to-image diffusion models with large language models. arXiv preprint arXiv:2305.13655, 2023. 1, 2, 3, 4, 5, 6, 7

    2023 arXiv

  19. [27]

    Videodirectorgpt: Consistent multi-scene video generation via llm-guided planning

    Han Lin, Abhay Zala, Jaemin Cho, and Mohit Bansal. Videodirectorgpt: Consistent multi-scene video generation via llm-guided planning. arXiv preprint arXiv:2309.15091,

    arXiv

  20. [28]

    Inference-time scaling for diffu- sion models beyond scaling denoising steps

    Nanye Ma, Shangyuan Tong, Haolin Jia, Hexiang Hu, Yu- Chuan Su, Mingda Zhang, Xuan Yang, Yandong Li, Tommi Jaakkola, Xuhui Jia, et al. Inference-time scaling for diffu- sion models beyond scaling denoising steps. arXiv preprint arXiv:2501.09732, 2025. 7

    2025 arXiv

  21. [29]

    Phybench: A physical common- sense benchmark for evaluating text-to-image models

    Fanqing Meng, Wenqi Shao, Lixin Luo, Yahong Wang, Yi- ran Chen, Quanfeng Lu, Yue Yang, Tianshuo Yang, Kaipeng Zhang, Yu Qiao, et al. Phybench: A physical common- sense benchmark for evaluating text-to-image models. arXiv preprint arXiv:2406.11802, 2024. 3

    2024 arXiv

  22. [30]

    Simple open-vocabulary object detection

    Matthias Minderer, Alexey Gritsenko, Austin Stone, Maxim Neumann, Dirk Weissenborn, Alexey Dosovitskiy, Aravindh Mahendran, Anurag Arnab, Mostafa Dehghani, Zhuoran Shen, et al. Simple open-vocabulary object detection. In European conference on computer vision , pages 728–755. ...

    2022

  23. [31]

    Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741, 2021

    Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741, 2021. 2

    2021 arXiv

  24. [32]

    Localizing object-level shape variations with text-to-image diffusion models

    Or Patashnik, Daniel Garibi, Idan Azuri, Hadar Averbuch- Elor, and Daniel Cohen-Or. Localizing object-level shape variations with text-to-image diffusion models. In Proceed- ings of the IEEE/CVF International Conference on Com- puter Vision, pages 23051–23061, 2023. 2

    2023

  25. [33]

    Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023. 2

    2023 arXiv

  26. [34]

    Diffusiongpt: Llm-driven text-to-image generation system

    Jie Qin, Jie Wu, Weifeng Chen, Yuxi Ren, Huixia Li, Hefeng Wu, Xuefeng Xiao, Rui Wang, and Shilei Wen. Diffusiongpt: Llm-driven text-to-image generation system. arXiv preprint arXiv:2401.10061, 2024. 3

    2024

  27. [35]

    Layoutllm-t2i: Eliciting layout guidance from llm for text-to-image generation

    Leigang Qu, Shengqiong Wu, Hao Fei, Liqiang Nie, and Tat- Seng Chua. Layoutllm-t2i: Eliciting layout guidance from llm for text-to-image generation. In Proceedings of the 31st ACM International Conference on Multimedia , pages 643– 654, 2023. 2, 3, 5

    2023

  28. [36]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  29. [37]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International confer- ence on machine learning, pages 8821–8831. Pmlr, 2021. 1, 2

    2021

  30. [38]

    Hierarchical text-conditional image gener- ation with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gener- ation with clip latents. arXiv preprint arXiv:2204.06125, 1 (2):3, 2022. 1, 2

    2022 arXiv

  31. [39]

    Generative ad- versarial text to image synthesis

    Scott Reed, Zeynep Akata, Xinchen Yan, Lajanugen Lo- geswaran, Bernt Schiele, and Honglak Lee. Generative ad- versarial text to image synthesis. InInternational conference on machine learning, pages 1060–1069. PMLR, 2016. 2

    2016

  32. [40]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 1, 2, 5, 6, 7

    2022

  33. [41]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information...

    2022

  34. [42]

    Benchmarking awesome diffusion mod- els

    Schuture. Benchmarking awesome diffusion mod- els. https : / / github . com / Schuture / Benchmarking - Awesome - Diffusion - Models ,

  35. [43]

    Df-gan: A simple and effec- tive baseline for text-to-image synthesis

    Ming Tao, Hao Tang, Fei Wu, Xiao-Yuan Jing, Bing-Kun Bao, and Changsheng Xu. Df-gan: A simple and effec- tive baseline for text-to-image synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16515–16525, 2022. 2

    2022

  36. [44]

    Qwen2.5: A party of foundation models, 2024

    Qwen Team. Qwen2.5: A party of foundation models, 2024. 2, 8

    2024

  37. [45]

    Visual chatgpt: Talking, drawing and editing with visual foundation models

    Chenfei Wu, Shengming Yin, Weizhen Qi, Xiaodong Wang, Zecheng Tang, and Nan Duan. Visual chatgpt: Talking, drawing and editing with visual foundation models. arXiv preprint arXiv:2303.04671, 2023. 3

    2023 arXiv

  38. [46]

    Self-correcting llm-controlled diffu- sion models

    Tsung-Han Wu, Long Lian, Joseph E Gonzalez, Boyi Li, and Trevor Darrell. Self-correcting llm-controlled diffu- sion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6327– 6336, 2024. 1, 3

    2024

  39. [47]

    Boxdiff: Text-to-image synthesis with training-free box-constrained diffusion

    Jinheng Xie, Yuexiang Li, Yawen Huang, Haozhe Liu, Wen- tian Zhang, Yefeng Zheng, and Mike Zheng Shou. Boxdiff: Text-to-image synthesis with training-free box-constrained diffusion. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7452–7461, 2023. 6, 7

    2023

  40. [48]

    Imagere- ward: Learning and evaluating human preferences for text- to-image generation

    Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagere- ward: Learning and evaluating human preferences for text- to-image generation. Advances in Neural Information Pro- cessing Systems, 36:15903–15935, 2023. 5, 7

    2023

  41. [49]

    Qwen2 technical report

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jian- wei Zhang, Jianxin Ma, Jin Xu, Jingren Zhou, Jinze Bai, Jinzhe...

    2024 arXiv

  42. [50]

    Paint by example: Exemplar-based image editing with diffusion mod- els

    Binxin Yang, Shuyang Gu, Bo Zhang, Ting Zhang, Xuejin Chen, Xiaoyan Sun, Dong Chen, and Fang Wen. Paint by example: Exemplar-based image editing with diffusion mod- els. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 18381–18391,

  43. [51]

    Reco: Region-controlled text-to-image genera- tion

    Zhengyuan Yang, Jianfeng Wang, Zhe Gan, Linjie Li, Kevin Lin, Chenfei Wu, Nan Duan, Zicheng Liu, Ce Liu, Michael Zeng, et al. Reco: Region-controlled text-to-image genera- tion. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 14246–14255,

  44. [52]

    Stack- gan: Text to photo-realistic image synthesis with stacked generative adversarial networks

    Han Zhang, Tao Xu, Hongsheng Li, Shaoting Zhang, Xiao- gang Wang, Xiaolei Huang, and Dimitris N Metaxas. Stack- gan: Text to photo-realistic image synthesis with stacked generative adversarial networks. In Proceedings of the IEEE international conference on computer vision , p...

    2017

  45. [53]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3836–3847, 2023. 2

    2023

  46. [54]

    Controllable text-to-image generation with gpt-

    Tianjun Zhang, Yi Zhang, Vibhav Vineet, Neel Joshi, and Xin Wang. Controllable text-to-image generation with gpt-

  47. [55]

    Dm-gan: Dynamic memory generative adversarial networks for text- to-image synthesis

    Minfeng Zhu, Pingbo Pan, Wei Chen, and Yi Yang. Dm-gan: Dynamic memory generative adversarial networks for text- to-image synthesis. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 5802–5810, 2019. 2 Test-time Prompt Refinement fo...

    2019

  48. [56]

    arXiv preprint arXiv:2305.18583, 2023. 3

    2023 arXiv

  49. [58]

    Benchmark Datasets We use three benchmark datasets to assess compositional fidelity, prompt comprehension, and generalization: 9.1.1

    Dataset and Evaluation Details 9.1. Benchmark Datasets We use three benchmark datasets to assess compositional fidelity, prompt comprehension, and generalization: 9.1.1. GENEV AL: Compositional Accuracy GENEV AL [14] consists of 553 prompts testing object pres- ence, count, co...

  50. [59]

    Qualitative Results Figs. 11 and 12 showcase qualitative results of our Test-time Iterative Refinement (TIR) method on Drawbench prompts, using GPT-4o as the MLLM for prompt refinement and DALL-E 3 as the text-to-image (T2I) generator. Each row illustrates the initial prompt a...

  51. [2023]

    Accessed: March 5, 2025. 4, 1

    2025

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.