Pith. sign in

REVIEW 4 major objections 5 minor 45 references

MCCD: Multi-Agent Collaboration-based Compositional Diffusion for Complex Text-to-Image Generation

T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a training-free wrapper combining multi-agent scene parsing with mask-based hierarchical denoising improves complex text-to-image generation in frozen diffusion models.

desk verdict Plausible training-free plug-in for compositional T2I, but reported gains rest on borrowed baselines and no error bars; deserves a serious referee with required revisions. read the letter →

arxiv 2505.02648 v2 pith:3N4CQY2E submitted 2025-05-05 cs.CV

Mingcheng Li , Xiaolu Hou , Ziyang Liu , Dingkang Yang , Ziyun Qian , Jiawei Chen , Jinjie Wei , Yue Jiang
show 2 more authors
Qingyao Xu Lihua Zhang
This is my paper · ORCID
classification cs.CV
keywords text-to-imagegenerationcomplexscenemulti-agentcollaborationparsingcompositionaldiffusiontraining-freeenhancementattributebindinglayout
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that complex text-to-image prompts—scenes with many objects, attributes, and relations—can be handled better by wrapping an existing frozen diffusion model in a training-free pipeline rather than retraining it. A multi-agent scene parser uses a multimodal large language model to split the prompt into object-plus-attribute prompts, a background prompt, relation descriptions, and depth-ordered bounding boxes, with an evaluator that catches and corrects parsing errors. A hierarchical composition stage then denoises each region in parallel and fuses them through a depth-aware Gaussian mask, regional enhancement, and Gaussian smoothing around box boundaries. On a standard compositional text-to-image benchmark, applying MCCD to SDXL-base raises the complex metric from 0.3130 to 0.3348 and the spatial metric from 0.1971 to 0.2350, for a reported 9.04% overall improvement.

What carries the argument

The load-bearing mechanism is the two-stage pipeline: Multi-agent Collaboration-based Scene Parsing (MCP) plus Hierarchical Compositional Diffusion (HCD). MCP's conductor routes subtasks to six specialized agents, and its evaluator triggers backward corrections, yielding a structured prompt set and a depth-ordered layout as a parse. HCD carries the spatial argument with a Gaussian mask whose weights decay from each bounding-box center (Eq. 10), a depth weight that lets nearer objects win overlapping regions (Eqs. 11–12), regional enhancement that pushes object latents up and background down (Eqs. 14–15), and Gaussian filtering that smooths the box boundary before the result is blended with the complex-prompt latent (Eqs. 16–18).

What would settle it

Re-run SDXL-base and the other base models with the same prompts, seeds, evaluation scripts, and number of images used for the MCCD runs, and check whether the published baseline values reproduce; if the baseline numbers move under identical protocols, the reported deltas cannot be attributed to MCCD.

Watch

Extended reading notes

Core claim

The central claim is that a structured two-module wrapper, MCCD, improves attribute binding, object relationships, layout fidelity, and realism of any frozen diffusion text-to-image model. The scene-parsing module builds a multi-agent system—object extraction, background extraction, action and spatial relation extraction, layout, and aesthetic enhancement—coordinated by a conductor and checked by an evaluator; forward chain-of-thought reasoning and backward feedback convert a single prompt into a structured prompt set plus bounding boxes. The compositional module runs the complex prompt, each object prompt, and the background prompt through the frozen denoiser in parallel at every timestep, resizes object latents to their boxes, fuses overlapping regions with depth-aware Gaussian masks, enhances foreground and suppresses background, and Gaussian-smooths box boundaries before mixing with the full-prompt latent. The paper reports consistent gains on four base models, with the largest on SDXL-base, and attributes the gains to the combination of accurate scene decomposition and hierarchical region-aware denoising.

Load-bearing premise

The reported gains assume the baseline scores were taken from the cited benchmark under the same prompts, seeds, scoring code, and image counts as the authors' MCCD runs; the baselines were not re-run by the authors, so if the protocols differed, the deltas may not measure MCCD alone.

Editorial extensions

If this is right

  • Any frozen diffusion model can adopt MCCD as a plug-in, so future improvements to base generators should compound with the method's gains without retraining.
  • Attribute binding, spatial relations, and overall complex-scene fidelity improve simultaneously rather than as a trade-off: for SDXL-base, the spatial metric rises from 0.1971 to 0.2350 while the complex metric rises from 0.3130 to 0.3348.
  • The scene parser is not tied to one multimodal model; the paper shows the framework works with several, suggesting the parsing quality, not the diffusion backbone, becomes the main lever for further gains.
  • Inference cost grows with the number of detected objects because each object adds a parallel denoising pass and fusion work, so the method trades latency for scene accuracy in dense prompts.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the protocol assumption holds, deployed text-to-image services could adopt MCCD-style wrappers as a low-cost upgrade path, since only the language-model calls and latent-space arithmetic change.
  • The same depth-aware Gaussian fusion could be ported to video generation by treating the layer depth as temporal ordering across frames, which might reduce attribute drift on objects that appear and disappear.
  • A possible cheaper variant would apply the regional enhancement and smoothing directly to cross-attention maps rather than latents, trading a little fusion fidelity for lower memory and latency.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes MCCD, a training-free framework for improving complex text-to-image generation in existing diffusion models. The method has two components: a Multi-agent Collaboration-based scene Parsing (MCP) module that uses MLLMs (GPT-4o-mini) with multiple specialized agents, a conductor, and an evaluator to extract objects, backgrounds, relations, and layouts; and a Hierarchical Compositional Diffusion (HCD) module that fuses object, background, and global prompts in latent space using Gaussian masks, regional enhancement, and Gaussian smoothing. The authors evaluate on T2I-CompBench and report quantitative gains (e.g., SDXL Complex from 0.3130 to 0.3348, and an overall metric improvement of 9.04%) for attribute binding, spatial relations, and complex scenes, supported by qualitative examples and an ablation over five components. The central claim is that MCCD significantly improves baseline diffusion models in a training-free manner.

Significance. If the quantitative claims are solid, MCCD is a practically useful training-free plug-in: it can be applied to any diffusion model, and the qualitative results show visible improvements in layout, attribute binding, and realism. The multi-agent scene parsing design is a nontrivial extension of prior MLLM-based layout methods such as RPG and LMD. The paper also shows some scalability across MLLMs (GPT-4o-mini, GPT-4o, LLaVA) and diffusion models (SD1.5, SD2, SDXL). However, the current evidence is not yet sufficient: the baseline numbers are borrowed from a previous benchmark without shared evaluation conditions, the ablation is qualitative only, no error bars or significance tests are provided, and the closest training-free baselines (RPG, LMD, BoxDiff) are not compared. These gaps prevent the central claim from being accepted as rigorously demonstrated.

major comments (4)
  1. [§4.3, Table 1] The quantitative comparison in Table 1 is not controlled: the baseline rows are explicitly credited to T2I-CompBench [15] (caption: 'Basic data is derived from [15]'), while the +MCCD rows were generated by the authors. The paper does not state that the baselines were rerun under identical seeds, sampler settings, detector versions, scorer implementations, or prompt counts. The claimed gains are small in absolute terms (e.g., SDXL Complex from 0.3130 to 0.3348, a 6.96% relative increase), and such deltas could arise from protocol differences. To support the central claim of significant improvement, the authors must either rerun all baselines under exactly the same evaluation pipeline or provide evidence that their protocol matches the one used to produce the borrowed numbers, including error bars or significance tests.
  2. [§4.4, Figure 3] The ablation study is qualitative only. The text states that removing each component yields visibly degraded images, but no quantitative results are reported on T2I-CompBench. This is insufficient to establish that each component is crucial, especially because the contribution of the MCP module (prompt rewriting and layout generation) cannot be separated from the HCD latent-space operations. A quantitative ablation using the same evaluation protocol as Table 1, with and without MCP and with and without HCD, is needed. The authors should also provide sensitivity analysis for the hyperparameters λpos, λneg, μ, σ, and α in Eqs. (11), (14), (15), (16), and (18), since all are set manually and the paper gives no evidence that the results are robust to their choice.
  3. [§2 and §4.3] The paper does not compare MCCD with the most relevant training-free baselines, even though several are cited in the related work: RPG [39], LMD [20], and BoxDiff [35]. Table 1 includes older methods and fine-tuned models, but not those closest to MCCD in design. Without direct comparison under identical evaluation conditions, the claimed advantage over the existing training-free state of the art is not established. The authors should add these methods to Table 1 or otherwise justify their omission.
  4. [Figure 1, §3.2] The aesthetics enhancement agent can alter the original attributes of objects, as shown in Figure 1: the input prompt contains 'A silver refrigerator', but the agent's output describes 'A sleek, black refrigerator'. This is a concrete semantic inconsistency that could break attribute binding if the generated image reflects the rewritten description. The paper does not explain how the original attributes are preserved or how the downstream evaluation—which uses the original T2I-CompBench prompts—handles such changes. This issue is load-bearing for the claimed improvements in Color and Texture metrics and must be addressed, e.g., by constraining the enhancement to preserve all attribute values or by discussing why the observed gains are not artifacts of this rewriting behavior.
minor comments (5)
  1. [§4.3] The claimed 'overall metric improvement of 9.04%' appears to be the average of the six columns in Table 1 (Color, Shape, Texture, Spatial, Non-Spatial, Complex), rather than the official T2I-CompBench Complex metric. The paper should clarify the aggregation method and also report the Complex metric separately, since that is the benchmark's own composite measure.
  2. [Supplementary, Table 5] The background extraction agent's prompt template says 'As an object extraction agent...', which appears to be a copy-paste error from the object extraction agent's template.
  3. [Supplementary, Table 9] The caption for Table 9 reads 'The prompt template for the layout agent', but the table itself is for the aesthetics enhancement agent. Please correct the caption.
  4. [§4.1] The text says 'Each metric corresponds to 300 prompts, and each prompt yields 10 images' but does not specify whether these are the same 300 prompts as in T2I-CompBench or whether the MLLM-based parsing is run with fixed temperature/seed. Because GPT-4o-mini outputs are stochastic, the paper should describe how the MLLM calls are controlled for reproducibility, or at least acknowledge this source of variance.
  5. [§3.3] Equations (14) and (15) use λpos and λneg on the right-hand side but also use the same symbol for the mask; please disambiguate the notation. Also, the value σ=1.0 in Eq. (16) is stated without explanation; a brief justification of the chosen parameters would improve clarity.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: MCCD's derivation is self-contained, and the quantitative claims are external-benchmark comparisons whose protocol mismatch is a verification caveat, not circular reasoning.

full rationale

The paper's derivation chain is not circular. The claimed contribution is a training-free pipeline whose components, MCP and HCD, are defined by explicit prompt templates and latent-space equations (Eqs. 8-18). The hyperparameters (lambda_pos/lambda_neg = 0.2, mu = 0.8, sigma = 1.0) are fixed constants and are not fitted to the evaluation metrics. No parameter is inferred from T2I-CompBench scores, and neither the MLLM outputs nor the sampled latents are adjusted using the evaluation labels. The comparative claim in Table 1 uses baseline numbers credited to the external T2I-CompBench [15] and +MCCD numbers produced by the authors; the lack of a fully shared evaluation protocol is a genuine correctness and reproducibility risk, but it is not circularity because the baselines are not generated by the method and the method does not optimize toward them. There is no load-bearing self-citation chain, no imported uniqueness theorem, and no ansatz smuggled in via citation: the related-work citations to RPG, BoxDiff, and similar methods are external prior art. The limitation section (Sec 11) concerns inference overhead and does not reveal a circular dependency. Therefore the paper receives a circularity score of 0.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

MCCD introduces no new physical or theoretical entities. Its load-bearing assumptions are that the MLLM multi-agent system parses prompts correctly, that the hand-set fusion constants work across scenes, and that the external benchmark numbers are directly comparable to the authors' runs. The free parameters are all hand-chosen and none are fitted to data, but they are also untested for sensitivity.

free parameters (5)
  • lambda_pos = 0.2
    Regional enhancement positive weight in Eq. 14, hand-set with no sensitivity analysis.
  • lambda_neg = 0.2
    Regional enhancement negative weight in Eq. 15, hand-set.
  • mu = 0.8
    Trade-off between complex prompt latent and smoothed latent in Eq. 18, hand-set.
  • sigma_filter = 1.0
    Standard deviation of Gaussian smoothing kernel in Eq. 16, hand-set.
  • alpha
    Sigmoid steepness in Eq. 11 for layer depth weighting; no value or sensitivity analysis is reported.
assumptions (4)
  • domain assumption A multi-agent system based on GPT-4o-mini can accurately extract objects, attributes, backgrounds, spatial relations, action relations, layouts, and aesthetic descriptions from complex prompts.
    Enters in Sec 3.2; the entire MCP stage relies on this, and the paper provides only qualitative examples.
  • domain assumption The Gaussian mask and depth-weighting scheme with sigma = max(w,h)/2 gives a fusion priority that preserves object identity in overlapping regions.
    Eqs. 10-12 in Sec 3.3; no ablation or alternative fusion rules are tested.
  • domain assumption T2I-CompBench metrics (BLIP-VQA, CLIP, UniDet) are valid proxies for compositional fidelity.
    Used as the sole quantitative evaluation in Sec 4.1.
  • domain assumption The baseline numbers from T2I-CompBench [15] are directly comparable to the authors' +MCCD runs.
    Table 1 mixes published baseline values with new runs; no identical-protocol verification is described.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MCCD: Multi-Agent Collaboration-based Compositional Diffusion for Complex Text-to-Image Generation." pith.science (2026). https://pith.science/paper/3N4CQY2E

@misc{pith2026250502648,
  author       = {Pith},
  title        = {Pith review of: MCCD: Multi-Agent Collaboration-based Compositional Diffusion for Complex Text-to-Image Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3N4CQY2E}},
  note         = {Machine review of arXiv:2505.02648}
}
read the original abstract

Diffusion models have shown excellent performance in text-to-image generation. Nevertheless, existing methods often suffer from performance bottlenecks when handling complex prompts that involve multiple objects, characteristics, and relations. Therefore, we propose a Multi-agent Collaboration-based Compositional Diffusion (MCCD) for text-to-image generation for complex scenes. Specifically, we design a multi-agent collaboration-based scene parsing module that generates an agent system comprising multiple agents with distinct tasks, utilizing MLLMs to extract various scene elements effectively. In addition, Hierarchical Compositional diffusion utilizes a Gaussian mask and filtering to refine bounding box regions and enhance objects through region enhancement, resulting in the accurate and high-fidelity generation of complex scenes. Comprehensive experiments demonstrate that our MCCD significantly improves the performance of the baseline models in a training-free manner, providing a substantial advantage in complex scene generation.

Figures

Figures reproduced from arXiv: 2505.02648 by the authors.

Figure 1
Figure 1. The overall framework of the proposed MCCD. MCCD consists of two core components: Multi-agent Collaboration-based scene Parsing (MCP) module and Hierarchical Compositional Diffusion (HCD) module. In MCP, the blue and green arrows indicate forward CoT reasoning and backward feedback processes, respectively 3.2. Multi-agent Collaboration based Scene Parsing Based on the sufficient consideration of the composition of c… view at source ↗
Figure 2
Figure 2. Qualitative results of MCCD improving diffusion models. MCCD enhances the attribute binding and spatial relationships of the base diffusion models. The generated results have reasonable backgrounds and detailed textures with great aesthetics and realism. decay from the center to the edges, represented as follows: M(x, y) = exp − (x − µx) 2 + (y − µy) 2 2σ 2 ! , (10) where (µx, µy) = (x0 + w/2, y0 + h/2) is the cente… view at source ↗
Figure 3
Figure 3. Ablation results of MCCD. The poor results after removing the critical components prove that each component is crucial. “The soft, warm glow of the campfire illuminated the faces of the hikers, as they roasted marshmallows and swapped stories. A starry sky stretches over a distant mountain range. Nearby, a tent suggests that the group is on a camping adventure.” “The dining area is filled with natural light. A silve… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Additional qualitative results. MCCD can handle complex text prompts with multiple objects and attribute binding relationships effectively, generating reasonable bounding box layouts and producing aesthetically pleasing images with high realism. 4. Experiments 4.1. Dat…
Figure 5
Figure 5. Figure 5: A case to illustrate the workflow of MCP. [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 6
Figure 6. Figure 6: A case to illustrate the workflow of MCP. In the agents’ outputs, orange text indicates errors, and green text indicates corrections. [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 7
Figure 7. Figure 7: Ablation results of MLLMs in MCP [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: Additional qualitative analysis. Chang Zhou, Da Yin, Junyang Lin, Xu Zou, Zhou Shao, Hongxia Yang, et al. Cogview: Mastering text-to-image generation via transformers. Advances in neural information processing systems, 34:19822–19835, 2021. 2 [9] Guian Fang, Zutao Jian…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

45 extracted references · 14 canonical work pages

  1. [15]

    T2i-compbench: A comprehensive bench- mark for open-world compositional text-to-image genera- tion

    Kaiyi Huang, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. T2i-compbench: A comprehensive bench- mark for open-world compositional text-to-image genera- tion. Advances in Neural Information Processing Systems , 36:78723–78747, 2023. 1, 6, 7

    work page 2023

  2. [39]

    Mastering text-to-image dif- fusion: Recaptioning, planning, and generating with multi- modal llms

    Ling Yang, Zhaochen Yu, Chenlin Meng, Minkai Xu, Ste- fano Ermon, and CUI Bin. Mastering text-to-image dif- fusion: Recaptioning, planning, and generating with multi- modal llms. In Forty-first International Conference on Ma- chine Learning, 2024. 2, 3, 4

    work page 2024

  3. [20]

    Llm- grounded diffusion: Enhancing prompt understanding of text-to-image diffusion models with large language models

    Long Lian, Boyi Li, Adam Yala, and Trevor Darrell. Llm- grounded diffusion: Enhancing prompt understanding of text-to-image diffusion models with large language models. arXiv preprint arXiv:2305.13655, 2023. 1, 2

    arXiv 2023

  4. [35]

    Boxdiff: Text-to-image synthesis with training-free box-constrained diffusion

    Jinheng Xie, Yuexiang Li, Yawen Huang, Haozhe Liu, Wen- tian Zhang, Yefeng Zheng, and Mike Zheng Shou. Boxdiff: Text-to-image synthesis with training-free box-constrained diffusion. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7452–7461, 2023. 4 15

    work page 2023

  5. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 ,

    arXiv

  6. [2]

    window”: “large, fills the room, offers a view of the peaceful outdoors

    James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce 12 Conductor Evaluator Prompt Set Object Extraction Agent Background Extraction Agent Spatial Relations Extraction Agent Layout Agent Aesthetics Enhancement Agent “window”: “large, fills the room, offers a view of the peaceful outdoors”, “bed”: “co...

    work page 2023

  7. [3]

    Attend-and-excite: Attention-based se- mantic guidance for text-to-image diffusion models

    Hila Chefer, Yuval Alaluf, Yael Vinker, Lior Wolf, and Daniel Cohen-Or. Attend-and-excite: Attention-based se- mantic guidance for text-to-image diffusion models. ACM Transactions on Graphics (TOG), 42(4):1–10, 2023. 6

    work page 2023

  8. [4]

    Pixart- alpha: Fast training of diffusion transformer for photorealistic text-to-image synthesis

    Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, et al. Pixart- alpha: Fast training of diffusion transformer for photorealistic text-to-image synthesis. arXiv preprint arXiv:2310.00426, 2023. 6

    arXiv

Show all 45 references
  1. [5]

    Llava-interactive: An all-in-one demo for image chat, segmentation, generation and editing

    Wei-Ge Chen, Irina Spiridonova, Jianwei Yang, Jianfeng Gao, and Chunyuan Li. Llava-interactive: An all-in-one demo for image chat, segmentation, generation and editing. arXiv preprint arXiv:2311.00571, 2023. 2

    2023 arXiv

  2. [6]

    Palm: Scaling language modeling with pathways

    Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240): 1–113, 2023. 2

    2023

  3. [7]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural informa- tion processing systems, 34:8780–8794, 2021. 1, 2

    2021

  4. [8]

    A woman in a pink shirt and jeans holds a white umbrella in the rain

    Ming Ding, Zhuoyi Yang, Wenyi Hong, Wendi Zheng, 13 “A woman in a pink shirt and jeans holds a white umbrella in the rain.” SDXL-base+MCCD (w/ GPT-4o-mini) +MCCD (w/ GPT-4o)+MCCD (w/ LLaVA-1.5-7b) ”A glass vase and a metallic watering can are placed beside each other, both fil...

    2021

  5. [9]

    Boosting text-to-image diffusion models with fine-grained semantic rewards

    Guian Fang, Zutao Jiang, Jianhua Han, Guangsong Lu, Hang Xu, and Xiaodan Liang. Boosting text-to-image diffusion models with fine-grained semantic rewards. arXiv preprint arXiv:2305.19599, 5, 2023. 1

    2023 arXiv

  6. [10]

    Training-free structured diffusion guidance for compositional text-to-image synthesis

    Weixi Feng, Xuehai He, Tsu-Jui Fu, Varun Jampani, Arjun Akula, Pradyumna Narayana, Sugato Basu, Xin Eric Wang, and William Yang Wang. Training-free structured diffusion guidance for compositional text-to-image synthesis. arXiv preprint arXiv:2212.05032, 2022. 1, 6

    2022 arXiv

  7. [11]

    Layoutgpt: Compositional visual plan- ning and generation with large language models

    Weixi Feng, Wanrong Zhu, Tsu-jui Fu, Varun Jampani, Ar- jun Akula, Xuehai He, Sugato Basu, Xin Eric Wang, and William Yang Wang. Layoutgpt: Compositional visual plan- ning and generation with large language models. Advances in Neural Information Processing Systems, 36, 2024. 2

    2024

  8. [12]

    Llm blueprint: Enabling 14 text-to-image generation with complex and detailed prompts

    Hanan Gani, Shariq Farooq Bhat, Muzammal Naseer, Salman Khan, and Peter Wonka. Llm blueprint: Enabling 14 text-to-image generation with complex and detailed prompts. arXiv preprint arXiv:2310.10640, 2023. 2

    2023 arXiv

  9. [13]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 2

    2020

  10. [14]

    Ella: Equip diffusion models with llm for enhanced semantic alignment

    Xiwei Hu, Rui Wang, Yixiao Fang, Bin Fu, Pei Cheng, and Gang Yu. Ella: Equip diffusion models with llm for enhanced semantic alignment. arXiv preprint arXiv:2403.05135, 2024. 2

    2024 arXiv

  11. [16]

    Aligning text- to-image models using human feedback

    Kimin Lee, Hao Liu, Moonkyung Ryu, Olivia Watkins, Yuqing Du, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, and Shixiang Shane Gu. Aligning text- to-image models using human feedback. arXiv preprint arXiv:2302.12192, 2023

    2023 arXiv

  12. [17]

    Parrot: Pareto-optimal multi-reward reinforce- ment learning framework for text-to-image generation

    Seung Hyun Lee, Yinxiao Li, Junjie Ke, Innfarn Yoo, Han Zhang, Jiahui Yu, Qifei Wang, Fei Deng, Glenn Entis, Jun- feng He, et al. Parrot: Pareto-optimal multi-reward reinforce- ment learning framework for text-to-image generation. In European Conference on Computer Vision, pag...

    2025

  13. [18]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In In- ternational conference on machine learning , pages 19730– 19742. PMLR, 2023. 2

    2023

  14. [19]

    Gligen: Open-set grounded text-to-image generation

    Yuheng Li, Haotian Liu, Qingyang Wu, Fangzhou Mu, Jian- wei Yang, Jianfeng Gao, Chunyuan Li, and Yong Jae Lee. Gligen: Open-set grounded text-to-image generation. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22511–22521, 2023. 1, 2

    2023

  15. [21]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26296–26306, 2024. 11

    2024

  16. [22]

    Llm4gen: Leveraging semantic representation of llms for text-to-image generation

    Mushui Liu, Yuhang Ma, Yang Zhen, Jun Dan, Yunlong Yu, Zeng Zhao, Zhipeng Hu, Bai Liu, and Changjie Fan. Llm4gen: Leveraging semantic representation of llms for text-to-image generation. arXiv preprint arXiv:2407.00737,

    arXiv

  17. [23]

    Compositional visual generation with composable diffusion models

    Nan Liu, Shuang Li, Yilun Du, Antonio Torralba, and Joshua B Tenenbaum. Compositional visual generation with composable diffusion models. In European Conference on Computer Vision, pages 423–439. Springer, 2022. 1, 6

    2022

  18. [24]

    T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models

    Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, and Ying Shan. T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 4296–4304, 2024. 2

    2024

  19. [25]

    Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741, 2021

    Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741, 2021. 2

    2021 arXiv

  20. [26]

    Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023. 2, 8

    2023 arXiv

  21. [27]

    Layoutllm-t2i: Eliciting layout guidance from llm for text-to-image generation

    Leigang Qu, Shengqiong Wu, Hao Fei, Liqiang Nie, and Tat- Seng Chua. Layoutllm-t2i: Eliciting layout guidance from llm for text-to-image generation. In Proceedings of the 31st ACM International Conference on Multimedia , pages 643– 654, 2023. 1, 2

    2023

  22. [28]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  23. [29]

    Hierarchical text-conditional image gener- ation with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gener- ation with clip latents. arXiv preprint arXiv:2204.06125, 1 (2):3, 2022. 1, 2, 6

    2022 arXiv

  24. [30]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 1, 2, 6, 8

    2022

  25. [31]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information...

    2022

  26. [32]

    Deep unsupervised learning using nonequilibrium thermodynamics

    Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International confer- ence on machine learning, pages 2256–2265. PMLR, 2015. 1, 2

    2015

  27. [33]

    Dreamsync: Aligning text- to-image generation with image understanding feedback

    Jiao Sun, Deqing Fu, Yushi Hu, Su Wang, Royi Rassin, Da-Cheng Juan, Dana Alon, Charles Herrmann, Sjoerd van Steenkiste, Ranjay Krishna, et al. Dreamsync: Aligning text- to-image generation with image understanding feedback. In Synthetic Data for Computer Vision Workshop@ CVPR ...

    2024

  28. [34]

    Galactica: A large language model for science

    Ross Taylor, Marcin Kardas, Guillem Cucurull, Thomas Scialom, Anthony Hartshorn, Elvis Saravia, Andrew Poul- ton, Viktor Kerkez, and Robert Stojnic. Galactica: A large language model for science. arXiv preprint arXiv:2211.09085, 2022. 2

    2022 arXiv

  29. [36]

    Imagere- ward: Learning and evaluating human preferences for text- to-image generation

    Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagere- ward: Learning and evaluating human preferences for text- to-image generation. Advances in Neural Information Pro- cessing Systems, 36, 2024. 1

    2024

  30. [37]

    Baichuan 2: Open large-scale language models

    Aiyuan Yang, Bin Xiao, Bingning Wang, Borong Zhang, Ce Bian, Chao Yin, Chenxu Lv, Da Pan, Dian Wang, Dong Yan, et al. Baichuan 2: Open large-scale language models. arXiv preprint arXiv:2309.10305, 2023. 2

    2023 arXiv

  31. [38]

    Diffusion models: A comprehensive survey of methods and applications

    Ling Yang, Zhilong Zhang, Yang Song, Shenda Hong, Run- sheng Xu, Yue Zhao, Wentao Zhang, Bin Cui, and Ming- Hsuan Yang. Diffusion models: A comprehensive survey of methods and applications. ACM Computing Surveys, 56(4): 1–39, 2023. 1, 2

    2023

  32. [40]

    Cross-modal con- textualized diffusion models for text-guided visual genera- tion and editing

    Ling Yang, Zhilong Zhang, Zhaochen Yu, Jingwei Liu, Minkai Xu, Stefano Ermon, and CUI Bin. Cross-modal con- textualized diffusion models for text-guided visual genera- tion and editing. In The Twelfth International Conference on Learning Representations, 2024. 2

    2024

  33. [41]

    Reco: Region-controlled text-to-image genera- tion

    Zhengyuan Yang, Jianfeng Wang, Zhe Gan, Linjie Li, Kevin Lin, Chenfei Wu, Nan Duan, Zicheng Liu, Ce Liu, Michael Zeng, et al. Reco: Region-controlled text-to-image genera- tion. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 14246–14255,

  34. [42]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3836–3847, 2023. 2

    2023

  35. [43]

    Realcompo: Dynamic equilibrium between realism and compositionality improves text-to-image diffusion models

    Xinchen Zhang, Ling Yang, Yaqi Cai, Zhaochen Yu, Jiake Xie, Ye Tian, Minkai Xu, Yong Tang, Yujiu Yang, and Bin Cui. Realcompo: Dynamic equilibrium between realism and compositionality improves text-to-image diffusion models. arXiv preprint arXiv:2402.12908, 2024. 1

    2024 arXiv

  36. [44]

    Sur-adapter: Enhancing text-to-image pre-trained diffusion models with large language models

    Shanshan Zhong, Zhongzhan Huang, Weushao Wen, Jinghui Qin, and Liang Lin. Sur-adapter: Enhancing text-to-image pre-trained diffusion models with large language models. In Proceedings of the 31st ACM International Conference on Multimedia, pages 567–578, 2023. 2

    2023

  37. [45]

    Minigpt-4: Enhancing vision-language understanding with advanced large language models

    Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mo- hamed Elhoseiny. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592, 2023. 2 16

    2023 arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.