Pith. sign in

REVIEW 5 major objections 5 minor 56 references

ReasonGen-R1: CoT for Autoregressive Image generation models through SFT and RL

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that autoregressive image generators can be trained to write a chain of thought before drawing, and that this two-stage SFT-plus-GRPO recipe improves instruction following on three text-to-image benchmarks.

desk verdict A plausible SFT+GRPO recipe for text-before-image generation in autoregressive models, but the headline gains are measured on the same distributions used to build the training prompts, so the current evidence does not yet support the CoT-driven claim: worth peer review, not a desk reject. read the letter →

arxiv 2505.24875 v2 pith:LNCMTB4Y submitted 2025-05-30 cs.CV cs.CL

Yu Zhang , Yunqi Li , Yifan Yang , Rui Wang , Yuqing Yang , Dai Qi , Jianmin Bao , Dongdong Chen
show 2 more authors
Chong Luo Lili Qiu
This is my paper
classification cs.CVcs.CL
keywords chain-of-thoughtautoregressiveimagegenerationreinforcementlearningGRPOsupervisedfine-tuningtext-to-imagealignmentvision-languagerewardmodeladaptiveentropyloss
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that an autoregressive image generator can be trained to emit a textual chain of thought before it draws, and that this makes the final image follow complex prompts more faithfully. The proposed recipe, ReasonGen-R1, fine-tunes Janus-Pro-7B in two stages: supervised fine-tuning on prompt–rationale–image triples built with GPT-4.1 captions, then Group Relative Policy Optimization in which a vision-language model scores each generated image and the score is propagated back through the entire text-and-image sequence. The reported result is that ReasonGen-R1 beats its base model on GenEval, DPG-Bench, and T2I-Benchmark, with the largest gain on compositional attribute binding. A sympathetic reader would take the central claim to be that reasoning text is a learnable intermediate representation for image generation, one that reinforcement learning can optimize directly.

What carries the argument

The central mechanism is Group Relative Policy Optimization (GRPO) applied to a single interleaved sequence of reasoning text and image tokens. In each rollout the model samples a group of responses; a pretrained vision-language model assigns each image a binary prompt-alignment reward; the rewards are normalized within the group into per-token advantages; and the image-level reward is propagated back through the reasoning tokens, so the policy is rewarded for rationales that lead to good images. Stability comes from an adaptive entropy loss with coefficient $\alpha = \arcsin(\phi)$, allowing positive or negative entropy pressure, with separate entropy targets for text and image tokens, plus DAPO-style batch sub-sampling that drops all-0 and all-1 reward groups.

What would settle it

Run the RL stage with the reward model's scores randomly shuffled or inverted across the group: if the benchmark gains survive, the reasoning text is not being rewarded for image quality. Alternatively, at inference, strip the generated reasoning and feed only the original prompt to the image head: if the images are unchanged, the chain of thought is not causally driving generation.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that a two-stage SFT-plus-GRPO pipeline can turn a standard autoregressive text-to-image model into a 'think-then-draw' model. Supervised fine-tuning teaches the model to interleave a reasoning passage with image tokens in one sequence; then GRPO, using Qwen-2.5-VL-7B as a reward model and an adaptive entropy loss, refines the reasoning so that higher-reward images become more likely. The paper reports gains of 6 percent on GenEval, 1.69 percent on DPG-Bench, and 13.38 percent on T2I-Benchmark over the Janus-Pro-7B baseline, and controlled substitutions inside the chain of thought produce corresponding visual changes, which the authors take as evidence that the reasoning causally guides generation.

Load-bearing premise

The load-bearing premise is that one overall score for the final image can be handed back evenly to every word of the reasoning text and every image pixel-token, so that the training correctly rewards the reasoning steps that actually caused the good image.

Editorial extensions

If this is right

  • Textual reasoning before image generation is trainable in two stages and improves instruction adherence on compositional and dense-prompt benchmarks.
  • A large vision-language reward model is necessary: swapping the 7B rewarder for a 3B one drops GenEval overall from 0.86 to 0.45 in the paper's ablation.
  • Adaptive entropy regularization keeps interleaved text-image RL stable, avoiding the entropy explosion seen without it and the mode collapse seen with a fixed entropy penalty.
  • Editing a phrase in the chain of thought produces a corresponding visual change, suggesting the reasoning is causally entangled with the generated image.
  • The SFT and RL stages are complementary: pure RL underperforms the full pipeline, and SFT alone is not sufficient.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication the paper leaves implicit is that the same two-stage recipe should transfer to other autoregressive multimodal generators, provided the model can switch from text to image tokens within one sequence.
  • If the CoT substitution evidence generalizes, the reasoning text becomes a user-facing control surface: editing lighting, background, or object phrases inside the CoT could steer images with sentence-level edits rather than full prompt rewrites.
  • A testable extension would use a reward model that returns per-aspect scores for count, color, position, and style instead of one binary alignment score; the ablations show reward quality matters, so finer-grained rewards might push the gains further.
  • Because the SFT rationales were generated from captions rather than from the images themselves, the dataset likely inherits caption biases; adding visually grounded rationale generation could be a natural next step.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes ReasonGen-R1, a two-stage framework that first fine-tunes an autoregressive image-generation model (Janus-Pro-7B) with supervised fine-tuning (SFT) on GPT-4.1-generated textual chain-of-thought (CoT) rationales paired with images, and then refines the model with GRPO using a Qwen-2.5-VL reward model and an adaptive entropy loss. The authors evaluate on GenEval, DPG-Bench, and T2I-Benchmark, reporting gains over the Janus-Pro baseline and several prior systems. Ablations examine the contributions of the SFT stage, the RL stage, the entropy loss, and the reward-model size.

Significance. If the central claim were established, the work would be a useful step toward endowing autoregressive image generators with text-based reasoning, and the released 200k CoT-image dataset would be a community resource. The adaptive entropy mechanism for interleaved text-image RL is a sensible engineering idea, and the paper provides extensive qualitative examples. However, as presented, the empirical evidence does not support the claim that CoT reasoning drives the reported improvements: the training prompts are drawn from the evaluation benchmarks, the credit-assignment mechanism is not shown to reward reasoning specifically, and the SFT-only ablation is far below the baseline. These issues are load-bearing, so the current significance is limited.

major comments (5)
  1. [4.1.2] The training prompts for both the SFT and RL stages are constructed from the same benchmarks used for evaluation, so the reported gains may reflect distribution memorization rather than a general effect of CoT+RL. For GenEval, the object vocabulary is enlarged to 308 and exact overlaps with the test set are filtered, but the prompt generator and distribution remain otherwise identical; for DPG-Bench, GPT-4.1 is prompted to produce new prompts by sampling five existing DPG prompts and matching their length and style, with no overlap filter reported; for T2I-Benchmark, the entire T2I-CompBench++ training split is used without modification. Since evaluation is performed on GenEval, DPG-Bench, and T2I-Benchmark, the main tables can be explained by the model learning benchmark-specific prompt statistics. Section 6 acknowledges benchmark specificity but does not address this train/eval overlap.
  2. [3.2.2] The credit-assignment mechanism is not established. The paper states that the image-level reward is "propagated" back through the entire sequence, but the objective in Eq. (6) applies the same normalized group advantage to every token in the prompt–reasoning–image sequence. This does not distinguish reasoning tokens from image tokens, so the RL update could improve image quality through image-token changes alone or through generic text-style shifts, without learning compositional planning in the CoT. A control experiment with the same SFT+RL recipe but without CoT tokens is needed to support the claim that the textual reasoning is the causal driver.
  3. [4.3.1, Table 4] The single-stage ablations undermine the CoT narrative. The w/o RL model (SFT only) scores 0.57 on GenEval, far below the Janus-Pro baseline of 0.80, and the w/o SFT model (RL only) scores 0.68, also below baseline. The full model reaches 0.86 only when SFT is followed by RL on benchmark-derived prompts. This pattern is consistent with RL fitting the evaluation distribution rather than with CoT reasoning improving instruction following. The authors should compare against a model trained with the same SFT and RL procedure but without the textual CoT stage, and should evaluate on a held-out prompt distribution.
  4. [1, Table 3] The claim of being "the first to integrate the reasoning process into autoregressive image generation via a two-stage SFT + GRPO training framework" is contradicted by the cited prior work [14], whose title indicates CoT-based image generation with verification and reinforcement (Show-o + PARM). The paper includes Show-o + PARM as a baseline in Table 3 but provides no discussion of how the proposed framework differs from or improves upon it, and it does not run PARM-style CoT+RL on the current base model. This overstates the novelty of the contribution.
  5. [4.2 / 4.3] No uncertainty quantification is provided for any of the reported scores. All tables and ablations appear to be single runs, and the differences between the full model and baselines (e.g., GenEval 0.86 vs 0.80, DPG 85.88 vs 84.19, T2I 0.3909 vs 0.3559) are reported without confidence intervals or significance tests. Given that the evaluation is automated and the training is stochastic, the reader cannot assess whether the reported gains are reliable.
minor comments (5)
  1. [3.2.1] Equation (2) includes a KL penalty term, but the text immediately afterward says "remove the KL loss" and Eq. (6) omits it; the relationship between the presented objective and the actual training objective should be clarified.
  2. [4.3.3] The text references "Figure 3" when discussing the entropy loss curves, but the figures showing entropy and reward over steps are in Figure 4.
  3. [3.1.1 / A.1] The captioning model is called GPT-4.1 mini in Section 3.1.1 but GPT-4.1 Small in Appendix A.1; the naming should be consistent.
  4. [A.3] The target entropy values (7.0 for image tokens, 2.0 for text tokens) are stated to come from the average rollout entropy after SFT. Since these are fitted to the specific model and dataset, the sensitivity of the results to these targets should be discussed or ablated.
  5. [Appendix B.3] There are minor typographical errors in the example prompts, such as "sandwichs" for "sandwiches" and "Monalisa" for "Mona Lisa" in Table 11 and surrounding captions.

Circularity Check

1 steps flagged · score 6.0 of 10

Benchmark gains partly reduce to training on the evaluation prompt distribution; no load-bearing self-citation chain is present.

  1. fitted input called prediction [Section 4.1.2 (Dataset Settings) vs. Section 4.1.1 (Evaluation Benchmarks)]
    "For GenEval, we enlarge its object vocabulary from 80 to 308 and extend its original generator with a new variant that specifies two distinct objects along with their respective counts. Using this augmented generator, we synthesized 12,552 unique prompts and filtered out any that overlap with the GenEval test set... For the DPG-Bench, we leveraged GPT-4.1 to produce 5,000 fresh prompts: for each draft, we sampled five existing DPG prompts at random and instructed GPT-4.1 to craft a final prompt matching their length and style."

    The evaluation benchmarks in Section 4.1.1 are GenEval, DPG-Bench, and T2I-Benchmark. The SFT and RL training prompts in Section 4.1.2 are constructed from those same benchmarks: GenEval via its own prompt generator with only exact test overlaps removed, DPG via new prompts explicitly made to match sampled DPG prompts in length and style, and T2I via the official training split of the same benchmark family. The reported GenEval (+6%), DPG (+1.69%), and T2I (+13.38%) gains are therefore in-distribution scores: the model is trained on the same object vocabulary, relation templates, and prompt statistics used for evaluation. The benchmark result is partly a measure of fit to the training distribution rather than an independent test of the CoT plus GRPO mechanism.

full rationale

The paper has no load-bearing self-citation chain: it builds on Janus-Pro-7B, Qwen-2.5-VL, GRPO, and SAC, all external, and the CoT substitution analysis and SFT/RL ablations provide some mechanism-level evidence. However, the central empirical claim is partially circular in evaluation: Section 4.1.2 constructs the SFT and RL prompt sets from the same benchmarks later used in Section 4.1.1 and Tables 1-3. For GenEval the training generator is an enlarged version of the evaluation generator, for DPG the new prompts are explicitly made to match the style and length of sampled DPG prompts, and for T2I the official training split of the same benchmark family is used. Thus the reported improvements are at least partly attributable to distribution memorization rather than to textual reasoning per se. The limitation section acknowledges benchmark specificity but does not address this train/eval overlap, and no held-out prompt distribution is evaluated. The adaptive-entropy targets are also taken from the model's own SFT rollouts, a minor self-referential calibration, but this does not drive the headline results. The score is 6, not higher, because the method's components are real, external machinery and are not defined in terms of the benchmark scores.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claims rest on three kinds of unpaid inputs: the VLM reward model's binary judgments, the GPT-4.1-generated detailed captions used as CoT supervision, and the credit-assignment rule that propagates one image-level reward to all tokens. The RL prompt distribution is also drawn from the evaluation benchmarks. No new physical entities are introduced.

free parameters (4)
  • Target entropy for image tokens = 7.0
    Set from the average rollout entropy observed after SFT (Appendix A.3); the adaptive entropy loss uses this target, so the regularization strength is fitted to the model's own output distribution.
  • Target entropy for text tokens = 2.0
    Same source as the image-token target, chosen separately because text tokens have a different vocabulary size and entropy scale (Appendix A.3).
  • Classifier-free guidance scale during RL rollouts = 1.0
    Chosen to speed up rollout (Appendix A.3); the evaluation uses a different scale of 5.0, so the policy is optimized under a different sampling distribution than the one scored.
  • Classifier-free guidance scale during inference/evaluation = 5.0
    Same as the base model default; this hand-chosen setting affects all benchmark numbers and is not swept or justified independently.
assumptions (4)
  • domain assumption Qwen-2.5-VL-7B provides a valid binary reward for prompt-image alignment
    Section 3.2.2 uses the VLM's boxed 0/1 score as the only training reward; no correlation with human judgment or downstream benchmark metrics is reported.
  • ad hoc to paper GPT-4.1-generated detailed captions, produced from concise captions alone, are valid chain-of-thought supervision for image generation
    Appendix A.1 defines the CoT as an expanded caption written by GPT-4.1 Nano from the concise caption; the paper provides no evidence that these verbose captions correspond to the model's planning process.
  • ad hoc to paper Uniform reward propagation from the image to all preceding reasoning tokens is a valid credit assignment
    Section 3.2.2 states the image-level reward is propagated back through the entire sequence; this assumes every reasoning token is equally responsible for image quality.
  • domain assumption Training prompts synthesized from GenEval, DPG-Bench, and T2I-CompBench++ are a fair preparation for evaluation on those same benchmarks
    Section 4.1.2 derives RL training prompts from the benchmark distributions used for testing; this assumption underlies the claimed generalization and is questionable.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ReasonGen-R1: CoT for Autoregressive Image generation models through SFT and RL." pith.science (2026). https://pith.science/paper/LNCMTB4Y

@misc{pith2026250524875,
  author       = {Pith},
  title        = {Pith review of: ReasonGen-R1: CoT for Autoregressive Image generation models through SFT and RL},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LNCMTB4Y}},
  note         = {Machine review of arXiv:2505.24875}
}
read the original abstract

Although chain-of-thought reasoning and reinforcement learning (RL) have driven breakthroughs in NLP, their integration into generative vision models remains underexplored. We introduce ReasonGen-R1, a two-stage framework that first imbues an autoregressive image generator with explicit text-based "thinking" skills via supervised fine-tuning on a newly generated reasoning dataset of written rationales, and then refines its outputs using Group Relative Policy Optimization. To enable the model to reason through text before generating images, We automatically generate and release a corpus of model crafted rationales paired with visual prompts, enabling controlled planning of object layouts, styles, and scene compositions. Our GRPO algorithm uses reward signals from a pretrained vision language model to assess overall visual quality, optimizing the policy in each update. Evaluations on GenEval, DPG, and the T2I benchmark demonstrate that ReasonGen-R1 consistently outperforms strong baselines and prior state-of-the-art models. More: aka.ms/reasongen.

Figures

Figures reproduced from arXiv: 2505.24875 by the authors.

Figure 1
Figure 1. Overall framework of ReasonGen-R1. We propose the first reinforcement learning post￾training framework that enables autoregressive image generation models to output both a chain-of￾thought reasoning process and the final image. Abstract Although chain-of-thought (CoT) reasoning and reinforcement learning (RL) have driven breakthroughs in NLP, their integration into generative vision models re￾mains underexplored. We… view at source ↗
Figure 2
Figure 2. Left: We show side-by-side visualizations of images generated by Janus-Pro-7B and [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The pipeline for reinforcement learning in ReasonGen-R1. In our experiment section, we aim to answer the fol￾lowing questions:(1) To what extent does incorporat￾ing textual reasoning improve instruction adherence in image generation? (2) How much does the RL benefit from the SFT training warmup? (2) How much does the RL training benefit from the size of reward model? (4) How does the adaptive entropy loss benefit th… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Comparison on Entropy Loss regularization. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Ranked word-frequency distribution across 1000 Chain-of-Thought (CoT) rollouts. Only [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

56 extracted references · 19 canonical work pages

  1. [14]

    Can we generate images with cot? let’s verify and reinforce image generation step by step.arXiv preprint arXiv:2501.13926, 2025

    Ziyu Guo, Renrui Zhang, Chengzhuo Tong, Zhizheng Zhao, Peng Gao, Hongsheng Li, and Pheng-Ann Heng. Can we generate images with cot? let’s verify and reinforce image generation step by step.arXiv preprint arXiv:2501.13926, 2025

    arXiv 2025

  2. [1]

    Qwen2.5-vl technical report, 2025

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report, 2025

    2025

  3. [2]

    Improving image generation with better captions

    James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al. Improving image generation with better captions. Computer Science. https://cdn. openai. com/papers/dall-e-3. pdf, 2(3):8, 2023

    2023

  4. [3]

    Pixart- α: Fast training of diffusion transformer for photorealistic text-to-image synthesis.arXiv preprint arXiv:2310.00426, 2023

    Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, et al. Pixart- α: Fast training of diffusion transformer for photorealistic text-to-image synthesis.arXiv preprint arXiv:2310.00426, 2023

    arXiv

  5. [4]

    Janus-pro: Unified multimodal understanding and generation with data and model scaling, 2025

    Xiaokang Chen, Zhiyu Wu, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, and Chong Ruan. Janus-pro: Unified multimodal understanding and generation with data and model scaling, 2025

    2025

  6. [5]

    Palm: Scaling language modeling with pathways.arXiv preprint arXiv:2204.02311, 2022

    Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways.arXiv preprint arXiv:2204.02311, 2022

    arXiv 2022

  7. [6]

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai D...

    work page 2025

  8. [7]

    Open- vlthinker: An early exploration to complex vision-language reasoning via iterative self- improvement.arXiv preprint arXiv:2503.17352, 2025

    Yihe Deng, Hritik Bansal, Fan Yin, Nanyun Peng, Wei Wang, and Kai-Wei Chang. Open- vlthinker: An early exploration to complex vision-language reasoning via iterative self- improvement.arXiv preprint arXiv:2503.17352, 2025

    arXiv 2025

Show all 56 references
  1. [8]

    Dreamllm: Synergistic multimodal comprehension and creation.arXiv preprint arXiv:2309.11499, 2023

    Runpei Dong, Chunrui Han, Yuang Peng, Zekun Qi, Zheng Ge, Jinrong Yang, Liang Zhao, Jian- jian Sun, Hongyu Zhou, Haoran Wei, et al. Dreamllm: Synergistic multimodal comprehension and creation.arXiv preprint arXiv:2309.11499, 2023

    2023 arXiv

  2. [9]

    Scaling rectified flow trans- formers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow trans- formers for high-resolution image synthesis. InForty-first international conference on machine...

    2024

  3. [10]

    Dpok: Reinforcement learning for fine-tuning text-to-image diffusion models, 2023

    Ying Fan, Olivia Watkins, Yuqing Du, Hao Liu, Moonkyung Ryu, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, Kangwook Lee, and Kimin Lee. Dpok: Reinforcement learning for fine-tuning text-to-image diffusion models, 2023

    2023

  4. [11]

    Puma: Empowering unified mllm with multi-granular visual generation, 2024

    Rongyao Fang, Chengqi Duan, Kun Wang, Hao Li, Hao Tian, Xingyu Zeng, Rui Zhao, Jifeng Dai, Hongsheng Li, and Xihui Liu. Puma: Empowering unified mllm with multi-granular visual generation, 2024

    2024

  5. [12]

    Geneval: An object-focused framework for evaluating text-to-image alignment, 2023

    Dhruba Ghosh, Hanna Hajishirzi, and Ludwig Schmidt. Geneval: An object-focused framework for evaluating text-to-image alignment, 2023

    2023

  6. [13]

    Gemini 2.0 flash | generative ai on vertex ai | google cloud

    Google Cloud. Gemini 2.0 flash | generative ai on vertex ai | google cloud. https: //cloud.google.com/vertex-ai/generative-ai/docs/models/gemini/2-0-flash , May23 2025. Last updated: May 23, 2025. Accessed: May 28, 2025

    2025

  7. [15]

    Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor, 2018

    Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor, 2018

    2018

  8. [16]

    Soft actor-critic algorithms and applications, 2019

    Tuomas Haarnoja, Aurick Zhou, Kristian Hartikainen, George Tucker, Sehoon Ha, Jie Tan, Vikash Kumar, Henry Zhu, Abhishek Gupta, Pieter Abbeel, and Sergey Levine. Soft actor-critic algorithms and applications, 2019

    2019

  9. [17]

    Classifier-free diffusion guidance, 2022

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance, 2022

    2022

  10. [18]

    Ella: Equip diffusion models with llm for enhanced semantic alignment, 2024

    Xiwei Hu, Rui Wang, Yixiao Fang, Bin Fu, Pei Cheng, and Gang Yu. Ella: Equip diffusion models with llm for enhanced semantic alignment, 2024

    2024

  11. [19]

    Kaiyi Huang, Chengqi Duan, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. T2i- compbench++: An enhanced and comprehensive benchmark for compositional text-to-image generation.IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(5):3563–3579, 2025

    2025

  12. [20]

    T2i-compbench: A compre- hensive benchmark for open-world compositional text-to-image generation.Advances in Neural Information Processing Systems, 36:78723–78747, 2023

    Kaiyi Huang, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. T2i-compbench: A compre- hensive benchmark for open-world compositional text-to-image generation.Advances in Neural Information Processing Systems, 36:78723–78747, 2023. 12

    2023

  13. [21]

    Comat: Aligning text-to-image diffusion model with image-to-text concept matching.Advances in Neural Information Processing Systems, 37:76177–76209, 2024

    Dongzhi Jiang, Guanglu Song, Xiaoshi Wu, Renrui Zhang, Dazhong Shen, Zhuofan Zong, Yu Liu, and Hongsheng Li. Comat: Aligning text-to-image diffusion model with image-to-text concept matching.Advances in Neural Information Processing Systems, 37:76177–76209, 2024

    2024

  14. [22]

    Flux.https://github.com/black-forest-labs/flux, 2024

    Black Forest Labs. Flux.https://github.com/black-forest-labs/flux, 2024

    2024

  15. [23]

    Parrot: Pareto-optimal multi-reward reinforcement learning framework for text-to-image generation, 2024

    Seung Hyun Lee, Yinxiao Li, Junjie Ke, Innfarn Yoo, Han Zhang, Jiahui Yu, Qifei Wang, Fei Deng, Glenn Entis, Junfeng He, Gang Li, Sangpil Kim, Irfan Essa, and Feng Yang. Parrot: Pareto-optimal multi-reward reinforcement learning framework for text-to-image generation, 2024

    2024

  16. [24]

    Adaptive group policy optimization: Towards stable training and token-efficient reasoning, 2025

    Chen Li, Nazhou Liu, and Kai Yang. Adaptive group policy optimization: Towards stable training and token-efficient reasoning, 2025

    2025

  17. [25]

    Optimizing safe and aligned language generation: A multi-objective grpo approach, 2025

    Xuying Li, Zhuo Li, Yuji Kosuga, and Victor Bian. Optimizing safe and aligned language generation: A multi-objective grpo approach, 2025

    2025

  18. [26]

    Mini-gemini: Mining the potential of multi-modality vision language models.arXiv preprint arXiv:2403.18814, 2024

    Yanwei Li, Yuechen Zhang, Chengyao Wang, Zhisheng Zhong, Yixin Chen, Ruihang Chu, Shaoteng Liu, and Jiaya Jia. Mini-gemini: Mining the potential of multi-modality vision language models.arXiv preprint arXiv:2403.18814, 2024

    2024 arXiv

  19. [27]

    Dual diffusion for unified image generation and understanding.arXiv preprint arXiv:2501.00289, 2024

    Zijie Li, Henry Li, Yichun Shi, Amir Barati Farimani, Yuval Kluger, Linjie Yang, and Peng Wang. Dual diffusion for unified image generation and understanding.arXiv preprint arXiv:2501.00289, 2024

    2024 arXiv

  20. [28]

    Mm-eureka: Exploring the frontiers of multimodal reasoning with rule-based reinforcement learning.arXiv preprint arXiv:2503.07365, 2025

    Fanqing Meng, Lingxiao Du, Zongkai Liu, Zhixiang Zhou, Quanfeng Lu, Daocheng Fu, Tiancheng Han, Botian Shi, Wenhai Wang, Junjun He, et al. Mm-eureka: Exploring the frontiers of multimodal reasoning with rule-based reinforcement learning.arXiv preprint arXiv:2503.07365, 2025

    2025 arXiv

  21. [29]

    Chang, Yiyi Zhang, Kianté Brantley, and Wen Sun

    Owen Oertell, Jonathan D. Chang, Yiyi Zhang, Kianté Brantley, and Wen Sun. Rl for consistency models: Faster reward guided text-to-image generation, 2024

    2024

  22. [30]

    Introducing openai o1

    OpenAI. Introducing openai o1. https://openai.com/o1/, December5 2024. Published: December 5, 2024. Accessed: May 28, 2025

    2024

  23. [31]

    Introducing gpt-4.1 in the api

    OpenAI. Introducing gpt-4.1 in the api. https://openai.com/index/gpt-4-1/ , April

  24. [32]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Floren- cia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    2023 arXiv

  25. [33]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis.arXiv preprint arXiv:2307.01952, 2023

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis.arXiv preprint arXiv:2307.01952, 2023

    2023 arXiv

  26. [34]

    High- resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10684–10695, June 2022

    2022

  27. [35]

    Laion-5b: An open large-scale dataset for training next generation image-text models, 2022

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski, Srivatsa Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, and Jenia Jitsev. Laion...

    2022

  28. [36]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y . K. Li, Y . Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024. 13

    2024

  29. [37]

    Hybridflow: A flexible and efficient rlhf framework.arXiv preprint arXiv: 2409.19256, 2024

    Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework.arXiv preprint arXiv: 2409.19256, 2024

    2024 arXiv

  30. [38]

    Metamorph: Multimodal under- standing and generation via instruction tuning.arXiv preprint arXiv:2412.14164, 2024

    Shengbang Tong, David Fan, Jiachen Zhu, Yunyang Xiong, Xinlei Chen, Koustuv Sinha, Michael Rabbat, Yann LeCun, Saining Xie, and Zhuang Liu. Metamorph: Multimodal under- standing and generation via instruction tuning.arXiv preprint arXiv:2412.14164, 2024

    2024 arXiv

  31. [39]

    Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288, 2023

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  32. [40]

    Diffusion model alignment using direct preference optimization, 2023

    Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. Diffusion model alignment using direct preference optimization, 2023

    2023

  33. [41]

    Illume: Illuminating your llms to see, draw, and self-enhance.arXiv preprint arXiv:2412.06673, 2024

    Chunwei Wang, Guansong Lu, Junwei Yang, Runhui Huang, Jianhua Han, Lu Hou, Wei Zhang, and Hang Xu. Illume: Illuminating your llms to see, draw, and self-enhance.arXiv preprint arXiv:2412.06673, 2024

    2024 arXiv

  34. [42]

    Emu3: Next-token prediction is all you need, 2024

    Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, Yingli Zhao, Yulong Ao, Xuebin Min, Tao Li, Boya Wu, Bo Zhao, Bowen Zhang, Liangdong Wang, Guang Liu, Zheqi He, Xi Yang, Jingjing Liu, Yonghua Lin, Tie...

    2024

  35. [43]

    Powerful and flexible: Personalized text-to-image generation via reinforcement learning, 2024

    Fanyue Wei, Wei Zeng, Zhenyang Li, Dawei Yin, Lixin Duan, and Wen Li. Powerful and flexible: Personalized text-to-image generation via reinforcement learning, 2024

    2024

  36. [44]

    Chain-of-thought prompting elicits reasoning in large language models, 2023

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models, 2023

    2023

  37. [45]

    Show-o: One single transformer to unify multimodal understanding and generation, 2024

    Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. Show-o: One single transformer to unify multimodal understanding and generation, 2024

    2024

  38. [46]

    Show-o: One single trans- former to unify multimodal understanding and generation.arXiv preprint arXiv:2408.12528, 2024

    Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. Show-o: One single trans- former to unify multimodal understanding and generation.arXiv preprint arXiv:2408.12528, 2024

    2024 arXiv

  39. [47]

    R1-onevision: Advancing generalized multimodal reasoning through cross-modal formalization.arXiv preprint arXiv:2503.10615, 2025

    Yi Yang, Xiaoxuan He, Hongkun Pan, Xiyan Jiang, Yan Deng, Xingtao Yang, Haoyu Lu, Dacheng Yin, Fengyun Rao, Minfeng Zhu, et al. R1-onevision: Advancing generalized multimodal reasoning through cross-modal formalization.arXiv preprint arXiv:2503.10615, 2025

    2025 arXiv

  40. [48]

    Dapo: An open-source llm reinforcement learning system at scale.arXiv preprint arXiv:2503.14476, 2025

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, et al. Dapo: An open-source llm reinforcement learning system at scale.arXiv preprint arXiv:2503.14476, 2025

    2025 arXiv

  41. [49]

    R1-vl: Learning to reason with multimodal large language models via step-wise group relative policy optimization.arXiv preprint arXiv:2503.12937, 2025

    Jingyi Zhang, Jiaxing Huang, Huanjin Yao, Shunyu Liu, Xikun Zhang, Shijian Lu, and Dacheng Tao. R1-vl: Learning to reason with multimodal large language models via step-wise group relative policy optimization.arXiv preprint arXiv:2503.12937, 2025. 14 A Method Details A.1 Datas...

    2025 arXiv

  42. [51]

    During this stage, we feed the GPT with image

    Concise Image Captioning We use GPT-4.1 Small to generate a short, accurate, and informative caption that highlights object counts, colors, positions, and other details. During this stage, we feed the GPT with image. You are a data annotation expert. Generate a concise image c...

  43. [52]

    concise_caption

    Concise Image Caption Augmentation We use GPT-4.1 Nano to augment the concise caption obtained from the previous API call. We augment the caption into several categories. The purpose for the augmentation is to prevent the model from overfitting to one specific prompt pattern d...

  44. [53]

    concise_caption

    Detailed Caption Generation We use GPT-4.1 Nano to generate a detailed caption from each concise caption. This detailed caption serves as the ground truth chain-of-thought (CoT) supervision during the supervised fine-tuning (SFT) stage, guiding the model to learn reasoning bas...

  45. [54]

    Describe the image thoroughly (objects, colors, layout, etc.), do not be affected by the prompt

  46. [55]

    Identify key visual elements and instructions from the prompt

  47. [56]

    thinking

    Evaluate how well the image follows the prompt: - Are all required elements present? - Are object counts, colors, and positions accurate? Be extremly strict and precise: Only if the image matches the prompt perfectly, respond with: \boxed{1}. Otherwise, respond with: \boxed{0}...

  48. [2025]

    Accessed: 2025-05-28

    2025

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.