Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Autoregressive model beats diffusion peers on layout-to-image control

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 16:38 UTC pith:ENEAJLAX

load-bearing objection Genuinely new masking strategy for AR layout control, but the GRPO reward likely overlaps with the evaluation metric, which makes the headline attribute gains hard to trust as-is. the 3 major comments →

arxiv 2509.12046 v3 pith:ENEAJLAX submitted 2025-09-15 cs.CV cs.AI

Layout-Conditioned Autoregressive Text-to-Image Generation via Structured Masking

Zirui Zheng , Takashi Isobe , Tong Shen , Xu Jia , Jianbin Zhao , Xiaomin Li , Mengmeng Ge , Baolu Li
show 7 more authors
Qinghe Wang Haiwen Diao Dong Li Dong Zhou Yunzhi Zhuge Huchuan Lu Emad Barsoum
This is my paper
classification cs.CV cs.AI
keywords layout-to-image generationautoregressive image generationstructured maskingattention maskingGRPOlayout controlattribute alignmenttext-to-image
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that autoregressive text-to-image models can handle spatial layout conditioning just as well as diffusion models, without adding heavy architectural components. It introduces SMARLI, which feeds a single sequence of prompt, layout, and image tokens into an autoregressive transformer and uses a structured attention mask to control which tokens can see each other. A second training stage applies GRPO with a layout reward that checks whether each region matches its description. On the LayoutSAM-Eval benchmark, SMARLI reports higher spatial and attribute-alignment scores than the diffusion-based SiamLayout baseline, suggesting AR models are a viable route for layout-controlled generation.

Core claim

SMARLI's central claim is that the reason layout control is hard for AR models is feature entanglement in attention: layout tokens for different objects pollute each other, and image tokens attend to irrelevant regions. The paper's fix is a structured mask that gives each layout token access to the global prompt and its own region's tokens only, and each image token access to the global prompt plus the layout tokens of the boxes that contain it. Combined with a next-set GRPO post-training that rewards both human-preference quality and CLIP-based region-description alignment, this produces layout-to-image generation that the authors report beats a diffusion baseline on spatial, color, texture

What carries the argument

The structured masking strategy is the load-bearing mechanism: a causal mask for prompt tokens, per-object causal masks within layout tokens, isolation between different objects' layout tokens, and image tokens restricted to the global prompt and the layout tokens of the boxes they fall in. The layout tokenizer encodes each box with Fourier coordinates concatenated with text-token embeddings through a zero-initialized MLP residual. The post-training is Layout-GRPO, a GRPO adaptation to next-set AR prediction with a CLIP-based layout reward averaged over cropped regions and an HPS quality reward.

Load-bearing premise

The layout reward that drives post-training uses CLIP similarity between cropped regions and their descriptions, and the benchmark's attribute-alignment evaluation is also CLIP-based, so the reported layout gains may partly come from directly optimizing the evaluation metric rather than from a general layout understanding.

What would settle it

Fine-tune SMARLI with the layout reward replaced by a non-CLIP alignment signal (e.g., answers from a VQA model or human preference judgments) and evaluate on LayoutSAM-Eval: if spatial/color/texture/shape scores drop sharply, the CLIP-reward optimization was the source of the reported performance.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If the claim holds, autoregressive T2I models are a viable alternative to diffusion for layout-conditioned generation, with simpler architecture and fewer added parameters.
  • The structured masking strategy is model-agnostic and can be transferred to next-token AR models, extending layout control beyond next-set paradigms.
  • The Layout-GRPO scheme can serve as a template for reward-based layout refinement in other generative architectures.
  • The observed trade-off that quality-only RL degrades layout accuracy, and vice versa, suggests multi-reward balancing is needed for controllable generation.
  • The SFT-only model already achieves high layout scores, so the gains of GRPO are relatively small (e.g., spatial 95.26 to 95.33) but they shift the Pareto frontier.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A straightforward test would compare SMARLI's layout reward against a non-CLIP alignment metric (e.g., human ratings or object-detector-based attribute checks) to see whether its gains generalize.
  • The same structured masking could be extended to other sparse conditioning signals (edge maps, keypoints) by adding token classes and corresponding masks.
  • The paper's ablation that removing global-prompt attention from layout tokens hurts attribute control suggests the global prompt serves as a context cache; one could try prompt compression to cut computation without losing that context.
  • If CLIP itself has a bias toward certain colors or materials, the layout reward could inherit it; testing with different CLIP backbones would indicate stability.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper presents SMARLI, a layout-conditioned text-to-image framework built on the next-set autoregressive model Show-o. It contributes a structured attention-masking scheme that controls interactions among global prompt, layout, and image tokens, and a GRPO-based post-training stage with a CLIP-based layout reward and the HPSv2.1 quality reward. The model is trained on a 1M subset of LayoutSAM and evaluated on LayoutSAM-Eval, where it reports improved layout-control metrics over diffusion-based baselines such as GLIGEN, HiCo, InstanceDiff, and SiamLayout-SD3, while retaining competitive image-quality scores. The central claim is that autoregressive models, with the proposed masking and reward design, can serve as a strong alternative to diffusion models for layout-conditioned generation.

Significance. If the empirical results are trustworthy, the paper would make a useful contribution: it is among the first to integrate layout conditions into an autoregressive T2I model, and the structured masking strategy is a simple, parameter-light mechanism that plausibly reduces region-description interference. The ablation studies in Tab. 2 and Tab. 3 are informative and directionally support the design choices. However, the main empirical claim rests on a load-bearing issue that the manuscript does not address: the GRPO layout reward appears to be the same type of CLIP-based crop–text similarity that LayoutSAM-Eval attribute metrics likely use. If so, the reported gains in color, texture, and shape largely reflect direct optimization of the evaluation metric, not a generalizable improvement in layout understanding. The paper also leaves the GRPO adaptation to next-set prediction imprecise, and reports all results as point estimates without uncertainty quantification. These gaps need to be closed before the central claim can be accepted.

major comments (3)
  1. [Method: Layout-GRPO; Experiments: Datasets and Evaluation Metrics] The layout reward is defined as the average CLIP similarity between each cropped region and its region description. The evaluation protocol is adopted from CreatiLayout on LayoutSAM-Eval, where attribute alignment for color, texture, and shape is measured in 'layout-controlled regions'. The manuscript does not disclose the implementation of these attribute metrics. If they are also CLIP-based crop–text similarities, then the GRPO reward directly optimizes the evaluation metric, and the large gains in Tab. 1 (e.g., color 86.84 vs. SiamLayout 74.45) may reflect reward overfitting rather than generalizable layout control. This issue is load-bearing because the paper's main claim is that AR models achieve superior layout control. Please state the exact metric implementation, show that it is independent of the reward, and provide an evaluation with a non-CLIP attribute metric (e.g., detection
  2. [Preliminary: GRPO and Method: Layout-GRPO] Eq. (1) is the standard token-level GRPO objective, with per-token log-probabilities πθ(oi,t|c, oi,<t). Next-set AR models predict multiple masked tokens simultaneously and do not factorize over tokens in the same way. The text merely says that πθ and πref 'traverse the same rollout trajectory through step-wise forward passes', but it never defines the action space for a set prediction, how per-token or per-set advantages are computed in Eq. (4), or how the group normalization is applied. Additionally, β is set to 0, so the KL term in Eq. (1) is vacuous and the reference model plays no role. Without a precise set-level objective or pseudo-code, the GRPO stage is under-specified and the reported improvement from Tab. 3 cannot be reproduced or fully attributed to the stated mechanism.
  3. [Experiments: Comparison with State-of-the-Arts] All quantitative results in Tab. 1 and the ablations in Tabs. 2–3 are reported as point estimates with no error bars, multiple seeds, or significance tests. Several layout-metric differences are small (for example, spatial 95.33 for SMARLI versus 95.26 for SMARLI-SFT, and 95.33 versus 95.29 for the HPS-only ablation), and the comparison with diffusion baselines uses a different backbone family without a parameter or compute comparison. The claim that SMARLI 'maintains the structural simplicity and generation efficiency of AR models' is not directly supported by any efficiency measurement. Please report confidence intervals or significance tests, and include at least FLOPs or latency comparisons with the diffusion baselines.
minor comments (5)
  1. [Abstract and Experiments] The abstract claims that the masking strategy and post-training scheme 'can also be transferred to standard next-token-based AR models', but no experiment or analysis in the paper supports this transferability claim. Either add results or soften the claim.
  2. [Method: Tokenization] The notation for layout tokens is slightly confusing: Eq. (2) concatenates text tokens and Fourier box embeddings in the sequence dimension, but Eq. (3) then applies a shared MLP with residual connection. It would help to state the resulting length of each object's token sequence and how the zero-initialized MLP preserves the Fourier embedding scale.
  3. [Training Details] The number of groups is set to 4 and batch size to 28; it is unclear whether this means G=4 sampled outputs per prompt and 7 prompts per batch, or another arrangement. Please clarify the rollout and group structure.
  4. [Figure 2] The global prompt in the figure says 'A wooden church...' but the example layout region says 'A Russian wooden church.' Minor inconsistency that could confuse readers.
  5. [Experiments: Datasets and Evaluation Metrics] There is a typo: 'generated through automated annotation methods to obtain detailed descriptions..' has double period. Also, the description of the LayoutSAM-Eval attribute metrics is too brief; please specify exactly how spatial, color, texture, and shape scores are computed, including which CLIP model or detector is used.

Circularity Check

0 steps flagged

No significant circularity found; the paper's claims rest on external benchmark comparisons and internal ablations, not on a derivation that reduces to its inputs.

full rationale

The paper's core contributions are empirical: a structured masking strategy for AR layout-to-image generation and a GRPO-based post-training scheme with a CLIP-based layout reward. Neither is presented as a prediction derived from first principles. The evaluation is conducted on the external LayoutSAM-Eval benchmark following CreatiLayout's protocol, with comparisons to external diffusion baselines. The masking strategy is ablated independently of the reward (Table 2), and the GRPO ablation (Table 3) isolates the reward's contribution. The only possible concern is that the CLIP crop-description similarity used as the layout reward may be similar to whatever implementation LayoutSAM-Eval uses for attribute alignment, but the paper does not specify the benchmark's implementation, so this is an evaluation-validity risk rather than a demonstrated circularity. The SFT-only baseline already outperforms SiamLayout on color/texture/shape, so the main reported advantage is not solely produced by the reward. No load-bearing self-citation, no imported uniqueness theorem, and no definitional equivalence between claimed results and inputs were found.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 0 invented entities

The paper introduces no new physical or conceptual entities. Its contributions are attention mask designs and a reward function built from existing CLIP and HPS models. The main unstated assumptions are that CLIP similarity is a good proxy for layout correctness, that the next-set GRPO adaptation is well-defined, and that the evaluation metrics are independent of the reward.

free parameters (3)
  • omega_layout = 1.1 for image tokens inside bounding boxes, else 1
    Hand-set advantage weight for layout reward; no sensitivity analysis is provided.
  • omega_hps = 1
    Weight for the HPS image quality reward, fixed at 1.
  • beta = 0
    KL regularization coefficient in the GRPO objective is set to 0, removing the constraint toward the reference policy (Eq. 1 in the paper).
axioms (5)
  • domain assumption Show-o is a next-set autoregressive model and its tokenizers can be reused for layout tokens.
    The method builds on Show-o, assuming its next-set prediction paradigm and tokenizers support layout conditioning (Preliminary and Method sections).
  • domain assumption CLIP similarity between region crops and text descriptions is a valid measure of layout fidelity.
    Used as the layout reward in GRPO post-training and likely correlates with the evaluation metrics (Layout-GRPO section).
  • domain assumption The attention masking described can be implemented within standard transformer blocks without architectural change.
    The Method section describes the mask design, but no implementation details or exact attention modification are given.
  • ad hoc to paper GRPO can be adapted to next-set predictions by traversing rollout trajectories.
    The paper states this adaptation but does not give the exact set-level policy objective or probability computation (Layout-GRPO section).
  • domain assumption The Qwen3-14B based filtering of redundant boxes improves reward signal fidelity.
    Data filtering in the Experiments section is motivated as enhancing reward reliability, but no analysis shows its effect on final metrics.

pith-pipeline@v1.3.0-alltime-deepseek · 12632 in / 12356 out tokens · 121485 ms · 2026-08-04T16:38:24.966711+00:00 · methodology

0 comments
read the original abstract

Although autoregressive (AR) models have demonstrated remarkable success in image generation, extending these models to layout-conditioned generation remains challenging due to the sparse nature of layout conditions and the risk of feature entanglement. We present \textbf{S}tructured \textbf{M}asking for \textbf{AR}-based \textbf{L}ayout-to-\textbf{I}mage (SMARLI), a novel framework that effectively integrates spatial layout constraints into the AR generation process. To equip AR models with layout control, a structured masking strategy is applied to the attention computation to govern the interaction among the global prompt, layout, and image tokens. This design prevents the misassociation of different regions with their corresponding descriptions while enabling the sufficient injection of layout constraints into the generation process. To alleviate the exposure bias of AR models and further enhance generation quality and layout accuracy, we incorporate a Group Relative Policy Optimization (GRPO) post-training scheme. We adapt it to the next-set-based paradigm and introduce a specifically designed layout reward, which is coordinated with an image quality reward to guide policy optimization in a balanced manner. Experimental results demonstrate that SMARLI seamlessly integrates layout tokens with text and image tokens without compromising generation quality, and the proposed masking strategy and post-training scheme can also be transferred to standard next-token-based AR models. The proposed framework achieves superior layout control while maintaining the structural simplicity and generation efficiency of AR models.

Figures

Figures reproduced from arXiv: 2509.12046 by Baolu Li, Dong Li, Dong Zhou, Emad Barsoum, Haiwen Diao, Huchuan Lu, Jianbin Zhao, Mengmeng Ge, Qinghe Wang, Takashi Isobe, Tong Shen, Xiaomin Li, Xu Jia, Yunzhi Zhuge, Zirui Zheng.

Figure 1
Figure 1. Figure 1: Example L2I generation results by SMARLI. We show that SMARLI could achieve fine-grained controllable genera [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: An overview of the proposed framework. In stage 1, the global prompt and image are tokenized using Show-o’s [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Qualitative results on the LayoutSAM-Eval. Conditioned on a complicated layout, prior works fail to faithfully [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Appearance Pointers -- Multimodal Region Control of Diffusion Transformers

    cs.CV 2026-07 conditional novelty 6.0

    Appearance pointers are compact tokens that let a diffusion transformer apply text, image, or combined prompts to specific image regions in a single pass.

Reference graph

Works this paper leans on

57 extracted references · 27 linked inside Pith · cited by 1 Pith paper

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Black, K.; Janner, M.; Du, Y.; Kostrikov, I.; and Levine, S. 2023. Training diffusion models with reinforcement learning. arXiv preprint arXiv:2305.13301

    Pith/arXiv arXiv 2023

  4. [4]

    T.; Rubinstein, M.; et al

    Chang, H.; Zhang, H.; Barber, J.; Maschinot, A.; Lezama, J.; Jiang, L.; Yang, M.-H.; Murphy, K.; Freeman, W. T.; Rubinstein, M.; et al. 2023. Muse: Text-to-image generation via masked generative transformers. arXiv preprint arXiv:2301.00704

    Pith/arXiv arXiv 2023

  5. [5]

    Chang, H.; Zhang, H.; Jiang, L.; Liu, C.; and Freeman, W. T. 2022. Maskgit: Masked generative image transformer. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 11315--11325

    2022

  6. [6]

    Chen, M.; Laina, I.; and Vedaldi, A. 2024. Training-free layout control with cross-attention guidance. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, 5343--5353

    2024

  7. [7]

    Chen, X.; Wu, Z.; Liu, X.; Pan, Z.; Liu, W.; Xie, Z.; Yu, X.; and Ruan, C. 2025 a . Janus-pro: Unified multimodal understanding and generation with data and model scaling. arXiv preprint arXiv:2501.17811

    Pith/arXiv arXiv 2025

  8. [8]

    Chen, Y.; Ma, Z.; Jia, G.; Jiang, C.; Li, J.; and Zhou, B. 2025 b . Context-Aware Autoregressive Models for Multi-Conditional Image Generation. arXiv preprint arXiv:2505.12274

    Pith/arXiv arXiv 2025

  9. [9]

    Cheng, B.; Ma, Y.; Wu, L.; Liu, S.; Ma, A.; Wu, X.; Leng, D.; and Yin, Y. 2024. Hico: Hierarchical controllable diffusion model for layout-to-image generation. arXiv preprint arXiv:2410.14324

    Pith/arXiv arXiv 2024

  10. [10]

    Chung, J.; Hyun, S.; Kim, H.; Koh, E.; Lee, M.; and Heo, J.-P. 2025. Fine-Tuning Visual Autoregressive Models for Subject-Driven Generation. arXiv preprint arXiv:2504.02612

    Pith/arXiv arXiv 2025

  11. [11]

    Clark, K.; Vicol, P.; Swersky, K.; and Fleet, D. J. 2023. Directly fine-tuning diffusion models on differentiable rewards. arXiv preprint arXiv:2309.17400

    Pith/arXiv arXiv 2023

  12. [12]

    Cui, X.; Sun, Q.; Wang, M.; Li, L.; Zhou, W.; and Li, H. 2025. LayoutEnc: Leveraging Enhanced Layout Representations for Transformer-based Complex Scene Synthesis. ACM Transactions on Multimedia Computing, Communications and Applications, 21(4): 1--21

    2025

  13. [13]

    Dahary, O.; Patashnik, O.; Aberman, K.; and Cohen-Or, D. 2024. Be yourself: Bounded attention for multi-subject text-to-image generation. In European Conference on Computer Vision, 432--448. Springer

    2024

  14. [14]

    Deng, H.; Pan, T.; Diao, H.; Luo, Z.; Cui, Y.; Lu, H.; Shan, S.; Qi, Y.; and Wang, X. 2024. Autoregressive video generation without vector quantization. arXiv preprint arXiv:2412.14169

    Pith/arXiv arXiv 2024

  15. [15]

    Esser, P.; Rombach, R.; and Ommer, B. 2021. Taming transformers for high-resolution image synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 12873--12883

    2021

  16. [16]

    Feng, Y.; Gong, B.; Chen, D.; Shen, Y.; Liu, Y.; and Zhou, J. 2024. Ranni: Taming text-to-image diffusion for accurate instruction following. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 4744--4753

    2024

  17. [17]

    Ge, M.; Jia, X.; Isobe, T.; Li, X.; Wang, Q.; Mu, J.; Zhou, D.; Wang, L.; Lu, H.; Tian, L.; et al. 2024. Customizing text-to-image generation with inverted interaction. In Proceedings of the 32nd ACM International Conference on Multimedia, 10901--10909

    2024

  18. [18]

    Heusel, M.; Ramsauer, H.; Unterthiner, T.; Nessler, B.; and Hochreiter, S. 2017. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30

    2017

  19. [19]

    J.; Shen, Y.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; Chen, W.; et al

    Hu, E. J.; Shen, Y.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; Chen, W.; et al. 2022. Lora: Low-rank adaptation of large language models. ICLR, 1(2): 3

    2022

  20. [20]

    Jia, C.; Luo, M.; Dang, Z.; Dai, G.; Chang, X.; Wang, M.; and Wang, J. 2024. Ssmg: Spatial-semantic map guided diffusion model for free-form layout-to-image generation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 2480--2488

    2024

  21. [21]

    Jiang, D.; Guo, Z.; Zhang, R.; Zong, Z.; Li, H.; Zhuo, L.; Yan, S.; Heng, P.-A.; and Li, H. 2025. T2i-r1: Reinforcing image generation with collaborative semantic-level and token-level cot. arXiv preprint arXiv:2505.00703

    Pith/arXiv arXiv 2025

  22. [22]

    Kirstain, Y.; Polyak, A.; Singer, U.; Matiana, S.; Penna, J.; and Levy, O. 2023. Pick-a-pic: An open dataset of user preferences for text-to-image generation. Advances in neural information processing systems, 36: 36652--36663

    2023

  23. [23]

    Lee, Y.; Yoon, T.; and Sung, M. 2024. Groundit: Grounding diffusion transformers via noisy patch transplantation. Advances in Neural Information Processing Systems, 37: 58610--58636

    2024

  24. [24]

    Li, T.; Tian, Y.; Li, H.; Deng, M.; and He, K. 2024 a . Autoregressive image generation without vector quantization. Advances in Neural Information Processing Systems, 37: 56424--56445

    2024

  25. [25]

    Li, X.; Liu, Y.; Isobe, T.; Jia, X.; Cui, Q.; Zhou, D.; Li, D.; He, Y.; Lu, H.; Wang, Z.; et al. 2025. Reneg: Learning negative embedding with reward guidance. In Proceedings of the Computer Vision and Pattern Recognition Conference, 23636--23645

    2025

  26. [26]

    Li, X.; Qiu, K.; Chen, H.; Kuen, J.; Lin, Z.; Singh, R.; and Raj, B. 2024 b . Controlvar: Exploring controllable visual autoregressive modeling. arXiv preprint arXiv:2406.09750

    Pith/arXiv arXiv 2024

  27. [27]

    Li, Y.; Liu, H.; Wu, Q.; Mu, F.; Yang, J.; Gao, J.; Li, C.; and Lee, Y. J. 2023. Gligen: Open-set grounded text-to-image generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 22511--22521

    2023

  28. [28]

    Li, Z.; Cheng, T.; Chen, S.; Sun, P.; Shen, H.; Ran, L.; Chen, X.; Liu, W.; and Wang, X. 2024 c . Controlar: Controllable image generation with autoregressive models. arXiv preprint arXiv:2410.02705

    Pith/arXiv arXiv 2024

  29. [29]

    Liu, J.; Liu, G.; Liang, J.; Li, Y.; Liu, J.; Wang, X.; Wan, P.; Zhang, D.; and Ouyang, W. 2025. Flow-grpo: Training flow matching models via online rl. arXiv preprint arXiv:2505.05470

    Pith/arXiv arXiv 2025

  30. [30]

    Loshchilov, I.; and Hutter, F. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101

    Pith/arXiv arXiv 2017

  31. [31]

    Mu, J.; Vasconcelos, N.; and Wang, X. 2025. Editar: Unified conditional generation with autoregressive models. In Proceedings of the Computer Vision and Pattern Recognition Conference, 7899--7909

    2025

  32. [32]

    W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al

    Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning, 8748--8763. PmLR

    2021

  33. [33]

    D.; Ermon, S.; and Finn, C

    Rafailov, R.; Sharma, A.; Mitchell, E.; Manning, C. D.; Ermon, S.; and Finn, C. 2023. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems, 36: 53728--53741

    2023

  34. [34]

    Ramesh, A.; Pavlov, M.; Goh, G.; Gray, S.; Voss, C.; Radford, A.; Chen, M.; and Sutskever, I. 2021. Zero-shot text-to-image generation. In International conference on machine learning, 8821--8831. Pmlr

    2021

  35. [35]

    Salimans, T.; Goodfellow, I.; Zaremba, W.; Cheung, V.; Radford, A.; and Chen, X. 2016. Improved techniques for training gans. Advances in neural information processing systems, 29

    2016

  36. [36]

    Shao, Z.; Wang, P.; Zhu, Q.; Xu, R.; Song, J.; Bi, X.; Zhang, H.; Zhang, M.; Li, Y.; Wu, Y.; et al. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300

    Pith/arXiv arXiv 2024

  37. [37]

    Sun, P.; Jiang, Y.; Chen, S.; Zhang, S.; Peng, B.; Luo, P.; and Yuan, Z. 2024. Autoregressive model beats diffusion: Llama for scalable image generation. arXiv preprint arXiv:2406.06525

    Pith/arXiv arXiv 2024

  38. [38]

    Tian, K.; Jiang, Y.; Yuan, Z.; Peng, B.; and Wang, L. 2024. Visual autoregressive modeling: Scalable image generation via next-scale prediction. Advances in neural information processing systems, 37: 84839--84865

    2024

  39. [39]

    Touvron, H.; Lavril, T.; Izacard, G.; Martinet, X.; Lachaux, M.-A.; Lacroix, T.; Rozi \`e re, B.; Goyal, N.; Hambro, E.; Azhar, F.; et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971

    Pith/arXiv arXiv 2023

  40. [40]

    Wallace, B.; Dang, M.; Rafailov, R.; Zhou, L.; Lou, A.; Purushwalkam, S.; Ermon, S.; Xiong, C.; Joty, S.; and Naik, N. 2024. Diffusion model alignment using direct preference optimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8228--8238

    2024

  41. [41]

    Wang, J.; Tian, Z.; Wang, X.; Zhang, X.; Huang, W.; Wu, Z.; and Jiang, Y.-G. 2025. Simplear: Pushing the frontier of autoregressive visual generation through pretraining, sft, and rl. arXiv preprint arXiv:2504.11455

    Pith/arXiv arXiv 2025

  42. [42]

    Wang, Q.; Liu, L.; Hua, M.; Zhu, P.; Zuo, W.; Hu, Q.; Lu, H.; and Cao, B. 2022. Hs-diffusion: Semantic-mixing diffusion for head swapping. arXiv preprint arXiv:2212.06458

    Pith/arXiv arXiv 2022

  43. [43]

    S.; Girdhar, R.; and Misra, I

    Wang, X.; Darrell, T.; Rambhatla, S. S.; Girdhar, R.; and Misra, I. 2024 a . Instancediffusion: Instance-level control for image generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 6232--6242

    2024

  44. [44]

    Wang, X.; Zhang, X.; Luo, Z.; Sun, Q.; Cui, Y.; Wang, J.; Zhang, F.; Wang, Y.; Li, Z.; Yu, Q.; et al. 2024 b . Emu3: Next-token prediction is all you need. arXiv preprint arXiv:2409.18869

    Pith/arXiv arXiv 2024

  45. [45]

    Wu, X.; Hao, Y.; Sun, K.; Chen, Y.; Zhu, F.; Zhao, R.; and Li, H. 2023. Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesis. arXiv preprint arXiv:2306.09341

    Pith/arXiv arXiv 2023

  46. [46]

    J.; Wang, W.; Lin, K

    Xie, J.; Mao, W.; Bai, Z.; Zhang, D. J.; Wang, W.; Lin, K. Q.; Gu, Y.; Chen, Z.; Yang, Z.; and Shou, M. Z. 2024. Show-o: One Single Transformer to Unify Multimodal Understanding and Generation. arXiv preprint arXiv:2408.12528

    Pith/arXiv arXiv 2024

  47. [47]

    Xu, J.; Liu, X.; Wu, Y.; Tong, Y.; Li, Q.; Ding, M.; Tang, J.; and Dong, Y. 2023. Imagereward: Learning and evaluating human preferences for text-to-image generation. Advances in Neural Information Processing Systems, 36: 15903--15935

    2023

  48. [48]

    Xue, Z.; Wu, J.; Gao, Y.; Kong, F.; Zhu, L.; Chen, M.; Liu, Z.; Liu, W.; Guo, Q.; Huang, W.; et al. 2025. DanceGRPO: Unleashing GRPO on Visual Generation. arXiv preprint arXiv:2505.07818

    Pith/arXiv arXiv 2025

  49. [49]

    Yang, A.; Li, A.; Yang, B.; Zhang, B.; Hui, B.; Zheng, B.; Yu, B.; Gao, C.; Huang, C.; Lv, C.; et al. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388

    Pith/arXiv arXiv 2025

  50. [50]

    Yang, L.; Yu, Z.; Meng, C.; Xu, M.; Ermon, S.; and Cui, B. 2024. Mastering text-to-image diffusion: Recaptioning, planning, and generating with multimodal llms. In Forty-first International Conference on Machine Learning

    2024

  51. [51]

    Yang, Z.; Liu, D.; Wang, C.; Yang, J.; and Tao, D. 2022. Modeling image composition for complex scene generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 7764--7773

    2022

  52. [52]

    Yao, Z.; Li, J.; Zhou, Y.; Liu, Y.; Jiang, X.; Wang, C.; Zheng, F.; Zou, Y.; and Li, L. 2024. Car: Controllable autoregressive modeling for visual generation. arXiv preprint arXiv:2410.04671

    Pith/arXiv arXiv 2024

  53. [53]

    Y.; Zhang, H.; Pang, R.; Qin, J.; Ku, A.; Xu, Y.; Baldridge, J.; and Wu, Y

    Yu, J.; Li, X.; Koh, J. Y.; Zhang, H.; Pang, R.; Qin, J.; Ku, A.; Xu, Y.; Baldridge, J.; and Wu, Y. 2021. Vector-quantized image modeling with improved vqgan. arXiv preprint arXiv:2110.04627

    Pith/arXiv arXiv 2021

  54. [54]

    B.; Versari, L.; Sohn, K.; Minnen, D.; Cheng, Y.; Birodkar, V.; Gupta, A.; Gu, X.; et al

    Yu, L.; Lezama, J.; Gundavarapu, N. B.; Versari, L.; Sohn, K.; Minnen, D.; Cheng, Y.; Birodkar, V.; Gupta, A.; Gu, X.; et al. 2023. Language Model Beats Diffusion--Tokenizer is Key to Visual Generation. arXiv preprint arXiv:2310.05737

    Pith/arXiv arXiv 2023

  55. [55]

    Zhang, H.; Hong, D.; Wang, Y.; Shao, J.; Wu, X.; Wu, Z.; and Jiang, Y.-G. 2024. Creatilayout: Siamese multimodal diffusion transformer for creative layout-to-image generation. arXiv preprint arXiv:2412.03859

    Pith/arXiv arXiv 2024

  56. [56]

    Zhang, L.; Rao, A.; and Agrawala, M. 2023. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, 3836--3847

    2023

  57. [57]

    Zhou, D.; Li, Y.; Ma, F.; Zhang, X.; and Yang, Y. 2024. Migc: Multi-instance generation controller for text-to-image synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 6818--6828

    2024