Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

Token-Shuffle: Towards High-Resolution Image Generation with Autoregressive Models

T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Token-Shuffle claims that autoregressive image generation can work with about a quarter of the visual tokens by fusing spatially local tokens along the channel dimension, and that this makes 2048×2048 text-to-image generation practical…

desk verdict A practical token-fusion trick that lets AR MLLMs reach 2048×2048 at ~25% of the visual-token compute, with a solid efficiency story and a weak head-to-head comparison against high-compression tokenizers. read the letter →

arxiv 2504.17789 v2 pith:AXXZFVWM submitted 2025-04-24 cs.CV

Xu Ma , Peize Sun , Haoyu Ma , Hao Tang , Chih-Yao Ma , Jialiang Wang , Kunpeng Li , Xiaoliang Dai
show 17 more authors
Yujun Shi Xuan Ju Yushi Hu Artsiom Sanakoyeu Felix Juefei-Xu Ji Hou Junjiao Tian Tao Xu Tingbo Hou Yen-Cheng Liu Zecheng He Zijian He Matt Feiszli Peizhao Zhang Peter Vajda Sam Tsai Yun Fu
This is my paper · ORCID
classification cs.CV
keywords token-shuffleautoregressiveimagegenerationmultimodallargelanguagemodelshigh-resolutionsynthesisvisualtokenreductionnext-tokenpredictionVQGANclassifier-freeguidance
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Autoregressive models generate images token by token, and high-resolution images need so many VQGAN tokens that training and inference quickly become impractical. Token-Shuffle attacks the token count itself: it takes $s\times s$ neighboring visual tokens, merges them along the channel dimension into one fused token, feeds only the fused tokens to the Transformer, and then unshuffles the output to recover the full spatial grid. The paper argues that the visual vocabulary inside a multimodal LLM has enough dimensional redundancy for this fusion to cost little in generation quality, and demonstrates that a 2.7B model with shuffle window size 2 reaches 1024×1024 and, for the first time in an autoregressive text-to-image model, 2048×2048 while keeping training and inference tractable. A sympathetic reader should care because this is a direct route to scaling discrete-token autoregressive image generation without changing the Transformer architecture or adding a separate text encoder.

What carries the argument

The token-shuffle / token-unshuffle pair is the load-bearing mechanism. Token-shuffle maps each $s\times s$ block of VQGAN tokens into one fused token by concatenating along the channel dimension and compressing with a shared MLP, so the Transformer sees $n/s^2$ tokens instead of $n$; token-unshuffle expands each predicted fused token back into $s\times s$ local tokens with a second MLP and reverses the spatial arrangement. Because the shuffle is performed before the Transformer and the unshuffle after it, the standard causal mask and cross-entropy next-token objective are unchanged, and the model effectively predicts the next fused token, generating several spatial tokens in one step.

What would settle it

Train a 16×-downsampling VQGAN and an 8× VQGAN on the identical dataset and compare generation at equal inference cost, Token-Shuffle at $s=2$ versus the high-compression tokenizer; if the high-compression tokenizer matches or beats Token-Shuffle on VQAScore and human preference, the local-redundancy premise fails. A second check is to extend training for shuffle sizes 4 and 8 and see whether the training-loss gap the paper reports ever closes.

Watch

Extended reading notes

Core claim

The central claim is that you do not need to predict every visual token individually: the model can predict fused tokens, each representing a small spatial window, and then disentangle them. With a shuffle window of size $s$, the visual token sequence shrinks by a factor of $s^2$, about 75% at $s=2$, while a shared-MLP compression and expansion pair keeps the fused representation in the same dimension as the original tokens. On GenAI-Bench hard prompts, the 2.7B model scores 0.77 VQAScore with Llama3-rewritten prompts and 0.67 on the original prompts, and it scores 0.62 overall on GenEval. The paper also reports human evaluations showing consistent wins over LlamaGen and Lumina-mGPT on text alignment, visual flaws, and visual appearance, and comparable or better results than LDM on text alignment and appearance, though slightly worse on visual flaws.

Load-bearing premise

The claim rests on the assumption that the redundancy in visual vocabulary is mostly spatial, so merging $s\times s$ neighboring VQGAN tokens along the channel dimension with an MLP loses little of what a high-fidelity generator needs; if neighboring tokens carry distinct local detail rather than redundant channels, the quality loss will grow quickly with $s$.

Editorial extensions

If this is right

  • At shuffle window size 2, visual token count and Transformer FLOPs drop by about 75%, making 2048×2048 generation feasible for autoregressive models for the first time.
  • The same operation can be inserted into existing discrete-token MLLMs without retraining the LLM from scratch, since the MLP layers are the only new parameters and the next-token prediction objective is untouched.
  • If the dimensional-redundancy premise holds, larger shuffle windows trade quality for efficiency along a smooth curve, giving a practical knob for deployment at different resolutions.
  • The reported results imply that pure autoregressive models can be competitive with diffusion models on text-to-image alignment benchmarks, a comparison the paper directly tests against LDM.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the same channel-dimension fusion could be applied to other dense modalities such as video frames or audio spectrograms, provided their tokenizers show similar dimensional redundancy.
  • Editorial inference: the local-window interaction introduced at unshuffle time is a mild spatial prior; combining it with global interaction mechanisms such as randomized autoregressive orders could address the visual-flaw weakness the paper itself documents.
  • Editorial inference: a controlled test with identically trained high-compression and low-compression tokenizers, which the paper admits its own comparison lacks, would cleanly separate the benefit of Token-Shuffle from tokenizer quality.
  • Editorial inference: if z-loss and QK-norm indeed stabilize 2048×2048 training, the same recipe should extend to 4096×4096, an easy scaling test of the method.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Token-Shuffle, a plug-and-play pair of operations for autoregressive multimodal LLMs that generate images from discrete VQGAN tokens. Spatially local s×s visual tokens are concatenated along the channel dimension, compressed by MLP layers, fed into the Transformer as a single fused token, and then expanded back by a token-unshuffle operation after the Transformer. This reduces the number of tokens seen by the Transformer by a factor of s^2 (about 75% at s=2) and, the paper argues, preserves generation quality by exploiting dimensional redundancy in the visual vocabulary. The method is evaluated with a 2.7B LLaMA backbone trained in stages from 512×512 to 2048×2048, using a half-linear CFG scheduler and z-loss for stability. The reported results include a GenAI-Bench VQAScore of 0.77 on hard prompts with Llama3-rewritten prompts, a GenEval overall score of 0.62, human evaluations against LlamaGen, Lumina-mGPT, and LDM, and visual examples at 1024×1024 and 2048×2048.

Significance. If the central quality-preservation claim holds, Token-Shuffle is a practically valuable contribution: it reduces the computational cost of discrete-token autoregressive image generation without changing the causal masking scheme or requiring a pretrained text encoder, and it enables 2048×2048 generation with a 2.7B model. The efficiency arithmetic is simple and the paper includes substantial empirical work: large-scale training, multiple ablations, a CFG-scheduler study, and both automatic and human evaluations. The paper is also unusually candid in its appendix about limitations, including visual flaws in AR models and the acknowledged unfairness of the high-compression tokenizer comparison. The main risk is that the paper's distinctive claim—that fusing spatially local tokens is better than simply using a higher-compression VQGAN—is not yet supported by a fair controlled experiment, and several headline quantitative claims are misreported or not traceable to the tables.

major comments (4)
  1. [Appendix B.4, Sec. 3.4] The paper's central qualitative claim is that Token-Shuffle preserves fine-grained information better than aggressive tokenizer compression, but the only direct test against that alternative is admitted to be unfair. In Sec. B.4, the D16 model uses a 16× VQGAN taken from the LlamaGen T2I checkpoint while D8+Token-Shuffle uses an internal 8× VQGAN, and the authors state that the tokenizers were not trained under identical conditions and that the dataset drifted during the project. This means the observed VQAScore and human-evaluation gaps (Tables 3-4, Fig. 18) could be due to tokenizer training conditions or data differences rather than to the Token-Shuffle operation itself. The central claim requires a controlled comparison in which a high-compression tokenizer and the low-compression tokenizer plus Token-Shuffle are trained and evaluated under matched data and training conditions; without it, the paper has not ruled out that a fairly trained 16× VQGAN with the same AR backbone would match or exceed Token-Shuffle at equal token count and FLOPs.
  2. [Sec. 3.3, Fig. 3; Sec. 4.5.1] The dimensional-redundancy evidence provided in Sec. 3.3 does not directly test the operation that Token-Shuffle actually performs. The experiment compresses the per-token embedding dimension by a factor r with two linear layers and shows that pretraining NLL is stable, but Token-Shuffle instead concatenates s×s spatially neighboring tokens along the channel dimension, compresses them with an MLP, and then requires the model to predict a fused token that must later be disentangled into s^2 spatial outputs. The latter operation involves a very different information bottleneck and a different prediction target. The ablations in Sec. 4.5.1 compare variants within the same tokenizer (Drop, Re-sampler, Simple), but none of them tests the natural alternative of reducing tokens at the tokenizer itself. As a result, the paper's motivation section is suggestive but not load-bearing evidence for the specific spatial-fusion design.
  3. [Abstract, Sec. 4.2, Table 1] Several headline quantitative claims are misreported or not supported by the tables. In Sec. 4.2 the text says Token-Shuffle 'surpasses DALL-E 3 by 0.7 in overall score on hard prompts,' but Table 1 shows 0.77 versus 0.70, a gap of 0.07, not 0.7. The abstract's claim of outperforming LDM by 0.15 is not traceable to any table: LDM does not appear in the GenAI-Bench table, and in the GenEval table the gap to LDM is 0.62 - 0.37 = 0.25. These are not mere typographical quibbles because the abstract and introduction use these numbers to support the state-of-the-art claim. The authors should correct the numbers and provide the exact source table for every comparative claim in the abstract.
  4. [Table 1, Sec. 4.2] The main GenAI-Bench comparison mixes prompt conditions. The headline Token-Shuffle values are marked with a dagger and use Llama3-rewritten prompts, while the baseline models are evaluated on the original prompts. The paper reports that Token-Shuffle without rewriting scores 0.67 on hard prompts, a full 0.10 lower than the rewritten-prompt score of 0.77. If prompt rewriting systematically improves VQAScore, then comparing the rewritten-prompt Token-Shuffle score to original-prompt baselines is not an apples-to-apples comparison. The authors should either evaluate the baselines with the same rewritten prompts, or restrict the superiority claims to the matched original-prompt condition, or clearly state that the comparison is between Token-Shuffle with rewriting and baselines without rewriting and justify why that is the correct protocol.
minor comments (5)
  1. [Sec. 3.4] The sentence 'we do not reduce the number of tokens during inference or training but instead reduce the token count during Transformer computation' is internally confusing; the paper should clarify that the input and output token sequences are restored by unshuffle, but the Transformer itself sees fewer tokens.
  2. [Fig. 8] In the first example of Fig. 8, the shuffle size 4 image receives the same VQAScore (0.956) as the shuffle size 1 image, which appears inconsistent with the text's statement that larger shuffle sizes cause a noticeable quality drop; please clarify whether these examples are representative or selected.
  3. [Appendix B.4] The text refers to 'Tab.17' when discussing reconstruction and generation results; this appears to be a typo for Table 4 or another appendix table, and the reference should be corrected.
  4. [Fig. 4] The caption states that Token-Shuffle gives 'approximately a 4× reduction in both training FLOPs and token number' at s=2; the token reduction is exact, but the FLOP reduction depends on the attention-to-MLP ratio and on the added Token-Shuffle MLP blocks, so the FLOP claim should be stated as approximate and preferably backed by a measured or computed FLOP count.
  5. [General] No code, model checkpoints, or evaluation scripts are mentioned as being released; providing these, or at least a detailed inference pseudocode with the CFG scheduler and sampling settings, would substantially improve reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the token-count reduction is definitional and quality is judged by external benchmarks; the admitted unfair comparison in B.4 weakens evidence but is not a circular step.

full rationale

Token-Shuffle's central claim is that spatially fusing s×s local tokens reduces the number of visual tokens by construction (n → n/s²); this is an architectural definition, not a prediction fitted to data. The quality-preservation claim is tested against external benchmarks (GenAI-Bench VQAScore, GenEval, human evaluation), which are independent of the method's parameters. Hyperparameters (window size s=2, CFG scale 7.5, half-linear scheduler) are selected by standard validation on those external metrics, so no fitted quantity is renamed as a prediction. The dimensional-redundancy experiment (Sec 3.3, Fig 3) compresses the codebook embedding dimension and supports the motivation by showing low NLL impact; it does not define the spatial token-fusion result. The head-to-head versus a 16× VQGAN in Sec B.4 is explicitly admitted to be 'not entirely fair' because tokenizers were not trained under identical conditions and the dataset drifted; this weakens the empirical support but is not circularity, since neither side of the comparison is an input to the other. Self-citations (LlamaGen's VQGAN, Emu training recipe) are used as off-the-shelf components and external baselines, not to validate the core claim. No derivation reduces to its own input.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim relies on a small number of design hyperparameters, all selected by the authors' own experiments, and on the empirical premise that visual vocabulary redundancy is spatially local. No new physical or mathematical entities are introduced.

free parameters (5)
  • shuffle window size s = 2 (default)
    Headline results use s=2, which reduces token count by 4x; s=4 blurs outputs (Sec 4.5.2).
  • CFG scale = 7.5
    Selected on GenAI-Bench VQAScore sweep (Fig 13); reported 0.77 hard-prompt score depends on this choice.
  • CFG scheduler = half-linear
    Chosen over linear, drop-first, sin, and sigmoid by visual quality and human evaluation (Sec 3.6).
  • z-loss weight = 1e-5
    Used to stabilize 2048x2048 training; no sensitivity analysis shown (Sec A).
  • number of MLP blocks = 2
    Ablation showed 4 or 6 blocks give no improvement (Fig 9a).
assumptions (4)
  • domain assumption VQGAN discrete tokens at 16x downsampling are a sufficient representation for high-fidelity AR generation
    The method builds on LlamaGen's VQGAN tokenizer (Sec 3.1); reconstruction quality of the tokenizer bounds generation quality.
  • ad hoc to paper Visual vocabulary dimensional redundancy is spatially local and can be exploited by channel-wise fusion of neighboring tokens
    Sec 3.3 shows rank compression up to 8x helps, but that experiment compresses the codebook embedding dimension, not spatially fused tokens; the spatial-locality assumption is never directly verified.
  • domain assumption Causal attention with local-window mutual interaction (modified mask after unshuffle) preserves trainability and quality
    Sec B.3 describes the mask change; no controlled comparison of this mask against the standard causal mask on final image quality is provided.
  • domain assumption GenAI-Bench VQAScore and GenEval are valid proxies for text-to-image quality
    These are the primary quantitative evidence (Tables 1 and 2); human evaluations are also used but vendor details are sparse.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Token-Shuffle: Towards High-Resolution Image Generation with Autoregressive Models." pith.science (2026). https://pith.science/paper/AXXZFVWM

@misc{pith2026250417789,
  author       = {Pith},
  title        = {Pith review of: Token-Shuffle: Towards High-Resolution Image Generation with Autoregressive Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AXXZFVWM}},
  note         = {Machine review of arXiv:2504.17789}
}
read the original abstract

Autoregressive (AR) models, long dominant in language generation, are increasingly applied to image synthesis but are often considered less competitive than Diffusion-based models. A primary limitation is the substantial number of image tokens required for AR models, which constrains both training and inference efficiency, as well as image resolution. To address this, we present Token-Shuffle, a novel yet simple method that reduces the number of image tokens in Transformer. Our key insight is the dimensional redundancy of visual vocabularies in Multimodal Large Language Models (MLLMs), where low-dimensional visual codes from visual encoder are directly mapped to high-dimensional language vocabularies. Leveraging this, we consider two key operations: token-shuffle, which merges spatially local tokens along channel dimension to decrease the input token number, and token-unshuffle, which untangles the inferred tokens after Transformer blocks to restore the spatial arrangement for output. Jointly training with textual prompts, our strategy requires no additional pretrained text-encoder and enables MLLMs to support extremely high-resolution image synthesis in a unified next-token prediction way while maintaining efficient training and inference. For the first time, we push the boundary of AR text-to-image generation to a resolution of 2048x2048 with gratifying generation performance. In GenAI-benchmark, our 2.7B model achieves 0.77 overall score on hard prompts, outperforming AR models LlamaGen by 0.18 and diffusion models LDM by 0.15. Exhaustive large-scale human evaluations also demonstrate our prominent image generation ability in terms of text-alignment, visual flaw, and visual appearance. We hope that Token-Shuffle can serve as a foundational design for efficient high-resolution image generation within MLLMs.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. VTBench: Evaluating Visual Tokenizers for Autoregressive Image Generation

    cs.CV 2025-05 conditional novelty 6.0 of 10

    VTBench evaluates visual tokenizers in isolation across reconstruction, detail, and text tasks, and finds discrete tokenizers lag continuous VAEs.

  2. Vision as a Dialect: Unifying Visual Understanding and Generation via Text-Aligned Representations

    cs.CV 2025-06 conditional novelty 5.0 of 10

    Images are encoded into discrete tokens projected from LLM embeddings, so a single autoregressive model does visual understanding and generation with matched or improved benchmark scores.

Reference graph

Works this paper leans on

42 extracted references · 9 canonical work pages · cited by 2 Pith papers

  1. [1]

    Cm3: A causal masked multimodal model of the internet.arXiv preprint arXiv:2201.07520,

    Armen Aghajanyan, Bernie Huang, Candace Ross, Vladimir Karpukhin, Hu Xu, Naman Goyal, Dmytro Okhonko, Mandar Joshi, Gargi Ghosh, Mike Lewis, et al. Cm3: A causal masked multimodal model of the internet.arXiv preprint arXiv:2201.07520,

    arXiv

  2. [2]

    The prompts are from GenEval Ghosh et al

    We show generated images focusing on position, color, counting, and combination. The prompts are from GenEval Ghosh et al. (2024) prompts. 21 a comic about two cats doing research A bowl of beef Pho a penguin standing on a sidewalkthree wolf moon but with cats instead of wolves A Christmas tree with lights and teddy bear a shibainuwearing a beret and blac...

    work page 2024

  3. [4]

    Longnet: Scaling transformers to 1,000,000,000 tokens.arXiv preprint arXiv:2307.02486,

    Jiayu Ding, Shuming Ma, Li Dong, Xingxing Zhang, Shaohan Huang, Wenhui Wang, Nanning Zheng, and Furu Wei. Longnet: Scaling transformers to 1,000,000,000 tokens.arXiv preprint arXiv:2307.02486,

    arXiv

  4. [5]

    The llama 3 herd of models.arXiv preprint arXiv:2407.21783,

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783,

    arXiv

  5. [6]

    Yuying Ge, Sijie Zhao, Jinguo Zhu, Yixiao Ge, Kun Yi, Lin Song, Chen Li, Xiaohan Ding, and Ying Shan

    https://arxiv.org/abs/2410.13863. Yuying Ge, Sijie Zhao, Jinguo Zhu, Yixiao Ge, Kun Yi, Lin Song, Chen Li, Xiaohan Ding, and Ying Shan. Seed-x: Multimodal models with unified multi-granularity comprehension and generation.arXiv preprint arXiv:2404.14396,

    arXiv

  6. [7]

    Emu video: Factorizing text-to-video generation by explicit image conditioning

    Rohit Girdhar, Mannat Singh, Andrew Brown, Quentin Duval, Samaneh Azadi, Sai Saketh Rambhatla, Akbar Shah, Xi Yin, Devi Parikh, and Ishan Misra. Emu video: Factorizing text-to-video generation by explicit image conditioning. arXiv preprint arXiv:2311.10709,

    arXiv

  7. [8]

    Better & faster large language models via multi-token prediction.arXiv preprint arXiv:2404.19737,

    Fabian Gloeckle, Badr Youbi Idrissi, Baptiste Rozière, David Lopez-Paz, and Gabriel Synnaeve. Better & faster large language models via multi-token prediction.arXiv preprint arXiv:2404.19737,

    arXiv

  8. [9]

    Mamba: Linear-time sequence modeling with selective state spaces

    Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752,

    arXiv

Show all 42 references
  1. [10]

    Elasticdiffusion: Training-free arbitrary size image generation

    Moayed Haji-Ali, Guha Balakrishnan, and Vicente Ordonez. Elasticdiffusion: Training-free arbitrary size image generation. arXiv preprint arXiv:2311.18822,

    arXiv

  2. [11]

    Clipscore: A reference-free evaluation metric for image captioning.arXiv preprint arXiv:2104.08718,

    Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation metric for image captioning.arXiv preprint arXiv:2104.08718,

    arXiv

  3. [12]

    However, as training progresses, the model consistently generates visual tokens up to the <|end_of_image|> token, resulting in complete images

    16 For mixed text-image tokens, we observe that during the early stages of training, the model is more prone to generating such outputs. However, as training progresses, the model consistently generates visual tokens up to the <|end_of_image|> token, resulting in complete imag...

    2023

  4. [13]

    Genai-bench: Evaluating and improving compositional text-to-visual generation.arXiv preprint arXiv:2406.13743, 2024a

    Baiqi Li, Zhiqiu Lin, Deepak Pathak, Jiayao Li, Yixin Fei, Kewen Wu, Tiffany Ling, Xide Xia, Pengchuan Zhang, Graham Neubig, et al. Genai-bench: Evaluating and improving compositional text-to-visual generation.arXiv preprint arXiv:2406.13743, 2024a. Hao Li, Changyao Tian, Jie ...

    2014 arXiv

  5. [15]

    Flow matching for generative modeling

    Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.02747,

    arXiv

  6. [17]

    Open-magvit2: An open-source project toward democratizing auto-regressive visual generation.arXiv preprint arXiv:2409.04410,

    Zhuoyan Luo, Fengyuan Shi, Yixiao Ge, Yujiu Yang, Limin Wang, and Ying Shan. Open-magvit2: An open-source project toward democratizing auto-regressive visual generation.arXiv preprint arXiv:2409.04410,

    arXiv

  7. [18]

    Star: Scale-wise text-to-image generation via auto-regressive representations.arXiv preprint arXiv:2406.10797,

    Xiaoxiao Ma, Mohan Zhou, Tao Liang, Yalong Bai, Tiejun Zhao, Huaian Chen, and Yi Jin. Star: Scale-wise text-to-image generation via auto-regressive representations.arXiv preprint arXiv:2406.10797,

    arXiv

  8. [19]

    Language models are few-shot learners.arXiv preprint arXiv:2005.14165, 1,

    Ben Mann, N Ryder, M Subbiah, J Kaplan, P Dhariwal, A Neelakantan, P Shyam, G Sastry, A Askell, S Agarwal, et al. Language models are few-shot learners.arXiv preprint arXiv:2005.14165, 1,

    2005 arXiv

  9. [21]

    Rwkv: Reinventing rnns for the transformer era.arXiv preprint arXiv:2305.13048,

    Bo Peng, Eric Alcaide, Quentin Anthony, Alon Albalak, Samuel Arcadinho, Stella Biderman, Huanqi Cao, Xin Cheng, Michael Chung, Matteo Grella, et al. Rwkv: Reinventing rnns for the transformer era.arXiv preprint arXiv:2305.13048,

    arXiv

  10. [22]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis.arXiv preprint arXiv:2307.01952,

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rom- bach. Sdxl: Improving latent diffusion models for high-resolution image synthesis.arXiv preprint arXiv:2307.01952,

    arXiv

  11. [23]

    Movie gen: A cast of media foundation models.arXiv preprint arXiv:2410.13720,

    13 Adam Polyak, Amit Zohar, Andrew Brown, Andros Tjandra, Animesh Sinha, Ann Lee, Apoorv Vyas, Bowen Shi, Chih- Yao Ma, Ching-Yao Chuang, et al. Movie gen: A cast of media foundation models.arXiv preprint arXiv:2410.13720,

    arXiv

  12. [24]

    Freescale: Unleashing the resolution of diffusion models via tuning-free scale fusion.arXiv preprint arXiv:2412.09626,

    Haonan Qiu, Shiwei Zhang, Yujie Wei, Ruihang Chu, Hangjie Yuan, Xiang Wang, Yingya Zhang, and Ziwei Liu. Freescale: Unleashing the resolution of diffusion models via tuning-free scale fusion.arXiv preprint arXiv:2412.09626,

    arXiv

  13. [26]

    Ultrapixel: Advancing ultra-high-resolution image synthesis to new peaks.arXiv preprint arXiv:2407.02158,

    Jingjing Ren, Wenbo Li, Haoyu Chen, Renjing Pei, Bin Shao, Yong Guo, Long Peng, Fenglong Song, and Lei Zhu. Ultrapixel: Advancing ultra-high-resolution image synthesis to new peaks.arXiv preprint arXiv:2407.02158,

    arXiv

  14. [27]

    Progressive distillation for fast sampling of diffusion models

    Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. arXiv preprint arXiv:2202.00512,

    arXiv

  15. [29]

    Visual autoregressive modeling: Scalable image generation via next-scale prediction.arXiv preprint arXiv:2404.02905,

    Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Liwei Wang. Visual autoregressive modeling: Scalable image generation via next-scale prediction.arXiv preprint arXiv:2404.02905,

    arXiv

  16. [30]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023a. Hugo Touvron, Lo...

    arXiv

  17. [31]

    Analysis of classifier-free guidance weight schedulers.arXiv preprint arXiv:2404.13040, 2024a

    14 Xi Wang, Nicolas Dufour, Nefeli Andreou, Marie-Paule Cani, Victoria Fernández Abrevaya, David Picard, and Vicky Kalogeiton. Analysis of classifier-free guidance weight schedulers.arXiv preprint arXiv:2404.13040, 2024a. Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun,...

    arXiv

  18. [32]

    Sana: Efficient high-resolution image synthesis with linear diffusion transformers.arXiv preprint arXiv:2410.10629, 2024a

    Enze Xie, Junsong Chen, Junyu Chen, Han Cai, Haotian Tang, Yujun Lin, Zhekai Zhang, Muyang Li, Ligeng Zhu, Yao Lu, et al. Sana: Efficient high-resolution image synthesis with linear diffusion transformers.arXiv preprint arXiv:2410.10629, 2024a. Jinheng Xie, Weijia Mao, Zechen ...

    arXiv

  19. [33]

    Language model beats diffusion–tokenizer is key to visual generation

    Lijun Yu, José Lezama, Nitesh B Gundavarapu, Luca Versari, Kihyuk Sohn, David Minnen, Yong Cheng, Vighnesh Birodkar, Agrim Gupta, Xiuye Gu, et al. Language model beats diffusion–tokenizer is key to visual generation. arXiv preprint arXiv:2310.05737, 2023a. Lili Yu, Bowen Shi, ...

    arXiv

  20. [34]

    Transfusion: Predict the next token and diffuse images with one multi-modal model

    Chunting Zhou, Lili Yu, Arun Babu, Kushal Tirumala, Michihiro Yasunaga, Leonid Shamis, Jacob Kahn, Xuezhe Ma, Luke Zettlemoyer, and Omer Levy. Transfusion: Predict the next token and diffuse images with one multi-modal model. arXiv preprint arXiv:2408.11039,

    arXiv

  21. [35]

    We provide detailed implementations in Sec

    15 Appendix This supplementary material provides more implementation details, ablation studies, visualization results, discussions and limitations. We provide detailed implementations in Sec. A to provide more insights. We also present more studies and visualization results in...

  22. [39]

    However, we admit that this comparison is not entirely fair for the following reasons: (1) The image tokenizers were not trained under identical conditions, and it is challenging to obtain fairly trained VQGAN models with different down-sampling ratios. (2) During the course o...

  23. [40]

    All images were generated with a shuffle window size of 2, half-linear CFG-scheduler with a scale of 7.5, as stated previously

    To our best knowledge, this is the first time AR-based models can generate such a high-resolution image efficiently and effectively. All images were generated with a shuffle window size of 2, half-linear CFG-scheduler with a scale of 7.5, as stated previously. C Discussions C....

    2024

  24. [42]

    The prompts are from our internal evaluation prompts

    We show two images of same prompt with different random seeds, focusing on complex scenarios or hard prompts. The prompts are from our internal evaluation prompts. 22 A majestic bald eagle is sitting at the top of a branch A sleek black cat with a dashing grey streak on its fo...

  25. [512]

    The 16× VQGAN model is taken from the previous LlamaGen T2I checkpoint, while the8×VQGAN is derived from our internal checkpoint

    For the comparison, we utilize two VQGAN models with different compression ratios:16× and 8×. The 16× VQGAN model is taken from the previous LlamaGen T2I checkpoint, while the8×VQGAN is derived from our internal checkpoint. We first benchmark both models on the MSCOCO-val data...

    2014

  26. [2014]

    Evaluating text-to-visual generation with image-to-text generation.arXiv preprint arXiv:2404.01291,

    Zhiqiu Lin, Deepak Pathak, Baiqi Li, Jiayao Li, Xide Xia, Graham Neubig, Pengchuan Zhang, and Deva Ramanan. Evaluating text-to-visual generation with image-to-text generation.arXiv preprint arXiv:2404.01291,

    arXiv

  27. [2016]

    Denoising diffusion implicit models.arXiv preprint arXiv:2010.02502, 2020a

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models.arXiv preprint arXiv:2010.02502, 2020a. Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models.arXiv preprint arXiv:2010.02502, 2020b. Peize Sun, Yi Jiang, Shoufa Che...

    2010 arXiv

  28. [2017]

    Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598,

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598,

    arXiv

  29. [2020]

    Transfer between modalities with metaqueries.arXiv preprint arXiv:2504.06256,

    Xichen Pan, Satya Narayan Shukla, Aashu Singh, Zhuokai Zhao, Shlok Kumar Mishra, Jialiang Wang, Zhiyang Xu, Jiuhai Chen, Kunpeng Li, Felix Juefei-Xu, et al. Transfer between modalities with metaqueries.arXiv preprint arXiv:2504.06256,

    arXiv

  30. [2021]

    Hierarchical text-conditional image generation with clip latents.arXiv preprint arXiv:2204.06125, 1(2):3,

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents.arXiv preprint arXiv:2204.06125, 1(2):3,

    arXiv

  31. [2022]

    Lumina-mgpt: Illu- minate flexible photorealistic text-to-image generation with multimodal generative pretraining.arXiv preprint arXiv:2408.02657,

    Dongyang Liu, Shitian Zhao, Le Zhuo, Weifeng Lin, Yu Qiao, Hongsheng Li, and Peng Gao. Lumina-mgpt: Illu- minate flexible photorealistic text-to-image generation with multimodal generative pretraining.arXiv preprint arXiv:2408.02657,

    arXiv

  32. [2023]

    Pixart-/sigma: Weak-to-strong training of diffusion transformer for 4k text-to-image generation

    Junsong Chen, Chongjian Ge, Enze Xie, Yue Wu, Lewei Yao, Xiaozhe Ren, Zhongdao Wang, Ping Luo, Huchuan Lu, and Zhenguo Li. Pixart-/sigma: Weak-to-strong training of diffusion transformer for 4k text-to-image generation. arXiv preprint arXiv:2403.04692,

    arXiv

  33. [2024]

    Emu: Enhancing image generation models using photogenic needles in a haystack

    Xiaoliang Dai, Ji Hou, Chih-Yao Ma, Sam Tsai, Jialiang Wang, Rui Wang, Peizhao Zhang, Simon Vandenhende, Xiaofang Wang, Abhimanyu Dubey, et al. Emu: Enhancing image generation models using photogenic needles in a haystack. arXiv preprint arXiv:2309.15807,

    arXiv

  34. [2048]

    To investigate the cause of unstable training, we analyze the training process in detail

    Training shows instability after approximately 20K iterations. To investigate the cause of unstable training, we analyze the training process in detail. Initially, we hypothesize that the instability arises from using a large learning rate, a common factor in such issues. To t...

    2024

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.