Pith. sign in

REVIEW 3 major objections 4 minor 38 references

MADFormer: Mixed Autoregressive and Diffusion Transformers for Continuous Image Generation

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Under constrained inference compute, allocating more transformer layers to autoregressive conditioning than to diffusion improves FID by up to 60-75% in MADFormer's controlled tests.

desk verdict A useful testbed for AR-diffusion hybrids whose headline 60-75% FID gain is not established because it compares at equal NFE rather than equal FLOPs. read the letter →

arxiv 2506.07999 v1 pith:PMFP7YDY submitted 2025-06-09 cs.CV cs.LG

Junhao Chen , Yulia Tsvetkov , Xiaochuang Han This is my paper
classification cs.CVcs.LG
keywords MADFormerautoregressivediffusionhybridimagegenerationcontinuouslatentspaceFIDlayerallocationinferencecomputebudgetblock-wisepartitioning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to answer a design question: when a transformer is split between autoregressive (AR) conditioning and diffusion refinement for image generation, how should the layers be allocated? The authors build MADFormer, a testbed that partitions an image into spatial blocks, uses early layers to autoregressively condition each block on earlier blocks, and uses later layers to denoise the current block. Across controlled experiments on FFHQ-1024 and ImageNet, they find that under constrained inference compute, AR-heavy splits outperform diffusion-heavy splits by up to 60-75% in FID; at high compute the advantage flips. They also find that block-wise partitioning helps at high resolution, with the best number of blocks depending on resolution. If these trends hold, they give a practical rule for laying out hybrid generators.

What carries the argument

The load-bearing mechanism is a layer-wise split of the transformer into an AR conditioning stage and a diffusion denoising stage. The first N-D layers compute a conditioning state from previously generated clean image blocks; the final D layers add that state to the noised latent of the current block and predict the clean latent. Images are mapped to continuous VAE latents and grouped into blocks, with bidirectional attention inside a block and causal attention across blocks. A second control knob is the AR length, the number of spatial blocks into which an image is partitioned, which sets the granularity of the autoregressive prior.

What would settle it

Compute-matched evaluation: hold total FLOPs or wall-clock time per sample constant and compare AR-heavy and diffusion-heavy splits at the same FID-measurement protocol; if the FID gap shrinks or vanishes, the paper's allocation rule is a cost artifact. Concretely, run the AR-heavy split with more denoising steps (or the diffusion-heavy split with fewer) until per-sample FLOPs are equal, then re-measure FID on the same checkpoint.

Watch

Extended reading notes

Core claim

The central discovery is a compute-dependent allocation rule. With a fixed 28-layer transformer, an AR-heavy split (e.g., 21 autoregressive layers and 7 diffusion layers) reaches a lower FID than an all-diffusion split when the number of denoising steps is small, improving FID by 60-75%; when the inference budget grows, the diffusion-heavy split wins. The paper interprets this as AR layers efficiently establishing global structure, while diffusion layers refine local detail at higher per-step cost. A second discovery is that block granularity matters: FFHQ-1024 performs best with 16 blocks, while ImageNet at 256x256 prefers a single block, indicating that the optimal AR length scales with image resolution.

Load-bearing premise

The comparison that produces the 60-75% headline keeps the number of denoising steps equal while the layer split changes, so an AR-heavy model uses fewer layers per step and thus less compute per sample; if that cost difference rather than the allocation itself drives the FID gap, the design rule collapses.

Editorial extensions

If this is right

  • With a fixed total depth and a small number of denoising steps, an AR-heavy layer split is the better choice for FID.
  • With a larger inference budget, diffusion-heavy splits pull ahead, so the optimal allocation is budget-dependent rather than universal.
  • High-resolution images benefit from finer block partitioning, while lower-resolution images prefer coarser or no partitioning.
  • Both clean-block prepending and AR conditioning contribute independently to image quality; removing either raises FID.
  • The loss design matters less than the architectural split: the hidden loss on conditioning helps, while separate parameter sets per modality give little gain.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication the authors do not draw: because Figure 4 compares layer splits at equal denoising steps, the AR-heavy model runs fewer layers per step and therefore fewer FLOPs per sample, so part of the 60-75% FID gain may reflect per-step compute cost rather than a superior inductive bias. A compute-matched comparison would settle this.
  • A testable extension of the testbed is an inference-time adaptive schedule that shifts layers or steps between AR conditioning and diffusion depending on the remaining compute budget, mirroring the paper's finding that the optimal split depends on available compute.
  • The resolution-dependent AR length result suggests a scaling rule for block partitioning that could be checked on higher-resolution images and larger models, where the authors' data point is limited to 256x256 and 1024x1024.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes MADFormer, a unified Transformer for continuous image generation that mixes autoregressive and diffusion modeling along two axes: spatial token blocks and network depth. Image latents are partitioned into spatial blocks; early transformer layers act as an AR conditioning module that processes previous clean blocks, while the remaining D layers denoise the current block with a diffusion objective. Controlled ablations on FFHQ-1024 and ImageNet vary block granularity, AR/diffusion layer split, auxiliary clean-block/AR-conditioning modules, parameter sharing, and loss terms. The paper's central claims are that block-wise AR partitioning helps at high resolution, and that under constrained inference compute an AR-heavy layer allocation improves FID by up to 60-75% relative to diffusion-heavy allocations, while diffusion-heavy allocations win at larger budgets.

Significance. This is a useful design-space study: it packages several architectural choices into one testbed, reports ablations over a broad set of axes, and is honest about the limited training budget (Sec. 3.3). If the NFE-vs-FLOPs confound were resolved, the qualitative guidance—AR layers are cheap sources of global conditioning, diffusion layers refine locally—would be a practical contribution. However, the headline quantitative claim is not yet supported: the 60-75% figure is computed at equal NFE, and because only D of 28 layers run per denoising step, equal NFE does not mean equal inference compute. The manuscript also contains a material configuration inconsistency (24 vs 28 layers) and an incomplete baseline for the block-partitioning claim. The paper is transparent about some of its limitations, but the central compute-efficiency conclusion needs additional evidence.

major comments (3)
  1. [Sec. 4.1, Fig. 4, Eqs. (1)-(5), Table 1 footnote] The central claim that AR-heavy allocations improve FID by 60-75% under 'constrained inference compute' is measured against NFE, but NFE is not a compute-neutral metric when D varies. In Eqs. (4)-(5) only the D diffusion layers execute per denoising step, while the N-D AR layers (Eqs. (1)-(3)) run once per block. At fixed NFE, an all-diffusion model incurs roughly 28 x NFE layer-passes, whereas a d=7 AR-heavy model incurs roughly 21 + 7 x NFE layer-passes per block; at NFE=280 this is about a 4x FLOPs difference. The footnote to Table 1 explicitly acknowledges that diffusion-heavy models have higher NFE in that setup, confirming that the two settings are not compute-matched. The paper does not report FLOPs-matched or wall-clock curves. Please add such curves, or reframe the conclusion narrowly as 'AR-heavy allocations help at equal number of denoising steps'; the practical claim about compute efficiency is not established by the present data.
  2. [Appendix A, Table 7 vs. Sec. 4.1 and Table 1] The appendix configuration table lists '# layers 24' and 'Diffusion Parameters: # diffusion layers 24', while the main text repeatedly uses a fixed 28-layer budget and reports splits d=7, 14, 21, 28 (Table 1 and Fig. 4). This is more than a typo: if the actual models have 24 layers, the reported layer-ratio comparisons are not reproducible and the '3:1 AR:Diffusion' description would be wrong; if they have 28, the appendix must be corrected. Please reconcile the configuration table with the model descriptions in Sec. 3.3 and Sec. 4.1.
  3. [Sec. 4.2, Table 2] The claim that block-wise partitioning significantly improves high-resolution image generation is not fully supported by the reported table. On FFHQ-1024 the best AR length is 16 blocks (FID 17.8) versus 4 blocks (18.9) and 64 blocks (21.9), but there is no l=1 (unpartitioned) baseline, so one cannot conclude that partitioning per se helps; on ImageNet, l=1 is best and partitioning degrades FID monotonically. Either add an FFHQ l=1 comparison or soften the conclusion to 'finer partitioning helps up to a point on FFHQ-1024.'
minor comments (4)
  1. [Sec. 4.6, Table 6] The FID score difference 17.76 versus 17.79 is very likely within the sampling noise of an 8,000-sample FID estimate, yet the table implies a quantitative ranking; please report confidence intervals or multiple seeds, or state that the clean-tower loss has no measurable effect.
  2. [Sec. 3.3, Evaluation] The text says FID scores are averaged over the last five checkpoints, but no per-checkpoint variance is reported; a sentence on the spread of those five values would help readers assess the reliability of small differences throughout the ablations.
  3. [Table 1 footnote] The footnote says diffusion-heavy models have higher NFE in that setup, but Table 1 does not list NFE; please state the exact NFE used for each row so the reader can see the confound explicitly.
  4. [Throughout] There are several formatting artifacts (e.g., 'MADF ormer' in the abstract, the stray example sentence in Fig. 1, and the unlabeled numerical values in Fig. 4); these should be cleaned up before a revised version is submitted.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's claims are empirical summaries of its own ablations, not derivations or fitted predictions; the only self-citation is peripheral and non-load-bearing.

full rationale

MADFormer is an empirical design-space study, not a derivation. The central claims—(1) block-wise partitioning helps high-resolution images and (2) AR-heavy layer splits improve FID by 60–75% under constrained inference compute—are direct summaries of ablation tables and Figure 4. No equation in the paper is constructed so that an output equals an input by definition; no parameter is fit to a subset and then reported as a prediction of that same subset. The NFE-versus-FLOPs confound identified by the skeptic is a genuine threat to the interpretation of Figure 4, but it is a measurement/control concern, not circularity: the reported FID values are measured, not derived from the model definition or from a fitted parameter. The paper even flags the related calibration issue in footnote 3 ('Since the diffusion process steps through more layers with diffusion-heavy models than AR-heavy models, the NFE in the setup for diffusion-heavy models is higher, a different setup from Figure 4'). The only self-citation is LMFusion [Shi et al., 2024], co-authored by Xiaochuang Han, used as motivation for trying separate parameter sets (Section 4.4); the ablation itself shows a trivial effect, and the central AR/diffusion allocation findings do not depend on that citation. Since the empirical results are externally checkable and not forced by construction, the circularity score is 0.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claims depend on tuned hyperparameters (loss weights, block lengths, diffusion depth) and on unproven measurement assumptions. There are no newly invented physical or conceptual entities; the 'clean tower' and 'noise tower' are parameter-sharing schemes, not new entities.

free parameters (5)
  • lambda_hidden = 0.1
    Tuned by FID ablation in Table 6; 0.1 gives best FFHQ FID 17.8 vs 19.4 at 0 and 18.4 at 1.
  • lambda_clean = 0.0
    Set to 0 after ablation showed negligible effect (FIDs 17.76, 17.79, 18.42 for 0, 0.1, 1).
  • AR block length for FFHQ = 16
    Best on FFHQ-1024 in Table 2 (FID 17.8 vs 18.9 for 4 blocks and 21.9 for 64 blocks); used to support the resolution-dependent block-size insight.
  • AR block length for ImageNet = 1
    Best on ImageNet in Table 2 (FID 28.4 vs 30.0 for 4 blocks); illustrates the resolution dependence.
  • Diffusion depth for headline AR-heavy split = d=7
    The 3:1 AR-to-diffusion split in Figure 4; the 60-75% improvement claim compares this to d=28 at low NFE.
assumptions (5)
  • domain assumption FID computed on the stated sample sizes is a reliable basis for ranking configurations.
    FID is the only metric; no confidence intervals are given, so small differences are treated as meaningful.
  • domain assumption Continuous VAE latents from Stable Diffusion support both autoregressive factorization and diffusion denoising.
    Section 3.1; this is inherited from prior work and not independently validated here.
  • domain assumption All ablation runs consume matched training compute (256 GPU hours each), so architecture, not compute, explains differences.
    Appendix A states this, but no logs or verification are provided.
  • ad hoc to paper NFE accurately represents inference compute for comparing layer splits.
    Figure 4 uses NFE as the budget axis without accounting for per-step FLOPs, which vary with diffusion depth.
  • domain assumption Findings from 50-epoch ImageNet training and 210k-step FFHQ training generalize to longer training.
    The authors disclaim absolute FID but still present design principles as generalizable.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MADFormer: Mixed Autoregressive and Diffusion Transformers for Continuous Image Generation." pith.science (2026). https://pith.science/paper/PMFP7YDY

@misc{pith2026250607999,
  author       = {Pith},
  title        = {Pith review of: MADFormer: Mixed Autoregressive and Diffusion Transformers for Continuous Image Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PMFP7YDY}},
  note         = {Machine review of arXiv:2506.07999}
}
read the original abstract

Recent progress in multimodal generation has increasingly combined autoregressive (AR) and diffusion-based approaches, leveraging their complementary strengths: AR models capture long-range dependencies and produce fluent, context-aware outputs, while diffusion models operate in continuous latent spaces to refine high-fidelity visual details. However, existing hybrids often lack systematic guidance on how and why to allocate model capacity between these paradigms. In this work, we introduce MADFormer, a Mixed Autoregressive and Diffusion Transformer that serves as a testbed for analyzing AR-diffusion trade-offs. MADFormer partitions image generation into spatial blocks, using AR layers for one-pass global conditioning across blocks and diffusion layers for iterative local refinement within each block. Through controlled experiments on FFHQ-1024 and ImageNet, we identify two key insights: (1) block-wise partitioning significantly improves performance on high-resolution images, and (2) vertically mixing AR and diffusion layers yields better quality-efficiency balances--improving FID by up to 75% under constrained inference compute. Our findings offer practical design principles for future hybrid generative models.

Figures

Figures reproduced from arXiv: 2506.07999 by the authors.

Figure 1
Figure 1. High-level overview of the MADFormer architecture. A single Transformer processes all modalities as a unified sequence. Text tokens follow a next-token prediction objective, while image tokens are grouped into blocks trained autoregressively with a diffusion objective. We use separate parameters (FFNs, QKVO projections, and layer norms) for each modality. The Transformer is divided into two stages: early layers prod… view at source ↗
Figure 2
Figure 2. Qualitative results of MADFormer with 14 diffusion layers (out of 28 total). The AR conditioning mechanism enables strong sample quality even under limited inference steps. More qualitative results can be found in Appendix B. Continuous Image Tokenizers In image generation, we operate on continuous VAE latents rather than discrete tokens, leveraging more mature and effective tokenizers without quantization in the la… view at source ↗
Figure 3
Figure 3. Architectural details of MADFormer’s image tower. The hidden loss and clean tower loss are computed using the clean tower output and conditioning input, compared against the ground truth image block. Loss Function. Prior work has emphasized the importance of loss function design for sta￾ble training and high-quality generation. In line with common practice, we apply negative log￾likelihood loss on text tokens and me… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: FID vs. NFE for different AR-Diffusion layer ratios. Under low NFE (i.e., constrained compute), AR-heavy configurations (e.g., 3:1 AR:Diffusion with d = 7) consistently outperform diffusion-heavy ones (e.g., all diffusion layers with d = 28), decreasing FID by up to 60…
Figure 5
Figure 5. Figure 5: Samples on FFHQ 1024 × 1024 with 9 inference steps [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: Samples on FFHQ 1024 × 1024 with 19 inference steps. 14 [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: Samples on FFHQ 1024 × 1024 with 49 inference steps [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Samples on ImageNet with 199 inference steps. 15 [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

38 extracted references · 11 canonical work pages

  1. [1]

    Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, Seyedeh Sara Mahdavi, Raphael Gontijo Lopes, Tim Salimans, Jonathan Ho, David J

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L. Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, Seyedeh Sara Mahdavi, Raphael Gontijo Lopes, Tim Salimans, Jonathan Ho, David J. Fleet, and Mohammad Norouzi. Photorealistic text-to-image diffusion models with deep language understanding. ArXiv, abs/2205.11487, 2022. UR...

    arXiv 2022

  2. [2]

    Semantic-conditional diffusion networks for image captioning*

    Jianjie Luo, Yehao Li, Yingwei Pan, Ting Yao, Jianlin Feng, Hongyang Chao, and Tao Mei. Semantic-conditional diffusion networks for image captioning*. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 23359--23368, 2022. URL https://api.semanticscholar.org/CorpusID:254275438

    work page 2023

  3. [3]

    Metaxas, and S

    Ligong Han, Jian Ren, Hsin-Ying Lee, Francesco Barbieri, Kyle Olszewski, Shervin Minaee, Dimitris N. Metaxas, and S. Tulyakov. Show me what and tell me how: Video synthesis via multimodal conditioning. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3605--3615, 2022. URL https://api.semanticscholar.org/CorpusID:247292765

    work page 2022

  4. [4]

    Moonshot: Towards controllable video generation and editing with multimodal conditions

    David Junhao Zhang, Dongxu Li, Hung Le, Mike Zheng Shou, Caiming Xiong, and Doyen Sahoo. Moonshot: Towards controllable video generation and editing with multimodal conditions. ArXiv, abs/2401.01827, 2024. URL https://api.semanticscholar.org/CorpusID:266741873

    arXiv 2024

  5. [5]

    Taming transformers for high-resolution image synthesis

    Patrick Esser, Robin Rombach, and Bj \"o rn Ommer. Taming transformers for high-resolution image synthesis. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12868--12878, 2020. URL https://api.semanticscholar.org/CorpusID:229297973

    work page 2021

  6. [6]

    Chameleon: Mixed-modal early-fusion foundation models

    Chameleon Team. Chameleon: Mixed-modal early-fusion foundation models. ArXiv, abs/2405.09818, 2024. URL https://api.semanticscholar.org/CorpusID:269791516

    arXiv 2024

  7. [7]

    Cosmos world foundation model platform for physical ai

    Nvidia Niket Agarwal, Arslan Ali, Maciej Bala, Yogesh Balaji, Erik Barker, Tiffany Cai, Prithvijit Chattopadhyay, Yongxin Chen, Yin Cui, Yifan Ding, and Daniel Dworakowski et al. Cosmos world foundation model platform for physical ai. ArXiv, abs/2501.03575, 2025. URL https://api.semanticscholar.org/CorpusID:275342594

    arXiv 2025

  8. [8]

    Blattmann, Dominik Lorenz, Patrick Esser, and Bj \"o rn Ommer

    Robin Rombach, A. Blattmann, Dominik Lorenz, Patrick Esser, and Bj \"o rn Ommer. High-resolution image synthesis with latent diffusion models. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10674--10685, 2021. URL https://api.semanticscholar.org/CorpusID:245335280

    work page 2022

Show all 38 references
  1. [9]

    Peebles and Saining Xie

    William S. Peebles and Saining Xie. Scalable diffusion models with transformers. 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 4172--4182, 2022. URL https://api.semanticscholar.org/CorpusID:254854389

    2023

  2. [10]

    Patrick Esser, Sumith Kulal, A. Blattmann, Rahim Entezari, Jonas Muller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, Kyle Lacey, Alex Goodwin, Yannik Marek, and Robin Rombach. Scaling rectified flow transformer...

    2024 arXiv

  3. [11]

    Announcing the flux pro finetuning api

    Black Forest Labs. Announcing the flux pro finetuning api. Black Forest Labs Announcements, January 2025. URL https://bfl.ai/announcements

    2025

  4. [12]

    Autoregressive image generation without vector quantization

    Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, and Kaiming He. Autoregressive image generation without vector quantization. ArXiv, abs/2406.11838, 2024. URL https://api.semanticscholar.org/CorpusID:270560593

    2024 arXiv

  5. [13]

    Acdit: Interpolating autoregressive conditional modeling and diffusion transformer

    Jinyi Hu, Shengding Hu, Yuxuan Song, Yufei Huang, Mingxuan Wang, Hao Zhou, Zhiyuan Liu, Wei-Ying Ma, and Maosong Sun. Acdit: Interpolating autoregressive conditional modeling and diffusion transformer. ArXiv, abs/2412.07720, 2024. URL https://api.semanticscholar.org/CorpusID:274610804

    2024

  6. [14]

    Transfusion: Predict the next token and diffuse images with one multi-modal model

    Chunting Zhou, Lili Yu, Arun Babu, Kushal Tirumala, Michihiro Yasunaga, Leonid Shamis, Jacob Kahn, Xuezhe Ma, Luke Zettlemoyer, and Omer Levy. Transfusion: Predict the next token and diffuse images with one multi-modal model. ArXiv, abs/2408.11039, 2024. URL https://api.semant...

    2024 arXiv

  7. [15]

    Show-o: One single transformer to unify multimodal understanding and generation

    Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. Show-o: One single transformer to unify multimodal understanding and generation. ArXiv, abs/2408.12528, 2024. URL https://api.s...

    2024 arXiv

  8. [16]

    Jonathan Ho, Ajay Jain, and P. Abbeel. Denoising diffusion probabilistic models. ArXiv, abs/2006.11239, 2020. URL https://api.semanticscholar.org/CorpusID:219955663

    2006 arXiv

  9. [17]

    Neural discrete representation learning

    A \"a ron van den Oord, Oriol Vinyals, and Koray Kavukcuoglu. Neural discrete representation learning. In Neural Information Processing Systems, 2017. URL https://api.semanticscholar.org/CorpusID:20282961

    2017

  10. [18]

    Minnen, Eirikur Agustsson, and Michael Tschannen

    Fabian Mentzer, David C. Minnen, Eirikur Agustsson, and Michael Tschannen. Finite scalar quantization: Vq-vae made simple. ArXiv, abs/2309.15505, 2023. URL https://api.semanticscholar.org/CorpusID:263153393

    2023 arXiv

  11. [19]

    Minnen, Yong Cheng, Agrim Gupta, Xiuye Gu, Alexander G

    Lijun Yu, Jos \'e Lezama, Nitesh Bharadwaj Gundavarapu, Luca Versari, Kihyuk Sohn, David C. Minnen, Yong Cheng, Agrim Gupta, Xiuye Gu, Alexander G. Hauptmann, Boqing Gong, Ming-Hsuan Yang, Irfan Essa, David A. Ross, and Lu Jiang. Language model beats diffusion -- tokenizer is ...

    2023

  12. [20]

    Givt: Generative infinite-vocabulary transformers

    Michael Tschannen, Cian Eastwood, and Fabian Mentzer. Givt: Generative infinite-vocabulary transformers. ArXiv, abs/2312.02116, 2023. URL https://api.semanticscholar.org/CorpusID:265610025

    2023 arXiv

  13. [21]

    Zettlemoyer, and Lili Yu

    Weijia Shi, Xiaochuang Han, Chunting Zhou, Weixin Liang, Xi Victoria Lin, Luke S. Zettlemoyer, and Lili Yu. Lmfusion: Adapting pretrained language models for multimodal generation. ArXiv, abs/2412.15188, 2024. URL https://api.semanticscholar.org/CorpusID:274860024

    2024 arXiv

  14. [22]

    Hartshorn, Aobo Yang, Archi Mitra, and Archie Sravankumar et al

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony S. Hartshorn, Aobo Yang, Archi Mitra, and Archie Sravankumar et al. The llama 3 herd of models. ArXiv, abs...

    2024 arXiv

  15. [23]

    Improved denoising diffusion probabilistic models

    Alex Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. ArXiv, abs/2102.09672, 2021. URL https://api.semanticscholar.org/CorpusID:231979499

    2021 arXiv

  16. [24]

    Zettlemoyer, and Xi Victoria Lin

    Weixin Liang, Lili Yu, Liang Luo, Srinivasan Iyer, Ning Dong, Chunting Zhou, Gargi Ghosh, Mike Lewis, Wen tau Yih, Luke S. Zettlemoyer, and Xi Victoria Lin. Mixture-of-transformers: A sparse and scalable architecture for multi-modal foundation models. ArXiv, abs/2411.04996, 20...

    2024 arXiv

  17. [25]

    Flex attention: A programming model for generating optimized attention kernels

    Juechu Dong, Boyuan Feng, Driss Guessous, Yanbo Liang, and Horace He. Flex attention: A programming model for generating optimized attention kernels. ArXiv, abs/2412.05496, 2024. URL https://api.semanticscholar.org/CorpusID:274598006

    2024 arXiv

  18. [26]

    Progressive distillation for fast sampling of diffusion models, 2022

    Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models, 2022. URL https://arxiv.org/abs/2202.00512

    2022 arXiv

  19. [27]

    A style-based generator architecture for generative adversarial networks

    Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4396--4405, 2018. URL https://api.semanticscholar.org/CorpusID:54482423

    2019

  20. [28]

    Berg, and Li Fei-Fei

    Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge . International Journal of Computer Vision (IJ...

    2015 doi

  21. [29]

    Gans trained by a two time-scale update rule converge to a nash equilibrium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, G \"u nter Klambauer, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a nash equilibrium. ArXiv, abs/1706.08500, 2017. URL https://api.semanticscholar.org/CorpusID:231697514

    2017 arXiv

  22. [30]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. ArXiv, abs/2010.02502, 2020. URL https://api.semanticscholar.org/CorpusID:222140788

    2010 arXiv

  23. [31]

    Hierarchical text-conditional image generation with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. ArXiv, abs/2204.06125, 2022. URL https://api.semanticscholar.org/CorpusID:248097655

    2022 arXiv

  24. [32]

    Monoformer: One transformer for both diffusion and autoregression

    Chuyang Zhao, Yuxin Song, Wenhao Wang, Haocheng Feng, Errui Ding, Yifan Sun, Xinyan Xiao, and Jingdong Wang. Monoformer: One transformer for both diffusion and autoregression. ArXiv, abs/2409.16280, 2024. URL https://api.semanticscholar.org/CorpusID:272832492

    2024 arXiv

  25. [33]

    Multimodal latent language modeling with next-token diffusion

    Yutao Sun, Hangbo Bao, Wenhui Wang, Zhiliang Peng, Li Dong, Shaohan Huang, Jianyong Wang, and Furu Wei. Multimodal latent language modeling with next-token diffusion. ArXiv, abs/2412.08635, 2024. URL https://api.semanticscholar.org/CorpusID:274638148

    2024 arXiv

  26. [34]

    Unified-io: A unified model for vision, language, and multi-modal tasks

    Jiasen Lu, Christopher Clark, Rowan Zellers, Roozbeh Mottaghi, and Aniruddha Kembhavi. Unified-io: A unified model for vision, language, and multi-modal tasks. ArXiv, abs/2206.08916, 2022. URL https://api.semanticscholar.org/CorpusID:249848272

    2022 arXiv

  27. [35]

    Unified-io 2: Scaling autoregressive multimodal models with vision, language, audio, and action

    Jiasen Lu, Christopher Clark, Sangho Lee, Zichen Zhang, Savya Khosla, Ryan Marten, Derek Hoiem, and Aniruddha Kembhavi. Unified-io 2: Scaling autoregressive multimodal models with vision, language, audio, and action. 2024 IEEE/CVF Conference on Computer Vision and Pattern Reco...

    2024

  28. [36]

    Addendum to gpt-4o system card: Native image generation, 2025

    OpenAI . Addendum to gpt-4o system card: Native image generation, 2025. URL https://cdn.openai.com/11998be9-5319-4302-bfbf-1167e093f1fb/Native_Image_Generation_System_Card.pdf

    2025

  29. [37]

    Visual autoregressive modeling: Scalable image generation via next-scale prediction

    Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Liwei Wang. Visual autoregressive modeling: Scalable image generation via next-scale prediction. ArXiv, abs/2404.02905, 2024. URL https://api.semanticscholar.org/CorpusID:268876071

    2024 arXiv

  30. [38]

    Hart: Efficient visual generation with hybrid autoregressive transformer

    Haotian Tang, Yecheng Wu, Shang Yang, Enze Xie, Junsong Chen, Junyu Chen, Zhuoyang Zhang, Han Cai, Yao Lu, and Song Han. Hart: Efficient visual generation with hybrid autoregressive transformer. ArXiv, abs/2410.10812, 2024. URL https://api.semanticscholar.org/CorpusID:273346791

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.