Pith. sign in

REVIEW 5 major objections 5 minor 1 cited by

Context-Aware Autoregressive Models for Multi-Conditional Image Generation

T0 review · 5 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Autoregressive model blends any subset of image conditions

desk verdict A solid autoregressive take on multi-condition control that deserves referee time, despite a rushed write-up and a few under-tested design choices. read the letter →

arxiv 2505.12274 v1 pith:FRTDBCI4 submitted 2025-05-18 cs.CV

Yixiao Chen , Zhiyuan Ma , Guoli Jia , Che Jiang , Jianjun Li , Bowen Zhou This is my paper · ORCID
classification cs.CV
keywords multi-conditionalimagegenerationautoregressivetransformersunifiedtokensequencehybridpositionalencodingattentionmaskingclassifier-freeguidancevectorquantizationcontrollable
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that an autoregressive transformer can solve multi-conditional image generation with one unified token stream, matching or beating diffusion-based systems that need task-specific adapters. It proposes ContextAR, which writes every visual condition—edges, depth maps, poses, subject images—plus text into a single sequence and predicts image tokens from that sequence. The key claim is versatility without retraining: after one joint training run, any subset of conditions can be switched on during inference. If the claim holds, multi-conditional control becomes a property of autoregressive next-token prediction rather than a collection of per-condition model patches.

What carries the argument

The load-bearing object is the unified sequence $S=[c_1,\dots,c_m,c_T,q]$ together with a mask that decides which tokens see which. Cross-Condition Perception Restriction removes dot-products between different condition types, cutting attention complexity from quadratic to linear in the number of conditions; Intra-Condition Bidirectional Perception drops the causal mask inside each condition block during prefill so the model sees each condition map as a whole. Hybrid positional encoding applies 2D rotary position embedding with image coordinates to every condition, then adds a learnable offset $P_k$ per condition type, so stacked condition maps stay pixel-aligned yet remain distinguishable. This combination is what makes subset selection possible: the model never depends on any particular pairing of conditions, because every condition is always processed as an independent, fully visible block.

What would settle it

Run each condition map through the shared VQ encoder and decoder and measure how faithfully it returns: canny F1, depth MSE, HED SSIM, and pose joint distance. If thin edges or fine pose lines are destroyed by discretization, ContextAR's control signal is already degraded before the transformer sees it, and the central claim would need a different condition representation.

Watch

Extended reading notes

Core claim

The central claim is that condition composition can be made arbitrary by construction rather than by exhaustive training on every combination. Three mechanisms carry this: separate embedding layers per condition type, initialized from the image embedding; hybrid positional encodings that add a condition-specific learnable offset on top of shared 2D rotary positions; and a conditional-context attention mask that blocks attention between different condition types while allowing bidirectional attention inside each condition. Because each visual condition is dropped independently during training with probability 0.25, the model learns to generate from any surviving subset, and at inference unused condition blocks are simply omitted. On the four-condition benchmark the paper reports FID 10.42 and SSIM 53.47, above the diffusion baseline's 11.85 and 43.99, and on subject-plus-spatial conditions it matches or exceeds the diffusion baseline on F1, depth MSE, and CLIP-I.

Load-bearing premise

The shared pretrained VQ encoder that turns both condition maps and target images into discrete tokens must preserve enough condition-specific structure—thin edges, depth gradients, pose lines—that the later embedding and attention layers can still read it; the paper does not directly test this link.

Editorial extensions

If this is right

  • A single jointly trained checkpoint can serve any subset of conditions at inference, so users can toggle conditions on and off without reloading or fine-tuning the model.
  • Attention cost grows linearly rather than quadratically with the number of condition types, making four or more simultaneous conditions practical for training and prefill.
  • The model can be trained once on a fixed set of condition types and then evaluated on every combination of them, which the paper demonstrates on single, pairs, triples, and all four spatial conditions.
  • Because condition tokens attend bidirectionally, each condition map is understood as a complete input; this is what allows all condition tokens to be prefilled and cached before image decoding begins.
  • The paper's classifier-free guidance results show a single guidance scale can balance fidelity and controllability, suggesting the condition-drop training strategy transfers across condition subsets.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A stress test the paper does not run: add condition types unseen in training, such as segmentation or normal maps. If the tokenizer preserves them, the architecture suggests each new type only needs an embedding table and a positional offset, not a new model.
  • The mask-based design implies a live-editing use case: a user could change one condition and re-prefill only that condition block while reusing cached keys and values for the others, making interactive correction cheaper than full regeneration.
  • The shared tokenizer assumption could be tested directly with a reconstruction benchmark, since the paper's own condition metrics would expose any information loss before the transformer runs.
  • The paper's linear-complexity argument suggests the framework should scale to many condition types, but the bottleneck may shift to the quality of the discrete tokenizer rather than the transformer itself; that is a testable boundary for future work.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes ContextAR, an autoregressive transformer framework for multi-conditional image generation. Visual conditions (canny, depth, HED, pose, subject) and text are embedded into a unified token sequence, with per-condition embedding layers, a hybrid RoPE-plus-learnable-positional-embedding scheme, a Cross-Condition Perception Restriction (CCPR) attention mask, and Intra-Condition Bidirectional Perception (ICBP). The central claim is that, without fine-tuning, the model can activate any subset of conditions at inference, and that it achieves controllability competitive with state-of-the-art diffusion-based multi-conditional methods. Experiments on MultiGen-20M and SubjectSpatial200K report FID, SSIM, MUSIQ, and condition-specific metrics, with ablations of CFG scale, positional embeddings, and attention design.

Significance. If the claims hold, ContextAR provides a genuinely different route to multi-conditional image generation: instead of adding adapters or task-specific diffusion branches, it treats all conditions as tokens in a single autoregressive sequence, enabling flexible subset selection at inference. The comparisons against strong diffusion baselines such as PixelPonder and UniCombine are appropriate and, if reliable, would be a notable result for the autoregressive paradigm. The attention design also has a concrete complexity benefit over naive self-attention over stacked conditions. However, the current evidence is weakened by the use of SSIM as the headline controllability metric, a short ICBP ablation, the absence of error bars, and no validation that the frozen VQ encoder preserves information in non-photographic condition maps. These issues do not invalidate the approach, but they need to be addressed before the main claims can be fully accepted.

major comments (5)
  1. [§4.2, Table 1] Table 1 uses SSIM between generated images and real images as the primary evidence for the claimed 21.55% controllability improvement, but this metric measures similarity to the reference image rather than adherence to the input conditions. A higher SSIM could also reflect memorization or reduced diversity. The all-conditions comparison should report condition-specific alignment metrics (e.g., canny F1, depth MSE, HED SSIM, and a pose metric), or the authors should justify SSIM as a controllability proxy with a correlation study.
  2. [§3.4, Eq. (CFG)] The classifier-free guidance equation uses an unconditional term l(q_t | q_<t, ∅, ∅), but the inference text states that unused condition tokens are omitted rather than replaced by an unconditional embedding. It is unclear how the unconditional logits are obtained: are all visual conditions dropped, is the text prompt emptied, or is some other protocol used? This needs to be specified precisely because CFG is used in all main experiments and the described omission scheme does not obviously define the ∅,∅ conditioning.
  3. [§3.2, shared VQ encoder] The paper applies a single pretrained VQ encoder, trained on natural images, to tokenize canny, depth, HED, and pose maps, and it explicitly acknowledges that these maps have token distributions differing markedly from natural images. Yet there is no reconstruction fidelity check or ablation (e.g., per-condition encoders, continuous tokens, or a larger codebook) to show that vector quantization does not destroy condition-specific spatial information. Because the separate embedding layers and attention masks can only adapt the surviving codes, the controllability results could be upper-bounded by the VQ encoder's fidelity; this load-bearing assumption should be validated.
  4. [§4.3, Figure 7] The ICBP ablation is trained for only 10k iterations, compared with 30k iterations for the main model, and no error bars or repeated runs are reported. The observed SSIM improvement of about 1.6 points and F1 improvement of about 0.27 points may not be statistically significant at this training budget. The claim that bidirectional perception improves controllability should be supported either by a full-length comparison or by a significance-aware analysis.
  5. [Tables 1–3] All headline results are reported as single numbers without standard deviations, confidence intervals, or multiple seeds. Given that the main comparisons involve FID differences of about 1–2 points (e.g., 10.42 vs. 11.85 in Table 1), the claim of competitive or superior performance would be more robust with repeated evaluations. At minimum, the authors should state how many seeds were used and whether the reported numbers are averages.
minor comments (5)
  1. [Abstract] The sentence 'show that the competitive perpormance than diffusion-based multi-conditional control approaches the existing autoregressive baseline' is grammatically garbled and should be rewritten for clarity.
  2. [Section 2.2 and References] The reference numbering appears inconsistent: the introduction cites ControlAR as [13], while [13] in the reference list is a survey by Cao et al., not the ControlAR paper (which appears as [33]). Please correct the citation.
  3. [Table 4 and Figure 6] There are typographical issues: 'differnet' in the Table 4 caption, 'Comparision' in Section 4.2, and '/glyph1197umber of Conditions' in the Figure 6 axis label.
  4. [§3.2, Equation (3)] In the positional-embedding equations, the learnable offset P_k is said to be shared across heads, but the notation 'P_k in R^{h x w x d/H}' is slightly confusing because h and w were used earlier to denote the latent spatial dimensions; please clarify the intended shape and whether P_k is added to both queries and keys as written.
  5. [§3.3, complexity statement] The complexity statement O((m+1)^2 N^2) for training and O(m^2 N^2) for prefill omits the text-token contribution and appears to assume each condition has the same length N as the image; a sentence clarifying the assumptions would make the complexity comparison more precise.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: ContextAR's central claims are supported by a trained cross-entropy objective and external benchmark evaluations.

full rationale

ContextAR's central claims—joint multi-condition control, arbitrary subset activation at inference, and competitive controllability—are not derived from their own assumptions by construction. The model is trained with the standard next-token cross-entropy loss (Eq. 2) over image tokens conditioned on the unified sequence S=[c1,...,cm,cT,q], and the 'arbitrary condition combinations' property is produced by independent per-condition dropout during training (Sec. 3.4: dropout 0.1 for text and 0.25 per visual condition) together with CCPR masking, which is an engineering mechanism rather than a fitted quantity renamed as a prediction. Tables 1–3 compare against external diffusion and autoregressive baselines on held-out benchmarks (MultiGen-20M and SubjectSpatial200K) using FID, SSIM, MUSIQ, and condition-specific metrics, so the main evidence is externally falsifiable. The only tuned hyperparameter explicitly discussed is the CFG scale, which is swept in Table 4 and then fixed at 3.0; selecting a hyperparameter by validation is standard practice, not circularity. No load-bearing argument relies on a self-citation: the paper cites LlamaGen, ControlAR, UniCombine, and diffusion control works as external baselines or backbones, and there is no appeal to the authors' own prior uniqueness theorem. The concern that a shared frozen VQ encoder may lose information in non-photographic condition maps is a legitimate robustness and correctness risk about an unablated design assumption, but it is not a case where a prediction reduces to its input by definition. Overall, the derivation chain is self-contained with respect to the claims made, so no significant circularity is found.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The framework rests on standard autoregressive modeling plus two unvalidated domain assumptions: the VQ tokenizer works on condition maps, and dropout during training is sufficient for arbitrary combinations. The only hand-set inference parameter is CFG scale, selected on the evaluation set.

free parameters (3)
  • CFG scale = 3.0
    Selected from the Table 4 sweep on MultiGen-20M evaluation metrics and used in all main results. This choice trades FID against SSIM and MUSIQ.
  • Visual condition dropout rate = 0.25
    Set by hand in Section 3.4 and applied independently to each visual condition during training. It directly enables arbitrary subset combinations but is not ablated.
  • Text dropout rate = 0.1
    Set by hand in Section 3.4 to reduce reliance on the text prompt. It is not ablated and its effect on the central claim is not demonstrated.
assumptions (5)
  • domain assumption A VQ-VAE encoder pretrained on natural images tokenizes non-photographic conditions such as canny, depth, HED, and pose without losing control information.
    Section 3.2 applies a shared pretrained VQ encoder to condition images. No ablation compares alternate condition tokenizers or continuous features; separate embedding layers cannot restore information lost during quantization.
  • domain assumption Independent random condition dropout during training suffices to support arbitrary subset combinations at inference.
    Section 3.4 sets dropout to 0.25 per visual condition. This is a training heuristic with no formal guarantee, and only a few subsets are evaluated in Tables 2 and 3.
  • domain assumption Bidirectional attention within condition tokens is consistent between training and inference and improves control.
    ICBP is introduced in Section 3.3 and validated only by a 10k-iteration ablation in Figure 7. The benefit is plausible but not conclusively established.
  • domain assumption Text tokens need no positional encoding because only their semantic content matters for image generation.
    Stated in Section 3.2. This is reasonable for image generation but remains an unstated assumption about text-image alignment.
  • standard math The autoregressive token factorization p(q|c) = product_t p(q_t | q_<t, c) is a valid model for image generation.
    Equation (1) in Section 3.1 is the standard next-token prediction paradigm used by LlamaGen and related autoregressive image models.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Context-Aware Autoregressive Models for Multi-Conditional Image Generation." pith.science (2026). https://pith.science/paper/FRTDBCI4

@misc{pith2026250512274,
  author       = {Pith},
  title        = {Pith review of: Context-Aware Autoregressive Models for Multi-Conditional Image Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FRTDBCI4}},
  note         = {Machine review of arXiv:2505.12274}
}
abstract

Autoregressive transformers have recently shown impressive image generation quality and efficiency on par with state-of-the-art diffusion models. Unlike diffusion architectures, autoregressive models can naturally incorporate arbitrary modalities into a single, unified token sequence--offering a concise solution for multi-conditional image generation tasks. In this work, we propose $\textbf{ContextAR}$, a flexible and effective framework for multi-conditional image generation. ContextAR embeds diverse conditions (e.g., canny edges, depth maps, poses) directly into the token sequence, preserving modality-specific semantics. To maintain spatial alignment while enhancing discrimination among different condition types, we introduce hybrid positional encodings that fuse Rotary Position Embedding with Learnable Positional Embedding. We design Conditional Context-aware Attention to reduces computational complexity while preserving effective intra-condition perception. Without any fine-tuning, ContextAR supports arbitrary combinations of conditions during inference time. Experimental results demonstrate the powerful controllability and versatility of our approach, and show that the competitive perpormance than diffusion-based multi-conditional control approaches the existing autoregressive baseline across diverse multi-condition driven scenarios. Project page: $\href{https://context-ar.github.io/}{https://context-ar.github.io/.}$

Figures

Figures reproduced from arXiv: 2505.12274 by the authors.

Figure 1
Figure 1. Visualization of generated results from our proposed ContextAR framework. The combinations of conditions are: (a) Canny + Depth + HED + Pose, (b) Canny + Depth + HED, (c) Canny + HED, (d) HED, (e) Canny, (f) Subject + Canny, (g) Subject + Depth. Our framework is im￾plemented on an autoregressive model, achieving excellent controllability while offering remarkable flexibility and versatility. Abstract Autoregressive … view at source ↗
Figure 2
Figure 2. Overview of our proposed ContextAR. (a) The overall process of training and inference. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Visualization of attention computation. (a) Normal attention with causal mask, (b) Cross-Condition Perception Restriction, (c) Intra-Condition Bidirectional Perception. 3.3 Conditional Context-aware Attention Cross-Condition Perception Restriction (CCPR). The goal of our approach is to enable the flexible combination of multiple conditions. Once the model is trained on m distinct conditions, it can dynamically selec… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Visualization comparison on MultiGen-20M. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Visualization comparison on SubjectSpatial200K. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: , as the number of conditions increases, the computational cost with the CCPR mechanism is significantly lower than that of standard attention, demonstrating the effectiveness of our approach. Moreover, the bidirectional attention introduced in ICBP adds almost no addi…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Layout-Conditioned Autoregressive Text-to-Image Generation via Structured Masking

    cs.CV 2025-09 conditional novelty 6.0 of 10

    SMARLI achieves strong layout control in autoregressive text-to-image generation via structured attention masks and GRPO post-training with a CLIP-based layout reward.

Reference graph

Works this paper leans on

45 extracted references · 3 canonical work pages · cited by 1 Pith paper

  1. [13]

    Controllable generation with text-to-image diffusion models: A survey

    Pu Cao, Feng Zhou, Qing Song, and Lu Yang. Controllable generation with text-to-image diffusion models: A survey. arXiv preprint arXiv:2403.04279, 2024

    arXiv 2024

  2. [1]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020

    arXiv 2010

  3. [2]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in NeurIPS, 33:6840–6851, 2020

    2020

  4. [3]

    Improved denoising diffusion probabilistic models

    Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In ICML, pages 8162–8171, 2021

    2021

  5. [4]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780–8794, 2021

    2021

  6. [5]

    High- resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022

    2022

  7. [6]

    Black Forest Labs. Flux. https://github.com/black-forest-labs/flux , 2024

    2024

  8. [7]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3836–3847, 2023

    2023

Show all 45 references
  1. [8]

    Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models

    Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models. arXiv preprint arXiv:2308.06721, 2023

    2023 arXiv

  2. [9]

    T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models

    Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, and Ying Shan. T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 42...

    2024

  3. [10]

    Attention is all you need

    A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017

    2017

  4. [11]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020

    1901

  5. [12]

    Editar: Unified conditional generation with autoregressive models

    Jiteng Mu, Nuno Vasconcelos, and Xiaolong Wang. Editar: Unified conditional generation with autoregressive models. arXiv preprint arXiv:2501.04699, 2025

    2025 arXiv

  6. [14]

    Roformer: Enhanced transformer with rotary position embedding

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568:127063, 2024

    2024

  7. [15]

    Generating diverse high-fidelity images with vq-vae-2

    Ali Razavi, Aaron Van den Oord, and Oriol Vinyals. Generating diverse high-fidelity images with vq-vae-2. Advances in neural information processing systems, 32, 2019

    2019

  8. [16]

    Taming transformers for high-resolution image synthesis

    Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12873–12883, 2021

    2021

  9. [17]

    Scaling autoregressive models for content-rich text-to-image generation

    Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yinfei Yang, Burcu Karagol Ayan, et al. Scaling autoregressive models for content-rich text-to-image generation. arXiv preprint arXiv:2206.10789, 2(3):5, 2022

    2022 arXiv

  10. [18]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International conference on machine learning, pages 8821–8831. Pmlr, 2021. 10

    2021

  11. [19]

    Autoregressive model beats diffusion: Llama for scalable image generation

    Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Autoregressive model beats diffusion: Llama for scalable image generation. arXiv preprint arXiv:2406.06525, 2024

    2024 arXiv

  12. [20]

    Autoregressive image generation without vector quantization

    Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, and Kaiming He. Autoregressive image generation without vector quantization. arXiv preprint arXiv:2406.11838, 2024

    2024 arXiv

  13. [21]

    Visual autoregressive mod- eling: Scalable image generation via next-scale prediction

    Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Liwei Wang. Visual autoregressive mod- eling: Scalable image generation via next-scale prediction. arXiv preprint arXiv:2404.02905, 2024

    2024 arXiv

  14. [22]

    Frequency autoregressive image generation with continuous tokens

    Hu Yu, Hao Luo, Hangjie Yuan, Yu Rong, and Feng Zhao. Frequency autoregressive image generation with continuous tokens. arXiv preprint arXiv:2503.05305, 2025

    2025

  15. [23]

    Simplear: Pushing the frontier of autoregressive visual generation through pretraining, sft, and rl

    Junke Wang, Zhi Tian, Xun Wang, Xinyu Zhang, Weilin Huang, Zuxuan Wu, and Yu-Gang Jiang. Simplear: Pushing the frontier of autoregressive visual generation through pretraining, sft, and rl. arXiv preprint arXiv:2504.11455, 2025

    2025 arXiv

  16. [24]

    Neural discrete representation learning

    Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Advances in neural information processing systems, 30, 2017

    2017

  17. [25]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timo- thée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  18. [26]

    Llama 2: Open foundation and fine-tuned chat models

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  19. [27]

    Instantid: Zero-shot identity-preserving generation in seconds

    Qixun Wang, Xu Bai, Haofan Wang, Zekui Qin, and Anthony Chen. Instantid: Zero-shot identity-preserving generation in seconds. arXiv preprint arXiv:2401.07519, 2024

    2024 arXiv

  20. [28]

    Unicontrol: A unified diffusion model for controllable visual generation in the wild

    Can Qin, Shu Zhang, Ning Yu, Yihao Feng, Xinyi Yang, Yingbo Zhou, Huan Wang, Juan Carlos Niebles, Caiming Xiong, Silvio Savarese, et al. Unicontrol: A unified diffusion model for controllable visual generation in the wild. arXiv preprint arXiv:2305.11147, 2023

    2023 arXiv

  21. [29]

    Uni-controlnet: All-in-one control to text-to-image diffusion models

    Shihao Zhao, Dongdong Chen, Yen-Chun Chen, Jianmin Bao, Shaozhe Hao, Lu Yuan, and Kwan-Yee K Wong. Uni-controlnet: All-in-one control to text-to-image diffusion models. Advances in Neural Information Processing Systems, 36, 2024

    2024

  22. [30]

    Cocktail: Mixing multi-modality control for text-conditional image generation

    Minghui Hu, Jianbin Zheng, Daqing Liu, Chuanxia Zheng, Chaoyue Wang, Dacheng Tao, and Tat-Jen Cham. Cocktail: Mixing multi-modality control for text-conditional image generation. In Thirty-seventh Conference on Neural Information Processing Systems, 2023

    2023

  23. [31]

    Unicombine: Unified multi-conditional combination with diffusion transformer

    Haoxuan Wang, Jinlong Peng, Qingdong He, Hao Yang, Ying Jin, Jiafu Wu, Xiaobin Hu, Yanjie Pan, Zhenye Gan, Mingmin Chi, et al. Unicombine: Unified multi-conditional combination with diffusion transformer. arXiv preprint arXiv:2503.09277, 2025

    2025 arXiv

  24. [32]

    Pixelponder: Dynamic patch adaptation for enhanced multi-conditional text-to-image generation

    Yanjie Pan, Qingdong He, Zhengkai Jiang, Pengcheng Xu, Chaoyi Wang, Jinlong Peng, Haox- uan Wang, Yun Cao, Zhenye Gan, Mingmin Chi, et al. Pixelponder: Dynamic patch adaptation for enhanced multi-conditional text-to-image generation. arXiv preprint arXiv:2503.06684, 2025

    2025 arXiv

  25. [33]

    Controlar: Controllable image generation with autoregressive models

    Zongming Li, Tianheng Cheng, Shoufa Chen, Peize Sun, Haocheng Shen, Longjin Ran, Xiaoxin Chen, Wenyu Liu, and Xinggang Wang. Controlar: Controllable image generation with autoregressive models. arXiv preprint arXiv:2410.02705, 2024

    arXiv

  26. [34]

    Janus: Decoupling visual encoding for unified multimodal understanding and generation

    Chengyue Wu, Xiaokang Chen, Zhiyu Wu, Yiyang Ma, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, Chong Ruan, et al. Janus: Decoupling visual encoding for unified multimodal understanding and generation. arXiv preprint arXiv:2410.13848, 2024

    arXiv

  27. [35]

    Vila-u: a unified foundation model integrating visual understanding and generation

    Yecheng Wu, Zhuoyang Zhang, Junyu Chen, Haotian Tang, Dacheng Li, Yunhao Fang, Ligeng Zhu, Enze Xie, Hongxu Yin, Li Yi, et al. Vila-u: a unified foundation model integrating visual understanding and generation. arXiv preprint arXiv:2409.04429, 2024

    2024 arXiv

  28. [36]

    Liquid: Language models are scalable multi-modal generators

    Junfeng Wu, Yi Jiang, Chuofan Ma, Yuliang Liu, Hengshuang Zhao, Zehuan Yuan, Song Bai, and Xiang Bai. Liquid: Language models are scalable multi-modal generators. arXiv preprint arXiv:2412.04332, 2024

    2024 arXiv

  29. [37]

    Chameleon: Mixed-modal early-fusion foundation models

    Chameleon Team. Chameleon: Mixed-modal early-fusion foundation models. arXiv preprint arXiv:2405.09818, 2024. 11

    2024 arXiv

  30. [38]

    Emu3: Next-token prediction is all you need

    Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, et al. Emu3: Next-token prediction is all you need. arXiv preprint arXiv:2409.18869, 2024

    2024 arXiv

  31. [39]

    Transfusion: Predict the next token and diffuse images with one multi-modal model

    Chunting Zhou, Lili Yu, Arun Babu, Kushal Tirumala, Michihiro Yasunaga, Leonid Shamis, Jacob Kahn, Xuezhe Ma, Luke Zettlemoyer, and Omer Levy. Transfusion: Predict the next token and diffuse images with one multi-modal model. arXiv preprint arXiv:2408.11039, 2024

    2024 arXiv

  32. [40]

    Show-o: One single trans- former to unify multimodal understanding and generation

    Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. Show-o: One single trans- former to unify multimodal understanding and generation. arXiv preprint arXiv:2408.12528, 2024

    2024 arXiv

  33. [41]

    Vision as lora

    Han Wang, Yongjie Ye, Bingru Li, Yuxiang Nie, Jinghui Lu, Jingqun Tang, Yanjie Wang, and Can Huang. Vision as lora. arXiv preprint arXiv:2503.20680, 2025

    2025 arXiv

  34. [42]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017

    2017

  35. [43]

    Musiq: Multi-scale image quality transformer

    Junjie Ke, Qifei Wang, Yilin Wang, Peyman Milanfar, and Feng Yang. Musiq: Multi-scale image quality transformer. In Proceedings of the IEEE/CVF international conference on computer vision, pages 5148–5157, 2021

    2021

  36. [44]

    Image quality assessment: from error visibility to structural similarity

    Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4):600– 612, 2004

    2004

  37. [45]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pa...

    2021

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.