Pith. sign in

REVIEW 3 major objections 4 minor 70 references

CycleVAR: Repurposing Autoregressive Model for Unsupervised One-Step Image Translation

T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that an autoregressive next-scale image model can be fine-tuned without paired data to translate images across domains, and that a parallel one-step decoding mode beats both serial decoding and a leading one-step…

desk verdict A plausible first use of pretrained AR image models for unpaired translation, but the SOTA claim is undercut by unmatched backbones and missing error bars. read the letter →

arxiv 2506.23347 v2 pith:JD2QMIMZ submitted 2025-06-29 cs.CV

Yi Liu , Shengqian Li , Zuzeng Lin , Feng Wang , Si Liu This is my paper
classification cs.CV
keywords unsupervisedimagetranslationautoregressivegenerationvisualmodelsoftmaxrelaxedquantizationvectorcycleconsistencyone-stepinferenceimage-to-image
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

CycleVAR tries to show that a pretrained autoregressive image-generation model can do unpaired image-to-image translation—no paired source/target images—by treating the source image as a multi-scale token prefix and predicting the target image as its continuation. The paper identifies the hard vector-quantization argmax as the blocker, because it cuts gradients between the VAE decoder and the transformer; its fix, Softmax Relaxed Quantization, replaces codebook selection with a softmax-weighted average of codebook vectors so image-space losses can train the model end-to-end. Two decoding modes are proposed, serial multi-step and parallel one-step, and the paper reports that the one-step mode gives better translation quality and faster inference. If the paper is right, autoregressive visual models can be fine-tuned with the same kind of unpaired data used by CycleGAN-style methods while keeping a single forward pass at inference.

What carries the argument

The load-bearing object is Softmax Relaxed Quantization (SRQ), a continuous relaxation of the VQ codebook lookup. Instead of computing $\arg\max$ over codebook logits and looking up a single code vector, SRQ applies $\hat{g}_v = \exp(\tilde{g}_v/\tau)/\sum_q \exp(\tilde{g}_q/\tau)$ and outputs $\hat{f} = \sum_v \hat{g}_v Z_v$, so the quantized feature is differentiable in the logits. This gives the frozen VAE decoder smooth inputs and gives image-space losses a gradient path into the transformer, which is what makes adversarial training possible despite discrete tokenization. The second mechanism is multi-scale token prefilling: the source image's residual maps $F_1,\dots,F_K$ are fed as contextual keys, so translation becomes next-scale prediction conditioned on the source, in the same spirit as a prefix in an autoregressive language model.

What would settle it

Train CycleVAR on horse↔zebra with SRQ replaced by straight-through hard quantization (or with $\tau=0.01$) while keeping everything else fixed, and measure both FID and the gradient norm flowing through the frozen VAE decoder; if translation quality does not collapse and gradients remain healthy, the paper's central claim that soft relaxation is what enables end-to-end unsupervised training is not supported.

Watch

Extended reading notes

Core claim

The paper claims that image-to-image translation can be recast as image-conditional visual autoregressive generation, and that a pretrained next-scale transformer can be fine-tuned for it with no paired data. The source image is tokenized into multi-scale residual maps that are prefilled as contextual keys, like a prefix prompt in a language model, and the transformer predicts the target image's tokens at all scales. Because the usual vector-quantization argmax blocks gradients, the paper replaces it with Softmax Relaxed Quantization: logits over the codebook are turned into a softmax distribution at temperature $\tau$, and the quantized feature is the convex combination $\sum_v \hat{g}_v Z_v$ of codebook vectors. With that differentiable path through the frozen VAE decoder, cycle-consistency, adversarial, and identity losses can be applied directly in image space. On horse↔zebra, day↔night, and an anime-scene dataset, the paper reports that the parallel one-step mode beats its own serial multi-step mode and, with the larger Infinity-2B model, matches or beats CycleGAN-Turbo on FID and DINO structure in all four translation directions.

Load-bearing premise

The whole training loop assumes the frozen VAE decoder, trained on single one-hot codebook vectors, produces meaningful images and useful gradients when fed softmax-weighted averages of codebook vectors blended across scales; if those soft mixtures decode to blurry or degenerate latents, the translation and its learning signal both fail.

Editorial extensions

If this is right

  • A pretrained autoregressive image generator can be adapted to a new visual domain with unpaired images only, using cycle, adversarial, and identity losses, without ever supervising the predicted token indices.
  • The parallel one-step decoding mode is preferable for unsupervised translation: it produces better FID and structure scores than serial multi-step decoding while using roughly a third of the inference time.
  • The approach carries over to text-conditional and bitwise-quantized autoregressive models, suggesting it is not tied to VAR's specific residual quantizer.
  • Removing either the multi-scale context or the multi-scale output degrades translation, so the coarse-to-fine token structure is doing essential work in preserving source structure.
  • On the reported benchmarks, CycleVAR with Infinity-2B matches or beats CycleGAN-Turbo on both distribution matching and structure preservation across horse↔zebra and day↔night.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If SRQ generalizes, the same soft relaxation should unlock image-space supervision for other discrete latent models, such as tokenizers based on finite scalar quantization or binary codes, without retraining the tokenizer.
  • The result that one parallel pass beats iterative scale-by-scale refinement suggests that when the source image already provides the content, the autoregressive prior may be bypassed entirely; this is worth testing on text-to-image and super-resolution tasks where conditioning is similarly content-rich.
  • Because temperature $\tau=2$ outperformed both sharper and much softer distributions, a natural extension is to anneal $\tau$ during training or make it per-scale, which the paper does not explore.
  • The reported speed advantage should grow at higher resolutions and larger models, since serial decoding cost scales with the number of scales while the parallel mode does not; measuring this on 512×512 or 1024×1024 translations would be a direct stress test.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes CycleVAR, an unsupervised image-to-image translation framework built on pre-trained visual autoregressive (VAR) models. The method tokenizes the source image into multi-scale residual maps, injects these as prefix context into a causal VAR transformer, and predicts target-domain tokens either serially across scales or in a parallel one-step mode. To enable end-to-end fine-tuning despite discrete quantization, the paper introduces Softmax Relaxed Quantization (SRQ), which replaces argmax codebook selection with a temperature-controlled softmax-weighted average of codebook vectors. Training uses cycle consistency, adversarial, and identity losses following CycleGAN-Turbo. Experiments on horse↔zebra, day↔night, and an anime scene dataset report FID and DINO-Structure metrics, and the paper claims state-of-the-art performance over prior unsupervised translation methods, especially CycleGAN-Turbo.

Significance. If the central claims hold, this is a meaningful contribution: it extends the autoregressive image generation paradigm to unsupervised translation with a single forward pass, potentially easing integration with LLM-style multimodal models. The paper's strengths include a clearly described framework, a concrete mechanism (SRQ) to address the gradient-truncation problem of VQ-based tokenizers, and ablations that isolate the benefits of the parallel one-step mode and multi-scale context/output (Tables 3 and 4). The temperature ablation (Table 5) gives direct evidence that soft codebook mixing is usable rather than catastrophically off-manifold. However, the headline comparison against CycleGAN-Turbo is confounded by backbone scale and lacks uncertainty quantification, which tempers the significance of the 'surpasses' claim.

major comments (3)
  1. [§4.2.1, Table 1] The claim that CycleVAR surpasses CycleGAN-Turbo is not adequately supported. The winning rows all use CycleVAR w/ Infinity, a 2.3B text-conditional model, whereas CycleGAN-Turbo is built on SD-Turbo 2.1 with about 1.1B parameters. With the matched 310M VAR backbone, CycleVAR is worse on horse→zebra FID (47.8 vs 41.0) and is not evaluated on day↔night at all (footnote 3). FID margins in the winning rows are small (0.7–8.5 points), and no seeds, confidence intervals, or significance tests are reported. This comparison therefore does not demonstrate that the CycleVAR mechanism itself, rather than a larger backbone, is responsible for the apparent superiority.
  2. [§4.3, Table 5] The temperature τ in SRQ is tuned on the same FID and DINO-Structure metrics used for the headline comparisons, with no held-out validation or selection protocol. Since τ strongly controls translation quality (FID ranges from 228.2 at τ=0.01 to 47.82 at τ=2), hyperparameter selection on the evaluation metric can inflate the reported gains. The mixing weight a in Eqs. (6) and (9) is never ablated, leaving a second free parameter that could drive the parallel-mode improvement. At minimum, the paper should report sensitivity to a and clarify whether τ was chosen on a validation split.
  3. [§3.3, Eq. (11) and Figure 6] The method assumes that the frozen VAE decoder, trained on one-hot codebook vectors, produces coherent images from soft convex combinations of codebook vectors (τ=2 in Table 5). The temperature ablation is helpful, but it reports only final image metrics; a more direct analysis of the decoded soft latents (e.g., reconstruction of soft-quantized features with the frozen decoder, or visual examples of decoded mixtures) would strengthen the claim that SRQ operates in a usable region of the decoder manifold and that gradients through the frozen decoder are not collapsed or degenerate.
minor comments (4)
  1. [Throughout] The transformer is repeatedly referred to as the 'casual V AR Transformer' (e.g., in §3.1 and Figure 1 caption); this should be 'causal'.
  2. [§2.2, footnote] The text-conditional model name is misspelled as 'Infininty' in the footnote in the Related Works section; it should be 'Infinity'.
  3. [§4.1, Training details] The paper states that 'the discriminator uses the CLIP model as the feature extractor and an MLP as the decoder' but does not specify the loss function (hinge, least-squares, etc.) or the discriminator input resolution; this should be stated in the main text or supplementary.
  4. [Eq. (5)] The notation in Eq. (5) uses K both as the number of scales in the sum and inside the summand, which is confusing; it should use k as the running index, e.g., sum_{k=1}^K up(R_k, (h,w)).

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: CycleVAR's architecture, objectives, and evaluation are defined independently of the quantities they are used to produce.

full rationale

CycleVAR's derivation chain is empirical rather than analytic. The prefilling equations (Eqs. 5-9), softmax relaxed quantization (Eqs. 10-11), and the cycle, adversarial, and identity losses (Eqs. 13-16) are defined from source images, codebook entries, and target-domain discriminators; none of these definitions presuppose the reported FID or DINO values. The translated image is obtained by decoding predicted tokens through the frozen VAE decoder, so the headline numbers are measured outputs, not identities. The comparisons in Tables 1 and 3 are against external baselines (CycleGAN-Turbo, CUT, SDEdit, etc.), so the central performance claim is not forced by a fitted parameter. The only hyperparameter chosen on the evaluation metrics is the SRQ temperature in Table 5, and the paper reports the resulting FID/DINO rather than deriving those metrics from the temperature formula; this is conventional model selection, not a statistically forced prediction. There are no load-bearing self-citations and no uniqueness claims imported from the authors' prior work. Consequently, no circular step is exhibited.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The method depends on pretrained VAR/Infinity priors, on the VAE decoder's tolerance for soft codebook mixtures, on cycle/adversarial losses being sufficient, and on FID/DINO as valid metrics. It adds one explicit tunable (tau) and at least one unreported weight a, plus unspecified loss weights.

free parameters (3)
  • temperature tau = 2.0 (selected from Table 5 ablation)
    Controls softmax sharpness in Eq. 10; chosen on the same FID/DINO metrics used for the main results.
  • mixing weight a = not reported
    Weights the fusion of source features and generated features in Eqs. 6 and 9; no ablation is provided.
  • loss weights for Lcycle, Lgan, Lidt = not reported
    Training details are promised in the supplementary material but the supplement does not list them.
assumptions (4)
  • domain assumption Pretrained VAR or Infinity provides a valid generative prior for both source and target domains.
    Section 4.1 uses VAR trained on ImageNet and Infinity trained on large text-image data, assuming these priors transfer to horse/zebra, day/night, and anime styles.
  • ad hoc to paper The frozen VAE decoder produces coherent images from soft convex combinations of codebook vectors.
    Eq. 11 with tau=2 is used at inference, but the decoder was trained on one-hot quantized features; the paper does not analyze this distribution shift.
  • domain assumption Cycle consistency, adversarial, and identity losses are sufficient to align unpaired domains.
    Section 3.4 adopts the CycleGAN-Turbo training setup without proving that the losses converge to the intended mapping.
  • domain assumption FID and DINO-Structure-Dist measure the relevant translation quality.
    Section 4.1 uses these two metrics as the primary evidence for superiority, but they capture only distribution matching and coarse structure.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CycleVAR: Repurposing Autoregressive Model for Unsupervised One-Step Image Translation." pith.science (2026). https://pith.science/paper/JD2QMIMZ

@misc{pith2026250623347,
  author       = {Pith},
  title        = {Pith review of: CycleVAR: Repurposing Autoregressive Model for Unsupervised One-Step Image Translation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JD2QMIMZ}},
  note         = {Machine review of arXiv:2506.23347}
}
read the original abstract

The current conditional autoregressive image generation methods have shown promising results, yet their potential remains largely unexplored in the practical unsupervised image translation domain, which operates without explicit cross-domain correspondences. A critical limitation stems from the discrete quantization inherent in traditional Vector Quantization-based frameworks, which disrupts gradient flow between the Variational Autoencoder decoder and causal Transformer, impeding end-to-end optimization during adversarial training in image space. To tackle this issue, we propose using Softmax Relaxed Quantization, a novel approach that reformulates codebook selection as a continuous probability mixing process via Softmax, thereby preserving gradient propagation. Building upon this differentiable foundation, we introduce CycleVAR, which reformulates image-to-image translation as image-conditional visual autoregressive generation by injecting multi-scale source image tokens as contextual prompts, analogous to prefix-based conditioning in language models. CycleVAR exploits two modes to generate the target image tokens, including (1) serial multi-step generation, enabling iterative refinement across scales, and (2) parallel one-step generation synthesizing all resolution outputs in a single forward pass. Experimental findings indicate that the parallel one-step generation mode attains superior translation quality with quicker inference speed than the serial multi-step mode in unsupervised scenarios. Furthermore, both quantitative and qualitative results indicate that CycleVAR surpasses previous state-of-the-art unsupervised image translation models, \textit{e}.\textit{g}., CycleGAN-Turbo.

Figures

Figures reproduced from arXiv: 2506.23347 by the authors.

Figure 1
Figure 1. Comparing conventional quantization methods with our Soft [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Illustration of CycleVAR. (a) The framework of repurposing VAR for image translation by prefilling multi-scale image tokens and making multi [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Qualitative comparison on horse ↔ zebra dataset(256 × 256). Our method, CycleVAR, is contrasted with GAN-based and diffusion methods. CycleVAR demonstrates strong performance in image translation while maintaining the integrity of the source image structure. Additionally, it ensures consistent transfer across foreground objects while preserving the chromatic harmony of the background. editing in standard CycleGAN da… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Qualitative comparison on day↔night dataset in the driving scene. based on a pre-trained text-to-image diffusion model with extensive training data, while CycleVAR w/ VAR employs VAR-310M, which is trained exclusively on ImageNet [5]. For a broader comparison, we also …
Figure 5
Figure 5. Figure 5: Anime Scene Translation via parallel one-step CycleVAR w/ [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Visual ablation of the temperature in Softmax Relaxed Quanti [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Qualitative comparison on Anime Scene Dataset. We compare parallel one-step CycleVAR w/ Infinity with other state-of-the-art baselines [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

70 extracted references · 36 canonical work pages

  1. [1]

    In- structpix2pix: Learning to follow image editing instructions

    Tim Brooks, Aleksander Holynski, and Alexei A Efros. In- structpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 18392–18402, 2023. 3, 6

    work page 2023

  2. [2]

    Maskgit: Masked generative image transformer

    Huiwen Chang, Han Zhang, Lu Jiang, Ce Liu, and William T Freeman. Maskgit: Masked generative image transformer. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 11315–11325, 2022. 2

    work page 2022

  3. [3]

    Animegan: A novel lightweight gan for photo animation

    Jie Chen, Gang Liu, and Xin Chen. Animegan: A novel lightweight gan for photo animation. In Artificial Intelli- gence Algorithms and Applications: 11th International Sym- posium, ISICA 2019, Guangzhou, China, November 16–17, 2019, Revised Selected Papers 11, pages 242–256. Springer,

    work page 2019

  4. [4]

    Cartoongan: Generative adversarial networks for photo cartoonization

    Yang Chen, Yu-Kun Lai, and Yong-Jin Liu. Cartoongan: Generative adversarial networks for photo cartoonization. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 9465–9474, 2018. 7

    work page 2018

  5. [5]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009. 7

    2009

  6. [6]

    Taming transformers for high-resolution image synthesis

    Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12873–12883, 2021. 2

    2021

  7. [7]

    Dual contrastive learning for unsu- pervised image-to-image translation

    Junlin Han, Mehrdad Shoeiby, Lars Petersson, and Mo- hammad Ali Armin. Dual contrastive learning for unsu- pervised image-to-image translation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 746–755, 2021. 3

    work page 2021

  8. [8]

    Infinity: Scaling bit- wise autoregressive modeling for high-resolution image syn- thesis

    Jian Han, Jinlai Liu, Yi Jiang, Bin Yan, Yuqi Zhang, Zehuan Yuan, Bingyue Peng, and Xiaobing Liu. Infinity: Scaling bit- wise autoregressive modeling for high-resolution image syn- thesis. arXiv preprint arXiv:2412.04431, 2024. 1, 2, 5, 12

    arXiv 2024

Show all 70 references
  1. [9]

    Multimodal unsupervised image-to-image translation

    Xun Huang, Ming-Yu Liu, Serge Belongie, and Jan Kautz. Multimodal unsupervised image-to-image translation. In Proceedings of the European conference on computer vision (ECCV), pages 172–189, 2018. 3

    2018

  2. [10]

    Image-to-image translation with conditional adver- sarial networks

    Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adver- sarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1125–1134,

  3. [11]

    Scenimefy: Learning to craft anime scene via semi- supervised image-to-image translation

    Yuxin Jiang, Liming Jiang, Shuai Yang, and Chen Change Loy. Scenimefy: Learning to craft anime scene via semi- supervised image-to-image translation. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 7357–7367, 2023. 5, 7

    2023

  4. [12]

    A style-based generator architecture for generative adversarial networks

    Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 4401–4410, 2019. 7

    2019

  5. [13]

    Analyzing and improv- ing the image quality of stylegan

    Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and improv- ing the image quality of stylegan. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8110–8119, 2020. 7

    2020

  6. [14]

    Learning to discover cross-domain relations with generative adversarial networks

    Taeksoo Kim, Moonsu Cha, Hyunsoo Kim, Jung Kwon Lee, and Jiwon Kim. Learning to discover cross-domain relations with generative adversarial networks. In International con- ference on machine learning, pages 1857–1865. Pmlr, 2017. 3

    2017

  7. [15]

    Autoregressive image generation using residual quantization

    Doyup Lee, Chiheon Kim, Saehoon Kim, Minsu Cho, and Wook-Shin Han. Autoregressive image generation using residual quantization. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 11523–11532, 2022. 2

    2022

  8. [16]

    Diverse image-to-image translation via disentangled representations

    Hsin-Ying Lee, Hung-Yu Tseng, Jia-Bin Huang, Maneesh Singh, and Ming-Hsuan Yang. Diverse image-to-image translation via disentangled representations. In Proceed- ings of the European conference on computer vision (ECCV), pages 35–51, 2018. 3

    2018

  9. [17]

    Con- trolnet++: Improving conditional controls with efficient 9 consistency feedback: Project page: liming-ai

    Ming Li, Taojiannan Yang, Huafeng Kuang, Jie Wu, Zhaoning Wang, Xuefeng Xiao, and Chen Chen. Con- trolnet++: Improving conditional controls with efficient 9 consistency feedback: Project page: liming-ai. github. io/controlnet plus plus. In European Conference on Com- puter Vis...

    2024

  10. [18]

    Autoregressive image generation without vec- tor quantization

    Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, and Kaiming He. Autoregressive image generation without vec- tor quantization. Advances in Neural Information Processing Systems, 37:56424–56445, 2025. 2

    2025

  11. [19]

    Controlvar: Exploring con- trollable visual autoregressive modeling

    Xiang Li, Kai Qiu, Hao Chen, Jason Kuen, Zhe Lin, Rita Singh, and Bhiksha Raj. Controlvar: Exploring con- trollable visual autoregressive modeling. arXiv preprint arXiv:2406.09750, 2024. 2, 3

    2024 arXiv

  12. [20]

    Gligen: Open-set grounded text-to-image generation

    Yuheng Li, Haotian Liu, Qingyang Wu, Fangzhou Mu, Jian- wei Yang, Jianfeng Gao, Chunyuan Li, and Yong Jae Lee. Gligen: Open-set grounded text-to-image generation. InPro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 22511–22521, 2023. 3

    2023

  13. [21]

    Controlar: Controllable image generation with autoregressive models

    Zongming Li, Tianheng Cheng, Shoufa Chen, Peize Sun, Haocheng Shen, Longjin Ran, Xiaoxin Chen, Wenyu Liu, and Xinggang Wang. Controlar: Controllable image generation with autoregressive models. arXiv preprint arXiv:2410.02705, 2024. 2, 3

    arXiv

  14. [22]

    Playground v3: Im- proving text-to-image alignment with deep-fusion large lan- guage models

    Bingchen Liu, Ehsan Akhgari, Alexander Visheratin, Aleks Kamko, Linmiao Xu, Shivam Shrirao, Chase Lambert, Joao Souza, Suhail Doshi, and Daiqing Li. Playground v3: Im- proving text-to-image alignment with deep-fusion large lan- guage models. arXiv preprint arXiv:2409.10695, 2024. 1

    2024 arXiv

  15. [23]

    Lumina-mgpt: Illuminate flexible photorealistic text-to-image generation with multimodal generative pretraining

    Dongyang Liu, Shitian Zhao, Le Zhuo, Weifeng Lin, Yu Qiao, Hongsheng Li, and Peng Gao. Lumina-mgpt: Illuminate flexible photorealistic text-to-image generation with multimodal generative pretraining. arXiv preprint arXiv:2408.02657, 2024. 2

    2024 arXiv

  16. [24]

    Open-magvit2: An open-source project toward democratizing auto-regressive visual gener- ation

    Zhuoyan Luo, Fengyuan Shi, Yixiao Ge, Yujiu Yang, Limin Wang, and Ying Shan. Open-magvit2: An open-source project toward democratizing auto-regressive visual gener- ation. arXiv preprint arXiv:2409.04410, 2024. 2

    2024 arXiv

  17. [25]

    Star: Scale-wise text-to- image generation via auto-regressive representations

    Xiaoxiao Ma, Mohan Zhou, Tao Liang, Yalong Bai, Tiejun Zhao, Huaian Chen, and Yi Jin. Star: Scale-wise text-to- image generation via auto-regressive representations. arXiv preprint arXiv:2406.10797, 2024. 1

    2024 arXiv

  18. [26]

    Sdedit: Guided image synthesis and editing with stochastic differential equa- tions

    Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jia- jun Wu, Jun-Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equa- tions. arXiv preprint arXiv:2108.01073, 2021. 6

    2021 arXiv

  19. [27]

    T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models

    Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, and Ying Shan. T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. In Proceedings of the AAAI conference on artificial intelligence, pages 4296–4304, 2024. 1, 3

    2024

  20. [28]

    Semantic image synthesis with spatially-adaptive nor- malization

    Taesung Park, Ming-Yu Liu, Ting-Chun Wang, and Jun-Yan Zhu. Semantic image synthesis with spatially-adaptive nor- malization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2337–2346,

  21. [29]

    Zero-shot image-to-image translation

    Gaurav Parmar, Krishna Kumar Singh, Richard Zhang, Yijun Li, Jingwan Lu, and Jun-Yan Zhu. Zero-shot image-to-image translation. In ACM SIGGRAPH 2023 conference proceed- ings, pages 1–11, 2023. 6

    2023

  22. [30]

    One-step image translation with text-to-image models

    Gaurav Parmar, Taesung Park, Srinivasa Narasimhan, and Jun-Yan Zhu. One-step image translation with text-to-image models. arXiv preprint arXiv:2403.12036, 2024. 3, 5, 6

    2024 arXiv

  23. [31]

    Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023. 1

    2023 arXiv

  24. [32]

    Efficiently scaling transformer inference

    Reiner Pope, Sholto Douglas, Aakanksha Chowdhery, Jacob Devlin, James Bradbury, Jonathan Heek, Kefan Xiao, Shiv- ani Agrawal, and Jeff Dean. Efficiently scaling transformer inference. Proceedings of Machine Learning and Systems, 5: 606–624, 2023. 8

    2023

  25. [33]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International confer- ence on machine learning, pages 8821–8831. Pmlr, 2021. 1

    2021

  26. [34]

    Gener- ating diverse high-fidelity images with vq-vae-2

    Ali Razavi, Aaron Van den Oord, and Oriol Vinyals. Gener- ating diverse high-fidelity images with vq-vae-2. Advances in neural information processing systems, 32, 2019. 2

    2019

  27. [35]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 1

    2022

  28. [36]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 2250...

    2023

  29. [37]

    Scribbler: Controlling deep image synthesis with sketch and color

    Patsorn Sangkloy, Jingwan Lu, Chen Fang, Fisher Yu, and James Hays. Scribbler: Controlling deep image synthesis with sketch and color. In Proceedings of the IEEE con- ference on computer vision and pattern recognition , pages 5400–5409, 2017. 3

    2017

  30. [38]

    Adversarial diffusion distillation

    Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach. Adversarial diffusion distillation. In European Conference on Computer Vision , pages 87–103. Springer,

  31. [39]

    Learning from simulated and unsupervised images through adversarial training

    Ashish Shrivastava, Tomas Pfister, Oncel Tuzel, Joshua Susskind, Wenda Wang, and Russell Webb. Learning from simulated and unsupervised images through adversarial training. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2107–2116, 2017. 3

    2017

  32. [40]

    Aligning latent and image spaces to connect the un- connectable

    Ivan Skorokhodov, Grigorii Sotnikov, and Mohamed Elho- seiny. Aligning latent and image spaces to connect the un- connectable. In Proceedings of the IEEE/CVF international conference on computer vision, pages 14144–14153, 2021. 5

    2021

  33. [41]

    Dual diffusion implicit bridges for image-to-image transla- tion

    Xuan Su, Jiaming Song, Chenlin Meng, and Stefano Ermon. Dual diffusion implicit bridges for image-to-image transla- tion. arXiv preprint arXiv:2203.08382, 2022. 3, 6

    2022 arXiv

  34. [42]

    Autoregressive model beats diffusion: Llama for scalable image generation

    Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Autoregressive model beats diffusion: Llama for scalable image generation. arXiv preprint arXiv:2406.06525, 2024. 1, 2, 3 10

    2024 arXiv

  35. [43]

    Unsu- pervised cross-domain image generation

    Yaniv Taigman, Adam Polyak, and Lior Wolf. Unsu- pervised cross-domain image generation. arXiv preprint arXiv:1611.02200, 2016. 3

    2016 arXiv

  36. [44]

    Hart: Efficient visual generation with hybrid au- toregressive transformer

    Haotian Tang, Yecheng Wu, Shang Yang, Enze Xie, Junsong Chen, Junyu Chen, Zhuoyang Zhang, Han Cai, Yao Lu, and Song Han. Hart: Efficient visual generation with hybrid au- toregressive transformer. arXiv preprint arXiv:2410.10812,

    arXiv

  37. [45]

    Visual autoregressive modeling: Scalable im- age generation via next-scale prediction

    Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Li- wei Wang. Visual autoregressive modeling: Scalable im- age generation via next-scale prediction. arXiv preprint arXiv:2404.02905, 2024. 2, 3, 5, 12

    2024 arXiv

  38. [46]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth´ee Lacroix, Baptiste Rozi`ere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023. 1

    2023 arXiv

  39. [47]

    Plug-and-play diffusion features for text-driven image-to-image translation

    Narek Tumanyan, Michal Geyer, Shai Bagon, and Tali Dekel. Plug-and-play diffusion features for text-driven image-to-image translation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 1921–1930, 2023. 6

    1921

  40. [48]

    Conditional image genera- tion with pixelcnn decoders

    Aaron Van den Oord, Nal Kalchbrenner, Lasse Espeholt, Oriol Vinyals, Alex Graves, et al. Conditional image genera- tion with pixelcnn decoders. Advances in neural information processing systems, 29, 2016. 2

    2016

  41. [49]

    Pixel recurrent neural networks

    A ¨aron Van Den Oord, Nal Kalchbrenner, and Koray Kavukcuoglu. Pixel recurrent neural networks. In Interna- tional conference on machine learning , pages 1747–1756. PMLR, 2016. 2

    2016

  42. [50]

    Neural discrete representation learning

    Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Advances in neural information pro- cessing systems, 30, 2017. 2

    2017

  43. [51]

    High-resolution image syn- thesis and semantic manipulation with conditional gans

    Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Andrew Tao, Jan Kautz, and Bryan Catanzaro. High-resolution image syn- thesis and semantic manipulation with conditional gans. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 8798–8807, 2018. 3

    2018

  44. [52]

    Learning to cartoonize us- ing white-box cartoon representations

    Xinrui Wang and Jinze Yu. Learning to cartoonize us- ing white-box cartoon representations. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8090–8099, 2020. 7

    2020

  45. [53]

    Emu3: Next-token prediction is all you need

    Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, et al. Emu3: Next-token prediction is all you need. arXiv preprint arXiv:2409.18869, 2024. 1, 2

    2024 arXiv

  46. [54]

    A latent space of stochastic diffusion models for zero-shot image editing and guidance

    Chen Henry Wu and Fernando De la Torre. A latent space of stochastic diffusion models for zero-shot image editing and guidance. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7378–7387, 2023. 3, 6

    2023

  47. [55]

    Dual- gan: Unsupervised dual learning for image-to-image trans- lation

    Zili Yi, Hao Zhang, Ping Tan, and Minglun Gong. Dual- gan: Unsupervised dual learning for image-to-image trans- lation. In Proceedings of the IEEE international conference on computer vision, pages 2849–2857, 2017. 3

    2017

  48. [56]

    Bdd100k: A diverse driving dataset for heterogeneous multitask learning

    Fisher Yu, Haofeng Chen, Xin Wang, Wenqi Xian, Yingying Chen, Fangchen Liu, Vashisht Madhavan, and Trevor Dar- rell. Bdd100k: A diverse driving dataset for heterogeneous multitask learning. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition ...

    2020

  49. [57]

    Scaling autoregres- sive models for content-rich text-to-image generation

    Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gun- jan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yin- fei Yang, Burcu Karagol Ayan, et al. Scaling autoregres- sive models for content-rich text-to-image generation. arXiv preprint arXiv:2206.10789, 2(3):5, 2022. 1, 2

    2022 arXiv

  50. [58]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, pages 3836–3847, 2023. 1, 3

    2023

  51. [59]

    Var-clip: Text-to-image gen- erator with visual auto-regressive modeling

    Qian Zhang, Xiangzi Dai, Ninghua Yang, Xiang An, Ziy- ong Feng, and Xingyu Ren. Var-clip: Text-to-image gen- erator with visual auto-regressive modeling. arXiv preprint arXiv:2408.01181, 2024. 1

    2024 arXiv

  52. [60]

    Uni-controlnet: All-in-one control to text-to-image diffusion models

    Shihao Zhao, Dongdong Chen, Yen-Chun Chen, Jianmin Bao, Shaozhe Hao, Lu Yuan, and Kwan-Yee K Wong. Uni-controlnet: All-in-one control to text-to-image diffusion models. Advances in Neural Information Processing Sys- tems, 36:11127–11150, 2023. 3

    2023

  53. [61]

    Unpaired image-to-image translation using cycle- consistent adversarial networks

    Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cycle- consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision , pages 2223– 2232, 2017. 3, 5, 6

    2017

  54. [62]

    Contrastive learning for unpaired image-to-image translation

    Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Contrastive learning for unpaired image-to-image translation. In ECCV, pages 319–345, 2020. 3, 6

    2020

  55. [63]

    Sean: Image synthesis with semantic region-adaptive nor- malization

    Peihao Zhu, Rameen Abdal, Yipeng Qin, and Peter Wonka. Sean: Image synthesis with semantic region-adaptive nor- malization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5104–5113,

  56. [65]

    Multi-Step Serial Generation Algorithm 1 Multi-Step Serial Generation Require: Source features {Fk}K k=1, class condition t, class em- bedding of s Ensure: Final output ˆHK 1: Initialize ˆH0 ← s 2: for k = 1to K do 3: ˜Gk−1 ← V ARTransformer( ˆHk−1, ( ˆH0, . . . ,ˆHk−1), t) 4:...

  57. [66]

    CycleV AR w/ Infinity Instead of using the original vector quantizer of vanilla V AR, Infinity uses a dimension-independent bitwise quan- tizer, which is implemented by LFQ and BSQ. The input feature f (i,j) is quantized to q(i,j) as follows: q(i,j) = Qsign(f (i,j)) = ( sign(f...

  58. [67]

    next-scale

    Training Details We employed two models with the “next-scale” prediction paradigm as baselines: the class-conditional image gener- ation model V AR [45] and the text-conditional generation model Infinity [8], with Infinity being an extension of V AR. CycleV AR w/ V AR.When tra...

  59. [68]

    Inference Time Comparison

    Inference Time Table 6. Inference Time Comparison. The unit of time is seconds. Method Time Method Time CycleGAN 0.004 SDEdit 1.900 CUT 0.004 Plug&Play 6.300 Pix2pix-Zero 14.20 CycleGAN-Turbo 0.080 Cycle-Diffusion 3.500 CycleV AR w/ V AR 0.030 DDIB 3.900 CycleV AR w/ Infinity ...

  60. [69]

    Qualitative and Quantitative Results Our method produces clearer images with richer texture details while capturing the light and dark color character- istics reminiscent of Shinkai’s style, as shown in Figure

  61. [70]

    The overall color scheme of Car- toonGAN’s images tends to be either too washed out or too dark

    The images generated by AnimeGAN focus too much on the edges, resulting in excessive contrast and a lack of Shinkai’s distinctive style. The overall color scheme of Car- toonGAN’s images tends to be either too washed out or too dark. Conversely, White-box-generated images exhi...

  62. [2020]

    3 11 CycleV AR: Repurposing Autoregressive Model for Unsupervised One-Step Image Translation Supplementary Material

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.