Pith. sign in

REVIEW 7 cited by

ViTGAN: Training GANs with Vision Transformers

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2107.04589 v2 pith:P5X42T44 submitted 2021-07-09 cs.CV cs.LGeess.IV

Kwonjoon Lee , Huiwen Chang , Lu Jiang , Han Zhang , Zhuowen Tu , Ce Liu This is my paper · ORCID
classification cs.CVcs.LGeess.IV
keywords gansperformancetrainingimageregularizationtransformersvisionvitgan
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Recently, Vision Transformers (ViTs) have shown competitive performance on image recognition while requiring less vision-specific inductive biases. In this paper, we investigate if such performance can be extended to image generation. To this end, we integrate the ViT architecture into generative adversarial networks (GANs). For ViT discriminators, we observe that existing regularization methods for GANs interact poorly with self-attention, causing serious instability during training. To resolve this issue, we introduce several novel regularization techniques for training GANs with ViTs. For ViT generators, we examine architectural choices for latent and pixel mapping layers to facilitate convergence. Empirically, our approach, named ViTGAN, achieves comparable performance to the leading CNN-based GAN models on three datasets: CIFAR-10, CelebA, and LSUN bedroom.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 7 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. The GAN is dead; long live the GAN! A Modern GAN Baseline

    cs.LG 2025-01 conditional novelty 7.0 of 10

    A minimalist GAN with a regularized relativistic loss and modern backbone matches or beats StyleGAN2 and several diffusion models on standard FID benchmarks.

  2. Scalable GANs with Transformers

    cs.CV 2025-09 conditional novelty 6.0 of 10

    A transformer-only GAN trained in VAE latent space with multi-level noise supervision and width-scaled learning rates achieves FID 2.96 on ImageNet-256 in 40 epochs.

  3. Dreamweaver: Learning Compositional World Models from Pixels

    cs.CV 2025-01 conditional novelty 6.0 of 10

    An unsupervised recurrent block-slot model that discovers static and dynamic concept blocks from raw video and recombines them to imagine novel future videos.

  4. Deeper Inside Deep ViT

    cs.CV 2025-08 conditional novelty 5.0 of 10

    Small-scale ViT-22B models outperform standard ViT under matched parameter counts on CIFAR, and a proposed ViTUnet runs image-to-image translation, though without strong quantitative validation.

  5. GM-LDM: Latent Diffusion Model for Brain Biomarker Identification through Functional Data-Driven Gray Matter Synthesis

    eess.IV 2025-06 conditional novelty 4.0 of 10

    A latent diffusion model pretrained on large MRI datasets generates subject-specific gray matter images from functional connectivity data, reporting improved similarity scores and schizophrenia-related regional differences.

  6. Landing Trajectory Prediction for UAS Based on Generative Adversarial Network

    cs.RO 2024-11 conditional novelty 4.0 of 10

    A Social-GAN-style LSTM generator predicts UAS landing trajectories and beats a Gaussian Mixture Regression baseline on real drone data, though the advantage disappears on simulated data beyond four steps.

  7. Texture Image Synthesis Using Spatial GAN Based on Vision Transformers

    cs.CV 2025-02 reject novelty 3.0 of 10

    ViT-SGAN modifies ViTGAN's self-attention with mean-variance and texton descriptors to synthesize textures, but the evaluation is too weak and the equations are too unclear to support the claimed gains.

Pith tools