Pith. sign in

REVIEW 6 cited by

Your ViT is Secretly a Hybrid Discriminative-Generative Diffusion Model

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2208.07791 v1 pith:REZP2W56 submitted 2022-08-16 cs.CV

Xiulong Yang , Sheng-Min Shih , Yinlin Fu , Xiaoting Zhao , Shihao Ji This is my paper · ORCID
classification cs.CV
keywords generativemodelsddpmdiffusionhybridtasksdiscriminativediscriminative-generative
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Diffusion Denoising Probability Models (DDPM) and Vision Transformer (ViT) have demonstrated significant progress in generative tasks and discriminative tasks, respectively, and thus far these models have largely been developed in their own domains. In this paper, we establish a direct connection between DDPM and ViT by integrating the ViT architecture into DDPM, and introduce a new generative model called Generative ViT (GenViT). The modeling flexibility of ViT enables us to further extend GenViT to hybrid discriminative-generative modeling, and introduce a Hybrid ViT (HybViT). Our work is among the first to explore a single ViT for image generation and classification jointly. We conduct a series of experiments to analyze the performance of proposed models and demonstrate their superiority over prior state-of-the-arts in both generative and discriminative tasks. Our code and pre-trained models can be found in https://github.com/sndnyang/Diffusion_ViT .

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Remix-DiT: Mixing Diffusion Transformers for Multi-Expert Denoising

    cs.CV 2024-12 conditional novelty 6.0 of 10

    Remix-DiT crafts many timestep-specialized diffusion experts by learnably mixing a small number of basis transformers, improving ImageNet generation FID at standard inference cost.

  2. Pretrained Reversible Generation as Unsupervised Visual Representation Learning

    cs.CV 2024-11 conditional novelty 6.0 of 10

    Reversing a pretrained flow or diffusion generator and fine-tuning it with a classification head yields strong image classifiers, reaching 78.1% top-1 on ImageNet-64.

  3. MetaFE-DE: Learning Meta Feature Embedding for Depth Estimation from Monocular Endoscopic Images

    eess.IV 2025-02 conditional novelty 5.0 of 10

    A temporal diffusion pretraining stage aligned with frame latents improves self-supervised monocular depth estimation in endoscopic video.

  4. Exploring Structured Semantic Priors Underlying Diffusion Score for Test-time Adaptation

    cs.CV 2025-01 conditional novelty 5.0 of 10

    DUSA adapts classifiers and segmenters at test time by matching their predictions to conditional noise estimates from a pre-trained diffusion model, using a single timestep and active class selection.

  5. LazyDiT: Lazy Learning for the Acceleration of Diffusion Transformers

    cs.LG 2024-12 conditional novelty 5.0 of 10

    LazyDiT learns small gates that decide when to reuse cached layer outputs, cutting diffusion transformer compute by up to half while matching or beating DDIM quality.

  6. Improving Joint Embedding Predictive Architecture with Diffusion Noise

    cs.CV 2025-07 conditional novelty 4.0 of 10

    Injecting EDM-style noise into masked-token position embeddings and adding two auxiliary losses improves I-JEPA's linear-probing accuracy by about 1.5 points on ImageNet-1K.

Pith tools