Pith. sign in

REVIEW 4 major objections 7 minor 77 references

Pretrained Reversible Generation as Unsupervised Visual Representation Learning

T0 review · 4 major / 7 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Turning a pretrained continuous-time flow/diffusion model backward yields an unsupervised feature extractor that, with a light fine-tuned head, reaches 78% top-1 accuracy on ImageNet at 64×64 and beats prior generative-model-based…

desk verdict A simple and plausible recipe for turning pretrained flow/diffusion models into strong classifiers, but the SOTA claim is only as solid as its unretrained baselines and single-run numbers. read the letter →

arxiv 2412.01787 v6 pith:YFLDRZ6F submitted 2024-11-29 cs.CV cs.AIcs.LG

Rongkun Xue , Jinouwen Zhang , Yazhe Niu , Dazhong Shen , Bingqi Ma , Yu Liu , Jing Yang This is my paper · ORCID
classification cs.CVcs.AIcs.LG
keywords representationlearningdiffusionmodelsflowmatchingreversiblegenerationimageclassificationout-of-distributionrobustnessmutualinformationunsupervisedpretraining
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes Pretrained Reversible Generation (PRG), a recipe for turning a pretrained continuous-time generative model into an unsupervised visual feature extractor: instead of reading internal layers, run the model's ODE backward and use the reverse trajectory as the representation. The authors argue that score-matching or flow-matching pretraining maximizes a lower bound on the mutual information between an image and its reversed latent, so the reverse process carries what the model learned about the data, and a small classifier fine-tuned jointly with the generative model outperforms prior generative-model-based approaches. The headline results are 97.65% accuracy on CIFAR-10, 71.33% on Tiny-ImageNet, and 78% top-1 on ImageNet at 64×64 resolution, with stronger out-of-distribution performance under corruptions. The contribution is a simple, model-agnostic way to convert the growing library of pretrained diffusion and flow checkpoints into classifiers without hand-designing which layers to read.

What carries the argument

The load-bearing object is the probability-flow ODE (Eq. 1), which makes the generative process of a diffusion or flow model a deterministic, exactly invertible map between data $x_0$ and latent $x_1$; PRG treats the reverse of this map, $F_\theta(x_0)=x_1$, as the encoder. The training objective is $L_{\mathrm{total}}=-\sum_i \log p_\phi(y_i|F_\theta(x_i))+\beta L_{\mathrm{FM}}(x)$ (Eq. 3), combining a classification head with a flow-matching term that preserves the generative structure. The theoretical spine is the mutual-information lower bound: pretraining by flow matching maximizes a lower bound on $I(X_0,X_1)$ through the score/flow-matching likelihood bound (Eq. 8), so the reversed latent is information-preserving; fine-tuning then prunes reconstruction-irrelevant detail. Since the ODE can be integrated with adaptive solvers, the network acts as an infinite-depth continuous feature extractor whose features are stable across step counts and solver choices.

What would settle it

Retrain or reuse the identical pretrained diffusion model and compare PRG against a DDAE-style internal-feature extractor at the same 64×64 resolution, same fine-tuning budget, and same evaluation protocol; if PRG does not beat the internal-feature baseline consistently across repeated runs, the claim that reversal is the operative ingredient is falsified. As a second check, track the paper's mutual-information indicator $I(x_0,x_1)+H(x_1)-H(x_0)$ during fine-tuning on a dataset with a known generative process: the explanation predicts it should decline while accuracy rises.

Watch

Extended reading notes

Core claim

The central claim is that the reverse of a pretrained continuous-time generative process is itself an unsupervised encoder: because the probability-flow ODE is exactly invertible, mapping an image $x_0$ through the same model backward to $x_1$ produces a latent that retains a large lower bound on the mutual information $I(X_0, X_1)$, and after light fine-tuning this latent supports strong downstream classification. The paper establishes the mechanism in two parts. Theoretically, score-matching and flow-matching pretraining are shown to increase data likelihood and therefore the mutual-information lower bound (Eqs. 4–8). Empirically, end-to-end fine-tuning with a two-layer MLP head and a small flow-matching regularizer (Eq. 3) is necessary—freezing the generative model drops accuracy to 47.10% on CIFAR-10 and 58.04% on Tiny-ImageNet, against roughly 97% and 71% when it is fine-tuned—and features at any time $t$ along the reverse trajectory remain usable, making the extractor continuous. On this basis PRG reports state-of-the-art accuracy among generative-model-based classifiers on three benchmarks and shows the approach transfers to a community text-to-image backbone.

Load-bearing premise

The headline comparisons assume that accuracy numbers reported by other papers at different resolutions and parameter counts are directly comparable to PRG's single-run results at 64×64, and the theory assumes that flow-matching pretraining genuinely raises the mutual-information lower bound the argument leans on.

Editorial extensions

If this is right

  • Any pretrained continuous-time diffusion or flow model can be converted into a downstream classifier by running it in reverse and fine-tuning a small head, with no access to internal features.
  • Stronger generative pretraining directly improves downstream classification: accuracy rises monotonically with pretraining epochs and with the mutual-information indicator the paper tracks.
  • Fine-tuning must update the generative parameters; freezing the flow model drops accuracy to 47.10% on CIFAR-10 and 58.04% on Tiny-ImageNet, far below the fine-tuned numbers.
  • The extracted features are continuous and stable to discretization: a checkpoint fine-tuned at the trajectory midpoint works across 20 to 1000 inference steps and ±20% offsets with negligible loss.
  • The same recipe transfers to community text-to-image backbones and improves out-of-distribution accuracy under common image corruptions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper demonstrates that features at different trajectory times $t$ are usable but does not prescribe how to choose $t$; a natural extension is to concatenate features from several $t$ values, pooling low-level detail near $x_0$ with semantic structure near $x_1$, which could beat any single cut point.
  • The straightness results (Tab. 8: OTCFM reaches 97% in 128 epochs versus 162 for the curved GVP path) suggest a selection principle the paper leaves implicit: among equally strong generative models, prefer the flow with straighter trajectories because it needs fewer pretraining steps to reach comparable representations; consistency and rectified-flow models should plug into PRG with even faster ad
  • Because the framework reads only the ODE trajectory, a pure-MLP flow model that satisfies the same invertibility should yield comparable representations, which would isolate whether the ODE structure rather than the image-backbone inductive bias is doing the work.
  • The same reversal could be applied to prompt-conditioned text-to-image models for zero-shot classification by comparing reconstruction losses of reverse runs under different text labels, bridging PRG's fine-tuned classifiers with the zero-shot generative classifier line the paper surveys.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. This paper introduces PRG (Pretrained Reversible Generation), a two-stage representation learning method. In the first stage, a continuous-time generative model (diffusion or flow matching variants GVP, ICFM, OTCFM) is pretrained on unlabeled images. In the second stage, the model is fine-tuned for downstream classification by running the generative ODE in the reverse direction (from data x0 to latent x1) and attaching a small MLP classifier to the terminal or intermediate states, with a combined cross-entropy and flow-matching loss. The paper reports accuracies of 97.59% on CIFAR-10, 71.12% on Tiny-ImageNet, and 78.1% on ImageNet at 64x64, claiming state-of-the-art among generative-model-based methods, plus OOD robustness on CIFAR-10-C and Tiny-ImageNet-C and a transfer experiment with SiT-XL. The paper also provides a theoretical analysis intended to explain why flow-matching pretraining benefits downstream tasks, and a set of ablations on model type, loss weight, scaling, ODE solvers, and trajectory length.

Significance. The core idea is simple and potentially broadly applicable: the reversibility of continuous-time flow/diffusion models means a pretrained generator can be reused as a feature extractor by reversing its trajectory, avoiding the need to access internal activations. The paper is strong on the empirical side in several respects: the method is model-agnostic; results are robust to ODE solver choice and inference step count (Tables 1, 2 and Appendix C.2); ablations cover model scaling, path type, and loss weight; and the transfer experiment with SiT-XL (Table 7) demonstrates compatibility with large pretrained generative models. The code is publicly available. The main weaknesses are the uncontrolled baseline comparison underlying the SOTA claim (especially on ImageNet), the absence of error bars, and a theoretical section whose information-theoretic argument is not valid for the stated deterministic invertible setting. If the controlled comparison and theory are fixed, the method would be a useful contribution.

major comments (4)
  1. [Section 3.2, Eqs. (4)-(9)] The mutual-information justification is not valid for the stated setting. The paper assumes 'deterministic mappings for both the encoder and decoder, which are invertible' (Sec. 3.2). For an invertible deterministic encoder z = F_theta(x), the conditional distribution p(x|z) is a Dirac delta, so the differential mutual information I(X,Z) is ill-defined (or infinite), and Eqs. (4)-(7) cannot be interpreted as a bound on I(X,Z). The argument then replaces the conditional likelihood E_{p(z,x)}[log p_theta'(x|z)] with a bound on the marginal likelihood D_KL(p(x0) || p_theta'(x0)) in Eq. (8); these are different quantities. The empirical proxy in Sec. 4.3.1 computed via the change-of-variables formula (10) is log p(x0) - log p(x1), i.e., the log-density ratio of the generative model's marginal, not a conditional density, so the identity 'E_p[log p(x0|x1)] = I(X0,X1) - H(X0)' does not hold as stated. I recommend either providing a correct information-theoretic statement (e.g., in terms of the ELBO of the generative model) or clearly labeling this as an analogy rather than a derivation.
  2. [Section 3.2, after Eq. (9); Eq. (2)] The assertion that flow matching 'also can be used to increase the likelihood, as shown in Eq. (2)' is unsupported. Eq. (2) is the flow-matching MSE objective; the references cited for likelihood bounds ([42,59,76]) address score-based diffusion ODEs, not the general flow-matching objective used for ICFM/OTCFM. No derivation or reference establishes a likelihood bound for these losses. This claim should be either proved, replaced by a precise citation, or explicitly labeled as a conjecture.
  3. [Sec. 4.4.1, Table 5] The headline claim of state-of-the-art performance among generative-model-based methods is not supported by a controlled comparison on ImageNet. Table 5 compares PRG at 64x64 (122M) with HybViT at 32x32 (43M), iGPT-L at 48x48 (1362M), and DMSZC at 256x256 (338M, zero-shot); DDAE, the strongest fine-tuned generative baseline on CIFAR/Tiny, is missing from the ImageNet table. All PRG numbers are from single runs without error bars. To substantiate the claim, the authors should provide a matched comparison at a common resolution (e.g., 64x64) using the same fine-tuning loss and augmentation protocol, and report the mean and standard deviation over at least three seeds.
  4. [Appendix C.1, Table 6; Sec. 4.2] Part of the margin over baselines may be due to fine-tuning protocol rather than the proposed representation. Appendix C.1 shows that label smoothing alone improves CIFAR-10 by +1.41 (from 96.18 to 97.59). Since the baseline numbers (e.g., DDAE's 97.2) are quoted from other papers without the same loss and augmentation configuration, the comparison is not like-for-like. The authors should either compare with baselines retrained under the same protocol or quantify the contribution of each training component (loss, augmentation, trajectory length) to the final accuracy.
minor comments (7)
  1. [Section 3.1] The notation 't ∈ [1, 0]' is non-standard and confusing; please use a standard interval and specify the integration direction and sign for the ODE used to compute F_theta(x0).
  2. [Eq. (3)] The summation notation '- NX i=1' appears to be a rendering error; it should be '-∑_{i=1}^N'. Also, the classifier p_phi(y|z) is defined for classification, but Sec. 3.1 also mentions regression; please specify the head and loss for regression.
  3. [Table 7] The baseline row 'MAE (ViT-B/16) [41]' is cited to Ref. [41] (Liu et al., 'Good helper is around you'), which is not the MAE paper; the correct reference for MAE is Ref. [24]. Please fix the citation.
  4. [Sec. 4.1 vs Appendix B.1] Sec. 4.1 states pre-training uses a fixed learning rate of 1e-4 for 1,200 epochs, while Appendix B.1 Table 2 lists 1000/1000/2000 epochs for CIFAR-10/Tiny-ImageNet/ImageNet; please make these consistent.
  5. [Abstract and Sec. 3.3] The abstract and Sec. 3.3 claim adaptation of 'community-developed text-to-image models' and refer to Sec. 4.4.3, but Sec. 4.4.3 only transfers SiT-XL, which is class-conditional rather than text-to-image; either add the claimed text-to-image experiment or correct the claim.
  6. [Appendix A.1.1 and Table 3] Appendix A.1.1 Table 1 and Table 3 both use labels such as 'PRG-onlyPretrain' and 'PRG w/o pre-training', but they appear to describe different settings (no fine-tuning vs. no pre-training); please clarify the terminology.
  7. [Table 9] The accuracy differences between β values are within 0.001 on CIFAR-10, and no error bars or seeds are reported; please report mean ± std over multiple runs or state that the differences are not statistically significant.

Circularity Check

1 steps flagged · score 2.0 of 10

No load-bearing circularity; the MI verification curves are self-referential diagnostics, but the central benchmark claims rest on external evaluations.

  1. other [Sec. 4.3.1 (Verification Analysis), Eqs. (2), (8), (10)-(11), Figs. 3-4]
    "During pre-training, we have E_{p(x1,x0)}[log p(x0 | x1)] = I(X0, X1) − H(X0) ... Hence, E_{p(x1,x0)}[log p(x0 | x1)] faithfully reflects the mutual information I(X0, X1) during pre-training. ... We compute log-probability density log p(x0|x1) using the adjoint method or Neural ODEs [11] for computational feasibility."

    The plotted 'mutual information' is E[log p(x0|x1)] computed with the same velocity network vθ minimized in Eq. (2). Since Eq. (8) states that score/flow-matching training increases this likelihood lower bound, the monotone rise in Fig. 3 is, by construction, the model's own pretraining objective rather than an independent measurement of I(X0,X1). The paper then cites this curve as evidence that better pre-training leads to better fine-tuning. The accuracy values come from external labels, so the central representation-learning claim is not forced; this is a minor, non-load-bearing self-referentiality.

full rationale

The core pipeline — pretrain a flow/diffusion generator, run it backward to define features Fθ(x0)=x1, and fine-tune a classifier with Ltotal = CE + βLFM — is evaluated on held-out CIFAR-10, Tiny-ImageNet, and ImageNet labels and against published baselines, so the main claims are not definitionally tied to the model's own outputs. The theoretical bound in Sec. 3.2 (Eqs. 4-8) is a standard likelihood/MI argument importing external results from Song et al., Lu et al., and Zheng et al., not a self-citation chain. The only citation of the authors' own prior work ([75]) is a passing reference to scientific modeling and is not load-bearing. The self-referential element is Sec. 4.3.1's MI diagnostic: its 'I(X0,X1)' curve is computed from the trained model's own log-likelihood, which is the pretraining objective, so Fig. 3's increase is partly by construction. The benchmark results and the pretraining-versus-no-pretraining accuracy comparison are independent of this diagnostic and remain meaningful. The SOTA comparison has a fairness weakness (baselines from other papers at different resolutions and training recipes, e.g., label smoothing contributes +1.41 on CIFAR-10 in Appendix C.1), but that is a correctness/experimental-design issue, not circularity. Overall, no load-bearing circularity; score 2 reflects the minor self-referential MI evidence.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The paper introduces no new physical entities. Its free parameters are standard training hyperparameters (loss weight, trajectory length, start point). The main unproven assumption is the extension of the score-matching likelihood bound to flow matching.

free parameters (4)
  • beta (loss weight in Eq. 3) = 10 (best on CIFAR/Tiny among 1, 10, 100)
    Weight of the flow-matching loss during fine-tuning, selected on validation (Table 9).
  • tspan (ODE trajectory length for fine-tuning) = 20 (CIFAR), 32 (Tiny), 64 (ImageNet)
    Chosen per dataset; longer spans give better accuracy up to a point (Appendix B Table 3, Appendix C.4).
  • fine-tuning start point along trajectory = e.g., near midpoint or later (x1/4 to x1)
    The paper reports task-dependent optimal start points (Sec 4.3.2) and uses cutoff-based schedules (Appendix B Table 4).
  • label smoothing vs cross-entropy = label smoothing chosen
    Chosen because it improves accuracy by about 1.4 points on CIFAR (Appendix C.1).
assumptions (6)
  • domain assumption Data lies on a low-dimensional manifold M; pretrained flow models map this manifold to semantically meaningful points in ambient space.
    Invoked in Appendix A and Sec 3.2 to justify why intermediate latents are useful for downstream tasks.
  • standard math The score-matching likelihood bound (Eq. 8) from prior work holds for the pretraining objective.
    Quoted from Song et al. 2021, Lu et al. 2022, Zheng et al. 2023; the paper cites [42, 59, 76].
  • ad hoc to paper The flow-matching objective (Eq. 2) also provides a likelihood bound in the same way score matching does.
    Asserted without derivation in Sec 3.2: 'Flow matching differs from score matching by employing a slightly different weighting factor... so it can also be used to increase the likelihood.'
  • standard math ODE trajectories of a trained flow model do not intersect, preserving topological structure during transformation.
    Used in Appendix A to argue reverse trajectories are stable and meaningful; based on neural ODE theory [11].
  • domain assumption The terminal distributions p(x1) and p_theta'(x1) are both approximately Gaussian during pre-training.
    Used in the Eq. 8 argument; standard for a well-trained diffusion or flow model that maps data to a Gaussian prior.
  • standard math Hutchinson trace estimator gives a sufficiently accurate estimate of the log-density change in Eq. (10).
    Used to compute the mutual-information proxy; standard Monte Carlo trace estimation [22, 29].

how reviews work

0 comments
Cite this review

Pith. "Pith review of Pretrained Reversible Generation as Unsupervised Visual Representation Learning." pith.science (2026). https://pith.science/paper/YFLDRZ6F

@misc{pith2026241201787,
  author       = {Pith},
  title        = {Pith review of: Pretrained Reversible Generation as Unsupervised Visual Representation Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YFLDRZ6F}},
  note         = {Machine review of arXiv:2412.01787}
}
read the original abstract

Recent generative models based on score matching and flow matching have significantly advanced generation tasks, but their potential in discriminative tasks remains underexplored. Previous approaches, such as generative classifiers, have not fully leveraged the capabilities of these models for discriminative tasks due to their intricate designs. We propose Pretrained Reversible Generation (PRG), which extracts unsupervised representations by reversing the generative process of a pretrained continuous generation model. PRG effectively reuses unsupervised generative models, leveraging their high capacity to serve as robust and generalizable feature extractors for downstream tasks. This framework enables the flexible selection of feature hierarchies tailored to specific downstream tasks. Our method consistently outperforms prior approaches across multiple benchmarks, achieving state-of-the-art performance among generative model based methods, including 78% top-1 accuracy on ImageNet at a resolution of 64*64. Extensive ablation studies, including out-of-distribution evaluations, further validate the effectiveness of our approach.PRG is available at https://github.com/opendilab/PRG.

Figures

Figures reproduced from arXiv: 2412.01787 by the authors.

Figure 1
Figure 1. Overview of the PRG as Unsupervised Visual Repre [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. CIFAR-10 Accuracy (y-axis) after fine-tuning checkpoints from different pre￾training stages (x-axis). 0 200 400 600 800 1000 1200 Pretrained Epoch 4.0 5.5 7.0 p(x1, x0)[lo g p(x 0|x 1)] GVP Model (Train) ICFM Model (Train) OTCFM Model (Train) GVP Model (Val) ICFM Model (Val) OTCFM Model (Val) [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 5
Figure 5. Reverse generation from x_0 to x_1 with full pre-training and fine-tuning. 0 2 4 6 8 10 12 14 16 18 20 Time Step 0 20 40 60 80 100 Acc. (%) 97 71 47.10% 58.04% CIFAR-10 No Frozen CIFAR-10 Frozen Tiny-ImageNet No Frozen Tiny-ImageNet Frozen [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figures from the paper (5 more)
Figure 6
Figure 6. Figure 6: Accuracy on CIFAR-10 and Tiny-ImageNet when fine [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 1
Figure 1. Figure 1: Reverse Generation Process on the Swiss Roll Dataset. Each color represents a different class. After diffusion, samples from the [PITH_FULL_IMAGE:figures/full_fig_p015_1.png]
Figure 2
Figure 2. Figure 2: Reverse Generation Process on the TinyImageNet val set. The first row represents the fully pretrained reverse generative process, [PITH_FULL_IMAGE:figures/full_fig_p015_2.png]
Figure 3
Figure 3. Figure 3: Reverse Generation Process on the TinyImageNet train set. The first row represents the fully pretrained reverse generative process, [PITH_FULL_IMAGE:figures/full_fig_p016_3.png]
Figure 4
Figure 4. Figure 4: Reverse Generation Process on the TinyImageNet-C Dataset. The first row represents the fully pretrained reverse generative [PITH_FULL_IMAGE:figures/full_fig_p016_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

77 extracted references · 48 canonical work pages

  1. [1]

    Ac- curate structure prediction of biomolecular interactions with alphafold 3

    Josh Abramson, Jonas Adler, Jack Dunger, Richard Evans, Tim Green, Alexander Pritzel, Olaf Ronneberger, Lindsay Willmore, Andrew J Ballard, Joshua Bambrick, et al. Ac- curate structure prediction of biomolecular interactions with alphafold 3. Nature, pages 1–3, 2024. 1

    work page 2024

  2. [2]

    Build- ing normalizing flows with stochastic interpolants

    Michael Samuel Albergo and Eric Vanden-Eijnden. Build- ing normalizing flows with stochastic interpolants. In The Eleventh International Conference on Learning Representa- tions, 2023. 1

    work page 2023

  3. [3]

    All are worth words: A vit backbone for diffusion models

    Fan Bao, Shen Nie, Kaiwen Xue, Yue Cao, Chongxuan Li, Hang Su, and Jun Zhu. All are worth words: A vit backbone for diffusion models. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 22669–22679, 2023. 13

    work page 2023

  4. [4]

    Label-efficient seman- tic segmentation with diffusion models

    Dmitry Baranchuk, Ivan Rubachev, Andrey V oynov, Valentin Khrulkov, and Artem Babenko. Label-efficient seman- tic segmentation with diffusion models. arXiv preprint arXiv:2112.03126, 2021. 2

    arXiv 2021

  5. [5]

    Rep- resentation learning: A review and new perspectives

    Yoshua Bengio, Aaron Courville, and Pascal Vincent. Rep- resentation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence, 35 (8):1798–1828, 2013. 2

    work page 2013

  6. [6]

    Generalized denoising auto-encoders as generative models

    Yoshua Bengio, Li Yao, Guillaume Alain, and Pascal Vincent. Generalized denoising auto-encoders as generative models. Advances in neural information processing systems, 26, 2013. 2

    work page 2013

  7. [7]

    Improving image generation with better captions

    James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al. Improving image generation with better captions. Computer Science. https://cdn. openai. com/papers/dall-e-3. pdf, 2(3):8, 2023. 4

    2023

  8. [8]

    Diffusion models are certifiably robust classifiers

    Huanran Chen, Yinpeng Dong, Shitong Shao, Zhongkai Hao, Xiao Yang, Hang Su, and Jun Zhu. Diffusion models are certifiably robust classifiers. In The Thirty-eighth Annual Conference on Neural Information Processing Systems. 2, 7

    work page

Show all 77 references
  1. [9]

    Robust clas- sification via a single diffusion model

    Huanran Chen, Yinpeng Dong, Zhengyi Wang, Xiao Yang, Chengqi Duan, Hang Su, and Jun Zhu. Robust clas- sification via a single diffusion model. arXiv preprint arXiv:2305.15241, 2023. 2

    2023 arXiv

  2. [10]

    Generative pretraining from pixels

    Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, and Ilya Sutskever. Generative pretraining from pixels. In International conference on machine learning, pages 1691–1703. PMLR, 2020. 1, 2, 7, 8

    2020

  3. [11]

    Ricky T. Q. Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. In Advances in Neural Information Processing Systems. Curran Associates, Inc., 2018. 2, 4, 5, 12

    2018

  4. [12]

    A simple framework for contrastive learning of visual representations

    Ting Chen, Simon Kornblith, Mohammad Norouzi, and Ge- offrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pages 1597–1607. PMLR, 2020. 1, 8

    2020

  5. [13]

    Infogan: Interpretable rep- resentation learning by information maximizing generative adversarial nets

    Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. Infogan: Interpretable rep- resentation learning by information maximizing generative adversarial nets. Advances in neural information processing systems, 29, 2016. 2

    2016

  6. [14]

    Text-to-image diffusion mod- els are zero shot classifiers

    Kevin Clark and Priyank Jaini. Text-to-image diffusion mod- els are zero shot classifiers. Advances in Neural Information Processing Systems, 36, 2024. 2

    2024

  7. [15]

    Autoaugment: Learning augmentation strategies from data

    Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasude- van, and Quoc V Le. Autoaugment: Learning augmentation strategies from data. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 113–123, 2019. 5, 7, 14

    2019

  8. [16]

    Large scale adversar- ial representation learning

    Jeff Donahue and Karen Simonyan. Large scale adversar- ial representation learning. Advances in neural information processing systems, 32, 2019. 2

    2019

  9. [17]

    An image is worth 16x16 words: Trans- formers for image recognition at scale

    Alexey Dosovitskiy. An image is worth 16x16 words: Trans- formers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020. 1, 7

    2010 arXiv

  10. [18]

    Your classifier is secretly an energy based model and you should treat it like one

    David Duvenaud, Jackson Wang, Jorn Jacobsen, Kevin Swer- sky, Mohammad Norouzi, and Will Grathwohl. Your classifier is secretly an energy based model and you should treat it like one. In ICLR 2020, 2020. 2, 5, 7

    2020

  11. [19]

    Understanding the limitations of conditional generative models

    Ethan Fetaya, J ¨orn-Henrik Jacobsen, Will Grathwohl, and Richard Zemel. Understanding the limitations of conditional generative models. arXiv preprint arXiv:1906.01171, 2019. 2, 5, 7

    1906 arXiv

  12. [20]

    Scheduled denoising autoencoders

    Krzysztof J Geras and Charles Sutton. Scheduled denoising autoencoders. arXiv preprint arXiv:1406.3269, 2014. 2

    2014 arXiv

  13. [21]

    Generative adversarial networks

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Communi- cations of the ACM, 63(11):139–144, 2020. 2

    2020

  14. [22]

    Will Grathwohl, Ricky T. Q. Chen, Jesse Bettencourt, and David Duvenaud. Scalable reversible generative models with free-form continuous dynamics. In International Conference on Learning Representations, 2019. 2, 5

    2019

  15. [23]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 1, 6, 7

    2016

  16. [24]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll´ar, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 16000– 16009, 2022. 2

    2022

  17. [25]

    Augmix: A simple data processing method to improve robustness and uncertainty

    Dan Hendrycks, Norman Mu, Ekin D Cubuk, Barret Zoph, Justin Gilmer, and Balaji Lakshminarayanan. Augmix: A simple data processing method to improve robustness and uncertainty. arXiv preprint arXiv:1912.02781, 2019. 5, 7, 14

    1912 arXiv

  18. [26]

    Denoising diffu- sion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 2 9

    2020

  19. [27]

    Video diffusion mod- els

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffusion mod- els. In Advances in Neural Information Processing Systems, pages 8633–8646. Curran Associates, Inc., 2022. 1

    2022

  20. [28]

    Dynamicid: Zero-shot multi-id image personalization with flexible facial editability

    Xirui Hu, Jiahao Wang, Hao Chen, Weizhan Zhang, Benqi Wang, Yikun Li, and Haishun Nan. Dynamicid: Zero-shot multi-id image personalization with flexible facial editability. arXiv preprint arXiv:2503.06505, 2025. 2

    2025 arXiv

  21. [29]

    Hutchinson

    M.F. Hutchinson. A stochastic estimator of the trace of the influence matrix for laplacian smoothing splines. Commu- nications in Statistics - Simulation and Computation, 19(2): 433–450, 1990. 5

    1990

  22. [30]

    Auto-encoding variational bayes

    Diederik P Kingma. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013. 2

    2013 arXiv

  23. [31]

    Adam: A method for stochastic opti- mization

    Diederik P Kingma. Adam: A method for stochastic opti- mization. arXiv preprint arXiv:1412.6980, 2014. 5

    2014 arXiv

  24. [32]

    On the effectiveness of adversarial training against common corruptions

    Klim Kireev, Maksym Andriushchenko, and Nicolas Flam- marion. On the effectiveness of adversarial training against common corruptions. In Uncertainty in Artificial Intelligence, pages 1012–1021. PMLR, 2022. 14

    2022

  25. [33]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky. Learning multiple layers of features from tiny images. 2009. 5

    2009

  26. [34]

    Ya Le and Xuan S. Yang. Tiny imagenet visual recognition challenge. 2015. 5

    2015

  27. [35]

    Your diffusion model is secretly a zero-shot classifier

    Alexander C Li, Mihir Prabhudesai, Shivam Duggal, Ellis Brown, and Deepak Pathak. Your diffusion model is secretly a zero-shot classifier. In CVPR, pages 2206–2217, 2023. 2, 7

    2023

  28. [36]

    An application of the principle of maximum in- formation preservation to linear systems

    Ralph Linsker. An application of the principle of maximum in- formation preservation to linear systems. Advances in neural information processing systems, 1, 1988. 3

    1988

  29. [37]

    Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maxim- ilian Nickel, and Matthew Le. Flow matching for genera- tive modeling. In The Eleventh International Conference on Learning Representations, 2023. 1

    2023

  30. [38]

    Towards robust neural networks via random self- ensemble

    Xuanqing Liu, Minhao Cheng, Huan Zhang, and Cho-Jui Hsieh. Towards robust neural networks via random self- ensemble. In Proceedings of the european conference on computer vision (ECCV), pages 369–385, 2018. 7, 14

    2018

  31. [39]

    Flow straight and fast: Learning to generate and transfer data with rectified flow

    Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint arXiv:2209.03003, 2022. 1, 8

    2022 arXiv

  32. [40]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012–10022, 2021. 5, 7

    2021

  33. [41]

    Good helper is around you: Attention-driven masked image modeling

    Zhengqi Liu, Jie Gui, and Hao Luo. Good helper is around you: Attention-driven masked image modeling. In Proceed- ings of the AAAI Conference on Artificial Intelligence, pages 1799–1807, 2023. 8

    2023

  34. [42]

    Maximum likelihood training for score- based diffusion odes by high order denoising score matching

    Cheng Lu, Kaiwen Zheng, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Maximum likelihood training for score- based diffusion odes by high order denoising score matching. In International Conference on Machine Learning , pages 14429–14460. PMLR, 2022. 4

    2022

  35. [43]

    Sit: Exploring flow and diffusion-based generative models with scalable interpolant transformers

    Nanye Ma, Mark Goldstein, Michael S Albergo, Nicholas M Boffi, Eric Vanden-Eijnden, and Saining Xie. Sit: Exploring flow and diffusion-based generative models with scalable interpolant transformers. arXiv preprint arXiv:2401.08740,

    arXiv

  36. [44]

    Towards deep learning models resistant to adversarial attacks

    Aleksander Madry. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083, 2017. 5, 7, 14

    2017 arXiv

  37. [45]

    Symbolic music generation with diffusion models

    Gautam Mittal, Jesse Engel, Curtis Hawthorne, and Ian Simon. Symbolic music generation with diffusion models. arXiv preprint arXiv:2103.16091, 2021. 1

    2021 arXiv

  38. [46]

    Diffusion based represen- tation learning

    Sarthak Mittal, Korbinian Abstreiter, Stefan Bauer, Bernhard Sch¨olkopf, and Arash Mehrjou. Diffusion based represen- tation learning. In International Conference on Machine Learning, pages 24963–24982. PMLR, 2023. 3

    2023

  39. [47]

    Probabilistic machine learning: Advanced topics

    Kevin P Murphy. Probabilistic machine learning: Advanced topics. MIT press, 2023. 12

    2023

  40. [48]

    On discriminative vs

    Andrew Ng and Michael Jordan. On discriminative vs. gen- erative classifiers: A comparison of logistic regression and naive bayes. In Advances in Neural Information Processing Systems. MIT Press, 2001. 2

    2001

  41. [49]

    Improved denoising diffusion probabilistic models

    Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International conference on machine learning, pages 8162–8171. PMLR,

  42. [50]

    Diffusion models for adversarial purification

    Weili Nie, Brandon Guo, Yujia Huang, Chaowei Xiao, Arash Vahdat, and Animashree Anandkumar. Diffusion models for adversarial purification. In Proceedings of the 39th Interna- tional Conference on Machine Learning, pages 16805–16827. PMLR, 2022. 2

    2022

  43. [51]

    Aram-Alexandre Pooladian, Heli Ben-Hamu, Carles Domingo-Enrich, Brandon Amos, Yaron Lipman, and Ricky T. Q. Chen. Multisample flow matching: Straightening flows with minibatch couplings. In Proceedings of the 40th Interna- tional Conference on Machine Learning, pages 28100–281...

    2023

  44. [52]

    Unsupervised representation learning with deep convolutional generative adversarial networks

    Alec Radford. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434, 2015. 2

    2015 arXiv

  45. [53]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. InCVPR, pages 10684– 10695, 2022. 1, 2, 4

    2022

  46. [54]

    Bernstein, Alexander C

    Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, San- jeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael S. Bernstein, Alexander C. Berg, and Li Fei-Fei. Imagenet large scale visual recognition challenge. International Journal of Computer Vision...

  47. [55]

    Rethinking the spatial inconsistency in classifier- free diffusion guidance

    Dazhong Shen, Guanglu Song, Zeyue Xue, Fu-Yun Wang, and Yu Liu. Rethinking the spatial inconsistency in classifier- free diffusion guidance. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 9370–9379. IEEE, 2024. 1

    2024

  48. [56]

    D2c: Diffusion-decoding models for few-shot con- ditional generation

    Abhishek Sinha, Jiaming Song, Chenlin Meng, and Stefano Ermon. D2c: Diffusion-decoding models for few-shot con- ditional generation. In Advances in Neural Information Pro- cessing Systems, 2021. 2

    2021

  49. [57]

    Deep unsupervised learning using 10 nonequilibrium thermodynamics

    Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using 10 nonequilibrium thermodynamics. In International confer- ence on machine learning, pages 2256–2265. PMLR, 2015. 2

    2015

  50. [58]

    Score-based generative modeling through stochastic differential equations

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020. 2, 3, 4, 12

    2011 arXiv

  51. [59]

    Maximum likelihood training of score-based diffusion models

    Yang Song, Conor Durkan, Iain Murray, and Stefano Ermon. Maximum likelihood training of score-based diffusion models. Advances in neural information processing systems, 34:1415– 1428, 2021. 2, 4

    2021

  52. [60]

    Improving and generalizing flow-based gener- ative models with minibatch optimal transport

    Alexander Tong, Kilian FATRAS, Nikolay Malkin, Guillaume Huguet, Yanlei Zhang, Jarrid Rector-Brooks, Guy Wolf, and Yoshua Bengio. Improving and generalizing flow-based gener- ative models with minibatch optimal transport. Transactions on Machine Learning Research, 2024. Expert...

    2024

  53. [61]

    Neural discrete representation learning

    Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Advances in neural information pro- cessing systems, 30, 2017. 2

    2017

  54. [62]

    Vladimir N. Vapnik. The nature of statistical learning theory. Springer,, 1995. 2

    1995

  55. [63]

    A connection between score matching and denoising autoencoders

    Pascal Vincent. A connection between score matching and denoising autoencoders. Neural Computation, 23(7):1661– 1674, 2011. 2

    2011

  56. [64]

    Extracting and composing robust features with denoising autoencoders

    Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre- Antoine Manzagol. Extracting and composing robust features with denoising autoencoders. In Proceedings of the 25th international conference on Machine learning, pages 1096– 1103, 2008. 2

    2008

  57. [65]

    Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion

    Pascal Vincent, Hugo Larochelle, Isabelle Lajoie, Yoshua Bengio, Pierre-Antoine Manzagol, and L´eon Bottou. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. Journal of machine learning research, 11(12), 2010. 2, 3

    2010

  58. [66]

    Enresnet: Resnets ensemble via the feynman–kac formal- ism for adversarial defense and beyond

    Bao Wang, Binjie Yuan, Zuoqiang Shi, and Stanley J Osher. Enresnet: Resnets ensemble via the feynman–kac formal- ism for adversarial defense and beyond. SIAM Journal on Mathematics of Data Science, 2(3):559–582, 2020. 5, 7, 14

    2020

  59. [67]

    Phased consis- tency models

    Fu-Yun Wang, Zhaoyang Huang, Alexander Bergman, Dazhong Shen, Peng Gao, Michael Lingelbach, Keqiang Sun, Weikang Bian, Guanglu Song, Yu Liu, et al. Phased consis- tency models. Advances in neural information processing systems, 37:83951–84009, 2024. 1

    2024

  60. [68]

    Denoising diffusion autoencoders are unified self-supervised learners

    Weilai Xiang, Hongyu Yang, Di Huang, and Yunhong Wang. Denoising diffusion autoencoders are unified self-supervised learners. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 15802–15812, 2023. 2, 4, 5, 7, 8

    2023

  61. [69]

    Aggregated residual transformations for deep neural networks

    Saining Xie, Ross Girshick, Piotr Doll´ar, Zhuowen Tu, and Kaiming He. Aggregated residual transformations for deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1492–1500,

  62. [70]

    Your vit is secretly a hybrid discriminative- generative diffusion model

    Xiulong Yang, Sheng-Min Shih, Yinlin Fu, Xiaoting Zhao, and Shihao Ji. Your vit is secretly a hybrid discriminative- generative diffusion model. arXiv preprint arXiv:2208.07791,

    arXiv

  63. [71]

    Pde+: Enhancing gen- eralization via pde with adaptive distributional diffusion

    Yige Yuan, Bingbing Xu, Bo Lin, Liang Hou, Fei Sun, Huawei Shen, and Xueqi Cheng. Pde+: Enhancing gen- eralization via pde with adaptive distributional diffusion. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 16614–16622, 2024. 5, 7, 14

    2024

  64. [72]

    Cutmix: Regu- larization strategy to train strong classifiers with localizable features

    Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regu- larization strategy to train strong classifiers with localizable features. In Proceedings of the IEEE/CVF international con- ference on computer vision, pages 6023–6032, 2019. 5

    2019

  65. [73]

    Wide residual networks

    Sergey Zagoruyko. Wide residual networks. arXiv preprint arXiv:1605.07146, 2016. 5, 7

    2016 arXiv

  66. [74]

    mixup: Beyond empirical risk minimization

    Hongyi Zhang. mixup: Beyond empirical risk minimization. arXiv preprint arXiv:1710.09412, 2017. 5

    2017 arXiv

  67. [75]

    Revisiting generative poli- cies: A simpler reinforcement learning algorithmic perspec- tive, 2024

    Jinouwen Zhang, Rongkun Xue, Yazhe Niu, Yun Chen, Jing Yang, Hongsheng Li, and Yu Liu. Revisiting generative poli- cies: A simpler reinforcement learning algorithmic perspec- tive, 2024. 1

    2024

  68. [76]

    Im- proved techniques for maximum likelihood estimation for diffusion odes

    Kaiwen Zheng, Cheng Lu, Jianfei Chen, and Jun Zhu. Im- proved techniques for maximum likelihood estimation for diffusion odes. In International Conference on Machine Learning, pages 42363–42389. PMLR, 2023. 4

    2023

  69. [77]

    Score-based generative classifiers

    Roland S Zimmermann, Lukas Schott, Yang Song, Ben- jamin A Dunn, and David A Klindt. Score-based generative classifiers. arXiv preprint arXiv:2110.00473, 2021. 2, 5, 7 11 Pretrained Reversible Generation as Unsupervised Visual Representation Learning Appendix A. Why PRG is eff...

    2021 arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.