Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Improving Generative Pre-Training: An In-depth Study of Masked Image Modeling and Denoising Models

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Noise-based pre-training improves recognition when corruption happens inside the encoder, in feature space, and masked and noisy tokens are explicitly disentangled.

desk verdict A plausible three-part recipe for combining masking and noising in pre-training, with useful analysis but single-run evidence that makes the headline margins unreliable. read the letter →

arxiv 2412.19104 v1 pith:ZDB7DRHL submitted 2024-12-26 cs.CV cs.LG

Hyesong Choi , Daeun Kim , Sungmin Cha , Kwang Moo Yi , Dongbo Min This is my paper · ORCID
classification cs.CVcs.LG
keywords maskedimagemodelingdenoisingpre-traininggenerativefine-grainedrecognitionfeature-levelnoisetaskdisentanglementdisruptionlossself-supervisedlearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that additive noise, the ingredient behind denoising diffusion models, can improve self-supervised pre-training for recognition, but only when applied in the right place and in the right way. The authors identify three conditions: corruption and restoration must happen inside the encoder that is later transferred, the noise must be added in feature space rather than pixel space, and the network must explicitly stop masked tokens from interfering with noisy visible tokens. Implemented together, these conditions produce a pre-training recipe that outperforms masked image modeling and recent generative baselines on fine-grained classification, ImageNet, semantic segmentation, and detection tasks, with reported gains up to 8.1% over MIM methods and 8.0% over generative baselines. The value of the paper is a concrete architectural guideline for combining masking and denoising in one pre-training framework.

What carries the argument

The carrying mechanism is an encoder-style generative pre-training framework with three coupled components: corruption and restoration inside the encoder, feature-level noise injection, and explicit task disentanglement. The disentanglement is enforced by a disruption loss, a variant of masked token optimization, that acts on the affinity matrix A (the attention-weight map) partitioned into visible-visible, visible-masked, masked-visible, and masked-masked quadrants; it recalibrates the row-wise attention distributions of the noisy visible tokens so that masked tokens have minimal influence on them. The paper also locates the optimal noise injection at encoder block 2 of a ViT-B, arguing that lower layers carry the high-frequency details that fine-grained tasks need.

What would settle it

Run the same pre-training with the disruption loss removed but feature-level noise kept at block 2, and check whether fine-grained accuracy on CUB-200-2011 and Stanford Cars drops; if it does not, the disentanglement claim is false. Alternatively, show that adding noise at block 4 or block 6 with the disruption loss matches block 2, which would contradict the claim that lower-layer feature-space injection is critical.

Watch

Extended reading notes

Core claim

The central discovery is that previous attempts to combine denoising with masked image modeling, such as DiffMAE's diffused masking and MaskDiT's hybrid masking, fail to help recognition because they add noise in the wrong place and let the two reconstruction tasks interfere with each other. The paper shows that an encoder-style framework, where both noisy and masked tokens are processed and reconstructed inside the encoder, combined with feature-level noise injection at a lower encoder block and a disruption loss that suppresses attention from masked tokens to noisy visible tokens, captures a broader range of frequency information and transfers better across fine-grained and dense recognition tasks. In the paper's own evaluation, this yields up to an 8.1% gain over MIM methods and an 8.0% improvement over recent generative baselines on CUB-200-2011, NABirds, iNaturalist 2017/2018, Stanford Cars, Aircraft, ImageNet, ADE20K, and COCO.

Load-bearing premise

The whole recipe rests on the empirical assumption that silencing the attention from masked tokens to noisy visible tokens inside the encoder improves the learned features, an effect the paper supports with ablations that lack error bars and with the noise layer chosen using the same benchmarks on which the final gains are reported.

Editorial extensions

If this is right

  • Encoder-style corruption and restoration should replace decoder-style placement in generative pre-training aimed at recognition, because the encoder is the part that gets transferred.
  • Feature-level noise injection is a transferable design choice: adding noise at the input image (pixel space) is measurably worse than adding it at an intermediate encoder block.
  • When both denoising and de-masking share the encoder, they must be explicitly disentangled; without the disruption loss, the gains largely disappear.
  • The recipe improves not only fine-grained classification but also ImageNet classification, semantic segmentation on ADE20K, and object and instance segmentation on COCO.
  • The failure of existing generative pre-training baselines like DiffMAE and MaskDiT on recognition tasks is explained by their violation of these three conditions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the frequency-diversity explanation is right, the same three conditions may transfer to other backbone architectures and to video or multi-modal pre-training, though the optimal noise-injection block might shift with the architecture.
  • The disruption loss can be viewed as a regularizer that prevents the masked-token branch from dominating the shared encoder; a testable variant would replace it with a softer penalty on cross-quadrant attention weights.
  • The block-2 choice is empirical; one could test whether the optimal block tracks the layer where high-frequency information is most accessible, which would turn the recipe into a predictable rule.
  • A practical extension is to apply the same conditions to latent diffusion pre-training at larger scale, potentially reducing the data hunger of foundational vision models.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper investigates why additive-noise-based pre-training has produced only marginal gains on recognition tasks and proposes three design conditions under which noise can help: (1) corruption and restoration should occur inside the encoder, (2) noise should be injected in feature space at a lower encoder block, and (3) masked and noisy tokens should be explicitly disentangled via a disruption loss. Based on these findings, the authors design an encoder-style hybrid masking framework with feature-level noise at block 2 and a disruption loss that minimizes row-wise entropy of the affinity matrix for noisy visible tokens. They report consistent improvements over MIM baselines (MAE, SimMIM) and generative baselines (DiffMAE, MaskDiT) on fine-grained datasets, ImageNet classification, ADE20K segmentation, and COCO detection/segmentation, with claimed gains up to 8.1% over MIM and 8.0% over generative baselines. The evidence is presented almost entirely through bar charts and radar plots in Figures 1-12, with no numerical tables in the main text.

Significance. If the three-condition recipe is validated, the paper would provide a practical and actionable guideline for combining masking and denoising in self-supervised pre-training, addressing a known weakness of MIM on fine-grained, high-frequency tasks. The authors should be credited for running all comparisons under a unified 400-epoch ViT-B setting, re-implementing baselines for fairness, and promising code in the supplementary material. The breadth of downstream tasks is a clear strength. However, the central claim is entirely empirical and currently rests on single-run, post hoc selected ablations without error bars, and the key mechanism (disruption loss) is not directly verified. These are fixable with additional experiments and tables, but until then the quantitative headline claims are not fully supported.

major comments (4)
  1. [Section 3.4, Figures 9 and 10] The design choices of noise-injection block (block 2) and the disruption loss are selected on the same transfer benchmarks used for the final claims, and all ablation results are reported as single runs without error bars or multiple seeds. Under typical ViT-B fine-tuning variance of 0.3-0.5%, the marginal gains attributed to feature-level noise and especially to the disruption loss in Figure 10 may lie within run-to-run noise. Please report means and standard deviations over at least three seeds for the ablations and the final comparisons, and ideally hold out a subset of downstream tasks for model selection.
  2. [Equation (9), Section 3.4] The disruption loss minimizes the row-wise entropy of the affinity matrix rows corresponding to noisy visible tokens. Entropy minimization can sharpen attention onto any token type; it does not by construction guarantee that masked tokens are specifically suppressed, contrary to the text's claim that it 'minimizes the impact of masked tokens xm on noisy visible tokens'. The paper never reports the actual attention mass assigned to masked tokens. Please measure the average attention weight from noisy visible tokens to masked tokens with and without Ld, or modify the loss to explicitly penalize the Avm/Amv entries, so that the disentanglement mechanism is directly verified.
  3. [Section 4.2, Figure 11; Section 1] All main quantitative results, including the headline 'up to an 8.1% gain over MIM methods and an 8.0% improvement over recent generative baselines', are presented only in figures without exact numerical values, standard deviations, or a comparison table. This makes it impossible for readers to verify individual dataset gains or assess statistical significance. Please include a full numeric table of fine-tuned accuracies (and detection/segmentation metrics) for every method and dataset, with the fine-tuning protocol and number of runs clearly stated.
  4. [Section 3.3, Figure 8; Section 3.4] The paper concludes that 'the encoder-style approach can indeed outperform the decoder-style in generative pre-training frameworks', but the only direct encoder-style vs decoder-style comparison (Figure 8) shows minimal difference. The later improvements come from adding feature-level noise and the disruption loss to the encoder-style variant only; a decoder-style variant with the same feature-level noise and disruption loss is not tested. Without that controlled comparison, the first condition (corruption and restoration must be applied within the encoder) is not independently supported.
minor comments (5)
  1. [Equation (1)] The notation is inconsistent: the input to the MIM framework is called Xmasked = M ⊙ X, but the reconstruction target in Eq. (1) is defined as ¯X = Xvis. Please clarify whether masked positions are zeroed out or removed, and define all symbols consistently.
  2. [Equation (9)] The symbol ˜p is described as 'an element of A' but then indexed as ˜pi,j and required to satisfy P j ˜pi,j = 1, suggesting it is a row-normalized submatrix. Please define whether the loss is applied per head, per layer, and how the row normalization is computed.
  3. [Figure 2] The 'KL divergence among attention distributions across different heads' is not defined. Please specify the exact formula, the layers and heads included, and the data over which the distributions are averaged.
  4. [Section 3.2] The discussion of diffused masking states that 'when the diffusion noise is weak, the pre-training task is trivial', but the noise schedule and timestep sampling distribution used in the comparisons are not given. Please state the schedule and sampling procedure for all variants.
  5. [Section 3.4] The disruption loss is described as 'a variant of masked token optimization proposed in MTO [7]' without specifying the exact modification. Please state how Ld differs from the loss in MTO and why the change is needed for the noisy visible tokens.

Circularity Check

0 steps flagged · score 2.0 of 10

No load-bearing circularity: the three design conditions are empirical ablations evaluated on external transfer benchmarks; the only self-reference is the MTO-derived disruption loss, which is independently ablated rather than assumed.

full rationale

The paper's central claim is an empirical recipe — encoder-side corruption, feature-level noise at a lower block, and explicit task disentanglement — each supported by controlled comparisons (Figures 6, 8, 9, 10, and 12) on external downstream datasets. No equation in the paper derives a reported accuracy number from a fitted parameter, and the final gains are measured on transfer tasks rather than being algebraic consequences of the loss definitions. The disruption loss in Eq. 9 is adapted from the authors' own MTO paper [7], which is a self-citation, but the paper does not rest the validity of the loss on that citation: Figure 10 directly ablates the loss against a no-disruption baseline and shows the transfer improvement. Thus the central claim does not reduce to a self-citation chain. The choice of block 2 from the same benchmark suite used for the final claims is a methodological limitation (selection on evaluation data, no error bars), but it is not a by-construction circularity: the reported superiority of the final configuration is not mathematically forced by the block selection, and no fitted value is renamed as a prediction. Concerns about single-run ablations and the mechanism of Eq. 9 are correctness and robustness risks, not circularity. Overall, the derivation is self-contained, with only one minor non-load-bearing self-citation.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central recipe depends on three empirical design choices: the noise injection location (block 2), the unspecified masking and noise schedule hyperparameters, and the attention-suppression assumption behind the disruption loss. No new physical entities or latent variables are introduced. The paper's contribution is an empirical method, not a derivation from first principles.

free parameters (3)
  • feature noise injection block = block 2 of ViT-B encoder
    Chosen after sweeping blocks 0, 2, 4, and 6 on the transfer benchmarks (Section 3.4, Figure 9). The best block is then used in the final method, making it a post hoc model-selection choice.
  • masking ratio gamma = not specified in main text
    The masking ratio is a core pre-training hyperparameter, but the main text does not report its value; the reconstruction difficulty and the balance between masking and noising depend on it.
  • noise schedule and timestep sampling = not specified in main text
    The noising process needs a schedule and a distribution over diffusion timesteps, which directly controls task difficulty and is not described in the main text.
assumptions (3)
  • domain assumption Feature-space noise, and denoising-based training in general, transfers high-frequency detail useful for recognition.
    The whole method rests on the premise that feature-level corruption teaches high-frequency representations; this is motivated by diffusion literature (Section 3.4, citing [6, 31]) rather than proven in this paper.
  • domain assumption The encoder is the component that matters for downstream transfer, so applying corruption and reconstruction inside the encoder is beneficial.
    The authors argue that since the encoder is transferred, it should be trained on reconstruction tasks (Section 3.3). Their own naive encoder-style baseline showed only minimal gains, exposing this assumption.
  • ad hoc to paper Suppressing attention from masked tokens to noisy visible tokens via per-row sparsity in the affinity matrix does not destroy useful cross-token information.
    The disruption loss is introduced to enforce this separation (Section 3.4), but its sufficiency is supported only by a single ablation without error bars, and it is adapted from the authors' prior MTO work [7].

how reviews work

0 comments
Cite this review

Pith. "Pith review of Improving Generative Pre-Training: An In-depth Study of Masked Image Modeling and Denoising Models." pith.science (2026). https://pith.science/paper/ZDB7DRHL

@misc{pith2026241219104,
  author       = {Pith},
  title        = {Pith review of: Improving Generative Pre-Training: An In-depth Study of Masked Image Modeling and Denoising Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZDB7DRHL}},
  note         = {Machine review of arXiv:2412.19104}
}
read the original abstract

In this work, we dive deep into the impact of additive noise in pre-training deep networks. While various methods have attempted to use additive noise inspired by the success of latent denoising diffusion models, when used in combination with masked image modeling, their gains have been marginal when it comes to recognition tasks. We thus investigate why this would be the case, in an attempt to find effective ways to combine the two ideas. Specifically, we find three critical conditions: corruption and restoration must be applied within the encoder, noise must be introduced in the feature space, and an explicit disentanglement between noised and masked tokens is necessary. By implementing these findings, we demonstrate improved pre-training performance for a wide range of recognition tasks, including those that require fine-grained, high-frequency information to solve.

Figures

Figures reproduced from arXiv: 2412.19104 by the authors.

Figure 1
Figure 1. We find that noise-based pre-training, when applied in [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. We display the KL divergence among attention distributions across different heads (indicated by small dots) and the mean KL [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. We visualized the self-attention maps for the image classification token in the final layer of our model on a fine-grained visual [PITH_FULL_IMAGE:figures/full_fig_p002_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Fine-grained visual categorization (FGVC) is a critical [PITH_FULL_IMAGE:figures/full_fig_p003_4.png]
Figure 5
Figure 5. Figure 5: Our evaluations reveal that recent generative pre-training approaches [ [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Hybrid masking achieved slightly better performance [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]
Figure 8
Figure 8. Figure 8: We advocate for an encoder-style approach, as it is the [PITH_FULL_IMAGE:figures/full_fig_p006_8.png]
Figure 10
Figure 10. Figure 10: We propose an explicit objective to disentangle the [PITH_FULL_IMAGE:figures/full_fig_p007_10.png]
Figure 11
Figure 11. Figure 11: The proposed method (Ours) consistently outperforms representative MIM [ [PITH_FULL_IMAGE:figures/full_fig_p008_11.png]
Figure 12
Figure 12. Figure 12: We present the ablation study results on the components of the proposed method. Feature-level noise addition and task [PITH_FULL_IMAGE:figures/full_fig_p008_12.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. TADFormer : Task-Adaptive Dynamic Transformer for Efficient Multi-Task Learning

    cs.CV 2025-01 conditional novelty 6.0 of 10

    A prompt-and-dynamic-filter PEFT design for multi-task dense prediction beats MTLoRA on PASCAL-Context with fewer trainable parameters.

Reference graph

Works this paper leans on

45 extracted references · 17 canonical work pages · cited by 1 Pith paper

  1. [1]

    Beit: Bert pre-training of image transformers

    Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. Beit: Bert pre-training of image transformers. In International Conference on Learning Representations, 2021. 1, 3, 5, 6

    work page 2021

  2. [2]

    End-to- end object detection with transformers

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to- end object detection with transformers. In European confer- ence on computer vision, pages 213–229. Springer, 2020. 1

    2020

  3. [3]

    Pre-trained image processing transformer

    Hanting Chen, Yunhe Wang, Tianyu Guo, Chang Xu, Yiping Deng, Zhenhua Liu, Siwei Ma, Chunjing Xu, Chao Xu, and Wen Gao. Pre-trained image processing transformer. InPro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12299–12310, 2021. 1

    work page 2021

  4. [4]

    A simple framework for contrastive learning of visual representations

    Ting Chen, Simon Kornblith, Mohammad Norouzi, and Ge- offrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on ma- chine learning, pages 1597–1607. PMLR, 2020. 1

    work page 2020

  5. [5]

    Context autoencoder for self- supervised representation learning

    Xiaokang Chen, Mingyu Ding, Xiaodi Wang, Ying Xin, Shentong Mo, Yunhao Wang, Shumin Han, Ping Luo, Gang Zeng, and Jingdong Wang. Context autoencoder for self- supervised representation learning. International Journal of Computer Vision, 132(1):208–223, 2024. 3, 5, 6, 7

    work page 2024

  6. [6]

    De- constructing denoising diffusion models for self-supervised learning

    Xinlei Chen, Zhuang Liu, Saining Xie, and Kaiming He. De- constructing denoising diffusion models for self-supervised learning. arXiv preprint arXiv:2401.14404, 2024. 7

    arXiv 2024

  7. [7]

    Emerging property of masked token for effective pre-training

    Hyesong Choi, Hunsang Lee, Seyoung Joung, Hyejin Park, Jiyeong Kim, and Dongbo Min. Emerging property of masked token for effective pre-training. arXiv preprint arXiv:2404.08330, 2024. 3, 7

    arXiv 2024

  8. [8]

    Salience-based adaptive masking: revisit- ing token dynamics for enhanced pre-training

    Hyesong Choi, Hyejin Park, Kwang Moo Yi, Sungmin Cha, and Dongbo Min. Salience-based adaptive masking: revisit- ing token dynamics for enhanced pre-training. In European Conference on Computer Vision , pages 343–359. Springer,

Show all 45 references
  1. [9]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009. 1, 2, 4, 5, 6, 7, 8

    2009

  2. [10]

    Bootstrapped masked autoencoders for vision bert pretraining

    Xiaoyi Dong, Jianmin Bao, Ting Zhang, Dongdong Chen, Weiming Zhang, Lu Yuan, Dong Chen, Fang Wen, and Nenghai Yu. Bootstrapped masked autoencoders for vision bert pretraining. In European Conference on Computer Vi- sion, pages 247–264. Springer, 2022. 3, 5, 6, 7

    2022

  3. [11]

    An image is worth 16x16 words: Trans- formers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, et al. An image is worth 16x16 words: Trans- formers for image recognition at scale. arXiv preprint a...

    2010 arXiv

  4. [12]

    Bootstrap your own latent-a new approach to self-supervised learning

    Jean-Bastien Grill, Florian Strub, Florent Altch ´e, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Ghesh- laghi Azar, et al. Bootstrap your own latent-a new approach to self-supervised learning. Advances in neur...

    2020

  5. [13]

    Momentum contrast for unsupervised visual rep- resentation learning

    Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual rep- resentation learning. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 9729–9738, 2020

    2020

  6. [14]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll´ar, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16000– 16009, 2022. 1, 3, 4, 5, 6, 7, 8

    2022

  7. [15]

    Unsupervised keypoints from pretrained diffusion models

    Eric Hedlin, Gopal Sharma, Shweta Mahajan, Xingzhe He, Hossam Isack, Abhishek Kar, Helge Rhodin, Andrea Tagliasacchi, and Kwang Moo Yi. Unsupervised keypoints from pretrained diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition...

    2024

  8. [16]

    Unsupervised semantic correspondence using stable diffu- sion

    Eric Hedlin, Gopal Sharma, Shweta Mahajan, Hossam Isack, Abhishek Kar, Andrea Tagliasacchi, and Kwang Moo Yi. Unsupervised semantic correspondence using stable diffu- sion. Advances in Neural Information Processing Systems , 36, 2024. 3

    2024

  9. [17]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 3

    2020

  10. [18]

    Simpler diffusion (sid2): 1.5 fid on imagenet512 with pixel-space diffusion

    Emiel Hoogeboom, Thomas Mensink, Jonathan Heek, Kay Lamerigts, Ruiqi Gao, and Tim Salimans. Simpler diffusion (sid2): 1.5 fid on imagenet512 with pixel-space diffusion. arXiv preprint arXiv:2410.19324, 2024. 7

    2024 arXiv

  11. [19]

    Elucidating the design space of diffusion-based generative models

    Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. Advances in neural information processing systems, 35:26565–26577, 2022. 3

    2022

  12. [20]

    3d object representations for fine-grained categorization

    Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine-grained categorization. In Proceedings of the IEEE international conference on com- puter vision workshops , pages 554–561, 2013. 1, 2, 4, 5, 8

    2013

  13. [21]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceeding...

    2014

  14. [22]

    Flow straight and fast: Learning to generate and transfer data with rectified flow

    Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint arXiv:2209.03003, 2022. 3

    2022 arXiv

  15. [23]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012–10022, 2021. 1

    2021

  16. [24]

    Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps

    Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in Neural Information Processing Systems , 35:5775–5787,

  17. [25]

    Diffusion hyperfeatures: Searching through time and space for semantic correspondence

    Grace Luo, Lisa Dunlap, Dong Huk Park, Aleksander Holyn- ski, and Trevor Darrell. Diffusion hyperfeatures: Searching through time and space for semantic correspondence. Ad- vances in Neural Information Processing Systems, 36, 2024. 3

    2024

  18. [26]

    Fine-grained visual classi- fication of aircraft

    Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi. Fine-grained visual classi- fication of aircraft. arXiv preprint arXiv:1306.5151 , 2013. 1, 2, 4, 5, 8

    2013 arXiv

  19. [27]

    Improved denoising diffusion probabilistic models

    Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International conference on machine learning, pages 8162–8171. PMLR,

  20. [28]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  21. [29]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International confer- ence on machine learning, pages 8821–8831. Pmlr, 2021. 1, 3

    2021

  22. [30]

    Vi- sion transformers for dense prediction

    Ren ´e Ranftl, Alexey Bochkovskiy, and Vladlen Koltun. Vi- sion transformers for dense prediction. In Proceedings of the IEEE/CVF international conference on computer vision, pages 12179–12188, 2021. 1

    2021

  23. [31]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 1, 3, 7

    2022

  24. [32]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information...

    2022

  25. [33]

    Laion-400m: Open dataset of clip-filtered 400 million image-text pairs

    Christoph Schuhmann, Richard Vencu, Romain Beaumont, Robert Kaczmarczyk, Clayton Mullis, Aarush Katta, Theo Coombes, Jenia Jitsev, and Aran Komatsuzaki. Laion-400m: Open dataset of clip-filtered 400 million image-text pairs. arXiv preprint arXiv:2111.02114, 2021. 1

    2021 arXiv

  26. [34]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020. 3

    2010 arXiv

  27. [35]

    Building a bird recognition app and large scale dataset with citizen scientists: The fine print in fine-grained dataset collection

    Grant Van Horn, Steve Branson, Ryan Farrell, Scott Haber, Jessie Barry, Panos Ipeirotis, Pietro Perona, and Serge Be- longie. Building a bird recognition app and large scale dataset with citizen scientists: The fine print in fine-grained dataset collection. In Proceedings of t...

  28. [36]

    The inaturalist challenge 2017 dataset.arXiv preprint arXiv:1707.06642, 1(2):4, 2017

    Grant Van Horn, Oisin Mac Aodha, Yang Song, Alexan- der Shepard, Hartwig Adam, Pietro Perona, and Serge Be- longie. The inaturalist challenge 2017 dataset.arXiv preprint arXiv:1707.06642, 1(2):4, 2017. 2, 4, 8

    2017 arXiv

  29. [37]

    The inaturalist species classification and de- tection dataset

    Grant Van Horn, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alex Shepard, Hartwig Adam, Pietro Perona, and Serge Belongie. The inaturalist species classification and de- tection dataset. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages...

  30. [38]

    The caltech-ucsd birds-200-2011 dataset

    Catherine Wah, Steve Branson, Peter Welinder, Pietro Per- ona, and Serge Belongie. The caltech-ucsd birds-200-2011 dataset. 2011. 1, 2, 4, 5, 8

    2011

  31. [39]

    Diffusion models as masked autoencoders

    Chen Wei, Karttikeya Mangalam, Po-Yao Huang, Yanghao Li, Haoqi Fan, Hu Xu, Huiyu Wang, Cihang Xie, Alan Yuille, and Christoph Feichtenhofer. Diffusion models as masked autoencoders. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision , pages 16284– 162...

    2023

  32. [40]

    Simmim: A simple framework for masked image modeling

    Zhenda Xie, Zheng Zhang, Yue Cao, Yutong Lin, Jianmin Bao, Zhuliang Yao, Qi Dai, and Han Hu. Simmim: A simple framework for masked image modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9653–9663, 2022. 1, 2, 3, 4, 5, 6, 8

    2022

  33. [41]

    Masked image modeling with denoising contrast

    Kun Yi, Yixiao Ge, Xiaotong Li, Shusheng Yang, Dian Li, Jianping Wu, Ying Shan, and Xiaohu Qie. Masked image modeling with denoising contrast. arXiv preprint arXiv:2205.09616, 2022. 3, 5, 6

    2022 arXiv

  34. [42]

    Fast training of diffusion models with masked transformers

    Hongkai Zheng, Weili Nie, Arash Vahdat, and Anima Anandkumar. Fast training of diffusion models with masked transformers. arXiv preprint arXiv:2306.09305, 2023. 1, 2, 4, 5, 6, 8

    2023 arXiv

  35. [43]

    Rethinking semantic segmen- tation from a sequence-to-sequence perspective with trans- formers

    Sixiao Zheng, Jiachen Lu, Hengshuang Zhao, Xiatian Zhu, Zekun Luo, Yabiao Wang, Yanwei Fu, Jianfeng Feng, Tao Xiang, Philip HS Torr, et al. Rethinking semantic segmen- tation from a sequence-to-sequence perspective with trans- formers. In Proceedings of the IEEE/CVF conference...

  36. [44]

    Scene parsing through ade20k dataset

    Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, and Antonio Torralba. Scene parsing through ade20k dataset. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 633–641,

  37. [45]

    Deformable detr: Deformable trans- formers for end-to-end object detection

    Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable detr: Deformable trans- formers for end-to-end object detection. arXiv preprint arXiv:2010.04159, 2020. 1

    2010 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.