REVIEW 3 major objections 6 minor 2 cited by
REGLUE claims that feeding a diffusion model a compact, nonlinearly compressed copy of a vision model's patch-level semantics—not a linear projection or a global token alone—is what unlocks faster convergence and better FID.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-03 15:28 UTC pith:WOUZTXV5
load-bearing objection A solid empirical ablation paper on injecting compressed DINOv2 semantics into latent diffusion; the mechanism claims are plausible but not fully disentangled from the smoother-statistics alternative. the 3 major comments →
REGLUE Your Latents with Global and Local Semantics for Entangled Diffusion
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
REGLUE rests on one central claim: patch-level VFM semantics, not global tokens or external alignment, are the primary engine of faster, better latent diffusion, provided they enter through a nonlinear compressor that preserves spatial structure. The ablation ladder: global [CLS] alone gives 25.7 FID; linear PCA patch features give 21.4; the same features through REGLUE's compressor give 14.3; adding multi-layer aggregation, the global token, and REPA-style alignment reaches 12.9 FID on SiT-B/2 at 400K. On SiT-XL/2, 200K steps reach 4.6 FID (REG: 5.0), 700K match REG's 1M-step 2.7, and 1M reach 2.5. Reading: spatial semantics are crucial, nonlinear compression unlocks them, global tokens and
What carries the argument
The load-bearing object is the frozen semantic compressor: a shallow convolutional autoencoder (input Conv 3072 to 256, one residual block, output Conv 256 to 16, symmetric decoder) trained offline with pure MSE to reconstruct channel-concatenated features of the last four DINOv2-B blocks. It produces a 16-channel, 16x16 spatially structured map that is resampled to the 4x32x32 VAE latent grid, noised under the same interpolant schedule, and added token-wise to the VAE latents inside the SiT backbone, so the transformer models latents and semantics as one sequence without extra sequence length. A global [CLS] token is concatenated as a separate token, and an auxiliary cosine alignment loss t
Load-bearing premise
The load-bearing premise is that a 16-channel representation trained only to reconstruct frozen DINOv2 features, with no coupling to the generative loss, stays semantically informative after being noised and added onto the image latents; the paper supports this indirectly with probing and segmentation accuracy, but the same pipeline gives 18.1 FID with CLIP-L features and 16.9 FID when early VFM layers are used.
What would settle it
Train SiT-B/2 with the same additive 16-channel semantic stream, but replace the frozen DINOv2-reconstruction compressor's output with channel-matched noise, or with features whose semantic content is destroyed by shuffling patch positions across images. If FID stays near 12.9 instead of rising toward the 21.4 of the linear-PCA baseline, semantic preservation is not what drives the gain.
If this is right
- Semantic guidance can be bought at preprocessing time: the compressor trains in under one hour on 8 A100s and is then frozen, so the gain costs almost nothing at diffusion training or inference time.
- Training budgets for class-conditional ImageNet diffusion shrink materially: SiT-XL/2+REGLUE at 200K steps (4.6 FID) beats a 7M-step SiT-XL/2 baseline (8.3 FID) and matches or beats REG at equal or far fewer steps.
- Nonlinear compression rather than dimensionality per se is the mechanism: at the same 8 channels, the learned compressor yields 14.3 FID and 67.1 mIoU on Cityscapes against 21.4 FID and 59.1 mIoU for PCA features.
- The method transfers to data-limited regimes: at 20% of ImageNet, REGLUE outperforms REG by 5.5 FID at the same 80-epoch budget, so semantic entanglement helps most when training data is scarce.
Where Pith is reading between the lines
- Because the compressor is trained purely for reconstruction and then frozen, the framework implies a decoupling: any frozen dense visual encoder could be wrapped the same way. A direct test would be sweeping VFMs and layer sets to map which feature statistics transfer; the paper's CLIP-L result (18.1 FID vs 12.9 for DINOv2-B) suggests contrastive language-image features carry a different trade-off
- The success of adding semantics onto latents token-wise suggests an interpretation of 'entanglement' as a learned prior that constrains the latent trajectory; if so, the same additive stream could be used as a controllable conditioning channel at sampling time, e.g., to steer generation toward a target patch-level semantic map.
- The 16-channel sweet spot (performance degrades at 20 channels) points to a capacity-balance law between reconstruction latents and semantic latents; a principled rule for choosing that dimension per VFM and per VAE would generalize REGLUE beyond ImageNet.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes REGLUE, a latent diffusion framework that jointly models VAE image latents, compact patch-level (local) VFM semantics, and a global [CLS] token inside a single SiT backbone. A lightweight convolutional autoencoder is trained offline, with an MSE objective, to compress multi-layer DINOv2 features into a 16-channel spatially structured representation; this representation is then noised with the same interpolant schedule as the VAE latents, added token-wise to the latent embedding sequence, and jointly denoised with a multimodal velocity objective. An optional REPA-style external alignment loss is added at an intermediate SiT block. On ImageNet 256x256, the method reports consistent FID improvements over SiT-B/2 and SiT-XL/2 baselines and over REPA, ReDi, and REG at matched training budgets, with the sharpest claim being that SiT-XL/2+REGLUE reaches the 1M-step performance of ReDi and REG using less than 30% and 80% of their iterations, respectively. The paper also includes ablations on compression channels, compressor capacity, VFM layer selection, loss variants, and VFM choice.
Significance. If the reported gains are robust, the paper makes a practically important contribution: a sub-hour-trained, frozen semantic compressor that is nearly parameter-free at diffusion inference time yields state-of-the-art-class FID on ImageNet 256x256 at a substantially reduced training budget. The experimental matrix in Table 1 is systematic and internally consistent, the arithmetic of the reported percentages checks out, and the code release supports reproducibility. The attentive-probing and Cityscapes mIoU analyses (Figs. 4 and 6) are a genuine effort to connect compression quality to generation quality, and the comparison between nonlinear compression and linear PCA is a clean, large-contrast experiment. The main weakness is that the causal link between 'semantic preservation' and 'generation usefulness' is asserted rather than tested: the paper does not rule out that the nonlinear compressor acts primarily as a learned smoother of feature statistics. This does not undermine the empirical recipe, but it does mean the mechanism claims (a) and (b) are stronger than the evidence supports.
major comments (3)
- [Sec. 3.3, Eq. (11); Sec. 4.2, Table 1; Sec. 4.4, Table 7] The central attribution of the 21.4 -> 14.3 FID gain to 'nonlinear compression unlocking semantics' assumes that an MSE-reconstructed VFM feature map, once frozen and noised, is useful to the diffusion model because it preserves semantics. The supporting evidence — attentive probing (Fig. 4), Cityscapes mIoU (Fig. 6), and the MSE+KL degradation (Table 7) — is indirect. It does not exclude the alternative that the compressor is a well-tuned nonlinear smoother of feature statistics and that the diffusion gain would persist (or even increase) with spatial/semantic structure destroyed. I ask for a control experiment: train the identical compressor architecture to reconstruct the same features with their spatial layout destroyed (e.g., channel-wise shuffled or randomly re-permuted patches), or use a fixed random nonlinear projection of the same channels, and run the same SiT-B/2 400K training
- [Sec. 3.2, Eqs. (3)-(5)] The shared noise schedule and the channel-wise additive fusion (z_tilde_t + s_tilde_t) are design choices that are load-bearing for the method but are not validated against alternatives. The paper states the two 'straightforward options' for combining latents and semantics but reports only the channel-wise merge, not the sequence-concatenation option (i). Since s* is a learned, unnormalized 16-channel representation, it is not a priori clear that the same (alpha_t, sigma_t) used for the SD-VAE latents is appropriate, nor that adding the projected semantics to the projected latents does not let the 16-channel semantics dominate the 4-channel VAE latents. Please add an ablation of the concatenation option (at least for SiT-B/2, 400K, matching Table 1 setting (l) or (m)) and/or per-modality schedule scaling, so the reader can see whether the reported gains are tied to the fusion mechanism r
- [Sec. 4.4, Table 6; Appendix A.3, Table 8] The VFM-dependence results are presented honestly, but the paper's wording sometimes overgeneralizes. Specifically, CLIP-L gives 18.1 FID versus 12.9 for DINOv2-B and 12.3 for DINOv3-B, and early-layer DINOv2 features (3,6,9,12) degrade to 16.9 FID versus 13.3 for last-four layers (9-12). These patterns are consistent with the 'semantics' story but are also consistent with feature statistics or scale differences. At minimum, the conclusion should explicitly bound the claim to self-supervised DINO-style features and note that the method does not improve over all VFMs. If the authors have a hypothesis for why CLIP-L underperforms, it would strengthen the paper to state and test it.
minor comments (6)
- [Table 1 caption] The checkmarks denote 'novel components proposed in our work' but do not distinguish, for example, the linear PCA baseline of ReDi from the nonlinear compressor of REGLUE. A clearer labeling (e.g., 'PCA' vs 'Ours') would make the table easier to parse.
- [Eq. (7)] 'where \tilde f_*^{(L)} denotes flattened spatial dimension to tokens' is grammatically unclear. Please rewrite, e.g., 'where \tilde f_*^{(L)} is the flattened patch feature sequence.'
- [Appendix A.2, Table 7] The MSE+KL and MSE+GAN variants are compared without mentioning whether they use the same random seed or multiple seeds. A single sentence on seed averaging would help assess the 17.2 vs 14.3 gap.
- [References] References [15] and [16] both cite the ImageNet paper (Deng et al.). Please merge or disambiguate.
- [Sec. 3.3] The notation 'D_s << \sum_l D_l' would be clearer if the sum were defined as the total input channel count, e.g., D_in = \sum_l D_l = 3072 in the default setting, before writing D_s = 16.
- [Figs. 4 and 6] The PCA point is labeled 'PCA 8 channels'; it would be helpful to state explicitly that this matches ReDi's default compression, and to mark the number of channels used in the Ours variants in the figure or caption.
Circularity Check
No significant circularity: central claims are benchmarked externally and the compressor is trained offline on frozen VFM features.
full rationale
REGLUE's quantitative claims are evaluated against external benchmarks (ImageNet FID/sFID/IS, attentive probing, Cityscapes mIoU) and against baselines that are reproduced in the paper's own tables. The semantic compressor is pretrained offline with a pure MSE reconstruction objective on frozen DINOv2 features (Eq. 11) and then frozen; at sampling, the VAE latents, compressed semantics, and global token are generated jointly from noise, so no fitted input is re-presented as a prediction. The ablations in Table 1 and the convergence comparisons in Tables 2–4 are empirical contrasts, not definitional identities. Self-citations (ReDi [33], attentive probing [52], DINO-foresight [28]) appear as baselines, related work, or measurement tools, but none is load-bearing for the central derivation: the FID improvements do not reduce to those citations. The paper's own limitations and ablations (e.g., MSE+KL degrading FID in Table 7, CLIP-L underperforming in Table 8) are validity/generalizability concerns, not evidence that an output is equivalent to an input by construction. The claim that reconstruction quality implies generation usefulness is an untested empirical assumption, which is a correctness risk, not circularity.
Axiom & Free-Parameter Ledger
free parameters (6)
- compression channels =
16
- compressor hidden size =
256
- VFM layers aggregated =
blocks 9-12 of DINOv2-B
- loss weights lambda_s, lambda_cls, lambda_rep =
1, 0.03, 0.5
- alignment block index k =
4 (SiT-B/2), 8 (SiT-XL/2)
- CFG scale and interval =
w=2.8, [0,0.9]
axioms (5)
- domain assumption DINOv2-B patch features from the last four layers carry semantic content useful for generating ImageNet images
- domain assumption Frozen SD-VAE-FT-EMA latents are a sufficient decode space for 256x256 ImageNet
- standard math Velocity-matching objective for stochastic interpolants trains a generative model
- ad hoc to paper A single shared noise schedule (alpha_t, sigma_t) is appropriate for three modalities of different data scale
- ad hoc to paper Channel-wise additive fusion z_t + s_t preserves both VAE and semantic information
invented entities (2)
-
16-channel compressed semantic space (output of E_psi)
independent evidence
-
Entangled joint diffusion state (z_t, s_t, cls_t)
no independent evidence
read the original abstract
Latent diffusion models (LDMs) achieve state-of-the-art image synthesis, yet their reconstruction-style denoising objective provides only indirect semantic supervision: high-level semantics emerge slowly, requiring longer training and limiting sample quality. Recent works inject semantics from Vision Foundation Models (VFMs) either externally via representation alignment or internally by jointly modeling only a narrow slice of VFM features inside the diffusion process, under-utilizing the rich, nonlinear, multi-layer spatial semantics available. We introduce REGLUE (Representation Entanglement with Global-Local Unified Encoding), a unified latent diffusion framework that jointly models (i) VAE image latents, (ii) compact local (patch-level) VFM semantics, and (iii) a global (image-level) [CLS] token within a single SiT backbone. A lightweight convolutional semantic compressor nonlinearly aggregates multi-layer VFM features into a low-dimensional, spatially structured representation, which is entangled with the VAE latents in the diffusion process. An external alignment loss further regularizes internal representations toward frozen VFM targets. On ImageNet 256x256, REGLUE consistently improves FID and accelerates convergence over SiT-B/2 and SiT-XL/2 baselines, as well as over REPA, ReDi, and REG. Extensive experiments show that (a) spatial VFM semantics are crucial, (b) non-linear compression is key to unlocking their full benefit, and (c) global tokens and external alignment act as complementary, lightweight enhancements within our global-local-latent joint modeling framework. The code is available at https://github.com/giorgospets/reglue .
Figures
Forward citations
Cited by 2 Pith papers
-
MaskAlign: Token-Subset Representation Alignment for Efficient Diffusion Training
MaskAlign uses random token-subset alignment and pre-mask mixing to reduce diffusion models' reliance on complete clean-image token sets during representation alignment.
-
Coevolving Representations in Joint Image-Feature Diffusion
CoReDi coevolves semantic representations with the diffusion model via a jointly learned linear projection stabilized by stop-gradient, normalization, and regularization, yielding faster convergence and higher sample ...
Reference graph
Works this paper leans on
-
[1]
Build- ing normalizing flows with stochastic interpolants
Michael Samuel Albergo and Eric Vanden-Eijnden. Build- ing normalizing flows with stochastic interpolants. InICLR, 2023
2023
-
[2]
Self-supervised learning from images with a joint-embedding predictive architecture
Mahmoud Assran, Quentin Duval, Ishan Misra, Piotr Bo- janowski, Pascal Vincent, Michael Rabbat, Yann LeCun, and Nicolas Ballas. Self-supervised learning from images with a joint-embedding predictive architecture. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15619–15629, 2023
2023
-
[3]
All are worth words: A vit backbone for diffusion models
Fan Bao, Shen Nie, Kaiwen Xue, Yue Cao, Chongxuan Li, Hang Su, and Jun Zhu. All are worth words: A vit backbone for diffusion models. InProceedings of the IEEE/CVF con- ference on computer vision and pattern recognition, pages 22669–22679, 2023
2023
-
[4]
Beit: Bert pre-training of image transformers.arXiv preprint arXiv:2106.08254, 2021
Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. Beit: Bert pre-training of image transformers.arXiv preprint arXiv:2106.08254, 2021
Pith/arXiv arXiv 2021
-
[5]
Springer, 2006
Christopher M Bishop.Pattern recognition and machine learning. Springer, 2006
2006
-
[6]
Sam Bond-Taylor, Adam Leach, Yang Long, and Chris G Willcocks. Deep generative modelling: A comparative re- view of V AEs, GANs, normalizing flows, energy-based and autoregressive models.IEEE transactions on pattern analy- sis and machine intelligence, 44(11):7327–7347, 2021
2021
-
[7]
Emerg- ing properties in self-supervised vision transformers
Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e Jegou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers. In ICCV, 2021
2021
-
[8]
Hila Chefer, Uriel Singer, Amit Zohar, Yuval Kirstain, Adam Polyak, Yaniv Taigman, Lior Wolf, and Shelly Sheynin. Videojam: Joint appearance-motion representations for en- hanced motion generation in video models.arXiv preprint arXiv:2502.02492, 2025
Pith/arXiv arXiv 2025
-
[9]
Masked autoencoders are effective tokenizers for diffusion models
Hao Chen, Yujin Han, Fangyi Chen, Xiang Li, Yidong Wang, Jindong Wang, Ze Wang, Zicheng Liu, Difan Zou, and Bhik- sha Raj. Masked autoencoders are effective tokenizers for diffusion models. InICML, 2025
2025
-
[10]
Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolu- tion, and fully connected crfs.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017
2017
-
[11]
A simple framework for contrastive learning of visual representations
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Ge- offrey Hinton. A simple framework for contrastive learning of visual representations. InInternational conference on ma- chine learning, pages 1597–1607. PmLR, 2020
2020
-
[12]
Masked-attention mask transformer for universal image segmentation
Bowen Cheng, Ishan Misra, Alexander G Schwing, Alexan- der Kirillov, and Rohit Girdhar. Masked-attention mask transformer for universal image segmentation. InCVPR, 2022
2022
-
[13]
The cityscapes dataset for semantic urban scene understanding
Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. InCVPR, 2016
2016
-
[14]
Vision transformers need registers.arXiv preprint arXiv:2309.16588, 2023
Timoth ´ee Darcet, Maxime Oquab, Julien Mairal, and Pi- otr Bojanowski. Vision transformers need registers.arXiv preprint arXiv:2309.16588, 2023
Pith/arXiv arXiv 2023
-
[15]
Imagenet: A large-scale hierarchical image database
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. InCVPR, pages 248–255, 2009
2009
-
[16]
Imagenet: A large-scale hierarchical image database
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255, 2009
2009
-
[17]
Diffusion models beat GANs on image synthesis
Prafulla Dhariwal and Alexander Quinn Nichol. Diffusion models beat GANs on image synthesis. InNeurIPS, 2021
2021
-
[18]
Alexey Dosovitskiy. An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020
Pith/arXiv arXiv 2010
-
[19]
Shanghua Gao, Pan Zhou, Ming-Ming Cheng, and Shuicheng Yan. Mdtv2: Masked diffusion transformer is a strong image synthesizer.arXiv preprint arXiv:2303.14389, 2023
Pith/arXiv arXiv 2023
-
[20]
Benyamin Ghojogh, Ali Ghodsi, Fakhri Karray, and Mark Crowley. Factor analysis, probabilistic principal component analysis, variational inference, and variational autoencoder: Tutorial and survey.arXiv preprint arXiv:2101.00734, 2021
Pith/arXiv arXiv 2021
-
[21]
Spyros Gidaris, Praveer Singh, and Nikos Komodakis. Un- supervised representation learning by predicting image rota- tions.arXiv preprint arXiv:1803.07728, 2018
Pith/arXiv arXiv 2018
-
[22]
Bootstrap your own latent-a new approach to self-supervised learning.Advances in neural information processing systems, 33:21271–21284, 2020
Jean-Bastien Grill, Florian Strub, Florent Altch ´e, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Ghesh- laghi Azar, et al. Bootstrap your own latent-a new approach to self-supervised learning.Advances in neural information processing systems, 33:21271–21284, 2020
2020
-
[23]
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InProceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016
2016
-
[24]
Masked autoencoders are scalable vision learners
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll´ar, and Ross Girshick. Masked autoencoders are scalable vision learners. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16000– 16009, 2022
2022
-
[25]
Gans trained by a two time-scale update rule converge to a local nash equilib- rium.Advances in neural information processing systems, 30, 2017
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilib- rium.Advances in neural information processing systems, 30, 2017
2017
-
[26]
Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598, 2022
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598, 2022
Pith/arXiv arXiv 2022
-
[27]
What to hide from your students: Attention-guided masked image modeling
Ioannis Kakogeorgiou, Spyros Gidaris, Bill Psomas, Yan- nis Avrithis, Andrei Bursuc, Konstantinos Karantzalos, and Nikos Komodakis. What to hide from your students: Attention-guided masked image modeling. InEuropean Conference on Computer Vision, pages 300–318. Springer, 2022
2022
-
[28]
DINO-foresight: Looking into the future with DINO
Efstathios Karypidis, Ioannis Kakogeorgiou, Spyros Gidaris, and Nikos Komodakis. DINO-foresight: Looking into the future with DINO. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. 10
2025
-
[29]
Kingma and Jimmy Ba
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization, 2017
2017
-
[30]
Glow: Generative flow with invertible 1x1 convolutions.Advances in neural information processing systems, 31, 2018
Durk P Kingma and Prafulla Dhariwal. Glow: Generative flow with invertible 1x1 convolutions.Advances in neural information processing systems, 31, 2018
2018
-
[31]
Ilias: Instance-level image retrieval at scale
Giorgos Kordopatis-Zilos, Vladan Stojni ´c, Anna Manko, Pavel Suma, Nikolaos-Antonios Ypsilantis, Nikos Efthymi- adis, Zakaria Laskar, Jiri Matas, Ondrej Chum, and Gior- gos Tolias. Ilias: Instance-level image retrieval at scale. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 14777–14787, 2025
2025
-
[32]
EQ-V AE: Equivariance regularized latent space for improved generative image modeling
Theodoros Kouzelis, Ioannis Kakogeorgiou, Spyros Gidaris, and Nikos Komodakis. EQ-V AE: Equivariance regularized latent space for improved generative image modeling. In Forty-second International Conference on Machine Learn- ing, 2025
2025
-
[33]
Boosting generative image modeling via joint image-feature synthe- sis
Theodoros Kouzelis, Efstathios Karypidis, Ioannis Kakoge- orgiou, Spyros Gidaris, and Nikos Komodakis. Boosting generative image modeling via joint image-feature synthe- sis. InNeurIPS, 2025
2025
-
[34]
Improved precision and recall met- ric for assessing generative models.Advances in neural in- formation processing systems, 32, 2019
Tuomas Kynk ¨a¨anniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Improved precision and recall met- ric for assessing generative models.Advances in neural in- formation processing systems, 32, 2019
2019
-
[35]
Applying guidance in a limited interval improves sample and distribution quality in diffusion models
Tuomas Kynk ¨a¨anniemi, Miika Aittala, Tero Karras, Samuli Laine, Timo Aila, and Jaakko Lehtinen. Applying guidance in a limited interval improves sample and distribution quality in diffusion models. InNeurIPS, 2024
2024
-
[36]
A path towards autonomous machine intelli- gence version 0.9
Yann LeCun. A path towards autonomous machine intelli- gence version 0.9. 2, 2022-06-27.Open Review, 62(1):1–62, 2022
2022
-
[37]
REPA-E: Unlocking V AE for end-to-end tuning with latent diffusion transform- ers
Xingjian Leng, Jaskirat Singh, Yunzhong Hou, Zhenchang Xing, Saining Xie, and Liang Zheng. REPA-E: Unlocking V AE for end-to-end tuning with latent diffusion transform- ers. InICCV, 2025
2025
-
[38]
Autoregressive image generation without vec- tor quantization
Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, and Kaiming He. Autoregressive image generation without vec- tor quantization. InNeurIPS, 2024
2024
-
[39]
Feature pyramid networks for object detection
Tsung-Yi Lin, Piotr Doll ´ar, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. Feature pyramid networks for object detection. InCVPR, 2017
2017
-
[40]
Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maxim- ilian Nickel, and Matthew Le. Flow matching for genera- tive modeling. InThe Eleventh International Conference on Learning Representations, 2023
2023
-
[41]
Fully convolutional networks for semantic segmentation
Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. In CVPR, 2015
2015
-
[42]
Decoupled weight decay regularization, 2019
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization, 2019
2019
-
[43]
Understanding diffusion models: A unified per- spective.arXiv preprint arXiv:2208.11970, 2022
Calvin Luo. Understanding diffusion models: A unified per- spective.arXiv preprint arXiv:2208.11970, 2022
Pith/arXiv arXiv 2022
-
[44]
Sit: Exploring flow and diffusion-based generative models with scalable in- terpolant transformers
Nanye Ma, Mark Goldstein, Michael S Albergo, Nicholas M Boffi, Eric Vanden-Eijnden, and Saining Xie. Sit: Exploring flow and diffusion-based generative models with scalable in- terpolant transformers. InECCV, page 23–40, 2024
2024
-
[45]
FA V AE-effective frequency aware latent tok- enizer
Tejaswini Medi, Hsien-Yi Wang, Arianna Rampini, and Mar- gret Keuper. FA V AE-effective frequency aware latent tok- enizer. InNeurIPS 2025 Workshop: Reliable ML from Unre- liable Data, 2025
2025
-
[46]
Generating images with sparse representations
Charlie Nash, Jacob Menick, Sander Dieleman, and Peter W Battaglia. Generating images with sparse representations. arXiv preprint arXiv:2103.03841, 2021
Pith/arXiv arXiv 2021
-
[47]
Unsupervised learning of visual representations by solving jigsaw puzzles
Mehdi Noroozi and Paolo Favaro. Unsupervised learning of visual representations by solving jigsaw puzzles. InEuro- pean conference on computer vision, pages 69–84. Springer, 2016
2016
-
[48]
Repre- sentation learning with contrastive predictive coding.arXiv preprint arXiv:1807.03748, 2018
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Repre- sentation learning with contrastive predictive coding.arXiv preprint arXiv:1807.03748, 2018
Pith/arXiv arXiv 2018
-
[49]
Maxime Oquab, Timoth ´ee Darcet, Th´eo Moutakanni, Huy V . V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel HAZIZA, Francisco Massa, Alaaeldin El-Nouby, Mido Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michael Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Herve Je- gou, Julien Mairal, Patr...
2024
-
[50]
Scalable diffusion models with transformers
William Peebles and Saining Xie. Scalable diffusion models with transformers. InProceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 4195–4205, 2023
2023
-
[51]
MIT press, 2023
Simon JD Prince.Understanding deep learning. MIT press, 2023
2023
-
[52]
Bill Psomas, Dionysis Christopoulos, Eirini Baltzi, Ioan- nis Kakogeorgiou, Tilemachos Aravanis, Nikos Komodakis, Konstantinos Karantzalos, Yannis Avrithis, and Giorgos To- lias. Attention, please! revisiting attentive probing for masked image modeling.arXiv preprint arXiv:2506.10178, 2025
arXiv 2025
-
[53]
Instance-level composed image retrieval
Bill Psomas, George Retsinas, Nikos Efthymiadis, Panagio- tis Filntisis, Yannis Avrithis, Petros Maragos, Ondrej Chum, and Giorgos Tolias. Instance-level composed image retrieval. InThe Thirty-ninth Annual Conference on Neural Informa- tion Processing Systems, 2025
2025
-
[54]
Learn- ing transferable visual models from natural language super- vision
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. InICML, 2021
2021
-
[55]
Vi- sion transformers for dense prediction
Ren ´e Ranftl, Alexey Bochkovskiy, and Vladlen Koltun. Vi- sion transformers for dense prediction. InCVPR, 2021
2021
-
[56]
High-resolution image syn- thesis with latent diffusion models
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. InCVPR, pages 10684– 10695, 2022
2022
-
[57]
Improved techniques for training gans.Advances in neural information processing systems, 29, 2016
Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans.Advances in neural information processing systems, 29, 2016
2016
-
[58]
Laion-5b: An open large-scale dataset for train- ing next generation image-text models
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo 11 Coombes, Aarush Katta, Clayton Mullis, Mitchell Worts- man, Patrick Schramowski, Srivatsa Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, and Je- nia Jitsev. Laion-5b: An open large-scale dataset for train- ing next generation image-tex...
2022
-
[59]
Latent diffusion model without variational autoen- coder.arXiv preprint arXiv:2510.15301, 2025
Minglei Shi, Haolin Wang, Wenzhao Zheng, Ziyang Yuan, Xiaoshi Wu, Xintao Wang, Pengfei Wan, Jie Zhou, and Ji- wen Lu. Latent diffusion model without variational autoen- coder.arXiv preprint arXiv:2510.15301, 2025
arXiv 2025
-
[60]
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equa- tions.arXiv preprint arXiv:2011.13456, 2020
Pith/arXiv arXiv 2011
-
[61]
Lposs: Label propagation over patches and pixels for open-vocabulary semantic segmentation
Vladan Stojni ´c, Yannis Kalantidis, Ji ˇr´ı Matas, and Giorgos Tolias. Lposs: Label propagation over patches and pixels for open-vocabulary semantic segmentation. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 9794–9803, 2025
2025
-
[62]
Any-to-any generation via composable diffu- sion.Advances in Neural Information Processing Systems, 36:16083–16099, 2023
Zineng Tang, Ziyi Yang, Chenguang Zhu, Michael Zeng, and Mohit Bansal. Any-to-any generation via composable diffu- sion.Advances in Neural Information Processing Systems, 36:16083–16099, 2023
2023
-
[63]
Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Li- wei Wang. Visual autoregressive modeling: Scalable im- age generation via next-scale prediction.arXiv preprint arXiv:2404.02905, 2024
Pith/arXiv arXiv 2024
-
[64]
Probabilistic principal component analysis.Journal of the Royal Statis- tical Society Series B: Statistical Methodology, 61(3):611– 622, 1999
Michael E Tipping and Christopher M Bishop. Probabilistic principal component analysis.Journal of the Royal Statis- tical Society Series B: Statistical Methodology, 61(3):611– 622, 1999
1999
-
[65]
Michael Tschannen, Alexey Gritsenko, Xiao Wang, Muham- mad Ferjad Naeem, Ibrahim Alabdulmohsin, Nikhil Parthasarathy, Talfan Evans, Lucas Beyer, Ye Xia, Basil Mustafa, Olivier H ´enaff, Jeremiah Harmsen, Andreas Steiner, and Xiaohua Zhai. Siglip 2: Multilingual vision- language encoders with improved semantic understand- ing, localization, and dense featu...
Pith/arXiv arXiv 2025
-
[66]
DDT: Decoupled diffusion transformer.arXiv preprint arXiv:2504.05741, 2025
Shuai Wang, Zhi Tian, Weilin Huang, and Limin Wang. DDT: Decoupled diffusion transformer.arXiv preprint arXiv:2504.05741, 2025
Pith/arXiv arXiv 2025
-
[67]
Representation entanglement for genera- tion: Training diffusion transformers is much easier than you think.NeurIPS, 2025
Ge Wu, Shen Zhang, Ruijing Shi, Shanghua Gao, Zhenyuan Chen, Lei Wang, Zhaowei Chen, Hongcheng Gao, Yao Tang, Jian Yang, et al. Representation entanglement for genera- tion: Training diffusion transformers is much easier than you think.NeurIPS, 2025
2025
-
[68]
Depth anything: Unleashing the power of large-scale unlabeled data
Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything: Unleashing the power of large-scale unlabeled data. InCVPR, 2024
2024
-
[69]
Depth any- thing v2
Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiao- gang Xu, Jiashi Feng, and Hengshuang Zhao. Depth any- thing v2. InNeurIPS, 2024
2024
-
[70]
Fasterdit: Towards faster diffusion transformers train- ing without architecture modification
Jingfeng Yao, Cheng Wang, Wenyu Liu, and Xinggang Wang. Fasterdit: Towards faster diffusion transformers train- ing without architecture modification. InNeurIPS, 2024
2024
-
[71]
Reconstruc- tion vs
Jingfeng Yao, Bin Yang, and Xinggang Wang. Reconstruc- tion vs. generation: Taming optimization dilemma in latent diffusion models. InProceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR), 2025
2025
-
[72]
Language model beats diffusion - tokenizer is key to visual generation
Lijun Yu, Jose Lezama, Nitesh Bharadwaj Gundavarapu, Luca Versari, Kihyuk Sohn, David Minnen, Yong Cheng, Agrim Gupta, Xiuye Gu, Alexander G Hauptmann, Boqing Gong, Ming-Hsuan Yang, Irfan Essa, David A Ross, and Lu Jiang. Language model beats diffusion - tokenizer is key to visual generation. InThe Twelfth International Conference on Learning Representati...
2024
-
[73]
Representation alignment for generation: Training diffusion transformers is easier than you think
Sihyun Yu, Sangkyung Kwak, Huiwon Jang, Jongheon Jeong, Jonathan Huang, Jinwoo Shin, and Saining Xie. Representation alignment for generation: Training diffusion transformers is easier than you think. InICLR, 2025
2025
-
[74]
Dif- fusion models with deterministic normalizing flow priors
Mohsen Zand, Ali Etemad, and Michael Greenspan. Dif- fusion models with deterministic normalizing flow priors. Transactions of Machine Learning Research, 2024
2024
-
[75]
Language- guided image tokenization for generation
Kaiwen Zha, Lijun Yu, Alireza Fathi, David A Ross, Cordelia Schmid, Dina Katabi, and Xiuye Gu. Language- guided image tokenization for generation. InCVPR, pages 15713–15722, 2025
2025
-
[76]
Normalizing flows are capable generative models
Shuangfei Zhai, Ruixiang Zhang, Preetum Nakkiran, David Berthelot, Jiatao Gu, Huangjie Zheng, Tianrong Chen, Miguel Angel Bautista, Navdeep Jaitly, and Josh Susskind. Normalizing flows are capable generative models. InICML, 2025
2025
-
[77]
Sigmoid Loss for Language Image Pre- Training
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid Loss for Language Image Pre- Training . InICCV, pages 11941–11952, 2023
2023
-
[78]
Diffusion normalizing flow
Qinsheng Zhang and Yongxin Chen. Diffusion normalizing flow. InNeurIPS, pages 16280–16291, 2021
2021
-
[79]
Pyramid scene parsing network
Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, and Jiaya Jia. Pyramid scene parsing network. In CVPR, 2017
2017
-
[80]
Fast training of diffusion models with masked transformers.arXiv preprint arXiv:2306.09305, 2023
Hongkai Zheng, Weili Nie, Arash Vahdat, and Anima Anandkumar. Fast training of diffusion models with masked transformers.arXiv preprint arXiv:2306.09305, 2023
Pith/arXiv arXiv 2023
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.