Pith. sign in

REVIEW 4 major objections 5 minor 48 references

Taming Diffusion for Dataset Distillation with High Representativeness

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that mapping VAE latents through DDIM inversion into a Gaussian noise domain, then selecting subsets by mean, standard deviation, and skewness, produces distilled datasets that outperform state-of-the-art baselines across…

desk verdict Strong empirical results with a genuinely useful new combination, but the central Gaussian-approximation lemma is under-supported and the T=31 operating point is chosen by accuracy, not by normality. read the letter →

arxiv 2505.18399 v1 pith:QOKQ75EH submitted 2025-05-23 cs.CV cs.AIcs.LG

Lin Zhao , Yushu Wu , Xinru Jiang , Jianyang Gu , Yanzhi Wang , Xiaolin Xu , Pu Zhao , Xue Lin This is my paper
classification cs.CVcs.AIcs.LG
keywords datasetdistillationdiffusionmodelsDDIMinversiondistributionmatchingGaussianapproximationlatentspacecross-architecturegeneralizationimagegeneration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to make diffusion-based dataset distillation reliable by moving the hard part of distribution matching into a noise domain where the distribution is nearly Gaussian. Its proposed framework, D3HR, runs deterministic DDIM inversion on the full dataset's VAE latents, fits a per-class Gaussian in the inverted domain, and then selects, from many candidate subsets, the few latents whose mean, standard deviation, and skewness best match that Gaussian. Those latents are decoded back through DDIM sampling into a small distilled dataset. The paper reports that this one-time generation yields accuracy above current baselines on CIFAR-10, CIFAR-100, Tiny-ImageNet, and ImageNet-1K, and that the same distilled set trains several architectures, so a separate distillation run per architecture is not needed.

What carries the argument

The load-bearing mechanism is the pair of a deterministic domain map and a moment-matched subset selection. DDIM inversion, the deterministic reverse of DDIM sampling, transfers each VAE latent $z_0$ into $z_T$ in a high-normality noise space, with the claim that for sufficiently large $T$ the per-class set $Z_{T,C}$ becomes approximately Gaussian with independent dimensions. On that Gaussian, the Ziggurat algorithm, a standard Gaussian sampler, generates candidate subsets, and a selection score $L_{T,C}=\lambda_\mu L_\mu+\lambda_\sigma L_\sigma+\lambda_{\gamma_1}L_{\gamma_1}$ picks the subset minimizing differences in mean, standard deviation, and skewness. The same inverse steps are then reversed to decode selected latents into images.

What would settle it

Take a single ImageNet-1K class, invert its VAE latents for 31 steps, and run a multivariate normality test or measure pairwise dimension correlations and multi-modality of $Z_{T,C}$; if the set deviates clearly from one Gaussian, then a Gaussian mixture fitted at the same step should produce a subset that matches the original class distribution better than D3HR's single-Gaussian subset, which would directly contradict the method's central claim.

Watch

Extended reading notes

Core claim

The central discovery is that the obstacle is not the diffusion model's generative power but where distribution matching is attempted: in the raw VAE latent space, each class is a multi-component Gaussian mixture that is hard to fit with a handful of samples. D3HR instead maps every latent through T steps of deterministic DDIM inversion into a noise space the paper argues is a single Gaussian per class, fits that Gaussian, and samples candidate subsets from it. Because DDIM inversion is bijective, the mapping preserves the structural relationship between latents, and because the selected subset is chosen to match the Gaussian's mean, variance, and skewness together, the resulting distilled images represent the whole class rather than isolated regions. The paper claims this yields lower distribution error, higher and more stable accuracy, and cross-architecture portability of one distilled dataset.

Load-bearing premise

The load-bearing premise is that after 31 DDIM inversion steps, each class's mapped latents are close enough to one bell-shaped Gaussian with independent dimensions that sampling from that Gaussian and matching three moments faithfully represents the class.

Editorial extensions

If this is right

  • One distilled dataset produced by D3HR can be reused across ResNet, MobileNet, VGG, EfficientNet, ShuffleNet, and DeiT students, eliminating per-architecture re-distillation.
  • Storing only per-class means and variances together with diffusion weights reproduces the distilled set, so storage for large images-per-class settings can be smaller than storing the images themselves.
  • Selecting on mean, standard deviation, and skewness simultaneously lowers variance across seeds and improves accuracy over random Gaussian sampling.
  • Accuracy peaks at 31 inversion steps, so the framework inherits a trade-off between how Gaussian the mapped distribution becomes and how much structural information survives the inversion.
  • D3HR outperforms both teacher-based and teacher-free baselines under soft-label and hard-label validation, including settings where baselines use multiple teacher models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the single-Gaussian approximation is the right lens, the same domain-mapping recipe could transfer to other modalities with a pretrained autoencoder and deterministic diffusion inversion, such as audio or video, though the paper does not test this.
  • The 31-step accuracy peak suggests inversion depth should be tuned per dataset or noise schedule rather than fixed; a normality diagnostic at each step could replace grid search for choosing $T$.
  • Because the selection metric stops at skewness, a natural extension is to include kurtosis or a full covariance distance; the paper's ablations show each added moment helps, so higher-order terms may continue the trend.
  • The method's dependence on one Gaussian per class means a class with genuinely multimodal latent content is described only by its moments; fitting a small mixture in the inverted space would be a testable alternative the paper does not explore.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes D3HR, a diffusion-based dataset distillation method. For each class, VAE latents of the full dataset are mapped by T-step DDIM inversion into a noise-like domain; a per-class diagonal Gaussian is fitted to the inverted latents; candidate subsets are sampled from this Gaussian; and the subset whose mean, standard deviation, and skewness are closest to the fitted Gaussian is selected and mapped back with DDIM sampling to produce the distilled images. Experiments on CIFAR-10, CIFAR-100, Tiny-ImageNet, and ImageNet-1K report accuracy improvements over SRe2L, DWA, D4M, RDED, and, under their evaluation settings, Minimax and D3S, across multiple architectures and images-per-class settings.

Significance. If the results hold, D3HR is a practically valuable contribution to dataset distillation: it is teacher-free, has a one-time generation cost, transfers across architectures, and reports consistent gains over strong baselines, often by several points at 10-100 IPC. The paper releases code and includes useful ablations of the domain-mapping choice, the group-sampling objective, the number of candidate subsets, and hyperparameters. The main weakness is that the theoretical foundation for the 'high representativeness' claim is not established at the operating point: Lemma 4.1 is stated for sufficiently large T, but the method uses T=31 chosen by accuracy, and the diagonal-Gaussian assumption in Eq. (4) is not verified. The empirical accuracy results are credible external evidence of utility, but the representativeness claim needs direct quantitative support.

major comments (4)
  1. [Section 4.3, Lemma 4.1, Appendix A.2, Section 6.2] The proof of Lemma 4.1 assumes 'sufficiently large T' and relies on alpha_t approaching zero and the network output having minimal effect, but the method operates at T=31, which Section 6.2 and Figure 5 select because accuracy peaks there, not because the Gaussian approximation is validated. No quantitative normality or independence check is provided at T=31; the t-SNE visualizations in Appendix C.4 (Figure A3) are qualitative and are not a statistical test. Since Eq. (4) and the entire sampling and selection procedure presuppose a diagonal Gaussian, the central representativeness claim is unverified at the operating point. Please report per-class and per-dataset diagnostics at T=31, such as a multivariate normality test, the magnitudes of off-diagonal covariance entries, and a distributional distance between Z_T,C and the fitted diagonal Gaussian.
  2. [Section 4.3, Eq. (4)] The statement that 'the dimensions in the noise space of DDIM inversion are independent (Song et al., 2020)' does not follow from the cited work. Song et al. initialize DDIM sampling from i.i.d. Gaussian noise, whereas here each z_T is a deterministic, class-conditioned, nonlinear function of a real VAE latent z_0 through the learned denoiser; off-diagonal correlations in Z_T,C may persist after inversion. The diagonal form of Eq. (4) therefore needs empirical support at T=31, for example by reporting the norm of the off-diagonal covariance block or by comparing the fitted diagonal Gaussian with a full-covariance Gaussian fit.
  3. [Appendix A.2, Eqs. (12)-(15)] The derivation of Lemma 4.1 is not valid as written. In Eq. (12), w_t is not a fixed additive term but a nonlinear function of z_t through epsilon_theta(z_t,t,C), so the recursive composition from z_T to z_0 is not a linear map with independent noise, and the variance recursion leading to Eq. (15) does not follow without additional assumptions. The proof should state explicitly when and why the network output can be treated as nearly constant and should show that this regime holds at T=31 for the DiT model used in the experiments.
  4. [Section 4.4, Eqs. (5)-(8)] The subset selection objective measures deviations from the fitted Gaussian \hat Z_T,C, whose mean and variance are themselves computed from Z_T,C in Section 4.3. Minimizing these statistics therefore enforces consistency with the fitted model by construction and is not, by itself, evidence that the selected subset represents the original class distribution Z_0,C. The downstream accuracy on held-out architectures is an external check and supports practical utility, but the paper should either directly evaluate representativeness, for example by comparing the distribution of decoded or re-encoded latents of the distilled set with the original full-data latent distribution, or should soften the claim that the method provably yields high representativeness.
minor comments (5)
  1. [Table 3 and Section 6.1] Table 3 reports no standard deviations, so the statement that 'each metric individually increases the accuracy' is not statistically substantiated; the differences between Base-RS (41.6) and L_sigma (42.3) or L_gamma1 (42.4) could be within run-to-run noise. Please report error bars or significance tests for these ablations.
  2. [Section 6.2] The text refers to 'Theorem 4.1' when the statement in Section 4.3 is labeled 'Lemma 4.1'; please unify the numbering and terminology.
  3. [Appendix C.3] The sentence 'only about 0.016 GB, which includes 320 MB from the VAE weights' is numerically inconsistent: 0.016 GB is 16 MB, which is smaller than 320 MB. Please clarify the units and the intended total storage figure.
  4. [Lemma 3.1 and Appendix A.1] The proof of Lemma 3.1 largely restates the VAE posterior structure and does not establish quantitatively that the Gaussian mixture is 'hard-to-fit' or that its components are 'distinct'; consider softening this claim or supporting it with a measured quantity.
  5. [Section 4.3] There is a typo in 'we compute the the mean mu_T,C and variance sigma^2_T,C' where 'the' is repeated; please fix.

Circularity Check

1 steps flagged · score 2.0 of 10

Subset selection is self-referential with respect to the fitted Gaussian, but final accuracy is externally benchmarked; score 2.

  1. fitted input called prediction [Section 4.3–4.4, Eqs. (4)–(9), Algorithm 1]
    "To obtain the statistical properties of ˆZT,C, we compute the the mean µT,C and variance σ2 T,C from ZT,C as the mean and variance of ˆZT,C ... To search for the most representative subset, we further propose an efficient algorithm that selects the subset statistically closest to ˆZT,C among m random subsets"

    The Gaussian \hat{Z}_T,C is not an independent target: its mean and variance are the empirical moments of the same Z_T,C that the subsets are drawn to match, and L_T,C (Eqs. 5–8) measures the deviation of candidate subsets from exactly those moments, with the skewness target set to 0 by the Gaussian assumption. Hence the selected subset is by construction the one closest to the fitted Gaussian; 'representativeness' with respect to \hat{Z}_T,C is tautological. The link from \hat{Z}_T,C to the true VAE latents rests entirely on Lemma 4.1's Gaussian approximation at T=31, which is asserted via a heuristic proof rather than quantitatively validated. The accuracy comparisons on held-out models are an external benchmark, so this is partial rather than total circularity.

full rationale

D3HR's headline results are external: the distilled images are trained on models (ResNet-101, MobileNet-V2, VGG-11, etc.) not used to fit the per-class Gaussian, so the reported accuracy gains are not forced by the construction. There is no load-bearing self-citation; the DDIM citation (Song et al., 2020) and the in-appendix proof of Lemma 4.1 are the main support for the Gaussian assumption. The independence claim imported from Song et al. is not actually established for deterministic DDIM inversion of real VAE latents, and Section 6.2 chooses T=31 because accuracy peaks there rather than because normality is verified; these are support gaps, not circular reductions. The only construction-level circularity is the group-sampling objective, which by design selects subsets closest to the Gaussian fitted from the same data; this makes the 'representativeness' framing self-referential unless Lemma 4.1 holds at the operating point. Since the empirical SOTA claim is independently falsifiable on held-out architectures and datasets, a low score of 2 is appropriate rather than a finding of forced circularity.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The method's contribution is an empirical recipe whose core modeling assumption, a single diagonal Gaussian per class in the inverted latent space, is fitted to data and not independently established. The fitted statistics and tuned hyperparameters carry most of the burden, while the Gaussianity lemma is asserted rather than proven.

free parameters (4)
  • T, number of DDIM inversion and sampling steps = 31
    Selected as the accuracy peak in Figure 5, not derived from theory. Lemma 4.1 requires 'sufficiently large T', but too large T destroys structural information.
  • lambda_mu, lambda_sigma, lambda_gamma1 = 1, 1, 0.5
    Weights in the subset selection objective Eq. (5), set by hand to put metrics on the same scale; Table A11 shows modest sensitivity.
  • m, number of candidate subsets = 1e5 to 5e7 depending on dataset
    Chosen at the saturation point from Tables A9 and A10; trades sampling cost against accuracy.
  • Per-class mean mu_T,C and variance sigma^2_T,C = Estimated from full class latents
    These statistics define the fitted Gaussian that generates the distilled set; the entire distilled dataset is produced from these fitted values.
assumptions (5)
  • domain assumption DDIM inversion with 31 steps is bijective and preserves structural information of the VAE latents.
    Section 4.2 claims information preservation and structural consistency; discrete inversion is only approximately invertible and the paper does not measure reconstruction error.
  • ad hoc to paper After DDIM inversion, per-class latents are i.i.d. samples from a single Gaussian with independent dimensions.
    Lemma 4.1 and Eq. (4); the proof in Appendix A.2 is informal and depends on 'sufficiently large T' while the implementation uses T=31.
  • domain assumption The pre-trained DiT and VAE, after 400 epochs of fine-tuning, generate images whose decoded latents represent the original class distribution.
    Used in all experiments; no independent metric of latent representativeness is provided.
  • domain assumption Moment matching on mean, standard deviation, and skewness is sufficient to select a representative subset.
    Section 4.4, Eqs. (5) to (8); higher moments and inter-sample correlations are ignored.
  • standard math The law of large numbers justifies treating the empirical Z_T,C as a continuous Gaussian distribution.
    Invoked in Section 4.3 and Appendix A.2; this is a misapplication because LLN concerns sample averages, not the distributional shape of a finite set.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Taming Diffusion for Dataset Distillation with High Representativeness." pith.science (2026). https://pith.science/paper/QOKQ75EH

@misc{pith2026250518399,
  author       = {Pith},
  title        = {Pith review of: Taming Diffusion for Dataset Distillation with High Representativeness},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QOKQ75EH}},
  note         = {Machine review of arXiv:2505.18399}
}
read the original abstract

Recent deep learning models demand larger datasets, driving the need for dataset distillation to create compact, cost-efficient datasets while maintaining performance. Due to the powerful image generation capability of diffusion, it has been introduced to this field for generating distilled images. In this paper, we systematically investigate issues present in current diffusion-based dataset distillation methods, including inaccurate distribution matching, distribution deviation with random noise, and separate sampling. Building on this, we propose D^3HR, a novel diffusion-based framework to generate distilled datasets with high representativeness. Specifically, we adopt DDIM inversion to map the latents of the full dataset from a low-normality latent domain to a high-normality Gaussian domain, preserving information and ensuring structural consistency to generate representative latents for the distilled dataset. Furthermore, we propose an efficient sampling scheme to better align the representative latents with the high-normality Gaussian distribution. Our comprehensive experiments demonstrate that D^3HR can achieve higher accuracy across different model architectures compared with state-of-the-art baselines in dataset distillation. Source code: https://github.com/lin-zhao-resoLve/D3HR.

Figures

Figures reproduced from arXiv: 2505.18399 by the authors.

Figure 1
Figure 1. t-SNE visualization of the low-normality VAE space and high-normality noise space for class “Goldfish”. The blue contour lines are the probability density curves of the distribution using kernel density estimation, highlighting the structure and concentration of the latents (blue dots). ★ in (b) marks the 10 representative latents generated in the noise space, corresponding to ★ in (a) after DDIM sampling, which pre… view at source ↗
Figure 2
Figure 2. Illustration of proposed D3HR framework. With the latents from the VAE, DDIM inversion is applied to map the latent embeddings to a Gaussian domain with better normality, which can further be matched to a Gaussian distribution. Then, we follow Sec￾tion 4.4 to sample representative latents based on different IPC requirements, and generate true images through DDIM sampling. ods generate images from the initial noise. … view at source ↗
Figure 3
Figure 3. t-SNE visualization of the representative latents in VAE space generated by Equation (1) and D3HR for class “Goldfish”. It can be observed that the latents accurately represent the VAE distribution in D3HR. of the random subset. As Gaussian distribution is perfectly symmetric, γ1,T ,C = 0. With the above evaluation metric function, for the k th subset, we can compute its metric L k T ,C . And we select the j th subs… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Validation of the effectiveness and stability of group sampling on ImageNet-1K. For our D3HR and Base-RS, we generate 3 distilled datasets at each IPC. Each dataset undergoes 3 rounds of validation, resulting in 9 data points per box plot. by comparing our results with…
Figure 6
Figure 6. Figure 6: Visualization of the distilled images for the class “gold￾fish” on ImageNet-1K, IPC = 5. mance loss. When t becomes large (e.g., t = 40), although our distribution matching can accurately represent the real distributions which becomes more normal, the real distribu￾tio…
Figure 7
Figure 7. Figure 7 [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

48 extracted references · 26 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey, D. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv: 2010.11929, 2020

    arXiv 2010

  3. [3]

    Azizi, S., Kornblith, S., Saharia, C., Norouzi, M., and Fleet, D. J. Synthetic data from diffusion models improves imagenet classification. arXiv preprint arXiv:2304.08466, 2023

    arXiv 2023

  4. [4]

    A., and Zhu, J.-Y

    Cazenavette, G., Wang, T., Torralba, A., Efros, A. A., and Zhu, J.-Y. Dataset distillation by matching training trajectories. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 4750--4759, 2022

    work page 2022

  5. [5]

    Scaling up dataset distillation to imagenet-1k with constant memory

    Cui, J., Wang, R., Si, S., and Hsieh, C.-J. Scaling up dataset distillation to imagenet-1k with constant memory. In International Conference on Machine Learning, pp.\ 6565--6590. PMLR, 2023

    work page 2023

  6. [6]

    Imagenet: A large-scale hierarchical image database

    Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp.\ 248--255. Ieee, 2009

    2009

  7. [7]

    Exploiting inter-sample and inter-feature relations in dataset distillation

    Deng, W., Li, W., Ding, T., Wang, L., Zhang, H., Huang, K., Huo, J., and Gao, Y. Exploiting inter-sample and inter-feature relations in dataset distillation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 17057--17066, 2024

    work page 2024

  8. [8]

    and Nichol, A

    Dhariwal, P. and Nichol, A. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34: 0 8780--8794, 2021

    2021

Show all 48 references
  1. [9]

    Y., Zhou, J

    Du, J., Jiang, Y., Tan, V. Y., Zhou, J. T., and Li, H. Minimizing the accumulated trajectory error to improve dataset distillation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 3749--3758, 2023

    2023

  2. [10]

    Du, J., Shi, Q., and Zhou, J. T. Sequential subset matching for dataset distillation. Advances in Neural Information Processing Systems, 36, 2024 a

    2024

  3. [11]

    Du, J., Zhang, X., Hu, J., Huang, W., and Zhou, J. T. Diversity-driven synthesis: Enhancing dataset distillation through directed weight adjustment. arXiv preprint arXiv:2409.17612, 2024 b

    2024 arXiv

  4. [12]

    Scaling rectified flow transformers for high-resolution image synthesis

    Esser, P., Kulal, S., Blattmann, A., Entezari, R., M \"u ller, J., Saini, H., Levi, Y., Lorenz, D., Sauer, A., Boesel, F., et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first International Conference on Machine Learning, 2024

    2024

  5. [13]

    Efficient Dataset Distillation via Minimax Diffusion

    Gu, J., Vahidian, S., Kungurtsev, V., Wang, H., Jiang, W., You, Y., and Chen, Y. Efficient Dataset Distillation via Minimax Diffusion . In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024

    2024

  6. [14]

    Deep residual learning for image recognition

    He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 770--778, 2016

    2016

  7. [15]

    Denoising diffusion probabilistic models

    Ho, J., Jain, A., and Abbeel, P. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33: 0 6840--6851, 2020

    2020

  8. [16]

    A smaller subset of 10 easily classified classes from imagenet, and a little more french

    Howard, J. A smaller subset of 10 easily classified classes from imagenet, and a little more french. URL https://github. com/fastai/imagenette, 4, 2019

    2019

  9. [17]

    and Robbins, H

    Hsu, P.-L. and Robbins, H. Complete convergence and the law of large numbers. Proceedings of the national academy of sciences, 33 0 (2): 0 25--31, 1947

    1947

  10. [18]

    P., Welling, M., et al

    Kingma, D. P., Welling, M., et al. An introduction to variational autoencoders. Foundations and Trends in Machine Learning , 12 0 (4): 0 307--392, 2019

    2019

  11. [19]

    Learning multiple layers of features from tiny images

    Krizhevsky, A., Hinton, G., et al. Learning multiple layers of features from tiny images. 2009

    2009

  12. [20]

    and Yang, X

    Le, Y. and Yang, X. Tiny imagenet visual recognition challenge. CS 231N, 7 0 (7): 0 3, 2015

    2015

  13. [21]

    Dataset condensation with contrastive signals

    Lee, S., Chun, S., Jung, S., Yun, S., and Yoon, S. Dataset condensation with contrastive signals. In International Conference on Machine Learning, pp.\ 12352--12364. PMLR, 2022

    2022

  14. [22]

    and Chung, H

    Lee, Y. and Chung, H. W. Selmatch: Effectively scaling up dataset distillation via selection-based initialization and partial updates by trajectory matching. In Forty-first International Conference on Machine Learning, 2024

    2024

  15. [23]

    Large scale dataset distillation with domain shift

    Loo, N., Maalouf, A., Hasani, R., Lechner, M., Amini, A., and Rus, D. Large scale dataset distillation with domain shift. In Forty-first International Conference on Machine Learning

  16. [24]

    Shufflenet v2: Practical guidelines for efficient cnn architecture design

    Ma, N., Zhang, X., Zheng, H.-T., and Sun, J. Shufflenet v2: Practical guidelines for efficient cnn architecture design. In Proceedings of the European conference on computer vision (ECCV), pp.\ 116--131, 2018

    2018

  17. [25]

    and Tsang, W

    Marsaglia, G. and Tsang, W. W. The ziggurat method for generating random variables. Journal of statistical software, 5: 0 1--7, 2000

    2000

  18. [26]

    and Xie, S

    Peebles, W. and Xie, S. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.\ 4195--4205, 2023

    2023

  19. [27]

    High-resolution image synthesis with latent diffusion models

    Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 10684--10695, 2022

    2022

  20. [28]

    Mobilenetv2: Inverted residuals and linear bottlenecks

    Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L.-C. Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 4510--4520, 2018

    2018

  21. [29]

    Generalized large-scale data condensation via various backbone and statistical matching

    Shao, S., Yin, Z., Zhou, M., Zhang, X., and Shen, Z. Generalized large-scale data condensation via various backbone and statistical matching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 16709--16718, 2024

    2024

  22. [30]

    Lazydit: Lazy learning for the acceleration of diffusion transformers

    Shen, X., Song, Z., Zhou, Y., et al. Lazydit: Lazy learning for the acceleration of diffusion transformers. In AAAI, 2025

    2025

  23. [31]

    and Zisserman, A

    Simonyan, K. and Zisserman, A. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014

    2014 arXiv

  24. [32]

    Denoising diffusion implicit models

    Song, J., Meng, C., and Ermon, S. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020

    2010 arXiv

  25. [33]

    Beyond neural scaling laws: beating power law scaling via data pruning

    Sorscher, B., Geirhos, R., Shekhar, S., Ganguli, S., and Morcos, A. Beyond neural scaling laws: beating power law scaling via data pruning. Advances in Neural Information Processing Systems, 35: 0 19523--19536, 2022

    2022

  26. [34]

    D 4M: Dataset Distillation via Disentangled Diffusion Model

    Su, D., Hou, J., Gao, W., Tian, Y., and Tang, B. D 4M: Dataset Distillation via Disentangled Diffusion Model . In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 5809--5818, June 2024

    2024

  27. [35]

    On the diversity and realism of distilled dataset: An efficient dataset distillation paradigm

    Sun, P., Shi, B., Yu, D., and Lin, T. On the diversity and realism of distilled dataset: An efficient dataset distillation paradigm. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 9390--9399, 2024

    2024

  28. [36]

    Data pruning via moving-one-sample-out

    Tan, H., Wu, S., Du, F., Chen, Y., Wang, Z., Wang, F., and Qi, X. Data pruning via moving-one-sample-out. Advances in Neural Information Processing Systems, 36, 2024

    2024

  29. [37]

    and Le, Q

    Tan, M. and Le, Q. Efficientnet: Rethinking model scaling for convolutional neural networks. In International conference on machine learning, pp.\ 6105--6114. PMLR, 2019

    2019

  30. [38]

    Training data-efficient image transformers & distillation through attention

    Touvron, H., Cord, M., Douze, M., Massa, F., Sablayrolles, A., and J \'e gou, H. Training data-efficient image transformers & distillation through attention. In International conference on machine learning, pp.\ 10347--10357. PMLR, 2021

    2021

  31. [39]

    Cafe: Learning to condense dataset by aligning features

    Wang, K., Zhao, B., Peng, X., Zhu, Z., Yang, S., Wang, S., Huang, G., Bilen, H., Wang, X., and You, Y. Cafe: Learning to condense dataset by aligning features. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 12196--12205, 2022

    2022

  32. [40]

    Wang, T., Zhu, J.-Y., Torralba, A., and Efros, A. A. Dataset distillation. arXiv preprint arXiv:1811.10959, 2018

    2018 arXiv

  33. [41]

    Squeeze, Recover and Relabel: Dataset Condensation at ImageNet Scale From A New Perspective

    Yin, Z., Xing, E., and Shen, Z. Squeeze, Recover and Relabel: Dataset Condensation at ImageNet Scale From A New Perspective . In Thirty-seventh Conference on Neural Information Processing Systems, 2023

    2023

  34. [42]

    Teddy: Efficient large-scale dataset distillation via taylor-approximated matching

    Yu, R., Liu, S., Ye, J., and Wang, X. Teddy: Efficient large-scale dataset distillation via taylor-approximated matching. In European Conference on Computer Vision, pp.\ 1--17. Springer, 2024

    2024

  35. [43]

    Echo: Efficient dataset condensation by higher-order distribution alignment

    Zhang, H., Li, S., Wang, P., Zeng, D., and Ge, S. Echo: Efficient dataset condensation by higher-order distribution alignment. arXiv preprint arXiv:2312.15927, 2023

    2023 arXiv

  36. [44]

    Zhang, X., Du, J., Li, Y., Xie, W., and Zhou, J. T. Spanning training progress: Temporal dual-depth scoring (tdds) for enhanced dataset pruning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 26223--26232, 2024

    2024

  37. [45]

    and Bilen, H

    Zhao, B. and Bilen, H. Dataset condensation with differentiable siamese augmentation. In International Conference on Machine Learning, pp.\ 12674--12685. PMLR, 2021

    2021

  38. [46]

    and Bilen, H

    Zhao, B. and Bilen, H. Dataset condensation with distribution matching. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp.\ 6514--6523, 2023

    2023

  39. [47]

    R., and Bilen, H

    Zhao, B., Mopuri, K. R., and Bilen, H. Dataset condensation with gradient matching. arXiv preprint arXiv:2006.05929, 2020

    2006 arXiv

  40. [48]

    Improved distribution matching for dataset condensation

    Zhao, G., Li, G., Qin, Y., and Yu, Y. Improved distribution matching for dataset condensation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 7856--7865, 2023

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.