MGD$^3$: Mode-Guided Dataset Distillation using Diffusion Models

Gaowen Liu; Gaurav Kumar Nayak; Jeffrey A. Chan-Santiago; Mubarak Shah; Praveen Tirupattur

REVIEW 4 major objections 6 minor 2 cited by

MGD$^3$: Mode-Guided Dataset Distillation using Diffusion Models

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A training-free procedure that steers denoising toward K-means latent modes yields distilled datasets that the paper reports as more diverse and representative than fine-tuned diffusion baselines, with accuracy gains of up to 4.4%.

desk verdict Plausible training-free distillation recipe with a load-bearing sign error in its central equation; the empirical gains look real but the paper isn't reproducible as written. read the letter →

arxiv 2505.18963 v1 pith:ISTUHZAG submitted 2025-05-25 cs.CV

Jeffrey A. Chan-Santiago , Praveen Tirupattur , Gaurav Kumar Nayak , Gaowen Liu , Mubarak Shah This is my paper

classification cs.CV

keywords datasetdistillationdiffusionmodelsmodeguidancelatentspaceclusteringK-meanssyntheticdiversityImageNettraining-freegenerative

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is trying to establish that a pre-trained diffusion model can produce a distilled training set that is both diverse and representative without any fine-tuning or distillation-loss training. The proposed method, MGD3, discovers the modes of each class by clustering the VAE latent space with K-means, then steers the denoising process toward a chosen mode with an additive guidance term, and finally disables that guidance at a fixed timestep to protect image quality. Against methods that do fine-tune a diffusion model with diversity and representativeness losses, the paper reports accuracy gains of 4.4%, 2.9%, 1.6%, and 1.6% on ImageNette, ImageIDC, ImageNet-100, and ImageNet-1K, respectively. If true, dataset distillation becomes much cheaper because the expensive fine-tuning step is replaced by clustering and a few lines of guidance during sampling.

What carries the argument

The load-bearing object is the mode guidance update, Eq. 6: $\hat{\epsilon}_\theta(x_t, t, c) = \tilde{\epsilon}_\theta(x_t, t, c) + \lambda \cdot (m_i - \hat{x}_0^t) \cdot \sigma_t$, where $\tilde{\epsilon}_\theta$ is the classifier-free guidance noise estimate and $\hat{x}_0^t$ is the predicted denoised latent at step $t$. This term is meant to push the reverse trajectory toward a K-means centroid $m_i$ computed in the VAE latent space of the diffusion model, so different random noises end up in different modes of a class. The second mechanism is Stop Guidance: after $t_{\mathrm{stop}} = 25$ steps the guidance weight is set to zero, leaving the refinement stage of denoising unguided; the paper motivates this with the claim that the last half of the reverse process is a refinement stage where strong mode pulling introduces artifacts and hurts class fidelity. In the DDIM variant the same steering is expressed differently, as a subtraction of $\sqrt{1 - \bar{\alpha}_t}\,\lambda\,g_t$ from the noise prediction, which is why the exact steering dynamics of the method are defined as a heuristic rather than derived from a score function.

What would settle it

Run MGD3 on a dataset whose VAE latent modes are visually uninformative (for example, a fine-grained class with high intra-class variation) with $\lambda=0.1$ and $t_{\mathrm{stop}}=25$; if downstream accuracy and per-class diversity fall to or below the plain pre-trained DiT baseline, the steering direction is not doing the work. A sharper test: replace the K-means centroids with random points in VAE latent space and keep everything else fixed; if accuracy is unchanged, the mode vectors are not the cause of the reported gains.

Watch

Extended reading notes

Core claim

The central claim is that the under-diversity of diffusion-based dataset distillation is a sampling problem, not a training problem, and can be fixed without touching the model weights. In the Mode Discovery stage, the paper runs K-means with $k = \text{IPC}$ on the VAE encoder's latent vectors for each class and treats the centroids as the modes of that class. In Mode Guidance, at every reverse step it computes $g_t = m_i - \hat{x}_0^t$ and adds $\lambda \cdot g_t \cdot \sigma_t$ to the noise prediction, pulling the trajectory toward centroid $m_i$; in Stop Guidance, it sets $\lambda = 0$ once $t < t_{\mathrm{stop}} = 25$ so that the refinement stage of denoising runs unguided. With only these components, the paper reports distilled datasets whose per-class diversity and representativeness scores exceed those of DiT and MinMax diffusion, and accuracy gains of 4.4%, 2.9%, 1.6%, and 1.6% over state-of-the-art methods on ImageNette, ImageIDC, ImageNet-100, and ImageNet-1K, while generating an ImageNet-100 IPC-10 set in 0.42 hours instead of 10 hours (MinMax) or over 100 hours (IDC).

Load-bearing premise

The load-bearing premise is that the additive guidance term in Eq. 6 genuinely steers the denoising trajectory toward a K-means centroid in VAE latent space and that stopping it at $t_{\mathrm{stop}}=25$ preserves class fidelity across all datasets; this is a heuristic, and the two fixed hyperparameters ($\lambda=0.1$, $t_{\mathrm{stop}}=25$) are tuned on ImageNette and assumed to transfer elsewhere.

Editorial extensions

If this is right

If the steering works as claimed, dataset distillation at ImageNet scale no longer requires a fine-tuned generative model: clustering the target data and guiding a pre-trained sampler is enough to beat fine-tuned baselines.
The reported 0.42 hours for ImageNet-100 IPC-10 (versus 10 hours for MinMax and over 100 hours for IDC) means the compute bottleneck of distillation moves from model fine-tuning to a one-time clustering pass.
Because the same guidance rule improves both DiT and LDM, and transfers to DDIM sampling and to a text-to-image Stable Diffusion model, the recipe is sampler- and architecture-agnostic within latent diffusion.
On ImageNet-1K IPC-50, the distilled set yields 60.2% top-1 with ResNet-18 and 67.7% with ResNet-101, showing the benefit holds for larger student models as well.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: the same mode-guidance idea should transfer to any generative prior with a meaningful latent space, such as GAN inversion codes, where 'modes' could be computed from the latent codes of real images rather than from a VAE.
Editorial inference: the appendix's finding that GMM slightly beats K-means suggests the quality of the centroid estimate matters more than the specific clustering algorithm, so a learned or density-based mode prior could push the accuracy gains further.
Editorial inference: the paper's per-class diversity and representativeness scores give a cheap, model-agnostic way to audit any distilled dataset before spending compute on downstream training; this audit could be applied to optimization-based distillation outputs as well.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things to know before you spend time on this one. First, the empirical story is credible and the method is genuinely training-free for the main benchmarks, which is a real step past MinMaxDiff's fine-tuning. Second, the central equation as printed has the wrong sign. That is not a cosmetic nit; it makes the described mechanism and the pseudo-code disagree about what the method actually does.

What's new: instead of fine-tuning a diffusion model with diversity losses, the authors run K-means in the VAE latent space to get class modes, steer sampling toward a mode for the first 25 steps, then switch to unguided denoising. The idea is simple, the ImageNette/ImageNet-1K gains over DiT and D4M are consistent across tables, the ablations (tstop, lambda, mode discovery algorithm) are informative, and the appendix reports ImageWoof honestly. If the guidance rule works as Algorithm 1 says, this is a useful, low-cost recipe for dataset distillation.

Soft spots, in rough order.

- Eq. 5-6: substituting Eq. 6 into the standard DDPM x0 estimator gives x0' = x0 + c(x0 - m_i), i.e. the predicted latent moves away from the mode. Algorithm 1 uses the opposite sign and a different scale. The text and Fig. 3 say the reverse. So the reader cannot tell from the paper what to implement. This is load-bearing because the whole contribution is this steering rule. The ablations show large gains, which suggests the implementation follows Algorithm 1, but then Eq. 6 and the surrounding text are wrong and must be fixed.

- The 86% statement in Section 5.1 is unsupported; Table 3 shows 60.2% for ResNet-18.

- Abstract accuracy gains (4.4/2.9/1.6/1.6) don't match the per-IPC numbers in the text. Minor, but sloppy.

- The "training-free" claim needs a footnote: ImageWoof requires training the base DiT, which the authors disclose in Appendix E.

- Diversity/representativeness metrics compare against the same original dataset used for mode discovery, so those plots are partly circular. They are supporting evidence, not the main claim.

On balance, the central claim (training-free mode guidance improves distillation accuracy) is probably right, but the paper is not reproducible as written. The sign error alone would sink a naive reimplementation. I'd want the authors to release the code, correct Eq. 6/Algorithm 1 to be consistent, and fix the numerical claims before I'd trust it further.

Who this is for: anyone working on diffusion-based dataset distillation or training-free guided sampling. Worth a serious referee, but only after the authors clean up the specification. I would not cite it in its current form.

Referee Report

4 major / 6 minor

Summary. The paper proposes MGD3, a three-stage training-free dataset distillation method built on a pre-trained latent diffusion model. In Mode Discovery, K-means clustering in the VAE latent space identifies class modes; in Mode Guidance, an additive correction term (Eq. 6) is intended to steer the denoising trajectory toward a chosen mode; in Stop Guidance, this correction is switched off below a timestep tstop to preserve class fidelity. The authors report accuracy gains over prior diffusion-based and optimization-based methods on ImageNette, ImageNet-IDC, ImageNet-100, and ImageNet-1K, and claim that no distillation-loss fine-tuning is needed.

Significance. If the reported results hold, the method is practically significant: it turns a pre-trained diffusion model into a dataset-distillation engine with a few lines of sampling-time code, avoiding the expensive fine-tuning used by prior generative-distillation methods such as MinMaxDiff. The paper has several concrete strengths: it reports mean accuracies with standard deviations over multiple runs, compares against external baselines on held-out test sets, provides ablations of each proposed component, and includes experiments across multiple backbone architectures and diffusion backbones (DiT, LDM, Stable Diffusion). The central accuracy claim is therefore not circular in the way many distillation evaluations are. However, the main mechanism is specified inconsistently between the equations, the algorithm, and the text, and several numerical and scope claims are not supported by the tables; these issues must be resolved before the contribution can be trusted.

major comments (4)

[§4.2, Eq. (6) and Algorithm 1]
[§5.1, Table 3 and Abstract]
[Appendix E and §1]
[§4.2–4.3 and Appendix A/F]

minor comments (6)

[§3, Eq. (4)]
[§3, Eq. (1)]
[Algorithm 1]
[Appendix D, Table 7]
[Throughout]
[§5.1 and Fig. 4]

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central accuracy claims are evaluated on held-out test sets against external baselines, and no load-bearing step of the method reduces to its own inputs.

full rationale

MGD3's contribution is an empirical sampling procedure: mode discovery via K-means over the VAE latent space of the original training set, mode guidance via the additive update in Eq. 5-6, and stop guidance at tstop = 25. The headline results are accuracies of independently trained classifiers on held-out test sets, compared with published baselines such as MinMaxDiff, D4M, and IDC-1. Those accuracies are not forced by construction from the mode-discovery inputs. Using the original training set to discover modes is standard in dataset distillation and does not by itself determine downstream test accuracy. The hyperparameters lambda = 0.1 and tstop = 25 are ablated on ImageNette and then transferred to other datasets; this is a generalization claim, not a fitted parameter being renamed as a prediction. The diversity and representativeness metrics are computed against the same original dataset used for mode discovery, which gives a mildly self-referential quality to those secondary analyses, but the paper's central contribution—outperforming prior methods in classification accuracy—does not depend on those metrics. Finally, there is a notable correctness and reproducibility concern: Eq. 6 as written (epsilon_hat = epsilon_tilde + lambda * (m_i - xhat_0_t) * sigma_t) would, under the DDPM reparameterization, move the predicted x0 away from the mode, whereas Algorithm 1 uses the opposite sign and a different scaling. That inconsistency is a substantive specification issue, but it is not an instance of circularity: the claimed accuracy gains are still externally measured and are not equivalent to the method's own definitions or inputs. Overall, no circular step can be exhibited from the paper's equations, so the circularity score is 0.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claim depends on the assumption that latent-space clusters of the original dataset are the right guidance targets and that a scalar additive correction to the noise prediction steers the diffusion sampler. These are heuristic modeling choices, not derived results.

free parameters (3)

Mode guidance scale lambda = 0.1
Selected by ablation on ImageNette IPC-10 (Appendix F, Fig. 10); values above 0.1 degrade accuracy sharply, and the chosen value is applied to all other datasets without per-dataset tuning.
Stop guidance timestep tstop = 25 of 50 steps
Chosen from ImageNette ablation (Appendix A, Fig. 7), which reports a peak near tstop=20 and a good range 10-30; the paper fixes tstop=25 for all experiments. This is a hand-selected hyperparameter, not derived.
Number of modes per class k = k = IPC
K-means is run with k equal to the images-per-class budget, so the number of guidance targets is tied to the distillation budget rather than to a data-driven estimate of the true number of modes. Section 4.1 and Implementation Details.

assumptions (5)

domain assumption K-means centroids in the VAE latent space correspond to the dominant semantic modes of each class in the original dataset.
Section 4.1 assumes the generative latent space 'captures the overall content' and that cluster centroids are representative modes; no validation that centroids are semantically distinct or that all modes are found.
ad hoc to paper Adding lambda times (m_i minus x_hat_0_t) times sigma_t to the noise prediction in Eq. 6 steers the reverse process toward mode m_i.
The guidance update is stated without derivation from a score function or energy model; the DDIM version in Algorithm 1 uses a different scaling, so the exact steering mechanism is underspecified.
domain assumption The pre-trained DiT-XL/2 trained on ImageNet covers the target datasets' class distributions, making fine-tuning unnecessary.
Used implicitly for ImageNette, IDC, ImageNet-100, and ImageNet-1K; Appendix E contradicts the generality by training a DiT on ImageWoof, and the text-to-image experiments rely on Stable Diffusion's pretraining data overlapping ImageNet.
ad hoc to paper Stopping guidance at tstop=25 preserves class fidelity while unguided sampling adds diversity.
Section 4.3 justifies this via the chaotic/semantic/refinement division from Yu et al. (2023), but the specific threshold is an empirical choice tuned on ImageNette.
standard math The diffusion reverse process and classifier-free guidance equations (Eqs. 1-4) hold for the pretrained models.
Standard DDPM and classifier-free guidance results from Ho et al. (2020) and Ho and Salimans (2021) are invoked as background.

how reviews work

0 comments

Cite this review

Pith. "Pith review of MGD$^3$: Mode-Guided Dataset Distillation using Diffusion Models." pith.science (2026). https://pith.science/paper/ISTUHZAG

@misc{pith2026250518963,
  author       = {Pith},
  title        = {Pith review of: MGD$^3$: Mode-Guided Dataset Distillation using Diffusion Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ISTUHZAG}},
  note         = {Machine review of arXiv:2505.18963}
}

read the original abstract

Dataset distillation has emerged as an effective strategy, significantly reducing training costs and facilitating more efficient model deployment. Recent advances have leveraged generative models to distill datasets by capturing the underlying data distribution. Unfortunately, existing methods require model fine-tuning with distillation losses to encourage diversity and representativeness. However, these methods do not guarantee sample diversity, limiting their performance. We propose a mode-guided diffusion model leveraging a pre-trained diffusion model without the need to fine-tune with distillation losses. Our approach addresses dataset diversity in three stages: Mode Discovery to identify distinct data modes, Mode Guidance to enhance intra-class diversity, and Stop Guidance to mitigate artifacts in synthetic samples that affect performance. Our approach outperforms state-of-the-art methods, achieving accuracy gains of 4.4%, 2.9%, 1.6%, and 1.6% on ImageNette, ImageIDC, ImageNet-100, and ImageNet-1K, respectively. Our method eliminates the need for fine-tuning diffusion models with distillation losses, significantly reducing computational costs. Our code is available on the project webpage: https://jachansantiago.github.io/mode-guided-distillation/

Figures

Figures reproduced from arXiv: 2505.18963 by the authors.

**Figure 1.** Optimization-based Dataset Distillation: Optimizes the distilled dataset to match the statistics of gradient/features of the Original Dataset. Generative Dataset Distillation: First, it learns the dataset distribution of the original dataset and then sample a dataset that approximates the original dataset distribution. the research community started to focus on developing techniques to address these limitations. W… view at source ↗

**Figure 2.** Overview of the gradient field (score function) during the denoising process in latent diffusion for a specific class c. The original data distribution, marked by blue dots, shows denser regions (orange shadow) in the gradient field. To generate an image Xˆi, noise xT i ∼ N(0, I) is sampled. In (a), a pre-trained diffusion model demonstrates imbalanced mode likelihood, leading to limited sample diversity and repeate… view at source ↗

**Figure 3.** Overview of the proposed method for distilled dataset synthesis using a diffusion model. Our approach consists of three key stages: Mode Discovery, Mode Guidance, and Stop Guidance. (Left) In the Mode Discovery stage, we estimate the N modes of the original dataset within the generative space of the latent diffusion model. (Right) Given a mode mtarget and a class c, the Mode-Guided Diffusion process directs the gene… view at source ↗

Figures from the paper (8 more)

**Figure 4.** Figure 4: Evaluation results across multiple datasets. (a–c) Accuracy of the Text-to-Image model using the soft-label protocol: (a) Nette dataset, (b) IDC dataset, and (c) ImageNet-1K dataset. (d) ImageNet-1K classification accuracy of the DiT + MGD3 model compared to other stat…

**Figure 5.** Figure 5: T-SNE plot showing the original samples (●) and the synthetic samples generated by different diffusion-based methods for two classes (English springer and cassette player) from ImageNet-1k. This visualization shows that DiT (Peebles & Xie, 2023) has limited diversity,…

**Figure 6.** Figure 6: Representative score versus Diversity score for each class on Nette for IPC 10 versus various models. Mode Guidance with DDIM. Our approach, similar to classifier guidance (Nichol & Dhariwal, 2021), can be incorporated into DDIM using Algorithm 1. In [PITH_FULL_IMAGE…

**Figure 7.** Figure 7: b. This makes tstop = 25 a good balance between generating a diverse set of backgrounds and maintaining class consistency. (a) tstop= 0 tstop= 25 tstop= 50 (b) [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]

**Figure 8.** Figure 8: shows the diversity and representativeness of each distilled sample for ten classes in the ImageNet-1k dataset for DiT, MinMax, and ours. This Figure shows that our method is consistently have higher representativeness across all the classes in comparison to the previo…

**Figure 9.** Figure 9: Representativeness versus Diversity versus tstop. Each point represents a distilled dataset. Diversity and representativeness are obtained by computing the mean across all the samples in the distilled dataset. Stopping the mode guidance early in the reverse process (ts…

**Figure 10.** Figure 10: Effect of guidance scale on performance. G. Diversity Class-Wise diversity score We calculate the diversity score for each class by averaging the diversity score across all the samples [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]

**Figure 11.** Figure 11: Generated images through the denoising process for different values of tstop. 16 [PITH_FULL_IMAGE:figures/full_fig_p016_11.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Self-Improving Diffusion Classifiers with Minority Preference Optimization
cs.CV 2026-07 conditional novelty 6.0 of 10

Fine-tuning a diffusion model with a reconstruction-error minority reward via LoRA+GRPO improves zero-shot diffusion classification by expanding low-density coverage.
GVD: Guiding Video Diffusion Model for Scalable Video Distillation
cs.CV 2025-07 conditional novelty 6.0 of 10

GVD guides a pre-trained video diffusion model with clustering-derived features to distill video datasets, outperforming prior methods on MiniUCF and HMDB51 while retaining over 70% of full-data accuracy using under 4...

Reference graph

Works this paper leans on

14 extracted references · 9 canonical work pages · cited by 2 Pith papers

[4]

Gener- alized large-scale data condensation via various backbone and statistical matching

Shao, S., Yin, Z., Zhou, M., Zhang, X., and Shen, Z. Gener- alized large-scale data condensation via various backbone and statistical matching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16709–16718, 2024a. Shao, S., Zhou, Z., Chen, H., and Shen, Z. Elucidating the design space of dataset condensation. In Adv...

work page 2022
[6]

Wang, T., Zhu, J.-Y ., Torralba, A., and Efros, A. A. Dataset distillation. arXiv preprint arXiv:1811.10959,

arXiv
[7]

Dataset Condensation via Generative Model

Zhang, D. J., Wang, H., Xue, C., Yan, R., Zhang, W., Bai, S., and Shou, M. Z. Dataset condensation via generative model. arXiv preprint arXiv:2309.07698,

work page Pith review arXiv
[8]

All the results are reproduced by us for the 256×256 resolution

Performance comparison with pre-trained diffusion models and other state-of-the-art methods on ImageWoof. All the results are reproduced by us for the 256×256 resolution. The missing results are due to out-of-memory. The best results are marked as bold. Higher is better.Results shown for the previous works are from (Gu et al., 2024). IPC (Ratio) Test Mode...

work page 2024
[9]

R., and Bilen, H

Zhao, B., Mopuri, K. R., and Bilen, H. Dataset conden- sation with gradient matching. In Ninth International Conference on Learning Representations 2021,

work page 2021
[10]

Hierarchical features matter: A deep ex- ploration of gan priors for improved dataset distillation

Zhong, X., Fang, H., Chen, B., Gu, X., Dai, T., Qiu, M., and Xia, S.-T. Hierarchical features matter: A deep ex- ploration of gan priors for improved dataset distillation. arXiv preprint arXiv:2406.05704,

arXiv
[12]

Due to its granularity of features, we trained DiT XL/2 on the ImageWoof dataset with just the simple loss mentioned in Eq

10 (0.8%)ConvNet-624.3±1.1 26.7±0.5 34.2±1.1 26.9±1.2 33.3±1.1 33.8±0.9 37.0±1.0 34.73±1.1 86.4±0.2 ResNetAP-1029.4±0.8 32.0±0.3 34.7±0.5 30.3±1.2 39.1±0.5 32.9±0.9 39.2±1.3 40.4±1.9 87.5±0.5 ResNet-1827.7±0.9 30.2±1.2 34.7±0.4 33.4±0.7 37.3±0.2 31.7±0.8 37.6±0.9 38.5±2.5 89.3±1.2 20 (1.6%)ConvNet-629.1±0.7 29.5±0.3 36.1±0.8 29.9±1.0 35.5±0.8 - 37.6 ±0.2 ...

work page 2024
[13]

Method Labels IPC10 IPC20 IPC50 IPC 100 MGD3 (Ours) Hard-Label 23.6±0.4 32.6±0.4 51.8±0.2 57.8±0.2 Soft-label 34.0±1.0 50.2±0.7 69.2±0.4 75.8±0.3 I

Evaluation of training with hard-labels versus soft labels in ImageNet-100 training with ResNet18. Method Labels IPC10 IPC20 IPC50 IPC 100 MGD3 (Ours) Hard-Label 23.6±0.4 32.6±0.4 51.8±0.2 57.8±0.2 Soft-label 34.0±1.0 50.2±0.7 69.2±0.4 75.8±0.3 I. Evaluation Technical Details For the hard-label protocol, we follow the evaluation method described in (Gu et...

work page 2024

Show all 14 references

[14]

We use a learning rate decay scheduler at the 2/3 and 5/6 points of the training process, with the decay factor (gamma) set to 0.2

We use Stochastic Gradient Descent (SGD) as the optimizer, setting the learning rate at 0.01. We use a learning rate decay scheduler at the 2/3 and 5/6 points of the training process, with the decay factor (gamma) set to 0.2. Cross-entropy was used as the Loss objective. For t...

2024
[2019]

and Salimans, T

Ho, J. and Salimans, T. Classifier-free diffusion guidance. In NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications,

2021
[2021]

and Bilen, H

Zhao, B. and Bilen, H. Synthesizing informative training samples with gan. In NeurIPS 2022 Workshop on Syn- thetic Data for Empowering ML Research,

2022
[2022]

B., Raue, F., Palacio, S., Frolov, S., and Dengel, A

Moser, B. B., Raue, F., Palacio, S., Frolov, S., and Dengel, A. Latent dataset distillation with diffusion models. arXiv preprint arXiv:2403.03881,
[2023]

Rebuffi, S.-A., Kolesnikov, A., Sperl, G., and Lampert, C. H. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 2001–2010,

2001
[2024]

Dim: Distilling dataset into generative model.arXiv preprint arXiv:2303.04707,

Wang, K., Gu, J., Zhou, D., Zhu, Z., Jiang, W., and You, Y . Dim: Distilling dataset into generative model.arXiv preprint arXiv:2303.04707,

arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [4]

Gener- alized large-scale data condensation via various backbone and statistical matching

Shao, S., Yin, Z., Zhou, M., Zhang, X., and Shen, Z. Gener- alized large-scale data condensation via various backbone and statistical matching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16709–16718, 2024a. Shao, S., Zhou, Z., Chen, H., and Shen, Z. Elucidating the design space of dataset condensation. In Adv...

work page 2022

[2] [6]

Wang, T., Zhu, J.-Y ., Torralba, A., and Efros, A. A. Dataset distillation. arXiv preprint arXiv:1811.10959,

arXiv

[3] [7]

Dataset Condensation via Generative Model

Zhang, D. J., Wang, H., Xue, C., Yan, R., Zhang, W., Bai, S., and Shou, M. Z. Dataset condensation via generative model. arXiv preprint arXiv:2309.07698,

work page Pith review arXiv

[4] [8]

All the results are reproduced by us for the 256×256 resolution

Performance comparison with pre-trained diffusion models and other state-of-the-art methods on ImageWoof. All the results are reproduced by us for the 256×256 resolution. The missing results are due to out-of-memory. The best results are marked as bold. Higher is better.Results shown for the previous works are from (Gu et al., 2024). IPC (Ratio) Test Mode...

work page 2024

[5] [9]

R., and Bilen, H

Zhao, B., Mopuri, K. R., and Bilen, H. Dataset conden- sation with gradient matching. In Ninth International Conference on Learning Representations 2021,

work page 2021

[6] [10]

Hierarchical features matter: A deep ex- ploration of gan priors for improved dataset distillation

Zhong, X., Fang, H., Chen, B., Gu, X., Dai, T., Qiu, M., and Xia, S.-T. Hierarchical features matter: A deep ex- ploration of gan priors for improved dataset distillation. arXiv preprint arXiv:2406.05704,

arXiv

[7] [12]

Due to its granularity of features, we trained DiT XL/2 on the ImageWoof dataset with just the simple loss mentioned in Eq

10 (0.8%)ConvNet-624.3±1.1 26.7±0.5 34.2±1.1 26.9±1.2 33.3±1.1 33.8±0.9 37.0±1.0 34.73±1.1 86.4±0.2 ResNetAP-1029.4±0.8 32.0±0.3 34.7±0.5 30.3±1.2 39.1±0.5 32.9±0.9 39.2±1.3 40.4±1.9 87.5±0.5 ResNet-1827.7±0.9 30.2±1.2 34.7±0.4 33.4±0.7 37.3±0.2 31.7±0.8 37.6±0.9 38.5±2.5 89.3±1.2 20 (1.6%)ConvNet-629.1±0.7 29.5±0.3 36.1±0.8 29.9±1.0 35.5±0.8 - 37.6 ±0.2 ...

work page 2024

[8] [13]

Method Labels IPC10 IPC20 IPC50 IPC 100 MGD3 (Ours) Hard-Label 23.6±0.4 32.6±0.4 51.8±0.2 57.8±0.2 Soft-label 34.0±1.0 50.2±0.7 69.2±0.4 75.8±0.3 I

Evaluation of training with hard-labels versus soft labels in ImageNet-100 training with ResNet18. Method Labels IPC10 IPC20 IPC50 IPC 100 MGD3 (Ours) Hard-Label 23.6±0.4 32.6±0.4 51.8±0.2 57.8±0.2 Soft-label 34.0±1.0 50.2±0.7 69.2±0.4 75.8±0.3 I. Evaluation Technical Details For the hard-label protocol, we follow the evaluation method described in (Gu et...

work page 2024