Pith. sign in

REVIEW 4 major objections 4 minor 98 references

Data-to-Model Distillation: Data-Efficient Learning Framework

T0 review · 4 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper proposes Data-to-Model Distillation (D2M), which distills a real dataset's knowledge into the parameters of a pre-trained generative model rather than into raw pixels, so one distilled generator can supply informative training…

desk verdict Genuinely new distillation target and impressive scaling, but the GAN provenance and latency-matched protocol must be clarified before the SOTA claim can be taken at face value. read the letter →

arxiv 2411.12841 v1 pith:FIGYC3YO submitted 2024-11-19 cs.CV cs.LG

Ahmad Sajedi , Samir Khaki , Lucy Z. Liu , Ehsan Amjadian , Yuri A. Lawryshyn , Konstantinos N. Plataniotis This is my paper · ORCID
classification cs.CVcs.LG
keywords datasetdistillationgenerativemodeldata-efficientlearningsyntheticdataimageclassificationneuralarchitecturesearchcross-architecturegeneralizationembeddingmatching
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Dataset distillation normally squeezes a large training set into a handful of synthetic pixel images. This paper argues that the right container for the distilled knowledge is not pixels but the parameter space of a pre-trained generative model: after a short refinement, the generator itself becomes the distilled dataset and can emit informative training images at any desired number per class. The authors show on 15 datasets that this data-to-model approach outperforms pixel-space distillation, transfers to architectures it never saw during distillation, needs no re-distillation when the image budget changes, and scales to 128x128 ImageNet-1K and 256x256 medical images. If correct, it turns dataset distillation from a storage problem into a model problem, with a single fixed-size artifact replacing many synthetic-image sets.

What carries the argument

The central object is a pre-trained generative model $G$ (BigGAN by default) serving as the distillation container; the identity is the two-term objective $G^* = \arg\min_G \mathcal{L}_{EM} + \lambda \mathcal{L}_{PM}$. $\mathcal{L}_{EM}$ aligns vectorized channel-wise attention maps between real and generated images across $L$ feature layers of a sampled network, while $\mathcal{L}_{PM}$ minimizes KL divergence between softened class predictions for real and generated pairs with the same label. A model pool of randomly initialized ConvNets and ResNets supplies the networks $\theta$, preventing overfitting to one architecture. This machinery moves the synthetic dataset out of pixel space into a fixed-size parameter vector, which is what makes re-distillation-free IPC changes and constant storage possible.

What would settle it

Distill with a generator that was pretrained without access to the target dataset's classes (for example, a BigGAN trained only on a disjoint subset of ImageNet, evaluated on CIFAR-10 or TinyImageNet classes), then measure IPC-10 accuracy; if it collapses toward random while the paper's numbers stand, the prior full-data pretraining of the generator is the source of the distilled knowledge.

Watch

Extended reading notes

Core claim

The central claim is that a generator refined with two matching losses—embedding matching on channel-wise attention maps and prediction matching on softened logits—captures enough of a real dataset's training signal that random draws from it outperform the best pixel-optimized synthetic sets. The learned generator solves $G^* = \arg\min_G \mathcal{L}_{EM} + \lambda \mathcal{L}_{PM}$, where the expectation over a pool of randomly initialized networks provides diverse supervision. Once distilled, one generator produces training images for any number of images per class, so re-distillation cost is zero and storage is constant; the paper reports state-of-the-art accuracy on CIFAR-10/100, TinyImageNet, ImageNet-1K and its subsets, and DermaMNIST, plus cross-architecture gains that reach 63.6% average on CIFAR-10 at IPC50.

Load-bearing premise

The method assumes a pre-trained class-conditional generator is already available for the target dataset's classes and resolution; if that generator was trained on the full dataset, the expensive pretraining has already encoded the data the method claims to distill.

Editorial extensions

If this is right

  • One distillation run serves all image-per-class settings; re-distilling from IPC1 to IPC50 costs 0 GPU hours, saving up to 42.5 GPU hours compared with pixel-space methods on CIFAR-10.
  • Storage and learnable-parameter count stay constant as IPC grows, up to roughly 35x fewer parameters than pixel-space distillation at IPC50.
  • The same distilled generator transfers to unseen architectures (AlexNet, VGG-11, ResNet-50, DenseNet-121, ViT), averaging 63.6% on CIFAR-10 with IPC50, 3.9 points above the best prior method.
  • D2M scales to 128x128 ImageNet-1K and 256x256 datasets such as ImageSquawk and DermaMNIST, where pixel-space distillation methods struggle.
  • Distilled images rank architectures for neural architecture search with a Spearman correlation of 0.80, approaching the whole-dataset ranking of 1.00.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the generator was pretrained on the full target dataset, the reported distillation cost excludes the dominant pretraining cost; the fair comparison would amortize BigGAN training or use a generator with no target-data exposure.
  • The same parameter-space container could work with diffusion backbones: the paper's single SD-XL experiment on ImageNette (52.1% at IPC1) hints the framework is not GAN-specific, but the authors do not develop this direction.
  • The evaluation protocol matches total train-plus-generate time rather than exact image counts (Appendix B), so headline IPC numbers are not directly comparable to fixed-IPC baselines without re-basing.
  • Because the generator can emit unlimited images, D2M may enable privacy-focused or continual-learning settings where storing pixels is undesirable, but memorization of training images would need to be tested.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes Data-to-Model Distillation (D2M), a framework that distills a large training dataset into the parameters of a pre-trained generative model (default: BigGAN) by matching channel attention maps and softened logits between real and generated images under a randomly sampled network from a model pool. After distillation, the generator can produce training images for arbitrary images-per-class (IPC) settings without re-distillation, and is claimed to achieve state-of-the-art results across 15 datasets, including 128×128 ImageNet-1K, while maintaining constant storage complexity. The paper also reports re-distillation efficiency, cross-architecture generalization, and a neural architecture search application.

Significance. If the central assumptions are clarified and the evaluation is made directly comparable, D2M is a conceptually interesting step: replacing pixel-space synthetic data with a tunable generator could genuinely address the re-distillation and scalability limitations of prior dataset distillation methods. The paper's strengths include extensive experiments on many datasets, a clear and reproducible algorithmic skeleton (Algorithm 1), public code, and ablations over temperature, task balance, batch size, and generator backbones. The cross-architecture generalization results (Table 5) and the NAS correlation experiment (Table 8) are valuable empirical contributions. However, the significance of the main efficiency and performance claims hinges on the provenance and cost of the pre-trained generator, which the manuscript leaves underspecified.

major comments (4)
  1. [Section 3.1, Eq. (1); Section 4.1; Table 4] The pre-trained generative model G is never specified for the non-ImageNet target datasets. BigGAN is a 1000-class ImageNet generator, while CIFAR-10/100 (10/100 classes, 32×32), TinyImageNet (200 classes), and DermaMNIST (7-class medical images) are outside its native class and resolution range. The manuscript only states 'we deploy the pre-trained BigGAN with the default hyperparameters' (Section 4.1). If G was trained on the full target dataset before distillation, then its parameters already encode the target classes and statistics, and the reported distillation cost in Table 4 (4.9 GPU hours total for CIFAR-10) omits the dominant pretraining expense. If instead an off-the-shelf ImageNet checkpoint is used without target-domain adaptation, the mechanics of class-conditional generation for 100 or 200 classes are unexplained. This is load-bearing for the central claim of distilling a dataset into a generative model and for the efficiency comparison against pixel-space distillation; the paper must state how G was obtained for each dataset, whether it was trained or fine-tuned on the full target data, and report the corresponding cost.
  2. [Section 4.1; Supplementary Section B] The evaluation protocol uses a wall-clock latency budget rather than a fixed image count. In Section 4.1, the authors state that they measure the end-to-end latency of training a model on IPC images and then divide this time between generating images and training for D2M. Supplementary Section B confirms that the 'raw amount of images is determined by the fixed time ratio per configuration.' This means the effective number of training images seen by a D2M-trained model may differ from the IPC value reported in Tables 1–3 and 6. Since all baselines are evaluated by training on exactly IPC pre-computed images, the 'superior performance' claim is not yet established under a strictly matched data budget. The paper should report the actual number of generated images used for each IPC and either match the image count or explicitly justify why matching latency, rather than image count, is the appropriate fairness criterion.
  3. [Section 4.3 vs. Section 4.1 and Supplementary Table 11] The default value of the task-balance parameter λ is inconsistent across the manuscript. Section 4.1 states 'setting the task balance λ and the temperature T to 100 and 4, respectively,' while Section 4.3 states 'Our default value of λ = 10 yields the best results,' and Supplementary Table 11 lists the default for λ as 100. Since the reported results in Tables 1–3 depend on the actual λ used, this inconsistency undermines reproducibility of all reported numbers.
  4. [Section 4.1; Section 5] The model pool used for feature matching includes Depth-n ConvNets, which are exactly the architectures used for evaluation in Tables 1–3. Because D2M optimizes the generator to match features extracted by networks randomly drawn from this pool, part of the performance on the ConvNet evaluation may be attributed to matching the evaluation architecture rather than to generalizable distilled knowledge. The cross-architecture experiments in Table 5 partially address this concern, but the primary comparisons on the seen architecture remain partially circular; the paper should quantify this effect, for example by evaluating on a ConvNet variant that is excluded from the model pool.
minor comments (4)
  1. [Section 4.4 and Supplementary D.1] The heading 'Visulization' contains a typo, and 'ImegeNette' in Supplementary D.1 (Diffusion Model Analysis) should be 'ImageNette'.
  2. [Supplementary Table 9] The dataset statistics table has mismatched rows: the row labeled 'ImageNet-1K (128 × 128)' lists resolution '64 × 64', and the row labeled 'ImageNet-1K Subsets (256×256)' lists resolution '128 × 128'. These are likely typographical errors and should be corrected to match the experimental setup described in Section 4.1.
  3. [Section 3, Algorithm 1] Algorithm 1 lists 'Pre-trained generative model G' as an input but does not mention the discriminator D, even though Eq. (1) in Section 3.1 describes training with a discriminator. Clarify whether D is used in the distillation stage or only in the pretraining stage.
  4. [Section 4.2, Table 2] The claim of state-of-the-art performance on 128×128 ImageNet-1K is supported only by a comparison to Random and Figure 3, which shows no other baseline results. Please provide numerical comparisons to any scalable baselines or explicitly state that no prior method reports results at this resolution.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the D2M optimization is an independent matching-loss refinement, and the pretrained-generator provenance is a cost/credit concern rather than a derivation loop.

full rationale

The derivation chain is self-contained and no step reduces to its own input by construction. The core optimization (Eq. 5) minimizes two matching losses (Eq. 2 and Eq. 4) over generator parameters, and the reported downstream accuracies are empirical results of that optimization, not fitted values or algebraic identities. The use of a model pool (Sec. 4.1) includes the evaluation ConvNet, but this is standard practice in dataset distillation, and the paper separately demonstrates cross-architecture generalization on AlexNet, VGG-11, ResNet-50, DenseNet-121, and ViT (Table 5), so the central comparisons do not reduce to a single fitted architecture. The pretrained-GAN input (Eq. 1) is a legitimate concern about provenance and cost accounting: Eq. 1 trains G on the real distribution P_T, so the final generator inherits information from the full dataset before the matching stage, and Table 4's 4.9 GPU hours excludes pretraining. This weakens the novelty and efficiency framing, but it is not a circular derivation, because the matching losses add measurable information (the ablation in Sec. 4.3 reports a 12.7% improvement over the base generator) and the paper does not define the output as equivalent to the pretrained input. Self-citations to DataDAM [64] and ATOM [36] are methodological precedents and baseline comparisons, not load-bearing uniqueness arguments. Appendix B's evaluation-protocol note is a transparency limitation about how IPC time is allocated, not a circular step. Overall, no prediction or result is equivalent by construction to its inputs.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The method uses two tuned loss weights (lambda, T) plus standard optimization hyperparameters; it relies on a pretrained GAN and a model pool whose provenance and training costs are partially unspecified. No new physical or conceptual entities are introduced.

free parameters (5)
  • lambda (task balance) = 100 (main text Sec. 4.1 and Table 11) or 10 (Sec. 4.3, inconsistent)
    Weights the prediction matching loss relative to embedding matching; ablated in Fig. 6(c) on CIFAR-10 IPC10.
  • temperature T = 4
    Controls softness of KL-based prediction matching; ablated in Fig. 6(a); default 4 in Sec. 4.1.
  • batch size B = 128
    Number of real/generated pairs per distillation iteration; ablated in Fig. 6(b).
  • generator learning rate = 1e-6
    SGD step size for the generator in the distillation stage; Table 11.
  • distillation epochs K = 60
    Number of generator update epochs; Table 11.
assumptions (4)
  • domain assumption A pre-trained class-conditional GAN (BigGAN) is available and can be refined for each target dataset.
    Invoked in Sec. 3.1 Eq. (1); no adaptation details for CIFAR-10/100 or TinyImageNet are provided.
  • domain assumption The model pool networks provide informative and diverse features; their training procedure is assumed rather than specified.
    Sec. 4.1 lists pool architectures and seeds but does not state whether the pool models are trained on real data or are randomly initialized.
  • standard math Mini-batch Monte Carlo estimates of the expectations in Eqs. (2) and (4) are valid surrogates for the true distribution matching objective.
    The method optimizes the generator against sampled batches and a sampled network; this is standard but an approximation.
  • domain assumption Matching wall-clock latency is a fair comparison protocol against fixed-IPC baselines.
    Sec. 4.1 and Appendix B define the evaluation by time budget rather than exact image count; the authors argue fairness, but the effective IPC differs.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Data-to-Model Distillation: Data-Efficient Learning Framework." pith.science (2026). https://pith.science/paper/FIGYC3YO

@misc{pith2026241112841,
  author       = {Pith},
  title        = {Pith review of: Data-to-Model Distillation: Data-Efficient Learning Framework},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FIGYC3YO}},
  note         = {Machine review of arXiv:2411.12841}
}
read the original abstract

Dataset distillation aims to distill the knowledge of a large-scale real dataset into small yet informative synthetic data such that a model trained on it performs as well as a model trained on the full dataset. Despite recent progress, existing dataset distillation methods often struggle with computational efficiency, scalability to complex high-resolution datasets, and generalizability to deep architectures. These approaches typically require retraining when the distillation ratio changes, as knowledge is embedded in raw pixels. In this paper, we propose a novel framework called Data-to-Model Distillation (D2M) to distill the real dataset's knowledge into the learnable parameters of a pre-trained generative model by aligning rich representations extracted from real and generated images. The learned generative model can then produce informative training images for different distillation ratios and deep architectures. Extensive experiments on 15 datasets of varying resolutions show D2M's superior performance, re-distillation efficiency, and cross-architecture generalizability. Our method effectively scales up to high-resolution 128x128 ImageNet-1K. Furthermore, we verify D2M's practical benefits for downstream applications in neural architecture search.

Figures

Figures reproduced from arXiv: 2411.12841 by the authors.

Figure 1
Figure 1. Different distillation frameworks for efficient learning. particular, transfers the informative knowledge from a large teacher model to a smaller student one to facilitate model compression ( [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. An overview of the proposed D2M framework. D2M distills the knowledge of large￾scale datasets into the parameter space of a pre-trained generator through embedding matching and prediction matching modules. The learned generator can then produce small yet informative training images for the downstream classification tasks. Z and Y represent the random noises and labels, respectively. 3.1 Data-to-Model Distillation Th… view at source ↗
Figure 3
Figure 3. Performance comparison and count of parameters on 128×128 ImageNet-1K. Distillation Cost Analysis. In efficient learning, it is critical to consider computational costs dur￾ing the distillation. We compare the re-distillation cost and the num￾ber of learnable parameters for D2M against priors in Tab. 4 and [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figures from the paper (35 more)
Figure 4
Figure 4. Figure 4: The effect of (a) temperature, (b) batch size, and (c) task balance on the D2M’s performance for CIFAR10 with IPC10. Exploring the effect of loss components in D2M. We conduct an ex￾periment to evaluate the effect of loss components, LEM and LPM, on D2M’s per￾formance.…
Figure 5
Figure 5. Figure 5: Example generated images from 32×32 CIFAR-100, 64×64 Tiny ImageNet, 128×128 ImageNet-1K, and 128×128 ImageSquawk [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: The effect of (a) temperature, (b) batch size, and (c) task balance on the D2M’s performance for CIFAR10 with IPC10. making it difficult to use in all our experiments. However, we conducted exper￾iments on CIFAR-10 IPC1, comparing three configurations of our distillati…
Figure 7
Figure 7. Figure 7: The GPU hours of the distillation time for FTD and D2M on CIFAR-10 with different IPCs. Computational costs of different generative models in D2M. To en￾hance the computational efficiency of D2M when applied to less complex datasets, we conducted experiments using CIFA…
Figure 8
Figure 8. Figure 8: Comparative analysis of the number of learnable parameters on 32×32 resolu￾tion CIFAR-100 with different IPCs [PITH_FULL_IMAGE:figures/full_fig_p028_8.png]
Figure 9
Figure 9. Figure 9: Number of learnable parameters for different generative models on 32×32 res￾olution CIFAR-10. erative model across all experiments. This choice was made to strike a balance between performance and computational costs, particularly when dealing with extensive and high-r…
Figure 10
Figure 10. Figure 10: Performance rank correlation between proxy-set and whole dataset training across all 720 architectures [PITH_FULL_IMAGE:figures/full_fig_p029_10.png]
Figure 11
Figure 11. Figure 11: Distributions of the synthetic images learned by six methods on the CIFAR10 dataset with IPC 50. The stars represent the synthetic data dispersed amongst the original dataset. The classes are as follows: plane, car, bird, cat, deer, dog, frog, horse, ship, truck. E Ad…
Figure 12
Figure 12. Figure 12: D2M-generated image visualization: ImageSquawk (256×256): (Top) ’ostrich’ class, (Bottom) ’eagle’ class [PITH_FULL_IMAGE:figures/full_fig_p032_12.png]
Figure 13
Figure 13. Figure 13: D2M-generated image visualization: ImageSquawk (256×256): (Top) ’pea￾cock’ class, (Bottom) ’macaw’ class [PITH_FULL_IMAGE:figures/full_fig_p033_13.png]
Figure 14
Figure 14. Figure 14: D2M-generated image visualization: ImageSquawk (256×256): (Top) ’cocka￾too’ class, (Bottom) ’toucan’ class [PITH_FULL_IMAGE:figures/full_fig_p034_14.png]
Figure 15
Figure 15. Figure 15: D2M-generated image visualization: ImageSquawk (256×256): (Top) ’black swan’ class, (Bottom) ’flamingo’ class [PITH_FULL_IMAGE:figures/full_fig_p035_15.png]
Figure 16
Figure 16. Figure 16: D2M-generated image visualization: ImageNet-1K (256×256): (Top) ’pelican’ class, (Bottom) ’penguin’ class [PITH_FULL_IMAGE:figures/full_fig_p036_16.png]
Figure 17
Figure 17. Figure 17: D2M-generated image visualization: DermaMNIST (256×256) 7 classes. Class names under respective columns [PITH_FULL_IMAGE:figures/full_fig_p036_17.png]
Figure 18
Figure 18. Figure 18: D2M-generated image visualization (IPC10) for the CIFAR-10 dataset with different temperatures (T) during distillation [PITH_FULL_IMAGE:figures/full_fig_p037_18.png]
Figure 19
Figure 19. Figure 19: D2M-generated image visualization (IPC10) for the CIFAR-10 dataset with different generative models. (a) CIFAR-10 (IPC10) (b) CIFAR-100 (IPC1) [PITH_FULL_IMAGE:figures/full_fig_p038_19.png]
Figure 20
Figure 20. Figure 20: D2M-generated image visualization: CIFAR datasets [PITH_FULL_IMAGE:figures/full_fig_p038_20.png]
Figure 21
Figure 21. Figure 21: D2M-generated image visualization: Tiny ImageNet (64×64) with IPC1 [PITH_FULL_IMAGE:figures/full_fig_p039_21.png]
Figure 22
Figure 22. Figure 22: D2M-generated image visualization: ImageNet-1K (64×64) Classes 0-500 [PITH_FULL_IMAGE:figures/full_fig_p040_22.png]
Figure 23
Figure 23. Figure 23: D2M-generated image visualization: ImageNet-1K (64×64) Classes 500-1000 [PITH_FULL_IMAGE:figures/full_fig_p041_23.png]
Figure 24
Figure 24. Figure 24: D2M-generated image visualization: ImageNet-1K (128×128) Classes 0-200 [PITH_FULL_IMAGE:figures/full_fig_p042_24.png]
Figure 25
Figure 25. Figure 25: D2M-generated image visualization: ImageNet-1K (128×128) Classes 200-400 [PITH_FULL_IMAGE:figures/full_fig_p043_25.png]
Figure 26
Figure 26. Figure 26: D2M-generated image visualization: ImageNet-1K (128×128) Classes 400-600 [PITH_FULL_IMAGE:figures/full_fig_p044_26.png]
Figure 27
Figure 27. Figure 27: D2M-generated image visualization: ImageNet-1K (128×128) Classes 600-800 [PITH_FULL_IMAGE:figures/full_fig_p045_27.png]
Figure 28
Figure 28. Figure 28: D2M-generated image visualization: ImageNet-1K (128×128) Classes 800- 1000 [PITH_FULL_IMAGE:figures/full_fig_p046_28.png]
Figure 29
Figure 29. Figure 29: D2M-generated image visualization: ImageNette Subset (128×128) with IPC1 [PITH_FULL_IMAGE:figures/full_fig_p047_29.png]
Figure 30
Figure 30. Figure 30: D2M-generated image visualization: ImageWoof Subset (128×128) with IPC1 [PITH_FULL_IMAGE:figures/full_fig_p048_30.png]
Figure 31
Figure 31. Figure 31: D2M-generated image visualization: ImageSquawk Subset (128×128) with IPC1 [PITH_FULL_IMAGE:figures/full_fig_p049_31.png]
Figure 32
Figure 32. Figure 32: D2M-generated image visualization: ImageFruit Subset (128×128) with IPC1 [PITH_FULL_IMAGE:figures/full_fig_p050_32.png]
Figure 33
Figure 33. Figure 33: D2M-generated image visualization: ImageMeow Subset (128×128) with IPC1 [PITH_FULL_IMAGE:figures/full_fig_p051_33.png]
Figure 34
Figure 34. Figure 34: D2M-generated image visualization: ImageNet-A Subset (128×128) with IPC1 [PITH_FULL_IMAGE:figures/full_fig_p052_34.png]
Figure 35
Figure 35. Figure 35: D2M-generated image visualization: ImageNet-B Subset (128×128) with IPC1 [PITH_FULL_IMAGE:figures/full_fig_p053_35.png]
Figure 36
Figure 36. Figure 36: D2M-generated image visualization: ImageNet-C Subset (128×128) with IPC1 [PITH_FULL_IMAGE:figures/full_fig_p054_36.png]
Figure 37
Figure 37. Figure 37: D2M-generated image visualization: ImageNet-D Subset (128×128) with IPC1 [PITH_FULL_IMAGE:figures/full_fig_p055_37.png]
Figure 38
Figure 38. Figure 38: D2M-generated image visualization: ImageNet-E Subset (128×128) with IPC1 [PITH_FULL_IMAGE:figures/full_fig_p056_38.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

98 extracted references · 40 canonical work pages

  1. [1]

    High Performance Convolution Using Sparsity and Patterns for Inference in Deep Convolutional Neural Networks

    Amer, H., Salamah, A.H., Sajedi, A., Yang, E.h.: High performance convolution using sparsity and patterns for inference in deep convolutional neural networks. arXiv preprint arXiv:2104.08314 (2021)

    work page Pith review arXiv 2021

  2. [2]

    In: Proceedings of the IEEE/CVF winter conference on applications of computer vision

    Belouadah, E., Popescu, A.: Scail: Classifier weights scaling for class incremental learning. In: Proceedings of the IEEE/CVF winter conference on applications of computer vision. pp. 1266–1275 (2020)

    2020

  3. [3]

    In: International Conference on Learning Representations (2018)

    Brock, A., Donahue, J., Simonyan, K.: Large scale gan training for high fidelity natural image synthesis. In: International Conference on Learning Representations (2018)

    2018

  4. [4]

    Advances in neural information processing systems33, 1877–1901 (2020)

    Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J.D., Dhariwal, P., Nee- lakantan, A., Shyam, P., Sastry, G., Askell, A., et al.: Language models are few-shot learners. Advances in neural information processing systems33, 1877–1901 (2020)

    2020

  5. [5]

    In: Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition

    Cazenavette, G., Wang, T., Torralba, A., Efros, A.A., Zhu, J.Y.: Dataset distilla- tion by matching training trajectories. In: Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition. pp. 4750–4759 (2022)

    2022

  6. [6]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Cazenavette, G., Wang, T., Torralba, A., Efros, A.A., Zhu, J.Y.: Generalizing dataset distillation via deep generative prior. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 3739–3748 (2023)

    2023

  7. [7]

    Advances in Neural Information Processing Systems35, 14678–14690 (2022)

    Chen, D., Kerkouche, R., Fritz, M.: Private set generation with discriminative information. Advances in Neural Information Processing Systems35, 14678–14690 (2022)

    2022

  8. [8]

    In:ProceedingsoftheIEEEconferenceoncomputervisionandpatternrecognition

    Chen, L., Zhang, H., Xiao, J., Nie, L., Shao, J., Liu, W., Chua, T.S.: Sca-cnn: Spatial and channel-wise attention in convolutional networks for image captioning. In:ProceedingsoftheIEEEconferenceoncomputervisionandpatternrecognition. pp. 5659–5667 (2017)

    2017

Show all 98 references
  1. [9]

    Advances in neural information processing systems29 (2016)

    Chen, X., Duan, Y., Houthooft, R., Schulman, J., Sutskever, I., Abbeel, P.: Info- gan: Interpretable representation learning by information maximizing generative adversarial nets. Advances in neural information processing systems29 (2016)

    2016

  2. [10]

    arXiv preprint arXiv:2310.06982 (2023)

    Chen, X., Yang, Y., Wang, Z., Mirzasoleiman, B.: Data distillation can be like vodka: Distilling more times for better quality. arXiv preprint arXiv:2310.06982 (2023)

    2023 arXiv

  3. [11]

    In: International Conference on Learning Representations (2021)

    Child, R.: Very deep vaes generalize autoregressive models and can outperform them on images. In: International Conference on Learning Representations (2021)

    2021

  4. [12]

    In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision

    Ciftci, U.A., Yuksek, G., Demir, I.: My face my choice: Privacy enhancing deep- fakes for social media anonymization. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. pp. 1369–1379 (2023)

    2023

  5. [13]

    In: International Conference on Machine Learning

    Cui, J., Wang, R., Si, S., Hsieh, C.J.: Scaling up dataset distillation to imagenet- 1k with constant memory. In: International Conference on Machine Learning. pp. 6565–6590. PMLR (2023)

    2023

  6. [14]

    In: 2009 IEEE conference on computer vision and pattern recognition

    Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: A large- scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition. pp. 248–255. Ieee (2009)

    2009

  7. [15]

    Dong, T., Zhao, B., Lyu, L.: Privacy for free: How does dataset condensation help privacy? In: International Conference on Machine Learning. pp. 5378–5396. PMLR (2022)

    2022

  8. [16]

    arXiv preprint arXiv:2010.11929 (2020) 16 Ahmad Sajedi *, Samir Khaki* et al

    Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020) 16 Ahmad Sajed...

    2020 arXiv

  9. [17]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Du, J., Jiang, Y., Tan, V.Y., Zhou, J.T., Li, H.: Minimizing the accumulated trajectory error to improve dataset distillation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 3749–3758 (2023)

    2023

  10. [18]

    Advances in Neural Information Processing Systems36 (2024)

    Du, J., Shi, Q., Zhou, J.T.: Sequential subset matching for dataset distillation. Advances in Neural Information Processing Systems36 (2024)

    2024

  11. [19]

    In: International Conference on Machine Learning

    Furlanello, T., Lipton, Z., Tschannen, M., Itti, L., Anandkumar, A.: Born again neural networks. In: International Conference on Machine Learning. pp. 1607–1616. PMLR (2018)

    2018

  12. [20]

    In:ProceedingsoftheIEEEconferenceoncomputervisionandpatternrecognition

    Gidaris, S., Komodakis, N.: Dynamic few-shot visual learning without forgetting. In:ProceedingsoftheIEEEconferenceoncomputervisionandpatternrecognition. pp. 4367–4375 (2018)

    2018

  13. [21]

    Advances in neural infor- mation processing systems27 (2014)

    Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. Advances in neural infor- mation processing systems27 (2014)

    2014

  14. [22]

    In: Proceedings of the AAAI Conference on Artificial Intelligence (AAAI) (2024)

    Gu, J., Wang, K., Jiang, W., You, Y.: Summarizing stream data for memory- restricted online continual learning. In: Proceedings of the AAAI Conference on Artificial Intelligence (AAAI) (2024)

    2024

  15. [23]

    In: The Twelfth In- ternational Conference on Learning Representations (2024),https://openreview

    Guo, Z., Wang, K., Cazenavette, G., LI, H., Zhang, K., You, Y.: Towards lossless dataset distillation via difficulty-aligned trajectory matching. In: The Twelfth In- ternational Conference on Learning Representations (2024),https://openreview. net/forum?id=rTBL8OhdhH

    2024

  16. [24]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    He, J., Shi, W., Chen, K., Fu, L., Dong, C.: Gcfsr: a generative and controllable face super resolution method without facial and gan priors. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 1889– 1898 (2022)

    2022

  17. [25]

    In: Proceedings of the IEEE interna- tional conference on computer vision

    He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: Surpassing human- level performance on imagenet classification. In: Proceedings of the IEEE interna- tional conference on computer vision. pp. 1026–1034 (2015)

    2015

  18. [26]

    He,K.,Zhang,X.,Ren,S.,Sun,J.:Deepresiduallearningforimagerecognition.In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770–778 (2016)

    2016

  19. [27]

    In: Proceedings of the Advances in Neural Information Processing Systems (NeurIPS) (2023)

    He, Y., Xiao, L., Zhou, T.J.: You Only Condense Once: Two rules for pruning con- densed datasets. In: Proceedings of the Advances in Neural Information Processing Systems (NeurIPS) (2023)

    2023

  20. [28]

    arXiv preprint arXiv:1503.02531 (2015)

    Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015)

    2015 arXiv

  21. [29]

    Advances in neural information processing systems29 (2016)

    Ho, J., Ermon, S.: Generative adversarial imitation learning. Advances in neural information processing systems29 (2016)

    2016

  22. [30]

    Howard, J.: Imagenette: A smaller subset of 10 easily classified classes from ima- genet, and a little more french (2019)

    2019

  23. [31]

    In: Proceedings of the IEEE conference on computer vision and pattern recognition

    Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 4700–4708 (2017)

    2017

  24. [32]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Jin, Y., Wang, J., Lin, D.: Multi-level logit distillation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 24276– 24285 (2023)

    2023

  25. [33]

    In: Proceedings of NAACL-HLT

    Kenton, J.D.M.W.C., Toutanova, L.K.: Bert: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of NAACL-HLT. pp. 4171–4186 (2019) Data-to-Model Distillation: Data-Efficient Learning Framework 17

    2019

  26. [34]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Khaki, S., Luo, W.: Cfdp: Common frequency domain pruning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 4714–4723 (2023)

    2023

  27. [35]

    arXiv preprint arXiv:2403.17921 (2024)

    Khaki, S., Plataniotis, K.N.: The need for speed: Pruning transformers with one recipe. arXiv preprint arXiv:2403.17921 (2024)

    2024 arXiv

  28. [36]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Khaki, S., Sajedi, A., Wang, K., Liu, L.Z., Lawryshyn, Y.A., Plataniotis, K.N.: Atom: Attention mixer for efficient dataset distillation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 7692– 7702 (2024)

    2024

  29. [37]

    In: Interna- tional Conference on Machine Learning

    Kim, J.H., Kim, J., Oh, S.J., Yun, S., Song, H., Jeong, J., Ha, J.W., Song, H.O.: Dataset condensation via efficient synthetic-data parameterization. In: Interna- tional Conference on Machine Learning. pp. 11102–11118. PMLR (2022)

    2022

  30. [38]

    Krizhevsky, A., Hinton, G., et al.: Learning multiple layers of features from tiny images (2009)

    2009

  31. [39]

    Communications of the ACM60(6), 84–90 (2017)

    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep con- volutional neural networks. Communications of the ACM60(6), 84–90 (2017)

    2017

  32. [40]

    CS 231N7(7), 3 (2015)

    Le, Y., Yang, X.: Tiny imagenet visual recognition challenge. CS 231N7(7), 3 (2015)

    2015

  33. [41]

    In: Proceedings of the IEEE conference on computer vision and pattern recognition

    Ledig, C., Theis, L., Huszár, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z., et al.: Photo-realistic single image super- resolution using a generative adversarial network. In: Proceedings of the IEEE conference on computer vision and...

    2017

  34. [42]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Lee, C.H., Liu, Z., Wu, L., Luo, P.: Maskgan: Towards diverse and interactive facial image manipulation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 5549–5558 (2020)

    2020

  35. [43]

    In: International Conference on Machine Learning

    Lee, S., Chun, S., Jung, S., Yun, S., Yoon, S.: Dataset condensation with con- trastive signals. In: International Conference on Machine Learning. pp. 12352– 12364. PMLR (2022)

    2022

  36. [44]

    In: European Conference on Computer Vision

    Li, C., Lin, M., Ding, Z., Lin, N., Zhuang, Y., Huang, Y., Ding, X., Cao, L.: Knowl- edge condensation distillation. In: European Conference on Computer Vision. pp. 19–35. Springer (2022)

    2022

  37. [45]

    In: International Conference on Learning Representations (2017)

    Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. In: International Conference on Learning Representations (2017)

    2017

  38. [46]

    In: Proceedings of the IEEE conference on computer vision and pattern recognition

    Li, J., Liang, X., Wei, Y., Xu, T., Feng, J., Yan, S.: Perceptual generative adver- sarial networks for small object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 1222–1230 (2017)

    2017

  39. [47]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Liu, L., Muelly, M., Deng, J., Pfister, T., Li, L.J.: Generative modeling for small- data object detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 6073–6081 (2019)

    2019

  40. [48]

    In: The Eleventh International Conference on Learning Representations (2022)

    Liu, P., Yu, X., Zhou, J.T.: Meta knowledge condensation for federated learning. In: The Eleventh International Conference on Learning Representations (2022)

    2022

  41. [49]

    Advances in Neural Information Processing Systems35, 1100–1113 (2022)

    Liu, S., Wang, K., Yang, X., Ye, J., Wang, X.: Dataset distillation via factorization. Advances in Neural Information Processing Systems35, 1100–1113 (2022)

    2022

  42. [50]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Liu, S., Ye, J., Yu, R., Wang, X.: Slimmable dataset condensation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 3759–3768 (2023)

    2023

  43. [51]

    arXiv preprint arXiv:2302.14416 (2023)

    Liu, Y., Gu, J., Wang, K., Zhu, Z., Jiang, W., You, Y.: Dream: Efficient dataset distillation by representative matching. arXiv preprint arXiv:2302.14416 (2023)

    2023 arXiv

  44. [52]

    Advances in Neural Information Processing Systems35, 13877–13891 (2022) 18 Ahmad Sajedi *, Samir Khaki* et al

    Loo, N., Hasani, R., Amini, A., Rus, D.: Efficient dataset distillation using random feature approximation. Advances in Neural Information Processing Systems35, 13877–13891 (2022) 18 Ahmad Sajedi *, Samir Khaki* et al

    2022

  45. [53]

    Loo,N.,Hasani,R.,Lechner,M.,Amini,A.,Rus,D.:Understandingreconstruction attacks with the neural tangent kernel and dataset distillation (2023)

    2023

  46. [54]

    In: Proceedings of the IEEE international conference on computer vision

    Ma, C., Huang, J.B., Yang, X., Yang, M.H.: Hierarchical convolutional features for visual tracking. In: Proceedings of the IEEE international conference on computer vision. pp. 3074–3082 (2015)

    2015

  47. [55]

    Journal of machine learning research 9(11) (2008)

    Van der Maaten, L., Hinton, G.: Visualizing data using t-sne. Journal of machine learning research 9(11) (2008)

    2008

  48. [56]

    arXiv preprint arXiv:1411.1784 (2014)

    Mirza, M., Osindero, S.: Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784 (2014)

    2014 arXiv

  49. [57]

    In: International Conference on Learning Representations (2019)

    Molchanov, P., Tyree, S., Karras, T., Aila, T., Kautz, J.: Pruning convolutional neural networks for resource efficient inference. In: International Conference on Learning Representations (2019)

    2019

  50. [58]

    In: International Conference on Learning Representations (2021), https: //openreview.net/forum?id=l-PrrQrK0QR

    Nguyen, T., Chen, Z., Lee, J.: Dataset meta-learning from kernel-ridge regres- sion. In: International Conference on Learning Representations (2021), https: //openreview.net/forum?id=l-PrrQrK0QR

    2021

  51. [59]

    Advances in Neural Information Processing Systems34, 5186–5198 (2021)

    Nguyen, T., Novak, R., Xiao, L., Lee, J.: Dataset distillation with infinitely wide convolutional networks. Advances in Neural Information Processing Systems34, 5186–5198 (2021)

    2021

  52. [60]

    In: Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Park, W., Kim, D., Lu, Y., Cho, M.: Relational knowledge distillation. In: Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 3967–3976 (2019)

    2019

  53. [61]

    In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition

    Rebuffi, S.A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition. pp. 2001–2010 (2017)

    2017

  54. [62]

    arXiv preprint arXiv:1412.6550 (2014)

    Romero, A., Ballas, N., Kahou, S.E., Chassang, A., Gatta, C., Bengio, Y.: Fitnets: Hints for thin deep nets. arXiv preprint arXiv:1412.6550 (2014)

    2014 arXiv

  55. [63]

    In: Proceedings of the IEEE conference on computer vision and pattern recognition

    Saito, K., Watanabe, K., Ushiku, Y., Harada, T.: Maximum classifier discrepancy for unsupervised domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 3723–3732 (2018)

    2018

  56. [64]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Sajedi, A., Khaki, S., Amjadian, E., Liu, L.Z., Lawryshyn, Y.A., Plataniotis, K.N.: Datadam: Efficient dataset distillation with attention matching. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 17097–17107 (2023)

    2023

  57. [65]

    In: ICASSP 2024- 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

    Sajedi, A., Khaki, S., Lawryshyn, Y.A., Plataniotis, K.N.: Probmcl: Simple prob- abilistic contrastive learning for multi-label visual classification. In: ICASSP 2024- 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). pp. 5115–5119. IEEE (2024)

    2024

  58. [66]

    arXiv preprint arXiv:2307.03967 (2023)

    Sajedi, A., Khaki, S., Plataniotis, K.N., Hosseini, M.S.: End-to-end supervised multilabel contrastive learning. arXiv preprint arXiv:2307.03967 (2023)

    2023 arXiv

  59. [67]

    In: 2022 IEEE 14th Image, Video, and Multidimen- sional Signal Processing Workshop (IVMSP)

    Sajedi, A., Lawryshyn, Y.A., Plataniotis, K.N.: Subclass knowledge distillation with known subclass labels. In: 2022 IEEE 14th Image, Video, and Multidimen- sional Signal Processing Workshop (IVMSP). pp. 1–5. IEEE (2022)

    2022

  60. [68]

    arXiv preprint arXiv:2109.05587 (2021)

    Sajedi, A., Plataniotis, K.N.: On the efficiency of subclass knowledge distillation in classification tasks. arXiv preprint arXiv:2109.05587 (2021)

    2021 arXiv

  61. [69]

    In: ACM SIGGRAPH 2022 conference proceedings

    Sauer, A., Schwarz, K., Geiger, A.: Stylegan-xl: Scaling stylegan to large diverse datasets. In: ACM SIGGRAPH 2022 conference proceedings. pp. 1–10 (2022)

    2022

  62. [70]

    In: International Conference on Learning Representations (2018)

    Sener, O., Savarese, S.: Active learning for convolutional neural networks: A core- set approach. In: International Conference on Learning Representations (2018)

    2018

  63. [71]

    arXiv preprint arXiv:1409.1556 (2014) Data-to-Model Distillation: Data-Efficient Learning Framework 19

    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014) Data-to-Model Distillation: Data-Efficient Learning Framework 19

    2014 arXiv

  64. [72]

    Advances in neural information processing systems 28 (2015)

    Sohn, K., Lee, H., Yan, X.: Learning structured output representation using deep conditional generative models. Advances in neural information processing systems 28 (2015)

    2015

  65. [73]

    In: International Conference on Machine Learning

    Such, F.P., Rawal, A., Lehman, J., Stanley, K., Clune, J.: Generative teaching networks: Accelerating neural architecture search by learning to generate synthetic training data. In: International Conference on Machine Learning. pp. 9206–9216. PMLR (2020)

    2020

  66. [74]

    In: International conference on machine learning

    Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International conference on machine learning. pp. 6105–6114. PMLR (2019)

    2019

  67. [75]

    In: International Conference on Learning Representations (2019)

    Toneva, M., Sordoni, A., des Combes, R.T., Trischler, A., Bengio, Y., Gordon, G.J.: An empirical study of example forgetting during deep neural network learning. In: International Conference on Learning Representations (2019)

    2019

  68. [76]

    Scientific data 5(1), 1–9 (2018)

    Tschandl, P., Rosendahl, C., Kittler, H.: The ham10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific data 5(1), 1–9 (2018)

    2018

  69. [77]

    arXiv preprint arXiv:1607.08022 (2016)

    Ulyanov, D., Vedaldi, A., Lempitsky, V.: Instance normalization: The missing in- gredient for fast stylization. arXiv preprint arXiv:1607.08022 (2016)

    2016 arXiv

  70. [78]

    arXiv preprint arXiv:2303.04707 (2023)

    Wang, K., Gu, J., Zhou, D., Zhu, Z., Jiang, W., You, Y.: Dim: Distilling dataset into generative model. arXiv preprint arXiv:2303.04707 (2023)

    2023 arXiv

  71. [79]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Wang, K., Zhao, B., Peng, X., Zhu, Z., Yang, S., Wang, S., Huang, G., Bilen, H., Wang, X., You, Y.: Cafe: Learning to condense dataset by aligning features. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 12196–12205 (2022)

    2022

  72. [80]

    In: Proceedings of the IEEE conference on computer vision and pattern recognition

    Wang, T.C., Liu, M.Y., Zhu, J.Y., Tao, A., Kautz, J., Catanzaro, B.: High- resolution image synthesis and semantic manipulation with conditional gans. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 8798–8807 (2018)

    2018

  73. [81]

    arXiv preprint arXiv:1811.10959 (2018)

    Wang, T., Zhu, J.Y., Torralba, A., Efros, A.A.: Dataset distillation. arXiv preprint arXiv:1811.10959 (2018)

    2018 arXiv

  74. [82]

    In: 2021 20th IEEE International Conference on Machine Learning and Applications (ICMLA)

    Wang, Y.R., Khaki, S., Zheng, W., Hosseini, M.S., Plataniotis, K.N.: Conetv2: Efficient auto-channel size optimization for cnns. In: 2021 20th IEEE International Conference on Machine Learning and Applications (ICMLA). pp. 998–1003 (2021). https://doi.org/10.1109/ICMLA52953.2021.00164

    2021

  75. [83]

    In: Proceedings of the IEEE conference on computer vision and pattern recognition

    Wu, J., Leng, C., Wang, Y., Hu, Q., Cheng, J.: Quantized convolutional neural networks for mobile devices. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 4820–4828 (2016)

    2016

  76. [84]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Xiong, Y., Wang, R., Cheng, M., Yu, F., Hsieh, C.J.: Feddm: Iterative distribution matching for communication-efficient federated learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 16323– 16332 (2023)

    2023

  77. [85]

    Advances in Neural Information Processing Systems 36 (2024)

    Yang, E., Shen, L., Wang, Z., Liu, T., Guo, G.: An efficient dataset condensation plugin and its application to continual learning. Advances in Neural Information Processing Systems 36 (2024)

    2024

  78. [86]

    Scientific Data 10(1), 41 (2023)

    Yang,J.,Shi,R.,Wei,D.,Liu,Z.,Zhao,L.,Ke,B.,Pfister,H.,Ni,B.:Medmnistv2- a large-scale lightweight benchmark for 2d and 3d biomedical image classification. Scientific Data 10(1), 41 (2023)

    2023

  79. [87]

    In: Proceedings of the Advances in Neural Information Processing Systems (NeurIPS) (2023) 20 Ahmad Sajedi *, Samir Khaki* et al

    Yin, Z., Xing, E., Shen, Z.: Squeeze, recover and relabel: Dataset condensation at imagenet scale from a new perspective. In: Proceedings of the Advances in Neural Information Processing Systems (NeurIPS) (2023) 20 Ahmad Sajedi *, Samir Khaki* et al

    2023

  80. [88]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Yuan, L., Tay, F.E., Li, G., Wang, T., Feng, J.: Revisiting knowledge distillation via label smoothing regularization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 3903–3911 (2020)

    2020

  81. [89]

    In: Interna- tional Conference on Learning Representations (2016)

    Zagoruyko, S., Komodakis, N.: Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer. In: Interna- tional Conference on Learning Representations (2016)

    2016

  82. [90]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Zhang, L., Zhang, J., Lei, B., Mukherjee, S., Pan, X., Zhao, B., Ding, C., Li, Y., Xu, D.: Accelerating dataset distillation via model augmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 11950–11959 (2023)

    2023

  83. [91]

    In: International Conference on Machine Learning

    Zhao, B., Bilen, H.: Dataset condensation with differentiable siamese augmenta- tion. In: International Conference on Machine Learning. pp. 12674–12685. PMLR (2021)

    2021

  84. [92]

    In: NeurIPS 2022 Workshop on Synthetic Data for Empowering ML Research (2022)

    Zhao, B., Bilen, H.: Synthesizing informative training samples with gan. In: NeurIPS 2022 Workshop on Synthetic Data for Empowering ML Research (2022)

    2022

  85. [93]

    In: Proceed- ings of the IEEE/CVF Winter Conference on Applications of Computer Vision

    Zhao, B., Bilen, H.: Dataset condensation with distribution matching. In: Proceed- ings of the IEEE/CVF Winter Conference on Applications of Computer Vision. pp. 6514–6523 (2023)

    2023

  86. [94]

    In: Ninth International Conference on Learning Representations 2021 (2021)

    Zhao, B., Mopuri, K.R., Bilen, H.: Dataset condensation with gradient matching. In: Ninth International Conference on Learning Representations 2021 (2021)

    2021

  87. [95]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Zhao, G., Li, G., Qin, Y., Yu, Y.: Improved distribution matching for dataset condensation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 7856–7865 (2023)

    2023

  88. [96]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Zhou, D., Wang, K., Gu, J., Peng, X., Lian, D., Zhang, Y., You, Y., Feng, J.: Dataset quantization. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 17205–17216 (2023)

    2023

  89. [97]

    In: Advances in Neural Information Processing Systems (2022)

    Zhou, Y., Nezhadarya, E., Ba, J.: Dataset distillation using neural feature regres- sion. In: Advances in Neural Information Processing Systems (2022)

    2022

  90. [98]

    airplane

    Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE interna- tional conference on computer vision. pp. 2223–2232 (2017) Data-to-Model Distillation: Data-Efficient Learning Fra...

    2017

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.