REVIEW 7 cited by
DiM: Distilling Dataset into Generative Model
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
abstract
Dataset distillation reduces the network training cost by synthesizing small and informative datasets from large-scale ones. Despite the success of the recent dataset distillation algorithms, three drawbacks still limit their wider application: i). the synthetic images perform poorly on large architectures; ii). they need to be re-optimized when the distillation ratio changes; iii). the limited diversity restricts the performance when the distillation ratio is large. In this paper, we propose a novel distillation scheme to \textbf{D}istill information of large train sets \textbf{i}nto generative \textbf{M}odels, named DiM. Specifically, DiM learns to use a generative model to store the information of the target dataset. During the distillation phase, we minimize the differences in logits predicted by a models pool between real and generated images. At the deployment stage, the generative model synthesizes various training samples from random noises on the fly. Due to the simple yet effective designs, the trained DiM can be directly applied to different distillation ratios and large architectures without extra cost. We validate the proposed DiM across 4 datasets and achieve state-of-the-art results on all of them. To the best of our knowledge, we are the first to achieve higher accuracy on complex architectures than simple ones, such as 75.1\% with ResNet-18 and 72.6\% with ConvNet-3 on ten images per class of CIFAR-10. Besides, DiM outperforms previous methods with 10\% $\sim$ 22\% when images per class are 1 and 10 on the SVHN dataset.
Forward citations
Cited by 7 Pith papers
-
Dataset Distillation Based on Saliency-Driven Prototype Alignment
Saliency-guided latent prototypes plus confidence-based hard-prototype refinement improve diffusion-based dataset distillation accuracy on ImageNet subsets, CIFAR, and ImageNet-1K without fine-tuning the generative backbone.
-
Dynamic-Aware Video Distillation: Optimizing Temporal Resolution Based on Video Semantics
DAViD uses Q-learning to pick a per-class temporal resolution for synthetic videos in dataset distillation, improving distilled-video recognition accuracy on UCF101, HMDB51, SSv2, and K400.
-
MGD$^3$: Mode-Guided Dataset Distillation using Diffusion Models
Guiding a pretrained diffusion model toward per-class K-means centroids during sampling improves distilled dataset accuracy on ImageNet subsets by up to 4.4% while avoiding diffusion fine-tuning.
-
CONCORD: Concept-Informed Diffusion for Dataset Distillation
A training-free concept-informed diffusion method, using LLM-retrieved and CLIP-filtered visual descriptions, improves dataset distillation accuracy on ImageNet subsets and ImageNet-1K.
-
Dataset Distillation by Influence Matching
Inf-Match distills datasets by matching estimated parameter influence of real and synthetic data, reporting SOTA classification and retrieval, but with an unsupported theoretical core.
-
Dataset Distillation via Vision-Language Category Prototype
A dataset distillation method that combines K-means image prototypes with LLM-generated text prototypes to synthesize small, high-accuracy training sets.
-
Diversity-Driven Generative Dataset Distillation Based on Diffusion Model with Self-Adaptive Memory
Diversity-driven diffusion dataset distillation with self-adaptive memory eviction achieves SOTA accuracy on ImageWoof, ImageNette, and ImageIDC.
Discussion (0). Sign in to comment.