Pith. sign in

REVIEW 5 cited by

Data-Free Adversarial Distillation

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 1912.11006 v3 pith:MYSXRZRC submitted 2019-12-23 cs.LG cs.CVstat.ML

Gongfan Fang , Jie Song , Chengchao Shen , Xinchao Wang , Da Chen , Mingli Song This is my paper · ORCID
classification cs.LGcs.CVstat.ML
keywords datadistillationmodelstudentadversarialapproachdata-drivendata-free
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Knowledge Distillation (KD) has made remarkable progress in the last few years and become a popular paradigm for model compression and knowledge transfer. However, almost all existing KD algorithms are data-driven, i.e., relying on a large amount of original training data or alternative data, which is usually unavailable in real-world scenarios. In this paper, we devote ourselves to this challenging problem and propose a novel adversarial distillation mechanism to craft a compact student model without any real-world data. We introduce a model discrepancy to quantificationally measure the difference between student and teacher models and construct an optimizable upper bound. In our work, the student and the teacher jointly act the role of the discriminator to reduce this discrepancy, when a generator adversarially produces some "hard samples" to enlarge it. Extensive experiments demonstrate that the proposed data-free method yields comparable performance to existing data-driven methods. More strikingly, our approach can be directly extended to semantic segmentation, which is more complicated than classification, and our approach achieves state-of-the-art results. Code and pretrained models are available at https://github.com/VainF/Data-Free-Adversarial-Distillation.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Towards Efficient and Effective Alignment of Large Language Models

    cs.CL 2025-06 conditional novelty 7.0 of 10

    A thesis presenting Lion, WebR, LTE, BMC, and FollowBench, five empirical methods that together address LLM alignment data, training, and evaluation.

  2. Stealix: Model Stealing via Prompt Evolution

    cs.CR 2025-06 conditional novelty 7.0 of 10

    An attacker with one seed image per class can steal a black-box image classifier by genetically evolving text prompts, guided only by the victim's hard-label predictions.

  3. When Data-Free Knowledge Distillation Meets Non-Transferable Teacher: Escaping Out-of-Distribution Trap is All You Need

    cs.LG 2025-07 conditional novelty 6.0 of 10

    Data-free distillation from non-transferable teachers fails because synthesized samples drift toward the OOD domain; ATEsc separates ID-like from OOD-like samples via adversarial robustness and improves distillation.

  4. Forget the Data and Fine-Tuning! Just Fold the Network to Compress

    cs.LG 2025-02 conditional novelty 6.0 of 10

    Model folding compresses a network by k-means clustering similar neurons across adjacent layers and repairing activation statistics without data (Fold-AR, Fold-DIR), surpassing prior data-free methods at high sparsity.

  5. Stabilizing Data-Free Model Extraction

    cs.LG 2025-09 conditional novelty 5.0 of 10

    MetaDFME stabilizes data-free model extraction by training the generator with Reptile-style meta-learning, achieving higher and less oscillating substitute accuracy.

Pith tools