Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Dataset Distillers Are Good Label Denoisers In the Wild

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Dataset distillation, which compresses a training set into a few synthetic images per class, can act as a label denoiser under symmetric and natural label noise, but it can also absorb structured asymmetric noise into the distilled samples.

desk verdict Plausible idea, thin evidence: interesting empirical trends, but missing baselines, no verification of the denoising mechanism, and a reversed inequality in Corollary-II mean this needs major revision. read the letter →

arxiv 2411.11924 v2 pith:WAM3AEWD submitted 2024-11-18 cs.LG cs.CV

Lechao Cheng , Kaifeng Chen , Jiyang Li , Shengeng Tang , Shufei Zhang , Meng Wang This is my paper · ORCID
classification cs.LGcs.CV
keywords datasetdistillationnoisylabellearningsymmetricnoiseasymmetricnaturalCIFAR-Nrobusttrainingdenoising
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that dataset distillation—compressing a large training set into a small set of synthetic images—can replace the usual estimate-then-correct pipeline of noisy-label learning. By benchmarking three representative distillers on symmetric, asymmetric, and human-annotation noise, it finds that distilled synthetic sets trained with plain cross-entropy often beat training on the full noisy dataset once the noise rate passes a threshold. The paper's central insight is that distillation keeps the common patterns of a class while discarding outliers, so random label noise is washed out before training begins. The same mechanism fails when noise is structured: visually similar classes flip into each other under asymmetric noise, and the distilled set absorbs the noise pattern. The payoff is a denoising step that is offline, privacy-preserving, and free of the feedback loop between noise evaluation and relabeling.

What carries the argument

The load-bearing mechanism is the distillation objective itself, which forces a tiny synthetic set to reproduce the behavior of the full dataset. Each of the three methods compresses by a different route—DATM aligns student model weights with expert trajectories, DANCE matches feature and label distributions of real and synthetic sets, and RCIG solves a meta-learning loop with convexified implicit gradients—but all share the effect that a few synthetic images per class must stand for the majority pattern in that class. That majority voting is what averages out symmetric label noise and drops rare mislabeled outliers, and it is also why structured asymmetric noise survives: when the noise is a consistent transition between similar classes, it becomes part of the common pattern. The IPC bound and the noise-rate bound translate this qualitative mechanism into quantitative statements about how much clean information survives.

What would settle it

Distill CIFAR-10 with known symmetric noise (say $\tau=0.8$) at one image per class using any of the three methods, then have a clean-label oracle judge the distilled image for each class: if the distilled sample carries the majority noisy transition rather than the true clean class, the denoising interpretation fails. Equivalently, compute the effective noise rate of the distilled set against clean labels; the claim requires it to be substantially below $\tau$.

Watch

Extended reading notes

Core claim

The central claim is that for symmetric noise, existing dataset distillation methods serve as effective denoising tools, and for real-world natural noise they still generalize well at fixed noisy ratios. The paper supports this with experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet using DATM (trajectory matching), DANCE (distribution matching), and RCIG (meta-learning), showing that distillation beats the full-noisy-dataset baseline at noise rates 0.2–0.8, sometimes with a single image per class. For asymmetric noise, the claim is the opposite: structured label transitions among similar classes are carried into the synthetic samples, so distillation does not recover the clean distribution. The paper also states a bound, $IPC \leq |\tilde{S}|(1-\tau)/C$, for how many distilled images per class are needed to preserve clean information under symmetric noise, and a probabilistic bound on the noise rate inferred from distillation accuracy. It further warns that clean but hard samples, such as tail-class images, can be lossily compressed.

Load-bearing premise

The paper assumes that the distilled synthetic images represent clean common patterns rather than noise, but it never verifies the labels of the distilled samples, so the denoising story stands on that unexamined assumption.

Editorial extensions

If this is right

  • At noise rates of 0.6 and 0.8, distilling even one image per class on CIFAR-10/100 and Tiny-ImageNet beats training on the full noisy dataset, so practitioners with heavily corrupted labels can distill first and train on the small synthetic set.
  • Under symmetric noise, the maximum useful distilled size per class is bounded by the expected number of clean samples per class, $|\tilde{S}|(1-\tau)/C$, and distilling beyond that point cannot preserve only-clean information.
  • On real natural noise (CIFAR-N), all three methods beat the full-data baseline under the Worst setting (about 40% noise) with fewer than 10 images per class, so an unknown fixed noise rate does not break the approach.
  • Asymmetric noise is a failure mode: methods generally underperform the full-noisy baseline unless the distilled set is large, so distillation should not be used when label corruption follows visually similar class pairs.
  • Clean but hard examples, like tail-class images, are at risk of lossy compression, meaning distillation-based denoising may trade overall accuracy for robustness in imbalanced settings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the denoising mechanism is majority-pattern averaging, then the denoising effect at a fixed noise rate should grow with the number of clean examples per class and shrink as the noise becomes class-correlated; a controlled experiment varying per-class noise correlation at constant $\tau$ would test this directly.
  • The crossing point where distilled accuracy overtakes full-data accuracy could be inverted to estimate the noise rate of an unlabeled noisy dataset, an estimator the paper raises as an open problem but does not develop.
  • Because distillation is offline and operates on the data itself rather than on model confidence, it is a natural fit for federated or privacy-sensitive settings: the noisy data never leaves the owner, and no relabeling model sees individual samples.
  • Natural human annotation noise contains semantic confusions similar to asymmetric noise, so the CIFAR-N results likely sit between the symmetric and asymmetric cases; methods that first detect structured transitions and exclude the affected pairs before distillation would test whether the failure is removable.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes an alternative to conventional noisy-label learning: instead of first estimating noise and then filtering/reweighting/relabeling, it suggests applying dataset distillation to a noisy dataset and training on the resulting synthetic set. The authors benchmark three dataset distillation methods (DATM, DANCE, RCIG) on CIFAR-10/100, Tiny-ImageNet, and CIFAR-N under symmetric, asymmetric, and natural label noise. Their central observations are that distillation outperforms cross-entropy training on the full noisy dataset under symmetric and natural noise at sufficiently high noise rates, but that structured asymmetric noise can be absorbed into the distilled samples. They also state two corollaries relating the number of distilled samples per class (IPC) to the noise rate. The paper is framed as an empirical study with a new perspective rather than a new algorithm.

Significance. If the central claim holds, the paper offers a practically appealing preprocessing strategy: a practitioner could distill a noisy dataset offline and train on the synthetic set, avoiding the iterative clean/noisy separation that characterizes most robust-learning methods. The benchmarking across three representative distillation families and several noise types is a useful service to the community, and the negative result for asymmetric noise is an honest and informative finding. The paper also releases source code. However, the significance is currently limited by the weakness of the empirical evidence: no error bars or multiple seeds, no comparison with state-of-the-art noisy-label methods, and no direct verification that distilled samples actually carry clean labels. The two corollaries, as stated, contain formal errors. The core observation is plausible and potentially interesting, but it is not yet convincingly established.

major comments (4)
  1. [Section 4, Experiment-I and Insight-I] The mechanism behind the central claim is not verified. The paper asserts that distillation 'captures common patterns while ignoring outliers such as noise,' but it never checks whether the distilled synthetic samples actually correspond to the clean labels they are assigned; Figure 3 is only illustrative. The observed accuracy gain could arise from the strong regularization of training on very few synthetic images rather than from label denoising. To support the mechanism, please compare distillation against training on same-size random subsets of the noisy dataset, and inspect the label correctness of distilled samples (e.g., by evaluating a clean-data classifier on the distilled images or by human inspection of the synthetic set).
  2. [Section 4, Figures 2, 4, 5 and Implementation Details] All reported results are single accuracy curves with no error bars or multiple seeds. Claims such as 'a single distilled sample per class surpasses the baseline' at τ=0.6/0.8 need variance estimates to be credible. Moreover, the only baseline is cross-entropy training on the full noisy dataset, which is known to be weak under high noise; standard noisy-label methods (e.g., Co-teaching, DivideMix, GCE) are not compared. Without such comparisons, the paper's claim that dataset distillation methods are 'good label denoisers' is not established relative to existing robust-learning techniques. Please report mean and standard deviation over at least three seeds and include at least one or two standard noisy-label baselines.
  3. [Section 4, Corollary-II, Eq. (11)] The inequality in Corollary-II is algebraically reversed. From Corollary-I, IPC ≤ |S|(1−τ)/C, rearranging gives τ ≤ 1 − C·IPC/|S|, not τ ≥ 1 − C·IPC/|S| as stated in Eq. (11). Furthermore, no derivation of the probability bound is provided, and the statement 'with at least high probability 1−δ' introduces a parameter δ that never appears in any antecedent or proof. The probability expression is also not well-formed, as the event is not clearly defined over a random draw of the dataset. This corollary should be corrected or removed.
  4. [Section 4, Corollary-I, Eq. (10)] Corollary-I is essentially a restatement of the expected number of clean samples per class under the symmetric noise model, not a derived consequence of distillation behavior. The step from 'at most |S|(1−τ)/C clean samples per class' to 'the maximum IPC needed to perfectly preserve sample information is this quantity' is unjustified; distillation could in principle preserve information from noisy samples or from fewer clean samples. Please provide a proof or rephrase the statement as a heuristic motivation, and explicitly state the assumption of balanced class sizes.
minor comments (5)
  1. [Throughout] There are multiple typos and formatting issues: Section 2 header 'Realted Works' should be 'Related Works'; the Contributions list says 'bellow' instead of 'below'; Section 1 has 'Y ou need generalizability'; Section 3.1 has 'that isSymmetric' missing a space; and 'DA TM' is inconsistently spaced in Section 3.2.
  2. [Section 4, Eq. (11)] The probability notation in Eq. (11) is confusing because the same symbol P is used for both the probability over the dataset and the claimed bound; please use separate notation and make the event inside the probability explicit.
  3. [Figure 5] The caption uses 'CIFAR10N' while the text uses 'CIFAR-10N'; please standardize the naming.
  4. [Section 4] The two corollaries are presented without proofs and without a clear connection to the experimental protocol; consider renaming them 'Remarks' and providing formal statements or derivations.
  5. [Title and Abstract] The phrase 'In the Wild' is broader than the experimental scope, which is limited to CIFAR-10/100, Tiny-ImageNet, and CIFAR-N; consider tempering the title or adding experiments on larger-scale noisy datasets such as Clothing1M or WebVision.

Circularity Check

2 steps flagged · score 6.0 of 10

Two theoretical corollaries restate the symmetric-noise model rather than deriving distillation bounds; the main empirical denoising claim is not circular.

  1. self definitional [Section 4, Corollary-I, Eq. (10)]
    "Corollary-I. Given a balanced dataset ˜S ∈ ˜D with symmetric noise τ, to perfectly preserve the sample information from the original dataset for each class during dataset distillation, the maximum number of distilled samples per class (Image Per Class, IPC) required is: IPC ≤ |S|·(1−τ)/C (10), where C is the total number of classes. This corollary succinctly states the upper bound for the number of distilled samples per class needed to retain clean data information under the given noise conditions."

    Under the symmetric-noise model in Eq. (1), each class contains |S|/C samples and each sample is clean with probability 1−τ, so the expected number of clean samples per class is exactly |S|(1−τ)/C. Eq. (10) simply rewrites this expectation and asserts it as the maximum IPC needed to 'perfectly preserve' clean information. No property of dataset distillation (trajectory matching, distribution matching, or meta-learning) enters the bound; the 'corollary' is the noise model itself, relabeled as a distillation result. Hence the claimed upper bound is true by definition of τ, not by any distillation analysis, and it does not answer the open question it is introduced to answer.

  2. self definitional [Section 4, Corollary-II, Eq. (11)]
    "Corollary-II. Given a noisy dataset ˜S ∈ ˜D from a real-world scenario with C classes, when applying dataset distillation such that each class is synthesized to a size of IPC, and the validation accuracy matches that of the original dataset, we can infer with at least high probability 1−δ that the noise rate τ of the dataset satisfies: P(τ ≥ (1 − C·IPC/|S|)) ≥ 1−δ. (11)"

    Eq. (11) is presented as a probabilistic inference about τ from distillation accuracy, but the inequality is just the algebraic converse of Eq. (10) rearranged for τ, and it is reversed: from IPC ≤ |S|(1−τ)/C one obtains τ ≤ 1 − C·IPC/|S|, not τ ≥. The conditions 'validation accuracy matches' and the confidence 1−δ are never connected to any distillation or concentration argument, so the bound is not derived from the distillation experiment; it is the input noise model restated with a sign error. The corollary therefore reduces to the paper's definition of symmetric noise and cannot support the claimed inference.

full rationale

The main empirical finding—that distillation improves over full-set cross-entropy training under symmetric and natural noise—is not circular: it is benchmarked against an external baseline and three independently published distillation methods. The paper does not verify the labels of the distilled images, and it omits standard noisy-label learners; those are evidential weaknesses, not definitional circularity, so they are excluded from the score under the review rules. The circularity is confined to the paper's two 'Corollary' results. Corollary-I is the expected clean-count formula under Eq. (1), written as an upper bound on IPC; it contains no distillation-specific derivation. Corollary-II is the same model manipulated into a probabilistic claim about τ, with the inequality direction reversed and no derivation of δ or of the role of matching accuracy. These two results are thus equivalent to the input noise model by construction, giving partial circularity (score 6). No self-citation chain or imported-uniqueness issue is present, so 8/10 would be disproportionate.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new entities or fitted parameters. Its corollaries rely on the noise model definitions and on an unverified memorization assumption about distillation.

assumptions (3)
  • domain assumption Symmetric noise follows p(y_tilde = y') = tau/(C-1) for y' != y and 1-tau otherwise
    Assumed in Eq (1) and used for Corollary-I.
  • domain assumption Dataset distillation captures common patterns and ignores outliers such as noisy labels
    Stated in Insight-I; the paper provides only indirect evidence via accuracy.
  • domain assumption Validation accuracy matching implies the distilled set captures all clean samples
    Used implicitly in Corollary-II.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Dataset Distillers Are Good Label Denoisers In the Wild." pith.science (2026). https://pith.science/paper/WAM3AEWD

@misc{pith2026241111924,
  author       = {Pith},
  title        = {Pith review of: Dataset Distillers Are Good Label Denoisers In the Wild},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WAM3AEWD}},
  note         = {Machine review of arXiv:2411.11924}
}
read the original abstract

Learning from noisy data has become essential for adapting deep learning models to real-world applications. Traditional methods often involve first evaluating the noise and then applying strategies such as discarding noisy samples, re-weighting, or re-labeling. However, these methods can fall into a vicious cycle when the initial noise evaluation is inaccurate, leading to suboptimal performance. To address this, we propose a novel approach that leverages dataset distillation for noise removal. This method avoids the feedback loop common in existing techniques and enhances training efficiency, while also providing strong privacy protection through offline processing. We rigorously evaluate three representative dataset distillation methods (DATM, DANCE, and RCIG) under various noise conditions, including symmetric noise, asymmetric noise, and real-world natural noise. Our empirical findings reveal that dataset distillation effectively serves as a denoising tool in random noise scenarios but may struggle with structured asymmetric noise patterns, which can be absorbed into the distilled samples. Additionally, clean but challenging samples, such as those from tail classes in imbalanced datasets, may undergo lossy compression during distillation. Despite these challenges, our results highlight that dataset distillation holds significant promise for robust model training, especially in high-privacy environments where noise is prevalent. The source code is available at https://github.com/Kciiiman/DD_LNL.

Figures

Figures reproduced from arXiv: 2411.11924 by the authors.

Figure 1
Figure 1. (a) for symmetric noise, existing dataset distillation [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The validation performance over symmetric noise for CIFAR-10, CIFAR-100, and Tiny-ImageNet datasets. The solid lines rep [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Visualization of images distilled from DATM on CIFAR-10 with one image per class. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: The validation performance over asymmetric noise for CIFAR-10, CIFAR-100. The solid lines represent the accuracy trend as [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: The results for CIFAR-10N/CIFAR-100N. The solid lines illustrate the trend of accuracy as Image Per-Class (IPC) increases. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. The Evolution of Dataset Distillation: Toward Scalable and Generalizable Solutions

    cs.CV 2025-02 conditional novelty 3.0 of 10

    A 2023-2025 survey of dataset distillation that organizes matching, generative, decoupling, and selective methods and tabulates ImageNet-scale accuracy comparisons.

Reference graph

Works this paper leans on

61 extracted references · 47 canonical work pages · cited by 1 Pith paper

  1. [1]

    A closer look at memorization in deep networks

    Devansh Arpit, Stanisław Jastrz˛ ebski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder S Kanwal, Tegan Maharaj, Asja Fischer, Aaron Courville, Yoshua Bengio, et al. A closer look at memorization in deep networks. In International conference on machine learning , pages 233–

  2. [2]

    From noisy predic- tion to true label: Noisy prediction calibration via generative model, 2022

    HeeSun Bae, Seungjae Shin, Byeonghu Na, JoonHo Jang, Kyungwoo Song, and Il-Chul Moon. From noisy predic- tion to true label: Noisy prediction calibration via generative model, 2022. 1, 2

    work page 2022

  3. [3]

    Dirichlet-based per-sample weighting by transition matrix for noisy label learning, 2024

    HeeSun Bae, Seungjae Shin, Byeonghu Na, and Il-Chul Moon. Dirichlet-based per-sample weighting by transition matrix for noisy label learning, 2024. 1, 2

    work page 2024

  4. [4]

    Flexible dataset distillation: Learn labels instead of images

    Ondrej Bohdal, Yongxin Yang, and Timothy Hospedales. Flexible dataset distillation: Learn labels instead of images. arXiv preprint arXiv:2006.08572, 2020. 3

    arXiv 2006

  5. [5]

    Efros, and Jun-Yan Zhu

    George Cazenavette, Tongzhou Wang, Antonio Torralba, Alexei A. Efros, and Jun-Yan Zhu. Dataset distillation by matching training trajectories, 2022. 6

    work page 2022

  6. [6]

    Data distillation can be like vodka: Distilling more times for better quality

    Xuxi Chen, Yu Yang, Zhangyang Wang, and Baharan Mirza- soleiman. Data distillation can be like vodka: Distilling more times for better quality. arXiv preprint arXiv:2310.06982 ,

    arXiv

  7. [7]

    Learning with instance-dependent label noise: A sample sieve approach

    Hao Cheng, Zhaowei Zhu, Xingyu Li, Yifei Gong, Xing Sun, and Yang Liu. Learning with instance-dependent label noise: A sample sieve approach. arXiv preprint arXiv:2010.02347,

    arXiv 2010

  8. [8]

    A framework using contrastive learning for classification with noisy labels, 2021

    Madalina Ciortan, Romain Dupuis, and Thomas Peel. A framework using contrastive learning for classification with noisy labels, 2021. 1, 2

    work page 2021

Show all 61 references
  1. [9]

    Scaling up dataset distillation to imagenet-1k with constant memory

    Justin Cui, Ruochen Wang, Si Si, and Cho-Jui Hsieh. Scaling up dataset distillation to imagenet-1k with constant memory. In International Conference on Machine Learning , pages 6565–6590. PMLR, 2023. 4

    2023

  2. [10]

    Robust classification via regression for learning with noisy labels

    Erik Englesson and Hossein Azizpour. Robust classification via regression for learning with noisy labels. In ICLR 2024- The Twelfth International Conference on Learning Represen- tations, Messe Wien Exhibition and Congress Center, Vienna, Austria, May 7-11t, 2024, 2024. 1

    2024

  3. [11]

    A survey on dataset distilla- tion: Approaches, applications and future directions

    Jiahui Geng, Zongxiong Chen, Yuandou Wang, Herbert Woisetschlaeger, Sonja Schimmler, Ruben Mayer, Zhiming Zhao, and Chunming Rong. A survey on dataset distilla- tion: Approaches, applications and future directions. arXiv preprint arXiv:2305.01975, 2023. 3

    2023 arXiv

  4. [12]

    Contrastive learning im- proves model robustness under label noise, 2021

    Aritra Ghosh and Andrew Lan. Contrastive learning im- proves model robustness under label noise, 2021. 2

    2021

  5. [13]

    Towards lossless dataset distillation via difficulty-aligned trajectory matching

    Ziyao Guo, Kai Wang, George Cazenavette, Hui Li, Kaipeng Zhang, and Yang You. Towards lossless dataset distillation via difficulty-aligned trajectory matching. arXiv preprint arXiv:2310.05773, 2023. 2, 3, 4

    2023 arXiv

  6. [14]

    Co- teaching: Robust training of deep neural networks with ex- tremely noisy labels

    Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. Co- teaching: Robust training of deep neural networks with ex- tremely noisy labels. Advances in neural information pro- cessing systems, 31, 2018. 1, 2, 4

    2018

  7. [15]

    Sigua: Forgetting may make learning with noisy labels more robust

    Bo Han, Gang Niu, Xingrui Yu, Quanming Yao, Miao Xu, Ivor Tsang, and Masashi Sugiyama. Sigua: Forgetting may make learning with noisy labels more robust. In Interna- tional Conference on Machine Learning , pages 4006–4016. PMLR, 2020. 2, 4

    2020

  8. [16]

    Multisize dataset condensation

    Yang He, Lingao Xiao, Joey Tianyi Zhou, and Ivor Tsang. Multisize dataset condensation. arXiv preprint arXiv:2403.06075, 2024. 3

    2024 arXiv

  9. [17]

    Meta-learning in neural networks: A survey

    Timothy Hospedales, Antreas Antoniou, Paul Micaelli, and Amos Storkey. Meta-learning in neural networks: A survey. IEEE transactions on pattern analysis and machine intelli- gence, 44(9):5149–5169, 2021. 3

    2021

  10. [18]

    Twin contrastive learning with noisy labels, 2023

    Zhizhong Huang, Junping Zhang, and Hongming Shan. Twin contrastive learning with noisy labels, 2023. 2

    2023

  11. [19]

    Mentornet: Learning data-driven curriculum for very deep neural networks on corrupted labels, 2018

    Lu Jiang, Zhengyuan Zhou, Thomas Leung, Li-Jia Li, and Li Fei-Fei. Mentornet: Learning data-driven curriculum for very deep neural networks on corrupted labels, 2018. 2

    2018

  12. [20]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. Technical report, Uni- versity of Toronto, 2009. Technical Report. 1, 6

    2009

  13. [21]

    Tiny imagenet visual recognition challenge

    Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N, 7(7):3, 2015. 6

    2015

  14. [22]

    Selmatch: Effectively scaling up dataset distillation via selection-based initializa- tion and partial updates by trajectory matching

    Yongmin Lee and Hye Won Chung. Selmatch: Effectively scaling up dataset distillation via selection-based initializa- tion and partial updates by trajectory matching. In Forty-first International Conference on Machine Learning, 2024. 3

    2024

  15. [23]

    A comprehensive survey of dataset distillation

    Shiye Lei and Dacheng Tao. A comprehensive survey of dataset distillation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023. 3

    2023

  16. [24]

    Dividemix: Learning with noisy labels as semi-supervised learning

    Junnan Li, Richard Socher, and Steven CH Hoi. Dividemix: Learning with noisy labels as semi-supervised learning. arXiv preprint arXiv:2002.07394, 2020. 1, 2, 3

    2002 arXiv

  17. [25]

    Gradient descent with early stopping is provably robust to label noise for overparameterized neural networks

    Mingchen Li, Mahdi Soltanolkotabi, and Samet Oymak. Gradient descent with early stopping is provably robust to label noise for overparameterized neural networks. In Inter- national conference on artificial intelligence and statistics , pages 4313–4324. PMLR, 2020. 2, 4

    2020

  18. [26]

    Selective-supervised contrastive learning with noisy labels,

    Shikun Li, Xiaobo Xia, Shiming Ge, and Tongliang Liu. Selective-supervised contrastive learning with noisy labels,

  19. [27]

    Webvision database: Visual learning and under- standing from web data

    Wen Li, Limin Wang, Wei Li, Eirikur Agustsson, and Luc Van Gool. Webvision database: Visual learning and under- standing from web data. arXiv preprint arXiv:1708.02862,

    arXiv

  20. [28]

    Disc: Learning from noisy labels via dynamic instance-specific se- lection and correction

    Yifan Li, Hu Han, Shiguang Shan, and Xilin Chen. Disc: Learning from noisy labels via dynamic instance-specific se- lection and correction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 24070–24079, 2023. 1, 2

    2023

  21. [29]

    Adaptive early-learning correc- tion for segmentation from noisy annotations, 2022

    Sheng Liu, Kangning Liu, Weicheng Zhu, Yiqiu Shen, and Carlos Fernandez-Granda. Adaptive early-learning correc- tion for segmentation from noisy annotations, 2022. 1

    2022

  22. [30]

    Efficient dataset distillation using random feature ap- proximation

    Noel Loo, Ramin Hasani, Alexander Amini, and Daniela Rus. Efficient dataset distillation using random feature ap- proximation. Advances in Neural Information Processing Systems, 35:13877–13891, 2022. 3

    2022

  23. [31]

    Evolution of neural tangent kernels under benign and adversarial training

    Noel Loo, Ramin Hasani, Alexander Amini, and Daniela Rus. Evolution of neural tangent kernels under benign and adversarial training. Advances in Neural Information Pro- cessing Systems, 35:11642–11657, 2022. 5

    2022

  24. [32]

    Dataset distillation with convexified implicit gradients, 2023

    Noel Loo, Ramin Hasani, Mathias Lechner, and Daniela Rus. Dataset distillation with convexified implicit gradients, 2023. 2, 3, 4

    2023

  25. [33]

    when to update

    Eran Malach and Shai Shalev-Shwartz. Decoupling "when to update" from "how to update", 2018. 1, 2

    2018

  26. [34]

    Learning with noisy labels

    Nagarajan Natarajan, Inderjit S Dhillon, Pradeep K Raviku- mar, and Ambuj Tewari. Learning with noisy labels. Ad- vances in neural information processing systems , 26, 2013. 1, 2

    2013

  27. [35]

    Dataset meta-learning from kernel ridge-regression

    Timothy Nguyen, Zhourong Chen, and Jaehoon Lee. Dataset meta-learning from kernel ridge-regression. arXiv preprint arXiv:2011.00050, 2020. 3

    2011 arXiv

  28. [36]

    Dataset distillation with infinitely wide convolutional networks

    Timothy Nguyen, Roman Novak, Lechao Xiao, and Jaehoon Lee. Dataset distillation with infinitely wide convolutional networks. Advances in Neural Information Processing Sys- tems, 34:5186–5198, 2021. 3

    2021

  29. [37]

    Making deep neural networks robust to label noise: a loss correction approach, 2017

    Giorgio Patrini, Alessandro Rozza, Aditya Menon, Richard Nock, and Lizhen Qu. Making deep neural networks robust to label noise: a loss correction approach, 2017. 6

    2017

  30. [38]

    Probabilistic outputs for support vector ma- chines and comparisons to regularized likelihood methods

    John Platt et al. Probabilistic outputs for support vector ma- chines and comparisons to regularized likelihood methods. Advances in large margin classifiers, 10(3):61–74, 1999. 5

    1999

  31. [39]

    Learning to reweight examples for robust deep learning,

    Mengye Ren, Wenyuan Zeng, Bin Yang, and Raquel Urta- sun. Learning to reweight examples for robust deep learning,

  32. [40]

    Ugur Guney, Yann Dauphin, and Leon Bottou

    Levent Sagun, Utku Evci, V . Ugur Guney, Yann Dauphin, and Leon Bottou. Empirical analysis of the hessian of over- parametrized neural networks, 2018. 6

    2018

  33. [41]

    Meta-weight-net: Learning an explicit mapping for sample weighting, 2019

    Jun Shu, Qi Xie, Lixuan Yi, Qian Zhao, Sanping Zhou, Zongben Xu, and Deyu Meng. Meta-weight-net: Learning an explicit mapping for sample weighting, 2019. 1, 2

    2019

  34. [42]

    Soft-label dataset distillation and text dataset distillation

    Ilia Sucholutsky and Matthias Schonlau. Soft-label dataset distillation and text dataset distillation. In2021 International Joint Conference on Neural Networks (IJCNN) , pages 1–8. IEEE, 2021. 3

    2021

  35. [43]

    Pi-dual: Using privileged information to distinguish clean from noisy labels, 2024

    Ke Wang, Guillermo Ortiz-Jimenez, Rodolphe Jenatton, Mark Collier, Efi Kokiopoulou, and Pascal Frossard. Pi-dual: Using privileged information to distinguish clean from noisy labels, 2024. 1, 2

    2024

  36. [44]

    Dataset distillation

    Tongzhou Wang, Jun-Yan Zhu, Antonio Torralba, and Alexei A Efros. Dataset distillation. arXiv preprint arXiv:1811.10959, 2018. 2, 3, 4

    2018 arXiv

  37. [45]

    Iterative learning with open-set noisy labels, 2018

    Yisen Wang, Weiyang Liu, Xingjun Ma, James Bailey, Hongyuan Zha, Le Song, and Shu-Tao Xia. Iterative learning with open-set noisy labels, 2018. 1

    2018

  38. [46]

    Learning with noisy labels re- visited: A study using real-world human annotations

    Jiaheng Wei, Zhaowei Zhu, Hao Cheng, Tongliang Liu, Gang Niu, and Yang Liu. Learning with noisy labels re- visited: A study using real-world human annotations. arXiv preprint arXiv:2110.12088, 2021. 3, 6, 8

    2021 arXiv

  39. [47]

    Robust early-learning: Hindering the memorization of noisy labels

    Xiaobo Xia, Tongliang Liu, Bo Han, Chen Gong, Nannan Wang, Zongyuan Ge, and Yi Chang. Robust early-learning: Hindering the memorization of noisy labels. InInternational conference on learning representations, 2020. 2, 4

    2020

  40. [48]

    Learning from massive noisy labeled data for im- age classification

    Tong Xiao, Tian Xia, Yi Yang, Chang Huang, and Xiaogang Wang. Learning from massive noisy labeled data for im- age classification. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2691–2699,

  41. [49]

    Estimating instance-dependent bayes-label transition matrix using a deep neural network,

    Shuo Yang, Erkun Yang, Bo Han, Yang Liu, Min Xu, Gang Niu, and Tongliang Liu. Estimating instance-dependent bayes-label transition matrix using a deep neural network,

  42. [50]

    Dual t: Reducing estimation error for transition matrix in label-noise learning,

    Yu Yao, Tongliang Liu, Bo Han, Mingming Gong, Jiankang Deng, Gang Niu, and Masashi Sugiyama. Dual t: Reducing estimation error for transition matrix in label-noise learning,

  43. [51]

    Dataset dis- tillation: A comprehensive review

    Ruonan Yu, Songhua Liu, and Xinchao Wang. Dataset dis- tillation: A comprehensive review. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023. 3

    2023

  44. [52]

    Learning with biased complementary labels, 2018

    Xiyu Yu, Tongliang Liu, Mingming Gong, and Dacheng Tao. Learning with biased complementary labels, 2018. 1, 2

    2018

  45. [53]

    How does disagreement help gener- alization against label corruption? In International confer- ence on machine learning, pages 7164–7173

    Xingrui Yu, Bo Han, Jiangchao Yao, Gang Niu, Ivor Tsang, and Masashi Sugiyama. How does disagreement help gener- alization against label corruption? In International confer- ence on machine learning, pages 7164–7173. PMLR, 2019. 2, 4

    2019

  46. [54]

    Dance: Dual-view distri- bution alignment for dataset condensation, 2024

    Hansong Zhang, Shikun Li, Fanzhao Lin, Weiping Wang, Zhenxing Qian, and Shiming Ge. Dance: Dual-view distri- bution alignment for dataset condensation, 2024. 2, 3, 4

    2024

  47. [55]

    Generalized cross entropy loss for training deep neural networks with noisy labels

    Zhilu Zhang and Mert Sabuncu. Generalized cross entropy loss for training deep neural networks with noisy labels. Ad- vances in neural information processing systems , 31, 2018. 1, 2, 3, 6

    2018

  48. [56]

    Dataset condensation with differ- entiable siamese augmentation, 2021

    Bo Zhao and Hakan Bilen. Dataset condensation with differ- entiable siamese augmentation, 2021. 6

    2021

  49. [57]

    Dataset condensation with dis- tribution matching

    Bo Zhao and Hakan Bilen. Dataset condensation with dis- tribution matching. In Proceedings of the IEEE/CVF Win- ter Conference on Applications of Computer Vision , pages 6514–6523, 2023. 3, 6

    2023

  50. [58]

    Dataset condensation with gradient matching

    Bo Zhao, Konda Reddy Mopuri, and Hakan Bilen. Dataset condensation with gradient matching. arXiv preprint arXiv:2006.05929, 2020. 3

    2006 arXiv

  51. [59]

    Im- proved distribution matching for dataset condensation, 2023

    Ganlong Zhao, Guanbin Li, Yipeng Qin, and Yizhou Yu. Im- proved distribution matching for dataset condensation, 2023. 3

    2023

  52. [60]

    Dataset distillation using neural feature regression.Advances in Neu- ral Information Processing Systems, 35:9813–9827, 2022

    Yongchao Zhou, Ehsan Nezhadarya, and Jimmy Ba. Dataset distillation using neural feature regression.Advances in Neu- ral Information Processing Systems, 35:9813–9827, 2022. 5 10

    2022

  53. [61]

    Lungren, and Lei Xing

    Yuyin Zhou, Xianhang Li, Fengze Liu, Qingyue Wei, Xuxi Chen, Lequan Yu, Cihang Xie, Matthew P. Lungren, and Lei Xing. L2b: Learning to bootstrap robust models for combat- ing label noise, 2024. 1, 2 11

    2024

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.