Pith. sign in

REVIEW 5 major objections 6 minor 21 references

Enhancing Deep Learning Model Robustness through Metamorphic Re-Training

T0 review · 5 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Adaptive metamorphic retraining—feeding failed metamorphic tests back as strong augmentations—raises the robustness of image models by an average of 17 percent.

desk verdict A well-intentioned student practicum with a real system built, but the circular robustness metric and unsupported 'flat 17%' claim make the central result unreliable. read the letter →

arxiv 2412.01958 v1 pith:L6OZBMVE submitted 2024-12-02 cs.CV cs.AI

Said Togru , Youssef Sameh Mostafa , Karim Lotfy This is my paper · ORCID
classification cs.CVcs.AI
keywords metamorphictestingrelationssemi-supervisedlearningmodelrobustnessadaptiveretrainingFixMatchCIFAR-10datascarcity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that retraining a model on data transformed by metamorphic relations—systematic changes to inputs whose expected effect on the output is known—can make the model more robust to input variation, particularly when labeled data is scarce. The authors build a loop that tests a model with metamorphic relations, collects the tests it fails, and feeds those failures back as strong augmentations for semi-supervised retraining, repeating the cycle. Across CIFAR-10, CIFAR-100, and MNIST with several architectures, they report an average flat 17 percent increase in their metamorphic-test success rate after retraining. The claim matters because it promises a way to improve model reliability without collecting new labeled data, by turning the model's own test failures into training signal.

What carries the argument

The mechanism is a feedback loop built on metamorphic relations (MRs), which specify how a transformed input's expected output relates to the original input's output. The GeMTest framework generates transformed inputs and checks whether the model's output follows the relation; tests that fail are extracted as functions and used to augment the training data. Four semi-supervised algorithms—FixMatch, FlexMatch, MixMatch, and FullMatch—provide the retraining objectives. The robustness metric is the success rate on metamorphic tests, $SR_{MT}=\frac{1}{N}\sum_{i=1}^{N} MTest(x_i, M)$, where $MTest$ returns 1 when the model's output after transformation is consistent with the relation.

What would settle it

Train one model with the adaptive metamorphic retraining loop and a matched control with the same loop but replacing metamorphic transformations with random augmentations of equal volume; then evaluate both on a held-out suite of metamorphic relations not used in training. If the control matches or beats the metamorphic-retrained model on those held-out relations, the reported robustness gains come from extra augmentation quantity rather than from the metamorphic structure.

Watch

Extended reading notes

Core claim

On the authors' terms, the central discovery is that adaptive metamorphic retraining—using failed metamorphic tests as strong augmentations in a semi-supervised retraining loop—reliably raises a model's success rate on metamorphic tests while keeping accuracy close to or above the base method in data-scarce settings. They report that every model they tested saw an additional flat 17 percent on average in this robustness metric. The adaptive scheme beats both the base semi-supervised algorithms and a static metamorphic retraining scheme that does not distinguish passed from failed tests. They also find that pretrained models gain more from the procedure than models trained from scratch, and that the benefit mostly disappears when labeled data is abundant, where all configurations reach 100 percent robustness.

Load-bearing premise

The load-bearing premise is that passing metamorphic tests that were used to generate the retraining data reflects genuine robustness to real-world input variation, rather than overfitting to the specific metamorphic relations in the test pool.

Editorial extensions

If this is right

  • In data-scarce conditions, adaptive metamorphic retraining raises robustness on metamorphic tests by roughly 17 percentage points on average over the base semi-supervised methods.
  • Pretrained models improve in both accuracy and robustness under adaptive retraining, suggesting the method can harden existing models without full retraining.
  • When labeled data is plentiful, the robustness advantage of metamorphic retraining vanishes: base algorithms already reach 100 percent robustness, so the framework is most valuable where labels are scarce.
  • For non-label-preserving transformations, such as a 180-degree rotation that changes a '2' into a '5', FullMatch with adaptive retraining handles the label changes best among the algorithms tested.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A decisive follow-up experiment the paper does not run is to hold out a set of metamorphic relations from the retraining loop and test robustness on those unseen relations, separating genuine generalization from memorization of the relation pool.
  • The reported 'flat 17 percent' is an average over heterogeneous baselines; comparing raw percentage-point gains without normalizing by each baseline's starting robustness could overstate the effect for already-robust models.
  • The loop is not image-specific: any task with cheaply definable metamorphic relations, such as paraphrase or negation in text or noise and speed changes in speech, could use the same adaptive retraining scheme.
  • Because the robustness metric is the success rate on metamorphic tests, a degenerate constant-output model would score perfectly; the paper acknowledges this and reports accuracy alongside, but a single combined accuracy-robustness score would make the trade-off more interpretable.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. This paper proposes a 'Metamorphic Retraining Framework' that uses metamorphic relations (MRs), taken from the GeMTest metamorphic-testing tool, to generate augmented training data for semi-supervised retraining of image classifiers. The pipeline iterates over 'robustness cycles': the model is evaluated by its success rate on metamorphic tests (SR_MT, Eq. 1), failing and passing tests are analyzed, and the next cycle's training data are generated either adaptively (failed tests become strong augmentations) or statically (a fixed mixture of MRs). Four semi-supervised algorithms (FixMatch, FlexMatch, MixMatch, FullMatch) are compared on CIFAR-10, CIFAR-100, and MNIST with ResNet-32/50, VGG16, and FCN architectures, under data-scarce and data-rich conditions, with pretrained and non-pretrained initializations, and with label-preserving and non-label-preserving transformations. The abstract claims that 'each model witnessed an increase of an additional flat 17 percent on average' in the robustness metric.

Significance. The intended contribution—an automated, algorithm-agnostic loop that converts metamorphic test failures into targeted semi-supervised retraining signal—is a plausible and practically attractive idea, and the breadth of the experiments (data scarcity, pretraining, non-label-preserving rotations such as the 2-to-5 and 6-to-9 mapping, static versus adaptive augmentation) is commendable for a first study. The paper does not, however, ship machine-checked proofs, reproducible code, or a falsifiable independent prediction; the core pipeline is stated to be closed-source, and the metamorphic relations used are never enumerated. Because the robustness metric and the training data derive from the same MR pool, the reported gains in SR_MT are expected by construction and uninterpretable as evidence of robustness to novel input variation. As presented, the contribution is a plausible recipe in need of a properly controlled evaluation rather than a demonstrated result.

major comments (5)
  1. [Abstract; Tables I-V] The headline claim—that 'each model witnessed an increase of an additional flat 17 percent on average in our robustness metric'—is contradicted by every table in the paper. Table I shows average robustness rising from 75.75% to 84.75% (+9 percentage points), Table II shows no average change (72.5% to 72.5%), Table III shows +10.5 points between random initialization and pretraining, Table IV shows 100% before and after (no gain), and Table V shows +9 points. No reported comparison yields a 17-point gain under any interpretation. The companion claim that each model improved is also false: in Table I, FlexMatch robustness drops from 90% to 83%; in Table II, FixMatch drops from 79% to 57% and FlexMatch from 100% to 80%. The paper's central quantitative claim is falsified by its own results, and the narrative in Sec. V.B.3 that Table II shows 'the same data patterns' is not supported by the numbers.
  2. [Sec. IV.A; Eq. (1); Sec. V.A] The evaluation design is circular with respect to the robustness claim. Robustness is defined as SR_MT, the success rate on metamorphic tests generated from a pool of MRs (Eq. 1), and the retraining data are generated from the same MR pool (Sec. IV.A.2); the adaptive method explicitly uses failed tests as strong augmentations for the next cycle (Sec. IV.A.3). A model retrained on those tests is expected to score higher on them by construction, regardless of any change in robustness to novel input variations. The paper never evaluates on held-out MRs, on transformation types absent from the training pool, or on an independent corruption benchmark, and the MR list is never given. A concrete remedy: hold out a subset of MRs (or use a standard corruption suite such as CIFAR-10-C) and report SR_MT there together with top-1 accuracy; without such a test, the load-bearing premise that MR test success indicates genuine robustness is untested.
  3. [Sec. V.A.1; Tables II and IV] The paper itself warns that a model that 'consistently outputs the same value for all inputs' achieves perfect robustness while being practically useless, yet it never applies this caveat to its own tables. In Table II, Base FlexMatch attains 100% robustness with 6% top-5 accuracy on CIFAR-100, where random guessing already yields 5% top-5 accuracy; this is exactly the degenerate-success pattern the paper describes. The same concern applies to Table IV, where every configuration reaches 'perfect robustification' (100% SR_MT) while top-5 accuracy is only 24-28%—values the text calls 'state-of-the-art,' which they are not. Table IV's robustness numbers are uninterpretable without evidence that the models' predictions remain informative.
  4. [Sec. III.C; Sec. IV.A] The method is not reproducible as described. The paper never enumerates the metamorphic relations used for each dataset (how many, which transformations, label-preserving versus non-label-preserving), and Sec. III.C states that 'the detailed framework and methods, including Figure 2, are part of a closed-source development and are not currently publicly available.' No code is released. Without the MR inventory and the pipeline implementation, no result in the paper can be reproduced, and the reader cannot even verify that the transformations used for evaluation are disjoint from those used for training—a condition that the paper's claims require.
  5. [Sec. V.C; Tables I-V] All results are single runs with no seeds, no variance, no repeated trials, and no statistical tests, so the word 'significant' in the claimed improvement is unsupported. In addition, Table III pools results from two datasets of very different difficulty (CIFAR-10 and MNIST) into a single average without a per-dataset breakdown, and the accuracy metric throughout is top-5, which for 10-class datasets has a 50% random baseline and for 100-class CIFAR-100 renders values of 6-22% nearly indistinguishable from chance. The paper should report top-1 accuracy, per-dataset results, and measures of dispersion.
minor comments (6)
  1. [Sec. III.F; Sec. V.B.3] Section titles contain stray spaces ('CIF AR-10', 'CIF AR-100'); 'test suits' should be 'test suites' (Sec. IV.A.2); and 'illustrated in I' and 'The II shows' should read 'Table I' and 'Table II'.
  2. [Eq. (2)] Equation (2) is labeled Acctop5 but the surrounding text defines top-N accuracy and the formula uses 'Top-N Predictions'; the notation should be made consistent.
  3. [Sec. V.E] Experiment 4 (non-label-preserving transformations) reports no quantitative results in the text; Figure 8 is referenced but its numbers are not described, so the conclusion that FullMatch 'performed the best' cannot be checked from the text.
  4. [Sec. V.A.1; Fig. 6] The 'hypothetical figure' of a constant-output model that achieves perfect robustness is promised but never shown, and Figure 6 is referenced without being discussed.
  5. [Related work; ref. [4]] The paper cites Chen et al. (ref. [4]), which already uses metamorphic relations as knowledge carriers to train deep neural networks, but never states how the proposed framework differs from or extends that prior work; an explicit comparison is needed to substantiate the novelty claim.
  6. [Sec. VII] The conclusion refers to 'employing a stopping criterion based on performance thresholds,' but no stopping criterion is defined or used in any experiment; all reported runs appear to use a fixed 30 cycles.

Circularity Check

2 steps flagged · score 8.0 of 10

The headline robustness gain is measured on the same metamorphic relations used to generate the retraining data, so SR_MT improvement is an in-distribution fit, not a prediction of robustness.

  1. self definitional [Sec. IV.A.1 and Sec. V.A.1 (Eq. 1)]
    "The key idea of our pipeline is to apply a repetitive feedback loop of Retraining with a data set augmented with the target metamorphic relations, evaluate the model, and test again on metamorphic tests formed from the available pool of metamorphic relations ... To quantify robustness, we use the success rate on metamorphic tests."

    Robustness is defined (Eq. 1) as SR_MT, the success rate on metamorphic tests built from the available MR pool. The retraining loop augments the training set with the same target metamorphic relations and then re-tests on metamorphic tests formed from the same pool. Thus the evaluation distribution equals the training augmentation distribution. An improved SR_MT after retraining shows that the model learned to satisfy these particular MRs; it is not evidence of robustness to unseen input variations. No held-out MRs or standard corruption benchmarks are reported, so the headline '17%' robustness gain is a fit to the evaluation distribution by construction.

  2. fitted input called prediction [Sec. IV.A.3 and Sec. V.B]
    "The Adaptive Method : which specifies the failed tests as strong augmentations for the semi-supervised algorithms."

    The adaptive retraining method takes the failed metamorphic tests — the very items scored by the SR_MT robustness metric — and uses them as strong augmentations for the next training cycle. The model is therefore explicitly trained on the failed cases of the evaluation set (or its transformation family), and the reported robustness improvement is the success rate on those same metamorphic tests after training on them. This is the 'fitted input called prediction' pattern: the failed tests are training inputs, and the subsequent SR_MT score is a measure of memorization/fitting rather than independent prediction of robustness.

full rationale

The central robustness claim is circular by design. The paper defines robustness (Eq. 1) as the success rate on metamorphic tests (SR_MT) and then trains the model, in the adaptive mode, on failed metamorphic tests used as strong augmentations (Sec. IV.A.3), with training data generated from the same MR pool (Sec. IV.A.1). Therefore the reported improvement on SR_MT is a measure of fit to the evaluation transformation distribution, not of generalization to new input variations. The paper never evaluates on held-out metamorphic relations or standard corruption benchmarks; even its own caveat (Sec. V.A.1) admits a constant-output model can achieve perfect SR_MT. The headline 'flat 17 percent' gain also contradicts the paper's own tables (Table I +9pp, Table II 0pp, Table III +10.5pp, Table V +9pp), which further undermines the claim, though that inconsistency is a correctness issue rather than additional circularity. No self-citation chain or imported uniqueness theorem is load-bearing, and the algorithm implementations are standard, so the circularity is confined to the evaluation/retraining overlap. Score 8: the central robustness result reduces by construction to training on the same metamorphic tests used for evaluation.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on several unstated assumptions: that metamorphic relations preserve the semantics needed for training, that the metamorphic-test success rate is a valid robustness proxy, and that gains on the MR pool transfer outside it. The MR-augmented training and the robustness evaluation share the same transformation distribution, so the evaluation is not independent. Free parameters are mostly hand-chosen dataset fractions and training budgets, with key SSL hyperparameters unreported.

free parameters (5)
  • Labeled/unlabeled/test split ratios = 10/70/20, 8%, 1%, 0.1%
    Dataset fractions are chosen by hand across experiments to simulate data scarcity, with no justification or ablation.
  • Robustness cycles and epochs per cycle = 30 cycles, 2 epochs per cycle (1 epoch in Exp. 4)
    Training budget chosen ad hoc; no convergence analysis or sensitivity study.
  • Pseudo-label confidence threshold (tau) = not reported
    FixMatch and FlexMatch rely on a confidence threshold for pseudo-labeling; the value is not stated, yet it directly affects training behavior.
  • Loss weights (lambda_u, lambda_p) = not reported
    Weights for unsupervised and penalty losses in the SSL objectives are standard but tuned; no values are given.
  • MixUp coefficient (gamma) = not reported
    MixMatch samples gamma from a Beta distribution; the distribution parameters are not reported.
assumptions (5)
  • domain assumption Metamorphic relations preserve or systematically alter the semantic label of an input.
    The framework relies on MRs producing valid follow-up inputs with known target labels (Sec. III.B). If an MR is invalid for a class, training on it injects label noise.
  • domain assumption Success rate on metamorphic tests is a valid measure of model robustness.
    Eq. 1 defines robustness as the MT success rate; the paper asserts this metric reflects real-world robustness (Sec. V.A), but a constant-output model can score perfectly, so accuracy must be coupled.
  • domain assumption Training on metamorphically augmented data transfers to improved robustness outside the MR pool.
    The central claim assumes the gains are not just memorization of the specific MRs used in training (Sec. IV.A).
  • standard math The cited semi-supervised algorithms are implemented faithfully.
    The paper uses published algorithms (FixMatch, FlexMatch, MixMatch, FullMatch) and does not provide implementation details or code to verify correctness.
  • domain assumption 180-degree rotation maps MNIST '2' to '5' and '6' to '9'.
    Sec. V.E uses this specific non-label-preserving transformation; if the mapping is wrong for the actual dataset, the labels in the augmented set are incorrect.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Enhancing Deep Learning Model Robustness through Metamorphic Re-Training." pith.science (2026). https://pith.science/paper/L6OZBMVE

@misc{pith2026241201958,
  author       = {Pith},
  title        = {Pith review of: Enhancing Deep Learning Model Robustness through Metamorphic Re-Training},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/L6OZBMVE}},
  note         = {Machine review of arXiv:2412.01958}
}
read the original abstract

This paper evaluates the use of metamorphic relations to enhance the robustness and real-world performance of machine learning models. We propose a Metamorphic Retraining Framework, which applies metamorphic relations to data and utilizes semi-supervised learning algorithms in an iterative and adaptive multi-cycle process. The framework integrates multiple semi-supervised retraining algorithms, including FixMatch, FlexMatch, MixMatch, and FullMatch, to automate the retraining, evaluation, and testing of models with specified configurations. To assess the effectiveness of this approach, we conducted experiments on CIFAR-10, CIFAR-100, and MNIST datasets using a variety of image processing models, both pretrained and non-pretrained. Our results demonstrate the potential of metamorphic retraining to significantly improve model robustness as we show in our results that each model witnessed an increase of an additional flat 17 percent on average in our robustness metric.

Figures

Figures reproduced from arXiv: 2412.01958 by the authors.

Figure 1
Figure 1. Metamorphic Testing Process [15] the resilience of neural networks under adversarial conditions [6] [9]. Our work aims to enhance robustness against a differ￾ent type of input variation—those captured by metamorphic relations. Metamorphic testing evaluates the robustness and reliability of neural networks by systematically transforming inputs and assessing model responses. While adversarial at￾tacks and metamorphic … view at source ↗
Figure 2
Figure 2. Workflow of the Metamorphic Testing Framework [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Architecture of FixMatch [18] 1) FixMatch: FixMatch simplifies the use of pseudo-labels by applying consistency regularization between weakly and strongly augmented versions of the same data. It enforces high-confidence predictions for unlabeled data to be treated as labels, guided by a static confidence threshold: L = Lsup + λu · Lunsup Where Lsup represents the loss on labeled data and Lunsup is the loss on pseudo… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Architecture of FlexMatch [18] 2) FlexMatch: FlexMatch extends FixMatch by dynami￾cally adjusting the confidence thresholds based on the class￾wise difficulty, thus addressing the class imbalance: L = Lsup + λu · X C c=1 τcLunsup,c where C is the number of classes, τc …
Figure 5
Figure 5. Figure 5: Architecture of our proposed pipeline A. Metrics 1) Robustness: Robustness is a critical metric for evaluating ML models, particularly in real-world applications where input data can vary significantly. According to [3], robustness denotes the capacity of a model to su…
Figure 6
Figure 6. Figure 6: Robustness and accuracy. 2) Accuracy: Accuracy is a fundamental metric for eval￾uating the performance of a machine-learning model. In our experiments, we use top-N accuracy on the test set as a key performance indicator. This metric is particularly useful for classifi…
Figure 7
Figure 7. Figure 7: Label preserving transformation vs non-label preserving [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 8
Figure 8. Figure 8: Accuracy on different sets. Comparing the performance on label [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

21 extracted references · 10 canonical work pages

  1. [1]

    Towards improving robustness of deep neural networks to adversarial perturbations

    Sajjad Amini and Shahrokh Ghaemmaghami. Towards improving robustness of deep neural networks to adversarial perturbations. IEEE Transactions on Multimedia , 22(7):1889–1903, 2020

    1903

  2. [2]

    Goodfellow, Nicolas Papernot, Avital Oliver, and Colin Raffel

    David Berthelot, Nicholas Carlini, Ian J. Goodfellow, Nicolas Papernot, Avital Oliver, and Colin Raffel. Mixmatch: A holistic approach to semi- supervised learning. CoRR, abs/1905.02249, 2019

    arXiv 1905

  3. [3]

    Machine learning robustness: A primer, 2024

    Houssem Ben Braiek and Foutse Khomh. Machine learning robustness: A primer, 2024

    work page 2024

  4. [4]

    Use of Metamorphic Relations as Knowledge Carriers to Train Deep Neural Networks

    Tsong Yueh Chen, Pak-Lok Poon, Kun Qiu, Zheng Zheng, and Jinyi Zhou. Use of metamorphic relations as knowledge carriers to train deep neural networks. CoRR, abs/2104.04718, 2021

    work page Pith review arXiv 2021

  5. [5]

    Boosting semi-supervised learning by exploiting all unlabeled data, 2023

    Yuhao Chen, Xin Tan, Borui Zhao, Zhaowei Chen, Renjie Song, Jiajun Liang, and Xuequan Lu. Boosting semi-supervised learning by exploiting all unlabeled data, 2023

    work page 2023

  6. [6]

    Zico Kolter

    Jeremy Cohen, Elan Rosenfeld, and J. Zico Kolter. Certified adversarial robustness via randomized smoothing. CoRR, abs/1902.02918, 2019

    arXiv 1902

  7. [7]

    The mnist database of handwritten digit images for machine learning research

    Li Deng. The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine , 29(6):141–142, 2012

    2012

  8. [8]

    Onnx runtime

    ONNX Runtime developers. Onnx runtime. https://onnxruntime.ai/,

    work page

Show all 21 references
  1. [9]

    Mann, and Pushmeet Kohli

    Sven Gowal, Krishnamurthy Dvijotham, Robert Stanforth, Rudy Bunel, Chongli Qin, Jonathan Uesato, Relja Arandjelovic, Timothy A. Mann, and Pushmeet Kohli. On the effectiveness of interval bound propagation for training verifiably robust models. CoRR, abs/1810.12715, 2018

    arXiv

  2. [10]

    Robust semi-supervised learning when not all classes have labels

    Lan-Zhe Guo, Yi-Ge Zhang, Zhi-Fan Wu, Jie-Jing Shao, and Yu-Feng Li. Robust semi-supervised learning when not all classes have labels. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, p...

    2022

  3. [11]

    Deep residual learning for image recognition, 2015

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition, 2015

    2015

  4. [12]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky. Learning multiple layers of features from tiny images. pages 32–33, 2009

    2009

  5. [13]

    Fully convolutional networks for semantic segmentation

    Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 3431–3440, 2015. 13

    2015

  6. [14]

    Towards deep learning models resistant to adversarial attacks, 2019

    Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks, 2019

    2019

  7. [15]

    Leveraging mutants for automatic prediction of metamorphic relations using machine learning, 08 2019

    Aravind Nair, Karl Meinke, and Sigrid Eldh. Leveraging mutants for automatic prediction of metamorphic relations using machine learning, 08 2019

    2019

  8. [16]

    Optimism in the face of adversity: Un- derstanding and improving deep learning through adversarial robustness

    Guillermo Ortiz-Jim ´enez, Apostolos Modas, Seyed-Mohsen Moosavi- Dezfooli, and Pascal Frossard. Optimism in the face of adversity: Un- derstanding and improving deep learning through adversarial robustness. Proceedings of the IEEE , 109(5):635–659, 2021

    2021

  9. [17]

    Very deep convolutional networks for large-scale image recognition

    Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. CoRR, abs/1409.1556

    arXiv

  10. [18]

    Cubuk, Alex Kurakin, Han Zhang, and Colin Raffel

    Kihyuk Sohn, David Berthelot, Chun-Liang Li, Zizhao Zhang, Nicholas Carlini, Ekin D. Cubuk, Alex Kurakin, Han Zhang, and Colin Raffel. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. CoRR, abs/2001.07685, 2020

    2001 arXiv

  11. [19]

    Apress, Berkeley, CA, 2020

    Andreas Franc ¸ois Vermeulen.Supervised Learning: Using Labeled Data for Insights, pages 63–136. Apress, Berkeley, CA, 2020

    2020

  12. [20]

    Flexmatch: Boosting semi-supervised learning with curriculum pseudo labeling

    Bowen Zhang, Yidong Wang, Wenxin Hou, Hao Wu, Jindong Wang, Manabu Okumura, and Takahiro Shinozaki. Flexmatch: Boosting semi-supervised learning with curriculum pseudo labeling. CoRR, abs/2110.08263, 2021

    2021 arXiv

  13. [21]

    Zhang, Mark Harman, Lei Ma, and Yang Liu

    Jie M. Zhang, Mark Harman, Lei Ma, and Yang Liu. Machine learning testing: Survey, landscapes and horizons. IEEE Transactions on Software Engineering, 48(1):1–36, 2022. 14

    2022

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.