REVIEW 4 cited by
Self-Training: A Survey
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Semi-supervised algorithms aim to learn prediction functions from a small set of labeled observations and a large set of unlabeled observations. Because this framework is relevant in many applications, they have received a lot of interest in both academia and industry. Among the existing techniques, self-training methods have undoubtedly attracted greater attention in recent years. These models are designed to find the decision boundary on low density regions without making additional assumptions about the data distribution, and use the unsigned output score of a learned classifier, or its margin, as an indicator of confidence. The working principle of self-training algorithms is to learn a classifier iteratively by assigning pseudo-labels to the set of unlabeled training samples with a margin greater than a certain threshold. The pseudo-labeled examples are then used to enrich the labeled training data and to train a new classifier in conjunction with the labeled training set. In this paper, we present self-training methods for binary and multi-class classification; as well as their variants and two related approaches, namely consistency-based approaches and transductive learning. We examine the impact of significant self-training features on various methods, using different general and image classification benchmarks, and we discuss our ideas for future research in self-training. To the best of our knowledge, this is the first thorough and complete survey on this subject.
Forward citations
Cited by 4 Pith papers
-
Embarrassingly Simple Self-Distillation Improves Code Generation
Simple self-distillation—fine-tuning a code model on its own temperature-sampled, truncated outputs—raises LiveCodeBench pass@1 substantially without verifiers, teachers, or RL.
-
Improving Data and Parameter Efficiency of Neural Language Models Using Representation Analysis
Representation smoothness can be used to regularize training, stop early without validation labels, and guide active learning combined with parameter-efficient fine-tuning, reducing data and compute.
-
Self-Boost via Optimal Retraining: An Analysis via Approximate Message Passing
For binary classification with noisy labels, the paper derives the Bayes-optimal function for combining a model's current predictions with the given labels during retraining, and shows a fitted version improves linear...
-
ADAPT: A Pseudo-labeling Approach to Combat Concept Drift in Malware Detection
A pseudo-labeling method with class-specific adaptive thresholds, label-consistent augmentation, and mixup reduces concept-drift performance loss in malware classifiers across five datasets.
Discussion (0). Continue with ORCID to comment.