Pith. sign in

REVIEW 4 major objections 5 minor 85 references

SST: Self-training with Self-adaptive Thresholding for Semi-supervised Learning

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Self-adaptive pseudo-label thresholds let a ViT-Huge trained on 10% of ImageNet's labels reach 84.9% Top-1 accuracy, beating the fully supervised 84.8% that uses all the labels.

desk verdict Per-cycle adaptive thresholding is a clean efficiency idea, but the '10% labels beats 100% labels' headline rests on an uncontrolled pretrained-vs-from-scratch comparison. read the letter →

arxiv 2506.00467 v1 pith:PZMUMZMU submitted 2025-05-31 cs.LG cs.AIcs.CV

Shuai Zhao , Heyan Huang , Xinge Li , Xiaokang Chen , Rui Wang This is my paper · ORCID
classification cs.LGcs.AIcs.CV
keywords semi-supervisedlearningself-trainingpseudo-labelingself-adaptivethresholdingconfirmationbiasImageNet-1KvisiontransformerEMA-teacher
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes SST, a semi-supervised learning framework built around a rule for deciding which automatically generated pseudo-labels to trust. That rule, Self-Adaptive Thresholding (SAT), derives a separate acceptance threshold for each class from the model's own confidence statistics and re-derives it only once per training cycle, when the model has converged. The paper's claim is that this selects pseudo-labels that are both accurate enough and plentiful enough that retraining on them rivals training on human labels, cutting annotation cost roughly tenfold. Support for the claim is the reported 84.9% Top-1 accuracy on ImageNet-1K with 10% labeled data and 80.7% with 1%, where the fully supervised DeiT-III-ViT-Huge achieves 84.8% with 100%.

What carries the argument

The load-bearing object is SAT, Self-Adaptive Thresholding, the mechanism that turns the model's current confidence statistics into a per-class acceptance threshold. For each class it sorts the predicted probabilities over the unlabeled pool, removes every probability at or below a cutoff $C$ (0.5 by default), averages the remaining probabilities, and multiplies by a scaling factor $S$ (about 0.8-0.9) to obtain that class's threshold $\tau_j = S\,\bar{p}_j$; a pseudo-label is accepted when its confidence exceeds the threshold of its predicted class. SAT sits inside a cyclic self-training loop: predict with the converged model, apply SAT, retrain on the original labeled data plus the accepted pseudo-labels, repeat. In Semi-SST the same thresholds also gate the online pseudo-labels produced by an EMA-Teacher. The defining design choice is that thresholds are updated only a handful of times per run (three updates on ImageNet-1K), whereas competing adaptive methods such as FlexMatch and FreeMatch update every iteration, which the paper argues is computationally wasteful and feeds confirmation bias because their thresholds are estimated from models still in training.

What would settle it

Freeze the class thresholds at the values SAT derives from the first converged checkpoint and run the rest of the Super-SST loop on ImageNet-1K with 1% labels without re-estimating them; if this frozen-threshold run reaches the same 70.4% Top-1 accuracy as the full algorithm, the cycle-by-cycle re-estimation that is the paper's core mechanism is not what carries the result, while a clear shortfall would confirm that tracking learning progress per cycle does real work. A companion check follows from the paper's own numbers: accepted-pseudo-label accuracy slides from 89.5% to 77.7% over cycles, so extending the run until that accuracy falls below the accuracy of the original labeled set should, by the paper's account, eventually produce a peak followed by a decline.

Watch

Extended reading notes

Core claim

The central claim is that the quantity-quality trade-off in pseudo-labeling is best managed by class-specific thresholds computed from the model's own confidence distribution and refreshed only when the model has converged, not at every training step. After each cycle, SAT sorts the softmax probabilities for each class, discards probabilities at or below a cutoff $C$, averages the survivors, and scales that average by a factor $S$ to set the class threshold $\tau_j = S\,\bar{p}_j$; an unlabeled sample is admitted to the training set when its top predicted probability exceeds the threshold of its predicted class. Classes the model has actually learned well therefore contribute many pseudo-labels, while uncertain classes contribute few, and because thresholds are recomputed from a well-trained model rather than a mid-training one, the paper argues that inaccurate pseudo-labels and the confirmation bias they reinforce are suppressed. The framework comes in two versions, Super-SST (offline SAT selection plus plain supervised training on the combined set) and Semi-SST (the same offline loop plus an EMA-Teacher, a teacher model updated as an exponential moving average of the student, that generates online pseudo-labels also gated by SAT), and the paper reports leading results with both: Semi-SST-ViT-Huge achieves 80.7% and 84.9% Top-1 accuracy on ImageNet-1K with 1% and 10% labeled data, against 84.8% for the fully supervised DeiT-III-ViT-Huge.

Load-bearing premise

The method assumes that any unlabeled image whose top predicted score clears its class threshold is as trustworthy as a human label and can be retrained on with plain cross-entropy, even though the paper's own measurements show the accuracy of accepted pseudo-labels falling from 89.5% to 77.7% across training cycles.

Editorial extensions

If this is right

  • Annotation cost falls by roughly tenfold for image classification: a ViT-Huge trained on 10% of ImageNet labels reaches 84.9% Top-1, slightly above the 84.8% of the same model family trained on all labels.
  • Thresholding becomes nearly free: Super-SST needs about three threshold updates per run versus over one million for FlexMatch and FreeMatch, while reporting higher accuracy on the 100K-label ImageNet protocol (64.42% versus 58.15% and 59.43%).
  • Gains concentrate where labels are scarcest: relative improvements over supervised and semi-supervised baselines are 0.7%-20.7% at 1% labels and 0.2%-12.1% at 10%, across ImageNet-1K, CIFAR-100, Food-101, and iNaturalist.
  • The gains survive distillation and imbalance: a ViT-Small distilled from a Super-SST ViT-Huge teacher reaches 80.3% Top-1 at 10% labels, above the fully supervised DeiT-ViT-Small's 79.8%, and Super-SST reaches 75.69% on Clothing-1M with 5% clean labels against a 78.99% fully supervised upper bound.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension the paper does not pursue: on very small labeled sets, which the paper's own limitation section flags with weak results on SVHN using only 40 labels, per-class confidence statistics are noisy, so a variance-aware correction that widens thresholds when a class has few confident predictions could extend SAT's gains into ultra-low-label regimes.
  • The paper's Figure 3 shows accepted-pseudo-label accuracy declining from 89.5% to 77.7% across cycles, so the method's ceiling is plausibly set by how long model improvement outruns the degradation of its own output; imposing a floor so that accepted pseudo-labels never drop below the accuracy of the original labeled set is a natural, untested variant.
  • The classical-benchmark comparison (Table 5) rests on a DINO-pretrained ViT-Small, whereas the prior methods it is ranked against typically use Wide ResNets, a mismatch the paper itself acknowledges; a same-architecture head-to-head would be the cleanest confirmation that SAT, rather than the backbone, drives the gains.
  • Because SAT needs only per-class confidence statistics and a converged checkpoint, the same machinery should transfer to object detection, segmentation, and NLP self-training, which the paper names as future work rather than demonstrated results.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Self-training with Self-adaptive Thresholding (SST), a semi-supervised learning framework with two variants: Super-SST, which repeatedly selects high-confidence pseudo-labels via class-specific thresholds and retrains on the expanded labeled set, and Semi-SST, which augments the offline pipeline with an EMA-teacher online pseudo-labeling component. The core contribution is the Self-Adaptive Thresholding (SAT) mechanism, which computes class-specific thresholds once per cycle from the model's own confidence distribution using a cutoff C and a scaling factor S. Experiments are reported on ImageNet-1K, CIFAR-10/100, SVHN, STL-10, Food-101, iNaturalist, and Clothing-1M, with claims of state-of-the-art performance, including 80.7%/84.9% Top-1 accuracy on ImageNet-1K with 1%/10% labels and a claim that 10%-label Semi-SST-ViT-Huge surpasses fully-supervised DeiT-III-ViT-Huge on 100% labels.

Significance. The paper makes a useful methodological contribution in simplifying adaptive thresholding: updating class-specific thresholds once per training cycle from a converged model is an elegant way to avoid the per-iteration overhead of FlexMatch and FreeMatch, and the pseudo-code and architecture-agnostic framework make the method easy to reproduce. The controlled comparisons in Table 3, where the same pretrained backbone is used for baseline and proposed method, show consistent gains on CIFAR-100, Food-101, iNaturalist, and ImageNet-1K, which is evidence that SST can improve over Semi-ViT in a matched setting. The efficiency claims (3 vs. over 1,000,000 threshold updates) are plausible and well demonstrated in Table 7. However, the paper's headline claims — 'best results on competitive ImageNet-1K SSL benchmarks' and 'tenfold reduction in human annotation costs' — are not supported by the evidence as presented, because the key comparisons are uncontrolled, the hyperparameters C and S are tuned on the same benchmark used for the state-of-the-art claim, and the multi-benchmark comparison in Table 5 is confounded by architecture differences.

major comments (4)
  1. [Section 4.4, Table 6, Abstract] The claim that Semi-SST-ViT-Huge achieves 84.9% with 10% labels and surpasses fully-supervised DeiT-III's 84.8% is an uncontrolled comparison, since Section 4.1 states that all transformer experiments start from DINO or MAE pretrained weights, while DeiT-III is a from-scratch training recipe; the controlled comparison in Table 3 shows only a 0.6-point gain over Semi-ViT-ViT-Huge with no error bars. To support the tenfold-annotation-saving claim, the authors must either train a fully supervised baseline with the same pretrained initialization and augmentation recipe, or substantially soften the claim.
  2. [Section 4.7, Figure 2] The cutoff C and scaling factor S are selected by ablation on the same ImageNet-1K benchmark (1% labeled data) and then used to produce the headline 1%/10% results, with no held-out validation split mentioned; this selection on the target benchmark can inflate the reported gains. Please report the chosen hyperparameters' performance on a validation split or demonstrate that the results are stable across a range of C and S on all datasets, not just ImageNet-1K.
  3. [Table 5 and Section 4.3] The Friedman test and mean-error-rate ranking are confounded by the architecture mismatch: the proposed methods use a DINO-pretrained ViT-Small while all baselines are WRN/ResNet models trained from scratch. The SVHN 40-label results (Super-SST 29.41%, Semi-SST 23.18%) are orders of magnitude worse than the baselines (e.g., FixMatch 3.37%), and the paper's own limitation section acknowledges this; as a result, the 'best overall' claim is not supported by a fair comparison.
  4. [Section 3, Eq. (9) and Figure 3] The paper treats SAT-selected pseudo-labels as 'equally reliable as human labels' when forming the combined training set, yet Figure 3 shows the accuracy of selected pseudo-labels declining from 89.5% to 77.7% across cycles; the manuscript provides no analysis of how error accumulation is controlled, and Section 5.1's promise of a 'detailed theoretical analysis' is not fulfilled anywhere in the paper. Please either provide such analysis or rephrase the confirmation-bias claims to be explicitly empirical.
minor comments (5)
  1. [Section 1, Table 8] The introduction states that on CIFAR-100 with 400/2500/10000 labels, FreeMatch attains a mean error rate of 28.71% using about 240 GPU hours, but Table 5 shows FreeMatch's CIFAR-100 mean is approximately 29.55%, and Table 8 reports FreeMatch's overall mean error rate as 12.51%; these numbers should be reconciled.
  2. [Section 5.1] The claim of 'Detailed theoretical analysis' is not backed by any theoretical section; either remove the phrase or add the analysis.
  3. [Algorithm 1, Section 3] The pseudo-code for Semi-SST is incomplete: it shows model.train(combined_labeled_data) but does not show the teacher model, weak/strong augmentations, or the online pseudo-labeling loss; please align Algorithm 1 with the description in Section 3 and Figure 1b.
  4. [Table 10, Section 4.2] The hyperparameters for Super-SST and Semi-SST differ in multiple respects (weight decay, drop path, layer-wise decay rate, batch size), so the relative improvement of Semi-SST over Super-SST in Table 3 is not a clean ablation of the online pseudo-labeling component.
  5. [Figures 2 and 3] The axis labels in Figure 2 are garbled (e.g., '6 0.36 6.36 8.66 9.57 0.07 0.37 0.4T'), likely a rendering artifact; please regenerate the figures with correct labels.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the SAT selection rule is confidence-based by design, but the reported accuracy gains are empirical and not forced by construction.

full rationale

The paper's pipeline is an empirical self-training scheme rather than a formal derivation, and none of its central claims reduce to their inputs by construction. SAT thresholds (Eq. 6) are computed from the model's own softmax probabilities, and selection (Eq. 9) uses the same model's confidence; this is self-referential in the generic pseudo-labeling sense, but it does not make the accuracy results tautological. The claim that SAT 'ensures the selection of high-quality pseudo-labeled data' is rendered empirically checkable by Figure 3, which separately measures the accuracy of selected pseudo-labels against ground truth and shows it declining across cycles; a claim that can fail is not a definitional equivalence. The hyperparameters C and S are chosen via ablation on ImageNet (Figure 2), which is a model-selection concern rather than a circularity, since no reported Top-1 number is algebraically determined by those choices. There are no load-bearing self-citations: the pretrained backbones (DINO, MAE) and the EMA-teacher protocol come from external prior work, and all baselines are external. The statement in Section 5.1 that 'detailed theoretical analysis ... strongly confirm[s]' SAT is unsupported by any derivation in the paper, but that is a missing-support issue, not a circularity. Overall, the derivation chain is self-contained and the empirical claims are independently evaluated.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central method relies on two hand-tuned thresholds (C and S), an unreported unlabeled-loss weight, and the domain assumption that model confidence tracks label correctness. No independent or machine-checked support is provided for the thresholding rule.

free parameters (3)
  • cutoff value C = 0.5 (ViT-Small and ViT-Huge, ImageNet-1K)
    Chosen from ablation on 1% ImageNet-1K (Figure 2a); it determines which probabilities enter the class threshold average in Eq. 4.
  • scaling factor S = 0.8 (ViT-Small), 0.9 (ViT-Huge), ImageNet-1K
    Chosen from ablation on 1% ImageNet-1K (Figure 2b); it directly multiplies the per-class mean to produce thresholds in Eq. 6.
  • unlabeled-loss weight mu = not reported
    Appears in the total loss Eq. 10 but no value is given in Table 10 or the text; required to reproduce Semi-SST.
assumptions (4)
  • domain assumption Model confidence on unlabeled data is a reliable proxy for pseudo-label correctness.
    Selection rule Eq. 9 accepts samples whose max softmax probability exceeds the threshold; Figure 3 shows selected-label accuracy declining from 89.5% to 77.7%, so the assumption is only partially met.
  • ad hoc to paper SAT-selected pseudo-labels can be treated as equally reliable as human labels in the combined set.
    Step 6 and Eq. 9 train with plain cross-entropy on original plus pseudo-labels without confidence weighting; this is a design choice specific to SST.
  • domain assumption Iterative self-training with per-cycle threshold updates converges to a better model rather than amplifying errors.
    Algorithm 1 repeats predict-select-train; the paper provides no convergence analysis, and decreasing pseudo-label accuracy over cycles means error amplification is a live risk.
  • domain assumption Pretrained ViT initialization is a fair basis for attributing observed gains to SAT.
    All ViT runs start from DINO or MAE pretrained weights (Section 4.1), and supervised baselines use the same backbones; the gains may partly reflect pretraining rather than the SAT mechanism.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SST: Self-training with Self-adaptive Thresholding for Semi-supervised Learning." pith.science (2026). https://pith.science/paper/PZMUMZMU

@misc{pith2026250600467,
  author       = {Pith},
  title        = {Pith review of: SST: Self-training with Self-adaptive Thresholding for Semi-supervised Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PZMUMZMU}},
  note         = {Machine review of arXiv:2506.00467}
}
read the original abstract

Neural networks have demonstrated exceptional performance in supervised learning, benefiting from abundant high-quality annotated data. However, obtaining such data in real-world scenarios is costly and labor-intensive. Semi-supervised learning (SSL) offers a solution to this problem. Recent studies, such as Semi-ViT and Noisy Student, which employ consistency regularization or pseudo-labeling, have demonstrated significant achievements. However, they still face challenges, particularly in accurately selecting sufficient high-quality pseudo-labels due to their reliance on fixed thresholds. Recent methods such as FlexMatch and FreeMatch have introduced flexible or self-adaptive thresholding techniques, greatly advancing SSL research. Nonetheless, their process of updating thresholds at each iteration is deemed time-consuming, computationally intensive, and potentially unnecessary. To address these issues, we propose Self-training with Self-adaptive Thresholding (SST), a novel, effective, and efficient SSL framework. SST introduces an innovative Self-Adaptive Thresholding (SAT) mechanism that adaptively adjusts class-specific thresholds based on the model's learning progress. SAT ensures the selection of high-quality pseudo-labeled data, mitigating the risks of inaccurate pseudo-labels and confirmation bias. Extensive experiments demonstrate that SST achieves state-of-the-art performance with remarkable efficiency, generalization, and scalability across various architectures and datasets. Semi-SST-ViT-Huge achieves the best results on competitive ImageNet-1K SSL benchmarks, with 80.7% / 84.9% Top-1 accuracy using only 1% / 10% labeled data. Compared to the fully-supervised DeiT-III-ViT-Huge, which achieves 84.8% Top-1 accuracy using 100% labeled data, our method demonstrates superior performance using only 10% labeled data.

Figures

Figures reproduced from arXiv: 2506.00467 by the authors.

Figure 1
Figure 1. Illustration of the Self-training with Self-adaptive Thresholding (SST) framework. Our SST can be integrated with either supervised or semi-supervised algorithms, referred to as Super-SST and Semi-SST, respectively. Self-Adaptive Thresholding (SAT) is a core mechanism in our SST frameworks, accurately and adaptively adjusting class-specific thresholds according to the learning progress of the model. This ensures bot… view at source ↗
Figure 2
Figure 2. Influence of cutoff value (𝐶) and scaling factor (𝑆) in SAT. All experiments are conducted with the same starting checkpoint (ViT-Small) that is supervised finetuned on 1% ImageNet-1K labels, achieving 60.3% accuracy. effects on performance. All experiments are conducted with the same starting checkpoint (i.e., ViT-Small) that is supervised finetuned on 1% ImageNet-1K labeled data, achieving 60.3% Top-1 accuracy. As… view at source ↗
Figure 3
Figure 3. Self-Adaptive Thresholding (SAT) vs. Fixed Thresholding (FTH). In this figure, we set the cutoff value 𝐶=0.5 and scaling factor 𝑆=0.8 for SAT. For FTH, we set the constant thresholds to 0.00, 0.25, 0.50, and 0.75, respectively. 5. Related work 5.1. Thresholding techniques Our work is closely related to SSL methods that utilize thresholding techniques [35, 29, 27, 31, 36, 6, 77, 78, 79, 80, 81, 82, 75]. Methods such … view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

85 extracted references · 66 canonical work pages

  1. [1]

    Russakovsky, J

    O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, et al., Imagenet large scale visual recognition challenge, International journal of computer vision 115 (2015) 211–252

    2015

  2. [2]

    J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, L. Fei-Fei, Imagenet: A large-scale hierarchical image database, in: 2009 IEEE conference on computer vision and pattern recognition, Ieee, 2009, pp. 248–255. : Preprint submitted to Elsevier Page 17 of 20 Table 10 Hyper-parameter settings for supervised initialization, Super-SST, and Semi-SST on ImageNet-1K. ...

    2009

  3. [3]

    T.-Y . Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, C. L. Zitnick, Microsoft coco: Common objects in context, in: Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, Springer, 2014, pp. 740–755

    2014

  4. [4]

    X. J. Zhu, Semi-supervised learning literature survey (2005)

    2005

  5. [5]

    X. Zhu, A. B. Goldberg, Introduction to semi-supervised learning, 2022

    2022

  6. [6]

    K. Sohn, D. Berthelot, N. Carlini, Z. Zhang, H. Zhang, C. A. Raffel, E. D. Cubuk, A. Kurakin, C.-L. Li, Fixmatch: Simplifying semi-supervised learning with consistency and confidence, Advances in neural information processing systems 33 (2020) 596–608

    work page 2020

  7. [7]

    J. E. Van Engelen, H. H. Hoos, A survey on semi-supervised learning, Machine learning 109 (2020) 373–440

    work page 2020

  8. [8]

    Laine, T

    S. Laine, T. Aila, Temporal ensembling for semi-supervised learning, arXiv preprint arXiv:1610.02242 (2016)

    arXiv 2016

Show all 85 references
  1. [9]

    Tarvainen, H

    A. Tarvainen, H. Valpola, Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results, Advances in neural information processing systems 30 (2017)

    2017

  2. [10]

    Miyato, S.-i

    T. Miyato, S.-i. Maeda, M. Koyama, S. Ishii, Virtual adversarial training: a regularization method for supervised and semi-supervised learning, IEEE transactions on pattern analysis and machine intelligence 41 (2018) 1979–1993

    2018

  3. [11]

    Q. Xie, Z. Dai, E. Hovy, T. Luong, Q. Le, Unsupervised data augmentation for consistency training, Advances in neural information processing systems 33 (2020) 6256–6268

    2020

  4. [12]

    Bachman, O

    P. Bachman, O. Alsharif, D. Precup, Learning with pseudo-ensembles, Advances in neural information processing systems 27 (2014)

    2014

  5. [13]

    Rasmus, M

    A. Rasmus, M. Berglund, M. Honkala, H. Valpola, T. Raiko, Semi-supervised learning with ladder networks, Advances in neural information processing systems 28 (2015)

    2015

  6. [14]

    Berthelot, N

    D. Berthelot, N. Carlini, I. Goodfellow, N. Papernot, A. Oliver, C. A. Raffel, Mixmatch: A holistic approach to semi-supervised learning, Advances in neural information processing systems 32 (2019)

    2019

  7. [15]

    Sajjadi, M

    M. Sajjadi, M. Javanmardi, T. Tasdizen, Regularization with stochastic transformations and perturbations for deep semi-supervised learning, Advances in neural information processing systems 29 (2016)

    2016

  8. [16]

    Scudder, Probability of error of some adaptive pattern-recognition machines, IEEE Transactions on Information Theory 11 (1965) 363–371

    H. Scudder, Probability of error of some adaptive pattern-recognition machines, IEEE Transactions on Information Theory 11 (1965) 363–371

    1965

  9. [17]

    G. J. McLachlan, Iterative reclassification procedure for constructing an asymptotically optimal rule of allocation in discriminant analysis, Journal of the American Statistical Association 70 (1975) 365–369

    1975

  10. [18]

    Yarowsky, Unsupervised word sense disambiguation rivaling supervised methods, in: 33rd annual meeting of the association for computational linguistics, 1995, pp

    D. Yarowsky, Unsupervised word sense disambiguation rivaling supervised methods, in: 33rd annual meeting of the association for computational linguistics, 1995, pp. 189–196

    1995

  11. [19]

    Riloff, Automatically generating extraction patterns from untagged text, in: Proceedings of the national conference on artificial intelligence, 1996, pp

    E. Riloff, Automatically generating extraction patterns from untagged text, in: Proceedings of the national conference on artificial intelligence, 1996, pp. 1044–1049

    1996

  12. [20]

    Riloff, J

    E. Riloff, J. Wiebe, Learning extraction patterns for subjective expressions, in: Proceedings of the 2003 conference on Empirical methods in natural language processing, 2003, pp. 105–112

    2003

  13. [21]

    Y . He, D. Zhou, Self-training from labeled features for sentiment analysis, Information Processing & Management 47 (2011) 606–616

    2011

  14. [22]

    D.-H. Lee, et al., Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks, in: Workshop on challenges in representation learning, ICML, volume 3, 2013, p. 896

    2013

  15. [23]

    W. Shi, Y . Gong, C. Ding, Z. M. Tao, N. Zheng, Transductive semi-supervised deep learning using min-max features, in: Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 299–315

    2018

  16. [24]

    Iscen, G

    A. Iscen, G. Tolias, Y . Avrithis, O. Chum, Label propagation for deep semi-supervised learning, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 5070–5079. : Preprint submitted to Elsevier Page 18 of 20

    2019

  17. [25]

    Arazo, D

    E. Arazo, D. Ortego, P. Albert, N. E. O’Connor, K. McGuinness, Pseudo-labeling and confirmation bias in deep semi-supervised learning, in: 2020 International joint conference on neural networks (IJCNN), IEEE, 2020, pp. 1–8

    2020

  18. [26]

    K. Sohn, Z. Zhang, C.-L. Li, H. Zhang, C.-Y . Lee, T. Pfister, A simple semi-supervised learning framework for object detection, arXiv preprint arXiv:2005.04757 (2020)

    2020 arXiv

  19. [27]

    Xie, M.-T

    Q. Xie, M.-T. Luong, E. Hovy, Q. V . Le, Self-training with noisy student improves imagenet classification, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 10687–10698

    2020

  20. [28]

    H. Pham, Z. Dai, Q. Xie, Q. V . Le, Meta pseudo labels, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 11557–11568

    2021

  21. [29]

    Z. Cai, A. Ravichandran, S. Maji, C. Fowlkes, Z. Tu, S. Soatto, Exponential moving average normalization for self-supervised and semi-supervised learning, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 194–203

    2021

  22. [30]

    Liu, C.-Y

    Y .-C. Liu, C.-Y . Ma, Z. He, C.-W. Kuo, K. Chen, P. Zhang, B. Wu, Z. Kira, P. Vajda, Unbiased teacher for semi-supervised object detection, arXiv preprint arXiv:2102.09480 (2021)

    2021 arXiv

  23. [31]

    Zhang, Y

    B. Zhang, Y . Wang, W. Hou, H. Wu, J. Wang, M. Okumura, T. Shinozaki, Flexmatch: Boosting semi-supervised learning with curriculum pseudo labeling, Advances in Neural Information Processing Systems 34 (2021) 18408–18419

    2021

  24. [32]

    C. Wei, K. Sohn, C. Mellina, A. Yuille, F. Yang, Crest: A class-rebalancing self-training framework for imbalanced semi-supervised learning, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 10857–10866

    2021

  25. [33]

    D. C. G. Pedronette, L. J. Latecki, Rank-based self-training for graph convolutional networks, Information Processing & Management 58 (2021) 102443

    2021

  26. [34]

    Alqahtani, N

    Y . Alqahtani, N. Al-Twairesh, A. Alsanad, Improving sentiment domain adaptation for arabic using an unsupervised self-labeling framework, Information Processing & Management 60 (2023) 103338

    2023

  27. [35]

    Z. Cai, A. Ravichandran, P. Favaro, M. Wang, D. Modolo, R. Bhotika, Z. Tu, S. Soatto, Semi-supervised vision transformers at scale, Advances in Neural Information Processing Systems 35 (2022) 25697–25710

    2022

  28. [36]

    Y . Wang, H. Chen, Q. Heng, W. Hou, Y . Fan, , Z. Wu, J. Wang, M. Savvides, T. Shinozaki, B. Raj, B. Schiele, X. Xie, Freematch: Self-adaptive thresholding for semi-supervised learning, International Conference on Learning Representations (ICLR) (2023)

    2023

  29. [37]

    B. Chen, J. Jiang, X. Wang, P. Wan, J. Wang, M. Long, Debiased self-training for semi-supervised learning, Advances in Neural Information Processing Systems 35 (2022) 32424–32437

    2022

  30. [38]

    X. Wang, Z. Wu, L. Lian, S. X. Yu, Debiased learning from naturally imbalanced pseudo-labels, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 14647–14657

    2022

  31. [39]

    Bossard, M

    L. Bossard, M. Guillaumin, L. Van Gool, Food-101–mining discriminative components with random forests, in: Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part VI 13, Springer, 2014, pp. 446–461

    2014

  32. [40]

    M. W. K. Grant Van Horn, macaodha, inaturalist 2019 at fgvc6, 2019. URL: https://kaggle.com/competitions/ inaturalist-2019-fgvc6

    2019

  33. [41]

    Krizhevsky, G

    A. Krizhevsky, G. Hinton, et al., Learning multiple layers of features from tiny images (2009)

    2009

  34. [42]

    Netzer, T

    Y . Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, A. Y . Ng, et al., Reading digits in natural images with unsupervised feature learning, in: NIPS workshop on deep learning and unsupervised feature learning, volume 2011, Granada, 2011, p. 4

    2011

  35. [43]

    Coates, A

    A. Coates, A. Ng, H. Lee, An analysis of single-layer networks in unsupervised feature learning, in: Proceedings of the fourteenth international conference on artificial intelligence and statistics, JMLR Workshop and Conference Proceedings, 2011, pp. 215–223

    2011

  36. [44]

    T. Xiao, T. Xia, Y . Yang, C. Huang, X. Wang, Learning from massive noisy labeled data for image classification, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 2691–2699

    2015

  37. [45]

    Dosovitskiy, L

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, et al., An image is worth 16x16 words: Transformers for image recognition at scale, arXiv preprint arXiv:2010.11929 (2020)

    2020 arXiv

  38. [46]

    Caron, H

    M. Caron, H. Touvron, I. Misra, H. Jégou, J. Mairal, P. Bojanowski, A. Joulin, Emerging properties in self-supervised vision transformers, in: Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 9650–9660

    2021

  39. [47]

    K. He, X. Chen, S. Xie, Y . Li, P. Dollár, R. Girshick, Masked autoencoders are scalable vision learners, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 16000–16009

    2022

  40. [48]

    Z. Liu, H. Mao, C.-Y . Wu, C. Feichtenhofer, T. Darrell, S. Xie, A convnet for the 2020s, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 11976–11986

    2022

  41. [49]

    Zhang, M

    H. Zhang, M. Cisse, Y . N. Dauphin, D. Lopez-Paz, mixup: Beyond empirical risk minimization, arXiv preprint arXiv:1710.09412 (2017)

    2017 arXiv

  42. [50]

    K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778

    2016

  43. [51]

    Loshchilov, F

    I. Loshchilov, F. Hutter, Decoupled weight decay regularization, arXiv preprint arXiv:1711.05101 (2017)

    2017 arXiv

  44. [52]

    Goyal, P

    P. Goyal, P. Dollár, R. Girshick, P. Noordhuis, L. Wesolowski, A. Kyrola, A. Tulloch, Y . Jia, K. He, Accurate, large minibatch sgd: Training imagenet in 1 hour, arXiv preprint arXiv:1706.02677 (2017)

    2017 arXiv

  45. [53]

    Loshchilov, F

    I. Loshchilov, F. Hutter, Sgdr: Stochastic gradient descent with warm restarts, arXiv preprint arXiv:1608.03983 (2016)

    2016 arXiv

  46. [54]

    Szegedy, V

    C. Szegedy, V . Vanhoucke, S. Ioffe, J. Shlens, Z. Wojna, Rethinking the inception architecture for computer vision, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 2818–2826

    2016

  47. [55]

    Huang, Y

    G. Huang, Y . Sun, Z. Liu, D. Sedra, K. Q. Weinberger, Deep networks with stochastic depth, in: Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV 14, Springer, 2016, pp. 646–661

    2016

  48. [56]

    S. Yun, D. Han, S. J. Oh, S. Chun, J. Choe, Y . Yoo, Cutmix: Regularization strategy to train strong classifiers with localizable features, in: Proceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 6023–6032

    2019

  49. [57]

    E. D. Cubuk, B. Zoph, J. Shlens, Q. V . Le, Randaugment: Practical automated data augmentation with a reduced search space, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, 2020, pp. 702–703. : Preprint submitted to Elsevier Page...

    2020

  50. [58]

    Zhong, L

    Z. Zhong, L. Zheng, G. Kang, S. Li, Y . Yang, Random erasing data augmentation, in: Proceedings of the AAAI conference on artificial intelligence, volume 34, 2020, pp. 13001–13008

    2020

  51. [59]

    T. Chen, S. Kornblith, K. Swersky, M. Norouzi, G. E. Hinton, Big self-supervised models are strong semi-supervised learners, Advances in neural information processing systems 33 (2020) 22243–22255

    2020

  52. [60]

    J. Li, C. Xiong, S. C. Hoi, Comatch: Semi-supervised learning with contrastive graph regularization, in: Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 9475–9484

    2021

  53. [61]

    Assran, M

    M. Assran, M. Caron, I. Misra, P. Bojanowski, A. Joulin, N. Ballas, M. Rabbat, Semi-supervised learning of visual features by non-parametrically predicting view assignments with support samples, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021,...

    2021

  54. [62]

    Zheng, S

    M. Zheng, S. You, L. Huang, F. Wang, C. Qian, C. Xu, Simmatch: Semi-supervised learning with similarity matching, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 14471–14481

    2022

  55. [63]

    Zheng, S

    M. Zheng, S. You, L. Huang, C. Luo, F. Wang, C. Qian, C. Xu, Simmatchv2: Semi-supervised learning with graph consistency, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 16432–16442

    2023

  56. [64]

    Z. Weng, X. Yang, A. Li, Z. Wu, Y .-G. Jiang, Semi-supervised vision transformers, in: European conference on computer vision, Springer Nature Switzerland Cham, 2022, pp. 605–620

    2022

  57. [65]

    Hinton, O

    G. Hinton, O. Vinyals, J. Dean, Distilling the knowledge in a neural network, arXiv preprint arXiv:1503.02531 (2015)

    2015 arXiv

  58. [66]

    S. I. Mirzadeh, M. Farajtabar, A. Li, N. Levine, A. Matsukawa, H. Ghasemzadeh, Improved knowledge distillation via teacher assistant, in: Proceedings of the AAAI conference on artificial intelligence, volume 34, 2020, pp. 5191–5198

    2020

  59. [67]

    Chen, J.-P

    D. Chen, J.-P. Mei, Y . Zhang, C. Wang, Z. Wang, Y . Feng, C. Chen, Cross-layer distillation with semantic calibration, in: Proceedings of the AAAI conference on artificial intelligence, volume 35, 2021, pp. 7028–7036

    2021

  60. [68]

    Passalis, A

    N. Passalis, A. Tefas, Learning deep representations with probabilistic knowledge transfer, in: Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 268–284

    2018

  61. [69]

    W. Park, D. Kim, Y . Lu, M. Cho, Relational knowledge distillation, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 3967–3976

    2019

  62. [70]

    J. Zhu, S. Tang, D. Chen, S. Yu, Y . Liu, M. Rong, A. Yang, X. Wang, Complementary relation contrastive distillation, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 9260–9269

    2021

  63. [71]

    C. Yang, Z. An, H. Zhou, L. Cai, X. Zhi, J. Wu, Y . Xu, Q. Zhang, Mixskd: Self-knowledge distillation from mixup for image recognition, in: European Conference on Computer Vision, Springer, 2022, pp. 534–551

    2022

  64. [72]

    Y . Wang, H. Chen, Y . Fan, W. Sun, R. Tao, W. Hou, R. Wang, L. Yang, Z. Zhou, L.-Z. Guo, et al., Usb: A unified semi-supervised learning benchmark for classification, Advances in Neural Information Processing Systems 35 (2022) 3938–3961

    2022

  65. [73]

    Touvron, M

    H. Touvron, M. Cord, M. Douze, F. Massa, A. Sablayrolles, H. Jégou, Training data-efficient image transformers & distillation through attention, in: International conference on machine learning, PMLR, 2021, pp. 10347–10357

    2021

  66. [74]

    Touvron, M

    H. Touvron, M. Cord, H. Jégou, Deit iii: Revenge of the vit, in: European conference on computer vision, Springer Nature Switzerland Cham, 2022, pp. 516–533

    2022

  67. [75]

    H. Chen, R. Tao, Y . Fan, Y . Wang, J. Wang, B. Schiele, X. Xie, B. Raj, M. Savvides, Softmatch: Addressing the quantity-quality trade-off in semi-supervised learning, International Conference on Learning Representations (ICLR) (2023)

    2023

  68. [76]

    Zagoruyko, N

    S. Zagoruyko, N. Komodakis, Wide residual networks, arXiv preprint arXiv:1605.07146 (2016)

    2016 arXiv

  69. [77]

    Y . Xu, L. Shang, J. Ye, Q. Qian, Y .-F. Li, B. Sun, H. Li, R. Jin, Dash: Semi-supervised learning with dynamic thresholding, in: International Conference on Machine Learning, PMLR, 2021, pp. 11525–11536

    2021

  70. [78]

    Berthelot, R

    D. Berthelot, R. Roelofs, K. Sohn, N. Carlini, A. Kurakin, Adamatch: A unified approach to semi-supervised learning and domain adaptation, arXiv preprint arXiv:2106.04732 (2021)

    2021 arXiv

  71. [79]

    Guo, Y .-F

    L.-Z. Guo, Y .-F. Li, Class-imbalanced semi-supervised learning with adaptive thresholding, in: International Conference on Machine Learning, PMLR, 2022, pp. 8082–8094

    2022

  72. [80]

    X. Lin, Z. Shou, S.-F. Chang, Towards train-test consistency for semi-supervised temporal action localization, arXiv preprint arXiv:1910.11285 (2019)

    2019 arXiv

  73. [81]

    Y . Chen, C. Wei, A. Kumar, T. Ma, Self-training avoids using spurious features under domain shift, Advances in Neural Information Processing Systems 33 (2020) 21061–21071

    2020

  74. [82]

    Kumar, T

    A. Kumar, T. Ma, P. Liang, Understanding self-training for gradual domain adaptation, in: International conference on machine learning, PMLR, 2020, pp. 5468–5479

    2020

  75. [83]

    Y . Zou, Z. Yu, B. Kumar, J. Wang, Unsupervised domain adaptation for semantic segmentation via class-balanced self-training, in: Proceedings of the European conference on computer vision (ECCV), 2018, pp. 289–305

    2018

  76. [84]

    L. Yang, W. Zhuo, L. Qi, Y . Shi, Y . Gao, St++: Make self-training work better for semi-supervised semantic segmentation, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 4268–4277

    2022

  77. [85]

    Bengio, J

    Y . Bengio, J. Louradour, R. Collobert, J. Weston, Curriculum learning, in: Proceedings of the 26th annual international conference on machine learning, 2009, pp. 41–48. : Preprint submitted to Elsevier Page 20 of 20

    2009

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.