Pith. sign in

REVIEW 4 major objections 5 minor 58 references

Transferring self-supervised pre-trained models for SHM data anomaly detection with scarce labeled data

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Self-supervised pre-training, especially a simple autoencoder, makes SHM anomaly detection accurate with only tens to hundreds of labeled samples.

desk verdict A useful, clearly reported empirical comparison of SSL pretraining for SHM anomaly detection, but the headline AE advantage is weakened by pretraining on the test split and by the lack of transfer experiments. read the letter →

arxiv 2412.03880 v1 pith:B3EAPRJ6 submitted 2024-12-05 cs.LG cs.CE

Mingyuan Zhou , Xudong Jian , Ye Xia , Zhilu Lai This is my paper · ORCID
classification cs.LGcs.CE
keywords self-supervisedlearningstructuralhealthmonitoringanomalydetectionautoencodercontrastivelow-shottransferdataimbalance
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that self-supervised pre-training, particularly a plain autoencoder, makes bridge monitoring data anomaly detection viable when labeled examples number only in the tens to hundreds. On data from two in-service bridges, pre-training an encoder on unlabeled one-hour acceleration summaries and then fine-tuning on small labeled sets raises F1 scores by roughly 3 to 8 points in Case 1 and 3 to 9 points in Case 2 over supervised training from scratch. The significance is that label scarcity is the main bottleneck for deep-learning structural health monitoring, while unlabeled monitoring data is abundant, so an autoencoder pretrained without labels is a practical first-pass data-cleaning tool. The paper also reports that contrastive methods (SimCLR and Mixup) and a GAN can underperform purely supervised training in this setting, which it attributes to the heavily imbalanced class distribution of real SHM data.

What carries the argument

The load-bearing object is the inverted envelope of the relative frequency histogram (IERFH), a 512-dimensional feature computed from each one-hour acceleration segment, which compresses high-dimensional time series before any learning. On top of that representation, the framework runs a two-stage procedure: unsupervised self-supervised pre-training on the full unlabeled dataset, followed by supervised fine-tuning on a low-shot labeled subset. The central mechanism is the autoencoder's reconstruction loss $L_{\text{SSL-AE}} = \frac{1}{B}\sum_i \|\hat{x}_i - x_i\|_2^2$, which forces the CNN encoder to preserve the information in the IERFH feature; the pre-trained encoder then initializes the classifier and reaches near-optimal F1 within roughly 10 fine-tuning epochs.

What would settle it

Run the same AE-versus-supervised comparison on raw one-hour acceleration segments, or on an alternative time-series feature, using the same low-shot label sets; if the 3–9 point F1 advantage of AE over supervised training disappears or reverses, the reported transfer gains are properties of the IERFH feature rather than of self-supervised pre-training. A second check: if fine-tuning from a randomly initialized encoder on the same feature matches AE's F1, then the pre-training stage itself is not the cause of the improvement.

Watch

Extended reading notes

Core claim

On its own terms, the paper discovers that an autoencoder trained with a simple reconstruction objective on unlabeled SHM data transfers better to low-shot anomaly classification than supervised training from scratch, SimCLR, Mixup, or a GAN. The evidence is F1 gains of roughly 3–8 points in Case 1 and 3–9 points in Case 2 across six balanced and unbalanced low-shot label configurations, with the largest gains appearing at the smallest label counts, plus 93.5% and 98.7% test accuracy for the best autoencoder models. The authors interpret this as showing that the generative pretext task is robust to the over-10:1 imbalance of real SHM monitoring data, whereas contrastive and generative-contrastive SSL methods can suffer negative transfer under the same conditions.

Load-bearing premise

The whole method presupposes that the compressed one-hour summary of each sensor signal, a 512-bin histogram shape called IERFH, keeps enough detail to tell every anomaly type apart; the summary was designed in earlier supervised work and is reused here without being re-validated for the self-supervised setting.

Editorial extensions

If this is right

  • If the claim holds, SHM operators can deploy anomaly detection with only a few hundred labeled samples, provided unlabeled historical data exists.
  • A plain autoencoder is the recommended default for SSL pre-training on SHM data, while contrastive and GAN-based pre-training may hurt performance under class imbalance.
  • The approach transfers across two structurally different bridges with different sensor configurations and anomaly types, suggesting it is not tied to one sensor layout.
  • Rare anomaly types such as outlier and drift remain hard for all methods, so gains concentrate on normal data and majority abnormal patterns.
  • Because autoencoder pre-training needs no augmentation or extra hyperparameters, it is the cheapest SSL option to adopt in practice.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reported gains could be partly an artifact of the IERFH feature: the same AE-versus-supervised comparison on raw acceleration segments or on an alternative time-series feature might show different or smaller improvements, separating feature effects from pre-training effects.
  • The authors' observations of negative pre-training imply a practical model-selection rule: on a new bridge, SSL pre-training should be validated against supervised training before being trusted, since not every pretext task transfers.
  • Fine-tuning reaching near-optimal performance in about 10 epochs suggests practitioners could use early stopping as a default, cutting compute without sacrificing accuracy.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a self-supervised learning (SSL) framework for anomaly detection in structural health monitoring (SHM) data with scarce labels. Acceleration time series are reduced to 512-dimensional IERFH features, then an encoder is pre-trained with one of four SSL methods (AE, SimCLR, Mixup, GAN) on unlabeled data, and finally fine-tuned on small labeled low-shot sets. Experiments on two in-service bridges compare SSL methods against purely supervised training (SUP), reporting that AE consistently improves F1 by about 3–9 percentage points over SUP. The paper concludes that SSL pre-training, especially AE, is a practical tool for low-label SHM anomaly detection.

Significance. If the reported gains are valid and transferable, the work would be a practically useful demonstration that SSL pre-training reduces annotation cost in SHM anomaly detection. The paper has concrete strengths: it uses real monitoring data from two bridges, includes four SSL methods, evaluates six low-shot configurations per bridge, reports means and standard deviations over five runs, and provides confusion matrices and loss curves. However, the central comparison is currently confounded by the use of the test split in pre-training, and the title's 'transferring' claim is not actually tested. The significance therefore depends on additional experiments that remove the transductive leakage and demonstrate transfer across data distributions.

major comments (4)
  1. [Section 3.1.3 and Tables 5–6] SSL pre-training is performed on the 'full original dataset' after the 20%/30%/50% split into Label, Validation, and Test sets, which means the Test split is included in the unlabeled pre-training data for AE, SimCLR, Mixup, and GAN. The SUP baseline never sees these inputs. This makes the headline comparison in Tables 5 and 6 transductive: the AE encoder is trained to reconstruct the very test-set features used later for evaluation, so the reported 3–9 point F1 advantage may reflect familiarity with the test distribution rather than a transferable benefit of SSL. The authors should either explicitly justify why test-set features are legitimate unlabeled pre-training data, or, preferably, rerun the comparison with pre-training on Label+Validation only (or on another disjoint unlabeled set) and report whether the AE advantage persists. A true transfer experiment, pre-training on one bridge or time period and fine-tuning/evaluating on another, is also needed to support the title's 'transferring' claim.
  2. [Section 3.2, Eqs. (18)–(21)] The evaluation metrics are defined only for binary classification, but the task is multiclass (5–6 classes). The F1 scores in Tables 5 and 6 are not accompanied by any definition of how multiclass F1 is averaged (macro, micro, or weighted). Given the strong class imbalance (e.g., Normal has 13,575 samples while Drift has 679 in Case 1), the averaging choice can materially change the reported numbers. The authors should specify the F1 averaging scheme and ideally report per-class precision/recall for all methods, not only for AE. In addition, no significance tests are provided; given that several standard deviations overlap (e.g., Case 2, D2_1: AE 75.38±3.20 vs SUP 68.31±3.16), paired tests or confidence intervals are needed to support the claim of consistent improvement.
  3. [Table 2 and Section 3.1.3] The dataset split numbers are internally inconsistent. In Case 1, summing the per-class rows gives Test = 12,663 (not 12,743), Validation = 7,599 (not 7,506), and Label = 5,072 (not 5,071); the three reported split sizes sum to 25,320, not the stated total of 25,330. This discrepancy affects reproducibility and also calls into question what 'full original dataset' means for pre-training. The authors should correct the table or explain the rounding/selection procedure.
  4. [Section 3.4.1 and title] The manuscript does not contain any experiment that actually transfers a model across data sources. All pre-training, fine-tuning, and evaluation are performed on the same bridge and the same time period, and only the low-shot label subset changes. The word 'transferring' in the title and the discussion of 'pre-trained models' imply that the learned encoder is useful for a different SHM dataset or monitoring campaign, but this is not tested. At minimum, the authors should either add a cross-bridge or cross-time-period transfer experiment or soften the title and claims to accurately describe same-distribution pre-training followed by fine-tuning.
minor comments (5)
  1. [Section 3.3] In the description of the discriminator, 'which amps 256-dimensional input to a single output' should read 'which maps 256-dimensional input to a single output'.
  2. [Table 3] The low-shot dataset labels in Table 3 are inconsistent in notation: for example, 'Low-shot 2,D2 3 30 30 30 30 30 150' mixes 'D2' and '3' without a subscript, unlike the clear D1_1 style in Table 2. The row labels should be made uniform.
  3. [Section 3.4.3] The statement that the SSL loss functions are 'effectively minimized' only shows that the pretext objective converges; it does not demonstrate that the learned representations are useful. The authors should avoid interpreting loss minimization alone as evidence of representation quality.
  4. [Section 3.3] Several hyperparameters (temperature τ, Mixup beta, augmentation choices, number of pre-training and fine-tuning epochs) are fixed without sensitivity analysis. A short ablation on at least one dataset would strengthen the claim that the reported gains are robust rather than sensitive to these specific settings.
  5. [Data availability] The data and code availability statement says they 'will be shared following the potential publication', but no repository link is provided. For a reproducibility-focused evaluation, the authors should make the code and processed features available or describe how to obtain them.

Circularity Check

1 steps flagged · score 6.0 of 10

SSL pre-training uses the full dataset, including the test split, so the reported AE advantage is partly a transductive fit rather than an independent prediction.

  1. fitted input called prediction [Section 3.1.3 (Dataset preparation), final paragraph; cf. Section 2.1 problem formulation]
    "In each case, the acquired SHM data is split into three subsets: labeled, validation, and test datasets, with proportion ratios of 20%, 30%, and 50%, respectively. Model performance is evaluated on the test dataset after training... For SSL pre-training, neural networks are trained on the full original dataset without access to label information. After pre-training, the low-shot labeled datasets are used to fine-tune the pre-trained SSL model."

    The 'full original dataset' is the union of the 20% labeled, 30% validation, and 50% test partitions. The AE reconstruction loss (Eq. 2), and likewise the SimCLR/Mixup/GAN losses, are therefore optimized on the very test IERFH features that are later used to compute the reported F1 scores in Tables 5 and 6. The SUP baseline (Eq. 17) is trained only on the low-shot labeled split and never sees test inputs. The claimed SSL advantage is thus not a clean predictive comparison: SSL models are fitted, in an unsupervised sense, to the evaluation distribution, while SUP is not. The reported 3-9 point AE improvement may reflect familiarity with test samples rather than transferable SSL pre-training.

full rationale

No formal derivation in the paper equates a predicted quantity to an input by construction, and the self-citations are not load-bearing in a uniqueness sense. The IERFH feature is adopted from the authors' prior work [18], but as an input representation rather than as the target result, so that is not circular. However, the central empirical claim is compromised by the dataset-preparation protocol: SSL pre-training is performed on the 'full original dataset', which includes the 50% test split, whereas the supervised baseline is restricted to the low-shot labeled data. Consequently, the F1 comparisons in Tables 5 and 6 mix the effect of SSL pre-training with transductive access to the test inputs. This makes the headline 'SSL boosts anomaly detection' finding partially a fitted-input-called-prediction artifact rather than an independent evaluation. The score of 6 reflects that the main result is partially circular due to this test-set contamination, while the underlying SSL loss definitions and fine-tuning procedure themselves are not circular.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim is an empirical comparison, not a derivation, so the ledger mainly records experimental design choices and domain assumptions. The most consequential assumptions are the sufficiency of the IERFH feature, the correctness of expert labels, and the transductive use of the full unlabeled dataset including the test split.

free parameters (5)
  • IERFH bin count = 512
    Set to 512 in Section 2.2 to balance smoothness and computational efficiency; the feature dimensionality is fixed by this choice and all downstream models depend on it.
  • Low-shot dataset sample counts = Tables 2 and 3 (50 to 550 samples)
    The balanced and unbalanced low-shot configurations are hand-designed by the authors, not randomly sampled from a defined distribution; performance differences across settings could reflect these choices.
  • Fine-tuning epochs = 50
    Used for all methods after 200 pre-training or supervised epochs; no sensitivity analysis is reported.
  • Contrastive hyperparameters = tau=0.5 (SimCLR), tau=0.1 (Mixup), lambda~Beta(0.2,0.2)
    Taken from prior SSL papers [30,43], but the relative ranking of SSL methods could change with tuning; AE needs none of these.
  • CNN encoder architecture = channels [16,32,64,128,256], kernels [5,3,3,3,3]
    Chosen without stated tuning; all methods share it, so it is not the source of AE versus contrastive differences, but it shapes the performance ceiling.
assumptions (4)
  • domain assumption IERFH features are sufficient to distinguish all anomaly classes.
    Adopted from prior work [18] without re-validation; Section 2.2 defines the feature and the experiments rely on it.
  • domain assumption The expert-defined labels in Table 1 are correct and consistent across the two bridges.
    Labels come from domain experts (Sections 3.1.1 and 3.1.2); label noise would bias F1 scores.
  • domain assumption Pre-training on the full unlabeled dataset, including the test split, is a fair transductive setup.
    Section 3.1.3 states SSL pre-training uses the full original dataset; the paper does not discuss the effect on evaluation.
  • domain assumption The multiclass F1 metric is computed in a consistent, meaningful way.
    Section 3.2 defines F1 only for binary classification; the averaging scheme for the multiclass tables is not stated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Transferring self-supervised pre-trained models for SHM data anomaly detection with scarce labeled data." pith.science (2026). https://pith.science/paper/B3EAPRJ6

@misc{pith2026241203880,
  author       = {Pith},
  title        = {Pith review of: Transferring self-supervised pre-trained models for SHM data anomaly detection with scarce labeled data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/B3EAPRJ6}},
  note         = {Machine review of arXiv:2412.03880}
}
read the original abstract

Structural health monitoring (SHM) has experienced significant advancements in recent decades, accumulating massive monitoring data. Data anomalies inevitably exist in monitoring data, posing significant challenges to their effective utilization. Recently, deep learning has emerged as an efficient and effective approach for anomaly detection in bridge SHM. Despite its progress, many deep learning models require large amounts of labeled data for training. The process of labeling data, however, is labor-intensive, time-consuming, and often impractical for large-scale SHM datasets. To address these challenges, this work explores the use of self-supervised learning (SSL), an emerging paradigm that combines unsupervised pre-training and supervised fine-tuning. The SSL-based framework aims to learn from only a very small quantity of labeled data by fine-tuning, while making the best use of the vast amount of unlabeled SHM data by pre-training. Mainstream SSL methods are compared and validated on the SHM data of two in-service bridges. Comparative analysis demonstrates that SSL techniques boost data anomaly detection performance, achieving increased F1 scores compared to conventional supervised training, especially given a very limited amount of labeled data. This work manifests the effectiveness and superiority of SSL techniques on large-scale SHM data, providing an efficient tool for preliminary anomaly detection with scarce label information.

Figures

Figures reproduced from arXiv: 2412.03880 by the authors.

Figure 1
Figure 1. The workflow of the self-supervised learning framework for data anomaly detection. (Details about the IERFH feature are provided in [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Categories of pretext tasks in self-supervised learning: di [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Sensor network of the bridge in Case 1 (image credits [ [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Sensor network of the bridge in Case 2 (image credits [ [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Illustration of SHM data patterns in Case 1. [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Illustration of SHM data patterns in Case 2. [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: Bar plot F1 scores (%) of different methods, utilizing labeled data from various low-shot datasets. (a) Case 1: AE, fine-tuned on D1 6 (b) Case 2: AE, fine-tuned on D2 6 [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: Confusion matrices of AE method on test datasets (numbers in the bottom right corner represent the accuracy). [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: Loss functions during the self-supervised pre-training stage. [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]
Figure 10
Figure 10. Figure 10: Analysis of the performance in the fine-tuning stage. [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

58 extracted references · 39 canonical work pages

  1. [1]

    J. Ko, Y . Q. Ni, Technology developments in structural health monitoring of large-scale bridges, Engineering structures 27 (12) (2005) 1715–1725

    work page 2005

  2. [2]

    O. Avci, O. Abdeljaber, S. Kiranyaz, M. Hussein, M. Gabbouj, D. J. Inman, A review of vibration-based damage detection in civil structures: From traditional methods to machine learning and deep learning applications, Mechanical systems and signal processing 147 (2021) 107077

    work page 2021

  3. [3]

    L. Sun, Z. Shang, Y . Xia, S. Bhowmick, S. Nagarajaiah, Review of bridge structural health monitoring aided by big data and artificial intelligence: From condition assessment to damage detection, Journal of Structural Engineering 146 (5) (2020) 04020073

    work page 2020

  4. [4]

    Z. Lai, S. Nagarajaiah, Semi-supervised structural linear /nonlinear damage detection and characterization using sparse identification, Struc- tural Control and Health Monitoring 26 (3) (2019) e2306

    work page 2019

  5. [5]

    Y . An, E. Chatzi, S.-H. Sim, S. Laflamme, B. Blachowski, J. Ou, Recent progress and future trends on damage identification methods for bridge structures, Structural Control and Health Monitoring 26 (10) (2019) e2416

    work page 2019

  6. [6]

    Z. Lai, C. Mylonas, S. Nagarajaiah, E. Chatzi, Structural identification with physics-informed neural ordinary di fferential equations, Journal of Sound and Vibration 508 (2021) 116196

    work page 2021

  7. [7]

    P. L. Green, E. J. Cross, K. Worden, Bayesian system identification of dynamical systems using highly informative training data, Mechanical systems and signal processing 56 (2015) 109–122

    work page 2015

  8. [8]

    X. Jian, Z. Lai, Y . Xia, L. Sun, A robust bridge weigh-in-motion algorithm based on regularized total least squares with axle constraints, Structural Control and Health Monitoring 29 (10) (2022) e3014

    work page 2022

Show all 58 references
  1. [9]

    H. Wang, T. Nagayama, D. Su, Static and dynamic vehicle load identification with lane detection from measured bridge acceleration and inclination responses, Structural Control and Health Monitoring 28 (11) (2021) e2823

    2021

  2. [10]

    D. M. Frangopol, M. Soliman, Life-cycle of structural systems: recent achievements and future directions, in: Structures and Infrastructure Systems, Routledge, 2019, pp. 46–65

    2019

  3. [11]

    X. Chen, B. Wang, J. Chen, X. Zhang, S. Liu, G. Zhou, P. Li, X. Zhao, Innovative life-cycle inspection strategy of civil infrastructure: Smartphone-based public participation, Structural Control and Health Monitoring 2023 (1) (2023) 8715784

    2023

  4. [12]

    Y . Deng, Y . Zhao, H. Ju, T.-H. Yi, A. Li, Abnormal data detection for structural health monitoring: State-of-the-art review, Developments in the Built Environment (2024) 100337

    2024

  5. [13]

    Y . Fu, C. Peng, F. Gomez, Y . Narazaki, B. F. Spencer Jr, Sensor fault management techniques for wireless smart sensor networks in structural health monitoring, Structural Control and Health Monitoring 26 (7) (2019) e2362

    2019

  6. [14]

    J. Kullaa, Distinguishing between sensor fault, structural damage, and environmental or operational e ffects in structural health monitoring, Mechanical Systems and Signal Processing 25 (8) (2011) 2976–2989

    2011

  7. [15]

    Chandola, A

    V . Chandola, A. Banerjee, V . Kumar, Anomaly detection: A survey, ACM computing surveys (CSUR) 41 (3) (2009) 1–58

    2009

  8. [16]

    Y . Bao, Z. Tang, H. Li, Y . Zhang, Computer vision and deep learning–based data anomaly detection method for structural health monitoring, Structural Health Monitoring 18 (2) (2019) 401–421

    2019

  9. [17]

    Z. Tang, Z. Chen, Y . Bao, H. Li, Convolutional neural network-based data anomaly detection method using multiple information for structural health monitoring, Structural Control and Health Monitoring 26 (1) (2019) e2296

    2019

  10. [18]

    X. Jian, H. Zhong, Y . Xia, L. Sun, Faulty data detection and classification for bridge structural health monitoring via statistical and deep- learning approach, Structural Control and Health Monitoring 28 (11) (2021) e2824

    2021

  11. [19]

    X. Lei, Y . Xia, A. Wang, X. Jian, H. Zhong, L. Sun, Mutual information based anomaly detection of monitoring data with attention mechanism and residual learning, Mechanical Systems and Signal Processing 182 (2023) 109607

    2023

  12. [20]

    F. Ni, J. Zhang, M. N. Noori, Deep learning for data anomaly detection and data compression of a long-span suspension bridge, Computer- Aided Civil and Infrastructure Engineering 35 (7) (2020) 685–700

    2020

  13. [21]

    J. Mao, H. Wang, B. F. Spencer Jr, Toward data anomaly detection for automated structural health monitoring: Exploiting generative adver- sarial nets and autoencoders, Structural Health Monitoring 20 (4) (2021) 1609–1626

    2021

  14. [22]

    Entezami, H

    A. Entezami, H. Sarmadi, B. Behkamal, C. De Michele, On continuous health monitoring of bridges under serious environmental variability by an innovative multi-task unsupervised learning method, Structure and Infrastructure Engineering (2023) 1–19

    2023

  15. [23]

    Sarmadi, A

    H. Sarmadi, A. Entezami, F. Magalh ˜aes, Unsupervised data normalization for continuous dynamic monitoring by an innovative hybrid feature weighting-selection algorithm and natural nearest neighbor searching, Structural Health Monitoring 22 (6) (2023) 4005–4026

    2023

  16. [24]

    Q. Pan, Y . Bao, H. Li, Transfer learning-based data anomaly detection for structural health monitoring, Structural Health Monitoring (2023) 14759217221142174

    2023

  17. [25]

    Zhang, Z

    Y . Zhang, Z. Tang, R. Yang, Data anomaly detection for structural health monitoring by multi-view representation based on local binary patterns, Measurement 202 (2022) 111804

    2022

  18. [26]

    X. Wang, Y . Du, X. Zhou, Y . Xia, et al., Data anomaly detection through semisupervised learning aided by customised data augmentation techniques, Structural Control and Health Monitoring 2023 (2023)

    2023

  19. [27]

    Balestriero, M

    R. Balestriero, M. Ibrahim, V . Sobal, A. Morcos, S. Shekhar, T. Goldstein, F. Bordes, A. Bardes, G. Mialon, Y . Tian, et al., A cookbook of self-supervised learning, arXiv preprint arXiv:2304.12210 (2023)

    2023 arXiv

  20. [28]

    J. Gui, T. Chen, J. Zhang, Q. Cao, Z. Sun, H. Luo, D. Tao, A survey on self-supervised learning: Algorithms, applications, and future trends, IEEE Transactions on Pattern Analysis and Machine Intelligence (2024)

    2024

  21. [29]

    K. He, H. Fan, Y . Wu, S. Xie, R. Girshick, Momentum contrast for unsupervised visual representation learning, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 9729–9738

    2020

  22. [30]

    T. Chen, S. Kornblith, M. Norouzi, G. Hinton, A simple framework for contrastive learning of visual representations, in: International conference on machine learning, PMLR, 2020, pp. 1597–1607

    2020

  23. [31]

    Misra, L

    I. Misra, L. v. d. Maaten, Self-supervised learning of pretext-invariant representations, in: Proceedings of the IEEE /CVF conference on computer vision and pattern recognition, 2020, pp. 6707–6717

    2020

  24. [32]

    Brown, B

    T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al., Language models are few-shot learners, Advances in neural information processing systems 33 (2020) 1877–1901. 17

    2020

  25. [33]

    Devlin, M.-W

    J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, Bert: Pre-training of deep bidirectional transformers for language understanding, arXiv preprint arXiv:1810.04805 (2018)

    2018 arXiv

  26. [34]

    Goyal, M

    P. Goyal, M. Caron, B. Lefaudeux, M. Xu, P. Wang, V . Pai, M. Singh, V . Liptchinsky, I. Misra, A. Joulin, et al., Self-supervised pretraining of visual features in the wild, arXiv preprint arXiv:2103.01988 (2021)

    2021 arXiv

  27. [35]

    X. Chen, K. He, Exploring simple siamese representation learning, in: Proceedings of the IEEE /CVF conference on computer vision and pattern recognition, 2021, pp. 15750–15758

    2021

  28. [36]

    X. Liu, F. Zhang, Z. Hou, L. Mian, Z. Wang, J. Zhang, J. Tang, Self-supervised learning: Generative or contrastive, IEEE Transactions on Knowledge and Data Engineering 35 (1) (2021) 857–876

    2021

  29. [37]

    T. Chen, S. Kornblith, K. Swersky, M. Norouzi, G. E. Hinton, Big self-supervised models are strong semi-supervised learners, Advances in neural information processing systems 33 (2020) 22243–22255

    2020

  30. [38]

    J. L. Devore, et al., Probability and Statistics for Engineering and the Sciences, V ol. 5, Duxbury Press Belmont, 1995

    1995

  31. [39]

    L. Dinh, J. Sohl-Dickstein, S. Bengio, Density estimation using real nvp, arXiv preprint arXiv:1605.08803 (2016)

    2016 arXiv

  32. [40]

    Yang, Xlnet: Generalized autoregressive pretraining for language understanding, arXiv preprint arXiv:1906.08237 (2019)

    Z. Yang, Xlnet: Generalized autoregressive pretraining for language understanding, arXiv preprint arXiv:1906.08237 (2019)

    2019 arXiv

  33. [41]

    LeCun, L

    Y . LeCun, L. Bottou, Y . Bengio, P. Haffner, Gradient-based learning applied to document recognition, Proceedings of the IEEE 86 (11) (1998) 2278–2324

    1998

  34. [42]

    Krizhevsky, I

    A. Krizhevsky, I. Sutskever, G. E. Hinton, Imagenet classification with deep convolutional neural networks, Advances in neural information processing systems 25 (2012)

    2012

  35. [43]

    Wickstrøm, M

    K. Wickstrøm, M. Kamp ffmeyer, K. Ø. Mikalsen, R. Jenssen, Mixing up contrastive learning: Self-supervised representation learning for time series, Pattern Recognition Letters 155 (2022) 54–61

    2022

  36. [44]

    Zhang, M

    H. Zhang, M. Cisse, Y . N. Dauphin, D. Lopez-Paz, mixup: Beyond empirical risk minimization, arXiv preprint arXiv:1710.09412 (2017)

    2017 arXiv

  37. [45]

    J. Zhu, H. Bai, L. Wang, Patch-mix transformer for unsupervised domain adaptation: A game perspective, in: Proceedings of the IEEE /CVF conference on computer vision and pattern recognition, 2023, pp. 3561–3571

    2023

  38. [46]

    S. Kim, G. Lee, S. Bae, S.-Y . Yun, Mixco: Mix-up contrastive learning for visual representation, arXiv preprint arXiv:2010.06300 (2020)

    2020 arXiv

  39. [47]

    S. Ren, H. Wang, Z. Gao, S. He, A. Yuille, Y . Zhou, C. Xie, A simple data mixing prior for improving self-supervised learning, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 14595–14604

    2022

  40. [48]

    Z. Shen, Z. Liu, Z. Liu, M. Savvides, T. Darrell, E. Xing, Un-mix: Rethinking image mixtures for unsupervised visual representation learning, in: Proceedings of the AAAI Conference on Artificial Intelligence, V ol. 36, 2022, pp. 2216–2224

    2022

  41. [49]

    Goodfellow, J

    I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, Y . Bengio, Generative adversarial nets, Advances in neural information processing systems 27 (2014)

    2014

  42. [50]

    Makhzani, J

    A. Makhzani, J. Shlens, N. Jaitly, I. Goodfellow, B. Frey, Adversarial autoencoders, arXiv preprint arXiv:1511.05644 (2015)

    2015 arXiv

  43. [51]

    C. Wu, J. Pfrommer, M. Zhou, J. Beyerer, Self-supervised generative-contrastive learning of multi-modal euclidean input for 3d shape latent representations: A dynamic switching approach, IEEE Transactions on Multimedia (2023)

    2023

  44. [52]

    Z. Qi, R. Dong, G. Fan, Z. Ge, X. Zhang, K. Ma, L. Yi, Contrast with reconstruct: Contrastive 3d representation learning guided by generative pretraining, in: International Conference on Machine Learning, PMLR, 2023, pp. 28223–28243

    2023

  45. [53]

    Y . Bao, J. Li, T. Nagayama, Y . Xu, B. F. Spencer Jr, H. Li, The 1st international project competition for structural health monitoring (ipc-shm, 2020): A summary and benchmark problem, Structural Health Monitoring 20 (4) (2021) 2229–2239

    2021

  46. [54]

    Grandini, E

    M. Grandini, E. Bagli, G. Visani, Metrics for multi-class classification: an overview, arXiv preprint arXiv:2008.05756 (2020)

    2020 arXiv

  47. [55]

    Paszke, S

    A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, et al., Pytorch: An imperative style, high-performance deep learning library, Advances in neural information processing systems 32 (2019)

    2019

  48. [56]

    D. P. Kingma, Adam: A method for stochastic optimization, arXiv preprint arXiv:1412.6980 (2014)

    2014 arXiv

  49. [57]

    X. Qiu, T. Sun, Y . Xu, Y . Shao, N. Dai, X. Huang, Pre-trained models for natural language processing: A survey, Science China technological sciences 63 (10) (2020) 1872–1897

    2020

  50. [58]

    Assran, R

    M. Assran, R. Balestriero, Q. Duval, F. Bordes, I. Misra, P. Bojanowski, P. Vincent, M. Rabbat, N. Ballas, The hidden uniform cluster prior in self-supervised learning, arXiv preprint arXiv:2210.07277 (2022). 18

    2022 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.