Pith. sign in

REVIEW 4 major objections 7 minor 103 references

A Unified and Scalable Membership Inference Method for Visual Self-supervised Encoder via Part-aware Capability

T0 review · 4 major / 7 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read PartCrop infers training membership from self-supervised vision encoders without knowing the training recipe.

desk verdict PartCrop is a genuinely novel recipe-free membership inference attack on visual SSL encoders, but its headline claim quietly assumes an API exposes spatial feature maps rather than pooled embeddings. read the letter →

arxiv 2505.10351 v1 pith:KCADEAI3 submitted 2025-05-15 cs.CV

Jie Zhu , Jirong Zha , Ding Li , Leye Wang This is my paper · ORCID
classification cs.CV
keywords membershipinferenceself-supervisedlearningpart-awarecapabilityblack-boxattackvisualencoderprivacycontrastivemaskedimagemodelingscalingrisk
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a single black-box attack, PartCrop, can decide whether a given image was part of the training set of a visual self-supervised encoder, even when the attacker has no knowledge of the training method or its hyperparameters. It builds on the observation that self-supervised models learn a part-aware capability and that training images produce stronger part-response signals than test images. A sympathetic reader would care because real encoder APIs are black boxes, while the previous state-of-the-art attack requires knowing the training recipe and fails on masked image modeling. PartCrop reports attack accuracy well above chance across MAE, DINO, and MoCo on three datasets, and the paper further studies how data and model scaling change the attack's success.

What carries the argument

The load-bearing construction is the part-response energy. An image is passed through the encoder to obtain a spatial feature map $\chi \in \mathbb{R}^{N \times D}$; $m$ random crops at scale 0.08–0.2 are encoded into part vectors $P$. Each part vector $p_i$ is multiplied against $\chi$ to form a similarity vector $v_i$, which is softmaxed into a probability distribution and then compared to a uniform and a gaussian reference distribution by KL divergence, giving $e_i = [e_i^u, e_i^g]$. The sorted energy vectors are concatenated into the membership feature $E \in \mathbb{R}^{2m}$ that trains a four-layer MLP attacker, so the method converts the part-response asymmetry between training and test data into a discriminative score without using any training recipe of the victim.

What would settle it

Run the PartCrop pipeline unchanged against an encoder API that returns only the pooled global embedding (no spatial map) in the partial setting: if attack accuracy falls to roughly 50 percent for MAE, DINO, and MoCo, the spatial part-response mechanism is confirmed as the active signal; if accuracy stays above chance, that premise is not necessary. A complementary check is to attack a model trained on a synthetic dataset of texture-only images that contain no object parts, where the claimed part-aware capability should have nothing to attach to and accuracy should collapse toward chance.

Watch

Extended reading notes

Core claim

The central claim is that membership leaves a measurable trace in how self-supervised encoders respond to object parts: for training images, the similarity between a cropped object part and the image's spatial feature map is more peaked than for test images, and this asymmetry persists across contrastive learning, masked image modeling, and hybrid paradigms. PartCrop harvests this trace without any training-recipe prior by randomly cropping many small patches, computing their response energy against the image feature map via KL divergence to uniform and gaussian reference distributions, and feeding the sorted energies to a small fully connected attacker. In the paper's experiments, this yields attack accuracies of 58.38/57.65/66.36 percent for MAE, 60.62/59.13/56.13 percent for DINO, and 77.20/78.84/73.77 percent for MoCo on CIFAR100/CIFAR10/Tinyimagenet, with the method also transferring across datasets in the shadow setting.

Load-bearing premise

The attack requires the black-box service to return a spatial feature map with per-location vectors; if only a single global embedding is exposed, the part-response energies cannot be constructed and PartCrop's core signal disappears.

Editorial extensions

If this is right

  • A black-box encoder service that exposes spatial feature maps can be probed for training membership even when the model, its paradigm, and its hyperparameters are hidden.
  • The attack transfers across datasets: an attacker trained on one public dataset can infer membership in encoders trained on other datasets, widening the threat beyond data-leakage scenarios.
  • Early stopping, differential privacy, and shrinking the crop scale range all reduce PartCrop's accuracy, but the paper reports only partial mitigation; MoCo remains substantially vulnerable.
  • Scaling behavior is non-trivial: larger encoders raise membership-inference risk, larger training corpora lower it (more slowly for contrastive learning), and attackers benefit from more training data.
  • PartCrop-v2, using Tanh activation and RMSNorm, stabilizes attacker scaling and avoids the random-guess collapse observed for narrow or deep attacker variants.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If a deployment exposes only a pooled global embedding, the spatial feature map that PartCrop needs is unavailable; a testable extension is whether attention maps or multi-crop consistency can reconstruct enough spatial response information to restore attack power.
  • The part-response gap between members and non-members may be a general memorization probe, useful for auditing datasets or detecting overfitting in self-supervised models beyond the binary membership question.
  • The scaling results imply that privacy risk tracks model capacity: publishing membership-inference accuracy alongside task accuracy in model cards could make the trade-off visible to deployers.
  • Because the paper shows PartCrop features are complementary to image-level EncoderMI features, a hybrid of both signals may push attack accuracy higher than either alone.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes PartCrop, a membership inference attack against visual self-supervised encoders under a black-box setting in which the adversary does not know the SSL training method or its hyperparameters. The attack crops multiple image parts, feeds the whole image and the crops through the victim encoder to obtain a spatial feature map and part features, computes per-location cosine similarities, converts them into a distribution, and uses KL divergence against uniform and Gaussian benchmarks as membership features (Eqs. 1–5). A simple MLP attacker is trained on these features. Experiments across MAE, DINO, and MoCo on CIFAR10/100 and Tinyimagenet report attack accuracies of roughly 53–79%, outperforming baselines including EncoderMI, and the paper evaluates defenses (early stopping, differential privacy, shrinking crop scale range) and scaling behavior, leading to the PartCrop-v2 attacker with Tanh and RMSNorm.

Significance. The paper addresses a timely and practically relevant question: membership inference against SSL encoders without training-recipe knowledge. The part-aware motivation is well grounded in prior work [20], and the evaluation is broad, covering multiple SSL paradigms, architectures, datasets, and threat models. The release of code and the scaling study are additional strengths. If the threat model is refined to match real API output granularity, the method could be practically valuable. However, the current manuscript overstates the generality of some results, and the missing error bars and a contradiction in the scaling narrative weaken several quantitative claims.

major comments (4)
  1. [Sections 3–4, Eq. (1)] The threat model in Section 3 defines only black-box access, but Stage 1 of PartCrop assumes the encoder outputs a spatial feature map χ ∈ R^{H×W×D} and the membership feature is built from per-location similarities (Eqs. 3–5). If a real black-box API exposes only a pooled or global embedding, the attack cannot be constructed. This output-granularity requirement should be stated explicitly in the threat model, justified with examples of APIs that expose feature maps, or evaluated with a pooled-embedding variant. As written, the central claim that PartCrop works 'without knowing how self-supervised model is trained' is conditional on an unstated API assumption.
  2. [Section 8.2.1, Fig. 7, Answer 1] The scaling experiments are reported without error bars even though the text states that three repeated runs were averaged, so the reader cannot assess whether the non-monotonic trends for DINO and MAE are significant. More seriously, Answer 1 states that scaling encoder training data reduces membership inference risk, with a more gradual reduction in contrastive learning, but Fig. 7(a) shows MoCo attack accuracy increasing with data scale (60.89% to 64.63% to 66.0%). This is a direct contradiction that is not resolved by the single ImageNet22K point (51.40%), which is also reported without variance. Please correct Answer 1 or provide the evidence that reconciles the stated conclusion with the plotted trend, and report error bars or confidence intervals.
  3. [Table 6, Section 6.3] Several reported cross-dataset successes are degenerate classifiers. For example, MAE CIFAR10→Tinyimagenet achieves Acc 52.13, Pre 99.51, Rec 4.14, and MoCo Tinyimagenet→CIFAR100 achieves Pre 100.00, Rec 12.96. With recall below 5%, an accuracy near 52% is essentially equivalent to an all-non-member classifier, and the F1 value is driven by the trivial precision-recall trade-off. The claim in Section 6.3 that 'PartCrop succeeds in all cross-dataset attacks' is therefore not supported by the reported metrics. Please report balanced measures such as AUC or balanced accuracy, or tune the decision threshold, and temper the generalization claim accordingly.
  4. [Section 6.2.3, Fig. 5, Appendix E] The method's performance is highly sensitive to the crop scale choice (e.g., MAE accuracy drops from 58.38% to 50.32% when the scale changes from (0.08, 0.2) to (0.01, 0.03)), and Appendix E acknowledges that the pre-defined crop scale may not suit all datasets. Since the adversary in the stated threat model does not know the training recipe, it also cannot know the optimal crop scale for a given victim model. The paper should specify how the attacker chooses the crop scale in practice (e.g., a fixed default across all datasets or a dataset-adaptive rule) and provide a sensitivity analysis across all evaluated datasets so that the 'unified' claim is not contingent on an unavailable oracle choice.
minor comments (7)
  1. [Section 7] The defense name 'PartCorp' appears in the text; it should be 'PartCrop'.
  2. [Section 6.2.1] The subsection heading 'Merbership Feature' contains a typo; it should be 'Membership Feature'.
  3. [Table 4 caption] The caption states 'Comparisions with SupervisedMI...'; the intended word is 'Comparisons'.
  4. [Section 8.3.2] The sentence 'To answer Question 2' should refer to Question 4, since the paragraph addresses attacker scaling rather than attacker training-data size.
  5. [Eq. (5), Section 4 Stage 2] The KL-divergence direction and the support of the Gaussian benchmark should be defined precisely: the distributions are over the N spatial locations, and it should be clarified whether the Gaussian vector g_i is regenerated for each query or fixed across queries, since this affects the stochasticity of the membership feature.
  6. [Section 8.2.2, Fig. 8] The text says that after scaling attacker training data, DINO performance 'is still higher than that of 2000 when the size is 10000.' This is true for accuracy (58.10 vs 57.99) but false for F1 (48.86 vs 54.58), so the claim should be qualified to refer to accuracy only, or the discrepancy should be explained.
  7. [Table 11] The variant definitions appear mislabeled: 'Narrow' replaces d=512 with d=1024 (a wider network) and 'Wide' replaces d=512 with d=256 (a narrower network). If the labels are correct, the accompanying discussion of 'too small' dimensions for the narrow variant is confusing; please clarify the naming or the definitions.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: PartCrop is an empirical attack evaluated on held-out member/non-member splits; its only self-citation, to the authors' prior part-aware work, is motivational rather than load-bearing.

full rationale

PartCrop does not derive its attack target from its inputs. Membership labels come from real data partitions (Table 3: known training/test data used to train the attacker, unknown splits used for evaluation), and the reported accuracies are measured on those held-out splits. The membership feature E in Eqs. (3)-(5) is a hand-designed feature extractor built from KL divergences between the similarity distribution and uniform/Gaussian benchmarks; it is not a fitted quantity that by construction equals the membership label, and no equation in the method reduces the evaluation result to a training objective or to the feature definition itself. The central premise that self-supervised encoders exhibit part-aware response differences is supported by the authors' prior work [20], which is a self-citation, but the paper also provides its own direct empirical evidence in Fig. 2 (training vs. test part-response curves) and verifies the attack across MAE, DINO, MoCo, multiple datasets, the shadow setting, and additional paradigms. The citation to [20] therefore supplies motivation, not the result. The disclosed conference extension [24] is not used as evidence for any claim. The main gap, that the threat model assumes the black-box API exposes a spatial feature map chi in R^{H x W x D} (Eq. 1), is an unstated capability assumption about the service interface rather than a circular derivation: even if that assumption were wrong, the method would fail for empirical/engineering reasons, not because its output was encoded in its input. Overall, the paper is self-contained against external benchmarks and shows no significant circularity.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on empirical design choices rather than on a derivation. The most fragile external premise is the spatial-feature-map oracle, and the most fragile internal premise is the part-aware-response mechanism, which is only qualitatively demonstrated.

free parameters (5)
  • crop scale range = (0.08, 0.2)
    Selected by ablation in Sec 6.2.3; larger or smaller ranges degrade attack accuracy.
  • crop count m = 128
    Chosen as trade-off between accuracy and compute in Sec 6.2.2.
  • KL benchmark distributions = Uniform and Gaussian N(0,1)
    Hand-chosen in Eq. 5; the Gaussian is randomly generated per query, with no principled justification given.
  • attacker width and depth = width 512, depth 4
    Default architecture in Tab 1 is claimed to work well; ablation shows variant sensitivity.
  • patch resize size = 16x16
    Implementation detail in Sec 5.4 for resizing crops before encoding.
assumptions (5)
  • domain assumption The black-box encoder exposes a spatial feature map of each input (Eq. 1).
    Required for computing patch-image similarity distributions; not justified by API design in the threat model (Sec 3).
  • domain assumption Self-supervised models share part-aware capability, and training images elicit stronger part responses than test images.
    Motivating mechanism from ref [20] and Fig 2, which shows only a handful of qualitative examples.
  • domain assumption Random small-scale crops (0.08-0.2) are adequate proxies for object parts.
    Used to generate part queries (Sec 4); ablation shows extreme scales fail, but no object-detection verification is used.
  • ad hoc to paper A Gaussian N(0,1) distribution over feature map locations is a meaningful KL-divergence benchmark.
    No theoretical reason is given; two benchmarks are claimed to work empirically (Eq. 5).
  • domain assumption Attack success is defined as better-than-50% accuracy on a balanced set.
    Sec 5.2; some reported successes are near 50% with degenerate precision and recall (Tab 6).

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Unified and Scalable Membership Inference Method for Visual Self-supervised Encoder via Part-aware Capability." pith.science (2026). https://pith.science/paper/KCADEAI3

@misc{pith2026250510351,
  author       = {Pith},
  title        = {Pith review of: A Unified and Scalable Membership Inference Method for Visual Self-supervised Encoder via Part-aware Capability},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KCADEAI3}},
  note         = {Machine review of arXiv:2505.10351}
}
read the original abstract

Self-supervised learning shows promise in harnessing extensive unlabeled data, but it also confronts significant privacy concerns, especially in vision. In this paper, we perform membership inference on visual self-supervised models in a more realistic setting: self-supervised training method and details are unknown for an adversary when attacking as he usually faces a black-box system in practice. In this setting, considering that self-supervised model could be trained by completely different self-supervised paradigms, e.g., masked image modeling and contrastive learning, with complex training details, we propose a unified membership inference method called PartCrop. It is motivated by the shared part-aware capability among models and stronger part response on the training data. Specifically, PartCrop crops parts of objects in an image to query responses within the image in representation space. We conduct extensive attacks on self-supervised models with different training protocols and structures using three widely used image datasets. The results verify the effectiveness and generalization of PartCrop. Moreover, to defend against PartCrop, we evaluate two common approaches, i.e., early stop and differential privacy, and propose a tailored method called shrinking crop scale range. The defense experiments indicate that all of them are effective. Finally, besides prototype testing on toy visual encoders and small-scale image datasets, we quantitatively study the impacts of scaling from both data and model aspects in a realistic scenario and propose a scalable PartCrop-v2 by introducing two structural improvements to PartCrop. Our code is at https://github.com/JiePKU/PartCrop.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

103 extracted references · 59 canonical work pages

  1. [20]

    Transactions on Machine Learning Research (2023)

    Zhu, J., Qi, J., Ding, M., Chen, X., Luo, P., Wang, X., Liu, W., Wang, L., Wang, J.: Understanding self-supervised pretrain- ing with part-aware representation learning. Transactions on Machine Learning Research (2023)

    2023

  2. [1]

    In: Inter- national Conference on Machine Learning, pp

    Chen, T., Kornblith, S., Norouzi, M., Hin- ton, G.: A simple framework for contrastive learning of visual representations. In: Inter- national Conference on Machine Learning, pp. 1597–1607 (2020). PMLR

    2020

  3. [2]

    Advances in Neural Information Processing Systems 33, 22243–22255 (2020) 19

    Chen, T., Kornblith, S., Swersky, K., Norouzi, M., Hinton, G.E.: Big self-supervised models are strong semi- supervised learners. Advances in Neural Information Processing Systems 33, 22243–22255 (2020) 19

    2020

  4. [3]

    IEEE transactions on pattern analysis and machine intelligence 43(11), 4037–4058 (2020)

    Jing, L., Tian, Y.: Self-supervised visual fea- ture learning with deep neural networks: A survey. IEEE transactions on pattern analysis and machine intelligence 43(11), 4037–4058 (2020)

    2020

  5. [4]

    IEEE Trans- actions on Pattern Analysis and Machine Intelligence (2024)

    Gui, J., Chen, T., Zhang, J., Cao, Q., Sun, Z., Luo, H., Tao, D.: A survey on self-supervised learning: Algorithms, appli- cations, and future trends. IEEE Trans- actions on Pattern Analysis and Machine Intelligence (2024)

    2024

  6. [5]

    https://www.bbc.com/news/ technology-51220654 (2020)

    Twitter demands AI company stops ’col- lecting faces’. https://www.bbc.com/news/ technology-51220654 (2020)

    2020

  7. [6]

    In: Proceedings of the 2021 ACM SIGSAC Con- ference on Computer and Communications Security, pp

    Liu, H., Jia, J., Qu, W., Gong, N.Z.: Enco- dermi: Membership inference against pre- trained encoders in contrastive learning. In: Proceedings of the 2021 ACM SIGSAC Con- ference on Computer and Communications Security, pp. 2081–2095 (2021)

    2021

  8. [7]

    International Journal of Computer Vision 130(6), 1459–1473 (2022)

    Wei, X., Yan, H., Li, B.: Sparse black-box video attack with reinforcement learning. International Journal of Computer Vision 130(6), 1459–1473 (2022)

    2022

Show all 103 references
  1. [8]

    International Journal of Computer Vision, 1–23 (2024)

    Ren, M., Wang, Y., Zhu, Y., Huang, Y., Sun, Z., Li, Q., Tan, T.: Artificial immune system of secure face recognition against adversarial attacks. International Journal of Computer Vision, 1–23 (2024)

    2024

  2. [9]

    In: 2017 IEEE Symposium on Security and Privacy, pp

    Shokri, R., Stronati, M., Song, C., Shmatikov, V.: Membership inference attacks against machine learning models. In: 2017 IEEE Symposium on Security and Privacy, pp. 3–18 (2017). IEEE

    2017

  3. [10]

    Proceed- ings of the ACM Conference on Com- puter and Communications Security, 634– 646 (2018)

    Nasr, M., Shokri, R., Houmansadr, A.: Machine learning with membership privacy using adversarial regularization. Proceed- ings of the ACM Conference on Com- puter and Communications Security, 634– 646 (2018)

    2018

  4. [11]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp

    Caron, M., Touvron, H., Misra, I., J´ egou, H., Mairal, J., Bojanowski, P., Joulin, A.: Emerging properties in self-supervised vision transformers. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9650–9660 (2021)

    2021

  5. [12]

    In: Proceed- ings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pp

    He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.: Momentum contrast for unsupervised visual representation learning. In: Proceed- ings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pp. 9729–9738 (2020)

    2020

  6. [13]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    He, K., Chen, X., Xie, S., Li, Y., Doll´ ar, P., Girshick, R.: Masked autoencoders are scal- able vision learners. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16000–16009 (2022)

    2022

  7. [14]

    In: 31st USENIX Security Symposium, pp

    Liu, Y., Wen, R., He, X., Salem, A., Zhang, Z., Backes, M., De Cristofaro, E., Fritz, M., Zhang, Y.: Ml-doctor: Holistic risk assess- ment of inference attacks against machine learning models. In: 31st USENIX Security Symposium, pp. 4525–4542 (2022)

    2022

  8. [15]

    In: Network and Distributed System Symposium (2019)

    Salem, A., Zhang, Y., Humbert, M., Berrang, P., Fritz, M., Backes, M.: Ml- leaks: Model and data independent mem- bership inference attacks and defenses on machine learning models. In: Network and Distributed System Symposium (2019)

    2019

  9. [16]

    In: Interna- tional Conference on Machine Learning, pp

    Touvron, H., Cord, M., Douze, M., Massa, F., Sablayrolles, A., J´ egou, H.: Training data-efficient image transformers & dis- tillation through attention. In: Interna- tional Conference on Machine Learning, pp. 10347–10357 (2021). PMLR

    2021

  10. [17]

    arXiv preprint arXiv:2202.03026 (2022)

    Chen, X., Ding, M., Wang, X., Xin, Y., Mo, S., Wang, Y., Han, S., Luo, P., Zeng, G., Wang, J.: Context autoencoder for self- supervised representation learning. arXiv preprint arXiv:2202.03026 (2022)

    2022 arXiv

  11. [18]

    In: Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pp

    Chen, X., Xie, S., He, K.: An empirical study of training self-supervised vision transform- ers. In: Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pp. 9640–9649 (2021)

    2021

  12. [19]

    In: Inter- national Conference on Learning Represen- tations (2022)

    Zhou, J., Wei, C., Wang, H., Shen, W., Xie, C., Yuille, A., Kong, T.: Image bert 20 pre-training with online tokenizer. In: Inter- national Conference on Learning Represen- tations (2022)

    2022

  13. [21]

    In: International Conference on Learning Rep- resentations (2022)

    Bao, H., Dong, L., Piao, S., Wei, F.: Beit: Bert pre-training of image transformers. In: International Conference on Learning Rep- resentations (2022)

    2022

  14. [22]

    CS 231N (2015)

    Le, Y., Yang, X.: Tiny imagenet visual recognition challenge. CS 231N (2015)

    2015

  15. [23]

    Inter- national Journal of Computer Vision (IJCV) 115(3), 211–252 (2015) https://doi.org/10

    Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A.C., Fei-Fei, L.: ImageNet Large Scale Visual Recognition Challenge. Inter- national Journal of Computer Vision (IJCV) 115(3), 211–252 (2015) https:/...

    2015

  16. [24]

    In: Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Com- munications Security, pp

    Zhu, J., Zha, J., Li, D., Wang, L.: A uni- fied membership inference method for visual self-supervised encoder via part-aware capa- bility. In: Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Com- munications Security, pp. 1241–1255 (2024)

    2024

  17. [25]

    the revolution will not be supervised

    Carrigan, C., Green, M.W., Rahman- Davies, A.: “the revolution will not be supervised”: Consent and open secrets in data science. Big data & society 8(2), 20539517211035673 (2021)

    2021

  18. [26]

    In: 37th IEEE/ACM Interna- tional Conference on Automated Software Engineering, pp

    Zhu, J., Wang, L., Han, X.: Safety and per- formance, why not both? bi-objective opti- mized model compression toward ai software deployment. In: 37th IEEE/ACM Interna- tional Conference on Automated Software Engineering, pp. 1–13 (2022)

    2022

  19. [27]

    IEEE Transactions on Software Engineering (2024)

    Zhu, J., Wang, L., Han, X., Liu, A., Xie, T.: Safety and performance, why not both? bi-objective optimized model compression against heterogeneous attacks toward ai software deployment. IEEE Transactions on Software Engineering (2024)

    2024

  20. [28]

    arXiv preprint arXiv:2003.04297 (2020)

    Chen, X., Fan, H., Girshick, R., He, K.: Improved baselines with momen- tum contrastive learning. arXiv preprint arXiv:2003.04297 (2020)

    2020 arXiv

  21. [29]

    Advances in Neural Information Processing Systems, 21271–21284 (2020)

    Grill, J.-B., Strub, F., Altch´ e, F., Tallec, C., Richemond, P., Buchatskaya, E., Doer- sch, C., Avila Pires, B., Guo, Z., Ghesh- laghi Azar, M., et al.: Bootstrap your own latent-a new approach to self-supervised learning. Advances in Neural Information Processing Systems, 2...

    2020

  22. [30]

    In: 28th USENIX Security Symposium, pp

    Carlini, N., Liu, C., Erlingsson, ´U., Kos, J., Song, D.: The secret sharer: Evaluat- ing and testing unintended memorization in neural networks. In: 28th USENIX Security Symposium, pp. 267–284 (2019)

    2019

  23. [31]

    In: 2019 IEEE Symposium on Security and Privacy, pp

    Nasr, M., Shokri, R., Houmansadr, A.: Com- prehensive privacy analysis of deep learn- ing: Passive and active white-box inference attacks against centralized and federated learning. In: 2019 IEEE Symposium on Security and Privacy, pp. 739–753 (2019). IEEE

    2019

  24. [32]

    ACM Computing Surveys, 1–37 (2022)

    Hu, H., Salcic, Z., Sun, L., Dobbie, G., Yu, P.S., Zhang, X.: Membership inference attacks on machine learning: A survey. ACM Computing Surveys, 1–37 (2022)

    2022

  25. [33]

    Radiology 226(3), 756–761 (2003)

    Swensen, S.J., Jett, J.R., Hartman, T.E., Midthun, D.E., Sloan, J.A., Sykes, A.-M., Aughenbaugh, G.L., Clemens, M.A.: Lung cancer screening with ct: Mayo clinic expe- rience. Radiology 226(3), 756–761 (2003)

    2003

  26. [34]

    In: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communica- tions Security, pp

    Abadi, M., Chu, A., Goodfellow, I., McMa- han, H.B., Mironov, I., Talwar, K., Zhang, L.: Deep learning with differential privacy. In: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communica- tions Security, pp. 308–318 (2016)

    2016

  27. [35]

    Rahman, M.A., Rahman, T., Lagani` ere, R., Mohammed, N., Wang, Y.: Membership inference attack against differentially pri- vate deep learning model. Trans. Data Priv., 21 61–79 (2018)

    2018

  28. [36]

    In: Proceedings of the 2019 ACM SIGSAC Conference on Com- puter and Communications Security, pp

    Jia, J., Salem, A., Backes, M., Zhang, Y., Gong, N.Z.: Memguard: Defending against black-box membership inference attacks via adversarial examples. In: Proceedings of the 2019 ACM SIGSAC Conference on Com- puter and Communications Security, pp. 259–274 (2019)

    2019

  29. [37]

    Ridnik, T., Ben-Baruch, E., Noy, A., Zelnik- Manor, L.: ImageNet-21K Pretraining for the Masses (2021)

    2021

  30. [38]

    : An image is worth 16x16 words: Transformers for image recognition at scale

    Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. : An image is worth 16x16 words: Transformers for image recognition at scale. In: International Conference on Learning Representat...

    2020

  31. [39]

    In: International Conference on Machine Learning, pp

    Dehghani, M., Djolonga, J., Mustafa, B., Padlewski, P., Heek, J., Gilmer, J., Steiner, A.P., Caron, M., Geirhos, R., Alabdul- mohsin, I., et al.: Scaling vision transformers to 22 billion parameters. In: International Conference on Machine Learning, pp. 7480– 7512 (2023). PMLR

    2023

  32. [40]

    https://en.wikipedia.org/wiki/ Facebook-Cambridge Analytica data scandal (2016)

    Facebook–Cambridge Analytica data scandal. https://en.wikipedia.org/wiki/ Facebook-Cambridge Analytica data scandal (2016)

    2016

  33. [41]

    Advances in Neural Information Processing Systems 30 (2017)

    Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. Advances in Neural Information Processing Systems 30 (2017)

    2017

  34. [42]

    Krizhevsky, A., Hinton, G., et al.: Learning multiple layers of features from tiny images (2009)

    2009

  35. [43]

    In: Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, pp

    Wang, Y., Wang, C., Wang, Z., Zhou, S., Liu, H., Bi, J., Ding, C., Rajasekaran, S.: Against membership inference attack: Prun- ing is all you need. In: Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, pp. 3141–3147 (2021)

    2021

  36. [44]

    In: Proceed- ings of the AAAI Conference on Artificial Intelligence, pp

    Shejwalkar, V., Houmansadr, A.: Member- ship privacy for machine learning models through knowledge transfer. In: Proceed- ings of the AAAI Conference on Artificial Intelligence, pp. 9549–9557 (2021)

    2021

  37. [45]

    In: International Confer- ence on Machine Learning, pp

    Choquette-Choo, C.A., Tramer, F., Carlini, N., Papernot, N.: Label-only membership inference attacks. In: International Confer- ence on Machine Learning, pp. 1964–1974 (2021). PMLR

    2021

  38. [46]

    In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp

    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    2016

  39. [47]

    In: International Conference on Learning Representations (2018)

    Zhang, H., Cisse, M., Dauphin, Y.N., Lopez- Paz, D.: mixup: Beyond empirical risk min- imization. In: International Conference on Learning Representations (2018)

    2018

  40. [48]

    arXiv preprint arXiv:2010.06300 (2020)

    Kim, S., Lee, G., Bae, S., Yun, S.-Y.: Mixco: Mix-up contrastive learning for visual repre- sentation. arXiv preprint arXiv:2010.06300 (2020)

    2020 arXiv

  41. [49]

    In: 2nd International Conference on Learn- ing Representations (2014)

    Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., Fergus, R.: Intriguing properties of neural networks. In: 2nd International Conference on Learn- ing Representations (2014)

    2014

  42. [50]

    In: Machine Learn- ing and Knowledge Discovery in Databases: European Conference, ECML PKDD 2013, Prague, Czech Republic, September 23-27, 2013, Proceedings, Part III 13, pp

    Biggio, B., Corona, I., Maiorca, D., Nel- son, B., ˇSrndi´ c, N., Laskov, P., Giacinto, G., Roli, F.: Evasion attacks against machine learning at test time. In: Machine Learn- ing and Knowledge Discovery in Databases: European Conference, ECML PKDD 2013, Prague, Czech Republic...

    2013

  43. [51]

    Advances in Neural Information Processing Systems 33, 2983–2994 (2020)

    Kim, M., Tack, J., Hwang, S.J.: Adver- sarial self-supervised contrastive learning. Advances in Neural Information Processing Systems 33, 2983–2994 (2020)

    2020

  44. [52]

    In: European Conference on Com- puter Vision, pp

    Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Doll´ ar, P., Zitnick, C.L.: Microsoft coco: Common objects in 22 context. In: European Conference on Com- puter Vision, pp. 740–755 (2014). Springer

    2014

  45. [53]

    arXiv preprint arXiv:2210.07224 (2022)

    Hu, R., Debnath, S., Xie, S., Chen, X.: Exploring long-sequence masked autoen- coders. arXiv preprint arXiv:2210.07224 (2022)

    2022 arXiv

  46. [54]

    Advances in Neural Information Processing Systems 33, 9912–9924 (2020)

    Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., Joulin, A.: Unsuper- vised learning of visual features by con- trasting cluster assignments. Advances in Neural Information Processing Systems 33, 9912–9924 (2020)

    2020

  47. [55]

    In: International Conference on Learning Representations (2022)

    Bardes, A., Ponce, J., Lecun, Y.: Vicreg: Variance-invariance-covariance regular- ization for self-supervised learning. In: International Conference on Learning Representations (2022)

    2022

  48. [56]

    In: Proceedings of the AAAI Conference on Artificial Intelligence, pp

    Qi, Y., Yang, F., Zhu, Y., Liu, Y., Wu, L., Zhao, R., Li, W.: Exploring stochas- tic autoregressive image modeling for visual representation. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 2074–2081 (2023)

    2023

  49. [57]

    arXiv preprint arXiv:2206.08954 (2022)

    Chen, Y., Bardes, A., Li, Z., LeCun, Y.: Intra-instance vicreg: Bag of self-supervised image patch embedding. arXiv preprint arXiv:2206.08954 (2022)

    2022 arXiv

  50. [58]

    In: 2022 IEEE Symposium on Security and Privacy, pp

    Jia, J., Liu, Y., Gong, N.Z.: Badencoder: Backdoor attacks to pre-trained encoders in self-supervised learning. In: 2022 IEEE Symposium on Security and Privacy, pp. 2043–2059 (2022). IEEE

    2022

  51. [59]

    In: 31st USENIX Security Symposium, pp

    Liu, H., Jia, J., Gong, N.Z.: Poisone- dencoder: Poisoning the unlabeled pre- training data in contrastive learning. In: 31st USENIX Security Symposium, pp. 3629–3645 (2022)

    2022

  52. [60]

    In: Proceedings of the 2022 ACM SIGSAC Conference on Com- puter and Communications Security, pp

    Liu, Y., Jia, J., Liu, H., Gong, N.Z.: Stole- nencoder: Stealing pre-trained encoders in self-supervised learning. In: Proceedings of the 2022 ACM SIGSAC Conference on Com- puter and Communications Security, pp. 2115–2128 (2022)

    2022

  53. [61]

    In: Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, pp

    Shokri, R., Shmatikov, V.: Privacy- preserving deep learning. In: Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, pp. 1310–1321 (2015)

    2015

  54. [62]

    Liu, Z., He, K.: A decade’s battle on dataset bias: Are we there yet? arXiv preprint arXiv:2403.08632 (2024)

    2024 arXiv

  55. [63]

    : Applied Predictive Modeling vol

    Kuhn, M., Johnson, K., et al. : Applied Predictive Modeling vol. 26. Springer, ??? (2013)

    2013

  56. [64]

    In: Proceedings of the 23rd Interna- tional Conference on Machine Learning, pp

    Caruana, R., Niculescu-Mizil, A.: An empir- ical comparison of supervised learning algo- rithms. In: Proceedings of the 23rd Interna- tional Conference on Machine Learning, pp. 161–168 (2006)

    2006

  57. [65]

    Journal of machine learning research 9(11) (2008)

    Maaten, L., Hinton, G.: Visualizing data using t-sne. Journal of machine learning research 9(11) (2008)

    2008

  58. [66]

    Advances in Neural Information Processing Systems 32 (2019)

    Zhang, B., Sennrich, R.: Root mean square layer normalization. Advances in Neural Information Processing Systems 32 (2019)

    2019

  59. [67]

    In: Proc

    Maas, A.L., Hannun, A.Y., Ng, A.Y., et al.: Rectifier nonlinearities improve neural net- work acoustic models. In: Proc. Icml, vol. 30, p. 3 (2013). Atlanta, GA

    2013

  60. [68]

    Neural networks 107, 3–11 (2018)

    Elfwing, S., Uchibe, E., Doya, K.: Sigmoid- weighted linear units for neural network function approximation in reinforcement learning. Neural networks 107, 3–11 (2018)

    2018

  61. [69]

    ArXiv e-prints, 1607 (2016)

    Lei Ba, J., Kiros, J.R., Hinton, G.E.: Layer normalization. ArXiv e-prints, 1607 (2016)

    2016

  62. [70]

    In: 2018 IEEE 31st Computer Security Foun- dations Symposium, pp

    Yeom, S., Giacomelli, I., Fredrikson, M., Jha, S.: Privacy risk in machine learning: Analyzing the connection to overfitting. In: 2018 IEEE 31st Computer Security Foun- dations Symposium, pp. 268–282 (2018). IEEE

    2018

  63. [71]

    In: 30th USENIX Security Symposium, pp

    Song, L., Mittal, P.: Systematic evaluation of privacy risks of machine learning models. In: 30th USENIX Security Symposium, pp. 2615–2632 (2021) 23

    2021

  64. [72]

    In: Proceed- ings of the 2020 ACM SIGSAC Conference on Computer and Communications Security, pp

    Song, C., Raghunathan, A.: Information leakage in embedding models. In: Proceed- ings of the 2020 ACM SIGSAC Conference on Computer and Communications Security, pp. 377–390 (2020)

    2020

  65. [73]

    Proceed- ings on Privacy Enhancing Technologies (2019)

    Hayes, J., Melis, L., Danezis, G., De Cristo- faro, E.: Logan: Membership inference attacks against generative models. Proceed- ings on Privacy Enhancing Technologies (2019)

    2019

  66. [74]

    Communications of the ACM, 139–144 (2020)

    Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adver- sarial networks. Communications of the ACM, 139–144 (2020)

    2020

  67. [75]

    In: MobiQuitous 2020-17th EAI International Conference on Mobile and Ubiquitous Systems: Computing, Network- ing and Services, pp

    Duddu, V., Boutet, A., Shejwalkar, V.: Quantifying privacy leakage in graph embedding. In: MobiQuitous 2020-17th EAI International Conference on Mobile and Ubiquitous Systems: Computing, Network- ing and Services, pp. 76–85 (2020)

    2020

  68. [76]

    Advances in Neural Information Processing Systems, 1867–1882 (2022)

    Hu, P., Wang, Z., Sun, R., Wang, H., Xue, M.: M 4i: Multi-modal models membership inference. Advances in Neural Information Processing Systems, 1867–1882 (2022)

    2022

  69. [77]

    arXiv preprint arXiv:1503.02531 2(7) (2015)

    Hinton, G., Vinyals, O., Dean, J., et al.: Distilling the knowledge in a neural net- work. arXiv preprint arXiv:1503.02531 2(7) (2015)

    2015 arXiv

  70. [78]

    In: 31st USENIX Security Symposium, pp

    Tang, X., Mahloujifar, S., Song, L., She- jwalkar, V., Nasr, M., Houmansadr, A., Mittal, P.: Mitigating membership inference attacks by self-distillation through a novel ensemble architecture. In: 31st USENIX Security Symposium, pp. 1433–1450 (2022)

    2022

  71. [79]

    In: 31st USENIX Security Symposium, pp

    Yuan, X., Zhang, L.: Membership inference attacks and defenses in neural network prun- ing. In: 31st USENIX Security Symposium, pp. 4561–4578 (2022)

    2022

  72. [80]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Xie, Z., Geng, Z., Hu, J., Zhang, Z., Hu, H., Cao, Y.: Revealing the dark secrets of masked image modeling. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14475– 14485 (2023)

    2023

  73. [81]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Kong, X., Zhang, X.: Understanding masked image modeling via learning occlusion invariant feature. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6241–6251 (2023)

    2023

  74. [82]

    arXiv preprint arXiv:2202.14037 (2022)

    Saunshi, N., Ash, J., Goel, S., Misra, D., Zhang, C., Arora, S., Kakade, S., Krish- namurthy, A.: Understanding contrastive learning requires incorporating inductive biases. arXiv preprint arXiv:2202.14037 (2022)

    2022 arXiv

  75. [83]

    Zhong, Y., Tang, H., Chen, J., Peng, J., Wang, Y.-X.: Is self-supervised learning more robust than supervised learning? arXiv preprint arXiv:2206.05259 (2022)

    2022 arXiv

  76. [84]

    arXiv preprint arXiv:2205.14141 (2022)

    Wei, Y., Hu, H., Xie, Z., Zhang, Z., Cao, Y., Bao, J., Chen, D., Guo, B.: Contrastive learning rivals masked image modeling in fine-tuning via feature distillation. arXiv preprint arXiv:2205.14141 (2022)

    2022 arXiv

  77. [85]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Saha, A., Tejankar, A., Koohpayegani, S.A., Pirsiavash, H.: Backdoor attacks on self- supervised learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13337–13346 (2022)

    2022

  78. [86]

    Advances in Neural Information Processing Systems 35, 12058– 12070 (2022)

    Dziedzic, A., Duan, H., Kaleem, M.A., Dhawan, N., Guan, J., Cattan, Y., Boenisch, F., Papernot, N.: Dataset inference for self-supervised models. Advances in Neural Information Processing Systems 35, 12058– 12070 (2022)

    2022

  79. [87]

    arXiv preprint arXiv:2210.07346 (2022)

    Li, C., Pang, R., Xi, Z., Du, T., Ji, S., Yao, Y., Wang, T.: Demystifying self- supervised trojan attacks. arXiv preprint arXiv:2210.07346 (2022)

    2022 arXiv

  80. [88]

    In: Proceed- ings of the 2022 ACM SIGSAC Conference on Computer and Communications Security, 24 pp

    Cong, T., He, X., Zhang, Y.: Sslguard: A watermarking scheme for self-supervised learning pre-trained encoders. In: Proceed- ings of the 2022 ACM SIGSAC Conference on Computer and Communications Security, 24 pp. 579–593 (2022)

    2022

  81. [89]

    : Laion-5b: An open large- scale dataset for training next generation image-text models

    Schuhmann, C., Beaumont, R., Vencu, R., Gordon, C., Wightman, R., Cherti, M., Coombes, T., Katta, A., Mullis, C., Worts- man, M., et al. : Laion-5b: An open large- scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems 3...

    2022

  82. [90]

    arXiv preprint arXiv:2111.02114 (2021)

    Schuhmann, C., Vencu, R., Beaumont, R., Kaczmarczyk, R., Mullis, C., Katta, A., Coombes, T., Jitsev, J., Komatsuzaki, A.: Laion-400m: Open dataset of clip-filtered 400 million image-text pairs. arXiv preprint arXiv:2111.02114 (2021)

    2021 arXiv

  83. [91]

    In: International Conference on Machine Learning, pp

    Tan, M., Le, Q.: Efficientnet: Rethink- ing model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR

    2019

  84. [92]

    In: Interna- tional Conference on Machine Learning, pp

    Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: Interna- tional Conference on Machine Learning, pp. 8748–8763 (2021). PMLR

    2021

  85. [93]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp

    Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A.C., Lo, W.-Y., et al.: Segment anything. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4015–4026 (2023)

    2023

  86. [94]

    arXiv preprint arXiv:1807.03748 (2018)

    Oord, A.v.d., Li, Y., Vinyals, O.: Represen- tation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 (2018)

    2018 arXiv

  87. [95]

    https: //doi.org/10.7910/DVN/DBW86T

    Tschandl, P.: The HAM10000 Dataset, a Large Collection of Multi-source Dermatoscopic Images of Com- mon Pigmented Skin Lesions. https: //doi.org/10.7910/DVN/DBW86T . https://doi.org/10.7910/DVN/DBW86T

    doi

  88. [96]

    In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp

    Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee

    2009

  89. [97]

    Advances in neural information processing systems 25 (2012)

    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolu- tional neural networks. Advances in neural information processing systems 25 (2012)

    2012

  90. [98]

    arXiv preprint arXiv:2304.12210 (2023)

    Geiping, J., Garrido, Q., Fernandez, P., Bar, A., Pirsiavash, H., LeCun, Y., Goldblum, M.: A cookbook of self-supervised learning. arXiv preprint arXiv:2304.12210 (2023)

    2023 arXiv

  91. [99]

    In: European Conference on Computer Vision, pp

    Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., Zagoruyko, S.: End-to-end object detection with transform- ers. In: European Conference on Computer Vision, pp. 213–229 (2020). Springer

    2020

  92. [100]

    arXiv preprint arXiv:2303.05499 (2023)

    Liu, S., Zeng, Z., Ren, T., Li, F., Zhang, H., Yang, J., Li, C., Yang, J., Su, H., Zhu, J., et al.: Grounding dino: Marrying dino with grounded pre-training for open-set object detection. arXiv preprint arXiv:2303.05499 (2023)

    2023 arXiv

  93. [101]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Wang, Z., Li, Y., Chen, X., Lim, S.-N., Torralba, A., Zhao, H., Wang, S.: Detect- ing everything in the open world: Towards universal object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11433– 11443 (2023)

    2023

  94. [102]

    In: The Eleventh International Conference on Learning Rep- resentations (2023)

    Zhu, J., Huang, H., Li, B., Wang, L.: E- crf: Embedded conditional random field for boundary-caused class weights confusion in semantic segmentation. In: The Eleventh International Conference on Learning Rep- resentations (2023)

    2023

  95. [103]

    Nature Communications, 654 (2024) 25 Appendix A Related Work Membership Inference and Defense

    Ma, J., He, Y., Li, F., Han, L., You, C., Wang, B.: Segment anything in med- ical images. Nature Communications, 654 (2024) 25 Appendix A Related Work Membership Inference and Defense. Cur- rently, membership inference methods mainly include binary-classifier-based methods [9,...

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.