pith. sign in

arxiv: 2604.12737 · v2 · submitted 2026-04-14 · 💻 cs.CR · cs.LG

Evaluating Differential Privacy Against Membership Inference in Federated Learning: Insights from the NIST Genomics Red Team Challenge

Gustavo de Carvalho Bertoli This is my paper

Pith reviewed 2026-05-10 16:18 UTC · model grok-4.3

classification 💻 cs.CR cs.LG
keywords differential privacymembership inference attacksfederated learninggenomicsstacking ensembleprivacy leakagered team challenge
0
0 comments X

The pith

Stacking attack achieves membership inference at ε=200 in differentially private federated learning where single-signal baselines fail.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper examines the effectiveness of differential privacy in defending federated learning models against membership inference attacks, using data and models from the NIST Genomics Privacy-Preserving Federated Learning Red Teaming Event. The authors develop a stacking attack that combines seven black-box estimators into a meta-classifier using prediction probabilities and loss values. In evaluations across no privacy, low privacy (ε=200), and high privacy (ε=10) settings, the stacking method outperforms existing baselines in the first two cases and continues to show leakage at ε=200, unlike the LiRA attack which drops to chance levels. These findings indicate that moderate levels of differential privacy may leave federated models vulnerable to advanced inference techniques in genomics applications.

Core claim

The central claim is that a stacking-based membership inference attack, by ensembling multiple estimators on model outputs and cross-entropy losses, maintains measurable success against differentially private federated learning models at ε=200, outperforming single-signal approaches like LiRA which collapse at that privacy level, while providing an empirical characterization of leakage degradation across DP tiers in the NIST challenge setup.

What carries the argument

The stacking meta-classifier that ensembles seven black-box estimators trained on the target model's prediction probabilities and cross-entropy losses.

If this is right

  • In the absence of differential privacy or with ε=200, stacking attacks can extract more membership information than standard methods.
  • At ε=10, higher privacy appears to reduce leakage more effectively.
  • The results highlight the need to consider ensemble attacks when setting privacy budgets in federated learning for sensitive data.
  • Independent third-party benchmarks confirm the attack's performance across the tested privacy configurations.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the stacking approach generalizes to other datasets and model architectures, then moderate differential privacy may be insufficient for protecting membership in federated learning generally.
  • Real-world deployments of DP in FL should account for the possibility of meta-classifiers combining multiple signals.
  • Further tests on non-genomics data could reveal whether the observed leakage pattern is domain-specific.

Load-bearing premise

The NIST genomics dataset and the specific DP implementations in the red team challenge accurately reflect the vulnerabilities present in real-world federated learning systems.

What would settle it

A demonstration that the stacking attack loses its advantage over LiRA at ε=200 when applied to a different dataset or under alternative DP mechanisms would falsify the claim of persistent measurable leakage.

Figures

Figures reproduced from arXiv: 2604.12737 by Gustavo de Carvalho Bertoli.

Figure 1
Figure 1. Figure 1: Each client’s model is evaluated across three privacy configurations: [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Architectural comparison across privacy tiers. Batch Normalization (Non [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Schematic effect of DP noise on member/non-member loss distributions. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Ensemble MIA inference pipeline. Solid arrows show the forward pass of [PITH_FULL_IMAGE:figures/full_fig_p010_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Attack accuracy across privacy tiers. Under High Privacy ( [PITH_FULL_IMAGE:figures/full_fig_p012_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Per-round classification accuracy over 50 FL rounds across three privacy [PITH_FULL_IMAGE:figures/full_fig_p015_6.png] view at source ↗
read the original abstract

While Federated Learning (FL) mitigates direct data exposure, the resulting trained models remain susceptible to membership inference attacks (MIAs). This paper presents an empirical evaluation of Differential Privacy (DP) as a defense mechanism against MIAs in FL, leveraging the environment of the 2025 NIST Genomics Privacy-Preserving Federated Learning (PPFL) Red Teaming Event. To improve inference accuracy, we propose a stacking attack strategy that ensembles seven black-box estimators to train a meta-classifier on prediction probabilities and cross-entropy losses. We evaluate this methodology against target models under three privacy configurations: an unprotected convolutional neural network (CNN, $\epsilon=\infty$), a low-privacy DP model ($\epsilon=200$), and a high-privacy DP model ($\epsilon=10$). The attack outperforms all baselines in the No DP and Low Privacy settings and, critically, maintains measurable membership leakage at $\epsilon=200$ where a single-signal LiRA baseline collapses. Evaluated on an independent third-party benchmark, these results provide an empirical characterisation of how stacking-based inference degrades across calibrated DP tiers in FL.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

0 major / 2 minor

Summary. This paper empirically evaluates the resilience of differential privacy (DP) in federated learning (FL) against membership inference attacks (MIAs) by leveraging the 2025 NIST Genomics Privacy-Preserving Federated Learning Red Teaming Event. The authors propose a stacking attack that trains a meta-classifier on outputs from seven black-box estimators using prediction probabilities and cross-entropy losses. They compare this attack to baselines across three DP levels: no DP (ε=∞), low privacy (ε=200), and high privacy (ε=10), finding that the stacking attack outperforms baselines in the first two settings and detects leakage at ε=200 where LiRA fails.

Significance. The results, if validated, offer important insights into how DP parameters affect MIA leakage in real FL deployments for sensitive genomics data. The paper's strength lies in its use of an independent third-party benchmark, which supports reproducibility and avoids self-referential biases in evaluation. This contributes to the understanding that moderate DP (ε=200) may not fully mitigate advanced ensemble attacks in FL.

minor comments (2)
  1. [Abstract] The abstract could include specific quantitative metrics (e.g., AUC or accuracy values) for the stacking attack versus baselines to immediately convey the magnitude of outperformance.
  2. [Methods] Clarify the selection criteria and training details for the seven black-box estimators, as well as the precise DP noise application mechanism within the FL training process, to improve reproducibility.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for their positive summary and significance assessment of our work. We appreciate the recognition that our use of the independent NIST Genomics benchmark strengthens reproducibility and that the results provide useful empirical insights into how moderate DP (ε=200) may still permit advanced ensemble attacks in FL. The recommendation for minor revision is noted.

Circularity Check

0 steps flagged

No significant circularity

full rationale

The paper is an empirical evaluation of a proposed stacking attack against membership inference in federated learning, using results from the external NIST Genomics PPFL Red Teaming Challenge benchmark. No derivation chain, equations, parameter fitting, or self-citations are present that would reduce any claim to its own inputs by construction. The central results (outperformance under No DP and ε=200) rest on external data and standard black-box estimators, remaining independent of internal self-referential logic.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Empirical paper with no new parameters or entities; the work relies on existing concepts from differential privacy and machine learning.

axioms (1)
  • domain assumption The differential privacy mechanisms and membership inference attack models used in the NIST challenge are correctly implemented and representative.
    The evaluation depends on the validity of these standard privacy and attack frameworks.

pith-pipeline@v0.9.0 · 5489 in / 1252 out tokens · 56631 ms · 2026-05-10T16:18:15.802649+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

24 extracted references · 24 canonical work pages · 1 internal anchor

  1. [1]

    Communication-Efficient Learning of Deep Net- works from Decentralized Data

    Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. “Communication-Efficient Learning of Deep Net- works from Decentralized Data”. In:Proceedings of the 20th International Conference on Artificial Intelligence and Statistics. Ed. by Aarti Singh and Jerry Zhu. Vol. 54. Proceedings of Machine Learning Research. PMLR, Differ...

    work page 2017

  2. [2]

    Feder- ated Learning in Healthcare: From Research to Real-World Deployment

    Spyridon Bakas, Xiaoxiao Li, Prashant Shah, and Holger R. Roth. “Feder- ated Learning in Healthcare: From Research to Real-World Deployment”. In:Annual Review of Biomedical Engineering(2026).issn: 1523-9829. doi:https : / / doi . org / 10 . 1146 / annurev - bioeng - 080125 - 041414. url:https://www.annualreviews.org/content/journals/10.1146/ annurev-bioen...

    work page 2026

  3. [3]

    Towards robust neurocomputing model in efficient federated brain tumour segmentation with sparsification and weights clus- tering

    Asaf Raza, Ciro Benito Raggio, Antonella Guzzo, Maria Francesca Spadea, and Giancarlo Fortino. “Towards robust neurocomputing model in efficient federated brain tumour segmentation with sparsification and weights clus- tering”. In:Neurocomputing677 (2026), p. 133142.issn: 0925-2312.doi: https://doi.org/10.1016/j.neucom.2026.133142.url:https:// www.science...

    work page doi:10.1016/j.neucom.2026.133142.url:https:// 2026

  4. [4]

    FedCrime: Zero-inflation adaptive federated learning for crime prediction

    Bhumika, Philippe Lalanda, German Vega, and Debasis Das. “FedCrime: Zero-inflation adaptive federated learning for crime prediction”. In:Neuro- computing679 (2026), p. 133217.issn: 0925-2312.doi:https://doi.org/ 10.1016/j.neucom.2026.133217.url:https://www.sciencedirect. com/science/article/pii/S0925231226006144

    work page doi:10.1016/j.neucom.2026.133217.url:https://www.sciencedirect 2026

  5. [5]

    275–283.doi:10.1109/ icdmw51313.2020.00046

    Geet Shingi. “A federated learning based approach for loan defaults pre- diction”. In:2020 International Conference on Data Mining Workshops (ICDMW). 2020, pp. 362–368.doi:10.1109/ICDMW51313.2020.00057

    work page doi:10.1109/icdmw51313.2020.00057 2020

  6. [7]

    Membership infer- ence attacks from first principles

    Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramèr. “Membership Inference Attacks From First Princi- ples”. In:2022 IEEE Symposium on Security and Privacy (SP). 2022, pp. 1897–1914.doi:10.1109/SP46214.2022.9833649

    work page doi:10.1109/sp46214.2022.9833649 2022

  7. [8]

    Differential Privacy

    Cynthia Dwork. “Differential Privacy”. In:Automata, Languages and Pro- gramming. Ed. by Michele Bugliesi, Bart Preneel, Vladimiro Sassone, and Ingo Wegener. Berlin, Heidelberg: Springer Berlin Heidelberg, 2006, pp. 1– 12.isbn: 978-3-540-35908-1

    work page 2006

  8. [9]

    Model-based development of QoS- aware reconfigurable autonomous robotic systems

    Samuel Yeom, Irene Giacomelli, Matt Fredrikson, and Somesh Jha. “Pri- vacy Risk in Machine Learning: Analyzing the Connection to Overfitting”. In:2018 IEEE 31st Computer Security Foundations Symposium (CSF). 2018, pp. 268–282.doi:10.1109/CSF.2018.00027

    work page doi:10.1109/csf.2018.00027 2018

  9. [10]

    arXiv preprint arXiv:1807.01069 , year=

    Maria-Irina Nicolae, Mathieu Sinn, Minh Ngoc Tran, Beat Buesser, Am- brish Rawat, Martin Wistuba, Valentina Zantedeschi, Nathalie Baracaldo, Bryant Chen, Heiko Ludwig, Ian M. Molloy, and Ben Edwards.Adver- sarial Robustness Toolbox v1.0.0. 2019. arXiv:1807.01069 [cs.LG].url: https://arxiv.org/abs/1807.01069. 20 G. de Carvalho Bertoli

    work page arXiv 2019

  10. [11]

    ML-leaks: Model and data independent mem- bership inference attacks and defenses on machine learning models

    Ahmed Salem, Yang Zhang, Mathias Humbert, Pascal Berrang, Mario Fritz, and Michael Backes. “ML-leaks: Model and data independent mem- bership inference attacks and defenses on machine learning models”. In: Proceedings 2019 Network and Distributed System Security Symposium. San Diego, CA: Internet Society, 2019

    work page 2019

  11. [12]

    Networks of spiking neurons: the third generation of neural network models,

    DavidH.Wolpert.“Stackedgeneralization”.In:Neural Networks5.2(1992), pp. 241–259.issn: 0893-6080.doi:https://doi.org/10.1016/S0893- 6080(05)80023- 1.url:https://www.sciencedirect.com/science/ article/pii/S0893608005800231

    work page doi:10.1016/s0893- 1992

  12. [13]

    Membership Inference Attacks Against Machine Learning Models

    Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. “Membership Inference Attacks Against Machine Learning Models”. In: 2017 IEEE Symposium on Security and Privacy (SP). 2017, pp. 3–18. doi:10.1109/SP.2017.41

    work page doi:10.1109/sp.2017.41 2017

  13. [14]

    2016, in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (ACM), 785–794, doi: 10.1145/2939672.2939785

    Tianqi Chen and Carlos Guestrin. “XGBoost: A Scalable Tree Boosting System”. In:Proceedings of the 22nd ACM SIGKDD International Confer- ence on Knowledge Discovery and Data Mining. KDD ’16. San Francisco, California, USA: Association for Computing Machinery, 2016, pp. 785– 794.isbn: 9781450342322.doi:10.1145/2939672.2939785.url:https: //doi.org/10.1145/2...

    work page doi:10.1145/2939672.2939785.url:https: 2016

  14. [15]

    Flower: A Friendly Federated Learning Research Framework

    DanielJ.Beutel,TanerTopal,AkhilMathur,XinchiQiu,JavierFernandez- Marques, Yan Gao, Lorenzo Sani, Kwing Hei Li, Titouan Parcollet, Pedro Porto Buarque de Gusmão, and Nicholas D. Lane.Flower: A Friendly Fed- erated Learning Research Framework. 2022. arXiv:2007.14390 [cs.LG]. url:https://arxiv.org/abs/2007.14390

    work page internal anchor Pith review arXiv 2022

  15. [16]

    Aharonov and A

    Milad Nasr, Reza Shokri, and Amir Houmansadr. “Comprehensive Pri- vacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning”. In:2019 IEEE Sym- posium on Security and Privacy (SP). 2019, pp. 739–753.doi:10.1109/ SP.2019.00065

    work page arXiv 2019

  16. [17]

    In: 2019 IEEE Symposium on Security and Privacy (SP), pp

    LucaMelis,CongzhengSong,EmilianoDeCristofaro,andVitalyShmatikov. “Exploiting Unintended Feature Leakage in Collaborative Learning”. In: 2019 IEEE Symposium on Security and Privacy (SP). 2019, pp. 691–706. doi:10.1109/SP.2019.00029

    work page doi:10.1109/sp.2019.00029 2019

  17. [18]

    Galen Andrew, Om Thakkar, H

    Martin Abadi, Andy Chu, Ian Goodfellow, H. Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. “Deep Learning with Differential Privacy”. In:Proceedings of the 2016 ACM SIGSAC Conference on Com- puter and Communications Security. CCS ’16. Vienna, Austria: Associa- tion for Computing Machinery, 2016, pp. 308–318.isbn: 9781450341394. doi:10 . 1145 / ...

    work page arXiv 2016

  18. [19]

    Evaluating Differentially Private Machine Learning in Practice

    Bargav Jayaraman and David Evans. “Evaluating Differentially Private Machine Learning in Practice”. In:28th USENIX Security Symposium (USENIX Security 19). Santa Clara, CA: USENIX Association, Aug. 2019, pp. 1895–1912.isbn: 978-1-939133-06-9.url:https://www.usenix.org/ conference/usenixsecurity19/presentation/jayaraman. Differential Privacy Against Member...

    work page 2019

  19. [20]

    meMIA: Multilevel Ensemble Membership Inference Attack

    Najeeb Ullah, Muhammad Naveed Aman, and Biplab Sikdar. “meMIA: Multilevel Ensemble Membership Inference Attack”. In:IEEE Transac- tions on Artificial Intelligence6.1 (2025), pp. 93–106.doi:10.1109/TAI. 2024.3445326

    work page doi:10.1109/tai 2025

  20. [21]

    Evaluating Differential Privacy in Federated Learn- ing Based on Membership Inference Attacks

    PengHe,XinyuWang,WeijiaoZhang,ZhongkaiWang,SongWang,Chuangxin Ou, and Guozheng Li. “Evaluating Differential Privacy in Federated Learn- ing Based on Membership Inference Attacks”. In:2024 10th International Conference on Big Data Computing and Communications (BigCom). 2024, pp. 196–203.doi:10.1109/BIGCOM65357.2024.00035

    work page doi:10.1109/bigcom65357.2024.00035 2024

  21. [22]

    Knapsack problems in products of groups

    Zirun Zhao, Zhaowen Lin, and Yi Sun. “ADPF: Anti-inference differen- tially private protocol for federated learning”. In:Computer Networks261 (2025), p. 111130.issn: 1389-1286.doi:https://doi.org/10.1016/j. comnet.2025.111130.url:https://www.sciencedirect.com/science/ article/pii/S1389128625000982

    work page doi:10.1016/j 2025

  22. [23]

    Enhance membership inference attacks in federated learning

    Xinlong He, Yang Xu, Sicong Zhang, Weida Xu, and Jiale Yan. “Enhance membership inference attacks in federated learning”. In:Computers & Se- curity136 (2024), p. 103535.issn: 0167-4048.doi:https://doi.org/10. 1016/j.cose.2023.103535.url:https://www.sciencedirect.com/ science/article/pii/S0167404823004455

    work page arXiv 2024

  23. [24]

    Enhancing black-box mem- bership inference attacks in federated learning

    Qiang Shi, Luzhen Ren, and Xinfeng He. “Enhancing black-box mem- bership inference attacks in federated learning”. In:Journal of Informa- tion Security and Applications96 (2026), p. 104302.issn: 2214-2126.doi: https://doi.org/10.1016/j.jisa.2025.104302.url:https://www. sciencedirect.com/science/article/pii/S2214212625003394

    work page doi:10.1016/j.jisa.2025.104302.url:https://www 2026

  24. [25]

    Chetan Pathade and Shubham Patil.Securing Genomic Data Against In- ference Attacks in Federated Learning Environments. 2025. arXiv:2505. 07188 [cs.CR].url:https://arxiv.org/abs/2505.07188

    work page arXiv 2025