REVIEW 4 major objections 5 minor 1 cited by
Metric Privacy in Federated Learning for Medical Imaging: Improving Convergence and Preventing Client Inference Attacks
T0 review · 4 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash
Pith's one-line read Metric privacy applied server-side improves convergence and matches differential privacy's protection against client inference attacks in federated medical imaging.
desk verdict Useful empirical comparison of server-side DP variants in FL, but the privacy claims are unsupported: no proof of the metric-privacy guarantee and no non-participating control for the client inference attack. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the per-round distance $d^{(n)}$, defined as the maximum over pairs of clients $i,j$ of the average Frobenius norm of the layer-wise differences between their local model weights. It is computed server-side from the received model updates without touching client data, and it is used to scale the standard deviation of the Gaussian noise applied to the aggregated model: the noise becomes $\mathcal{N}\left(0, \frac{\text{ne}\cdot C}{n_c\cdot d^{(n)}}\right)$ instead of depending on a fixed sensitivity. This adaptivity is what metric privacy requires, and what lets the method tune the noise more finely than fixed-clipping global-DP.
What would settle it
Train the identical federated setup with the target client excluded, then measure the same relative loss difference between shadow data and aggregated test set; if non-participating targets show a gap comparable to participating ones, the claimed protection against client inference attacks does not follow.
Extended reading notes
Core claim
The central claim is that metric privacy, applied from the server side, can improve the convergence and final accuracy of a federated model compared with standard global differential privacy while offering similar resistance to client inference attacks. The privacy guarantee is scaled by a distance computed each round as the maximum, over pairs of clients, of the average Frobenius norm of the differences between their local model weights. That distance is then used to scale the Gaussian noise added to the aggregated model, replacing the fixed-clipping sensitivity used by global-DP. Across homogeneous and non-i.i.d. client settings and all six tested aggregation strategies, the paper reports that metric privacy achieves higher accuracy and smoother convergence curves than global-DP, while the relative loss gap between a target client's shadow dataset and the aggregated test set remains similar for the two privacy methods.
Load-bearing premise
The client-inference-attack evaluation rests on the assumption that a larger relative difference between loss on a target client's shadow dataset and loss on the aggregated test set reveals whether that client participated, yet the experiments never train a model without the target client to serve as a non-participation control.
Editorial extensions
If this is right
- A trusted aggregator can calibrate privacy noise per round using only the received model updates, adding less noise when client models agree and more when they diverge.
- Under the reported experimental conditions, metric privacy yields higher test accuracy than global-DP for all six aggregation strategies in homogeneous and non-i.i.d. settings, while keeping convergence curves closer to the non-private baseline.
- The Client Inference Attack gives federated learning deployments a concrete, lightweight way to audit whether a semi-honest participant can infer another client's presence.
- Because the defense is server-side, it also protects the final global model if it is later published, without requiring changes in client training.
Reading between the lines
- If the attack statistic were validated with non-participating controls and ROC-style thresholds, metric privacy's protection could be quantified as a true detection rate; the current relative-loss comparison is at best a screening heuristic.
- The distance-calibration recipe generalizes: any server-side privacy mechanism whose output space has a meaningful metric could be made adaptive in the same way, not just Gaussian noise on model weights.
- Because $d^{(n)}$ is computed from the same updates the noise is meant to hide, a malicious client that can perturb its local model might distort the distance and therefore the noise level; checking that robustness would be a natural next step.
- The reported gains are likely to grow with client heterogeneity, since standard DP adds noise independent of how far client models have drifted; testing on more heterogeneous non-i.i.d. medical data would show the boundary.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a server-side metric-privacy mechanism for federated learning. Instead of the standard global-DP Gaussian mechanism with a fixed noise multiplier, the server computes, in each round, the maximum pairwise Frobenius distance d(n) among the received local model updates and scales the Gaussian noise standard deviation by 1/d(n). The approach is evaluated on an Alzheimer MRI dataset with six aggregation strategies (FedAvg, FedAvgM, FedMedian, FedProx, FedOpt, FedYogi) under homogeneous and non-i.i.d. client splits, and the authors introduce a 'Client Inference Attack' (CIA) in which a semi-honest client tries to determine whether another client participated in training. The reported experiments show better accuracy and convergence for metric-privacy than for global-DP, with similar or better CIA protection measured by the relative difference between loss on a target shadow dataset and loss on an aggregated test set.
Significance. If the central claim were established, the paper would be a useful practical contribution: it extends the small literature on server-side DP in federated learning to six aggregation strategies and proposes an attack scenario, CIA, that is relevant to cross-silo medical collaborations. The experimental setup is described in detail, the data are openly available, and the reproducibility appendix lists library versions and the modified Flower file. The systematic comparison across aggregation strategies and the explicit trust model are strengths. However, the significance is heavily limited by two load-bearing problems: the metric-privacy guarantee is asserted rather than proved and, as stated, is not compatible with the formal definition, and the CIA evaluation lacks the control condition needed to support the 'similar protection' conclusion.
major comments (4)
- [Section 6, Definition 7] The claim that the proposed mechanism satisfies metric-privacy in each round is unsupported and, as stated, incompatible with Definition 7. In Definition 7, d must be a fixed metric on the input domain and the mechanism must satisfy a likelihood-ratio bound uniformly over all pairs of inputs at a given distance. Here d(n) is a statistic of the current multiset of client updates, so the Gaussian noise variance depends on the very input being privatized. Concretely, consider one round with clipped updates: let W consist of four nearly identical updates, so d(W)=delta, and let W' consist of two updates at +C e1 and two at -C e1, so d(W')=2C while the FedAvg aggregate is the same. The output distributions have variances K/delta and K/(2C) respectively. As delta approaches 0, the likelihood ratio P(M(W)=y)/P(M(W')=y) is unbounded in the tail, even though the distance between the two inputs under any finite metric is finite. Thus the inequality in Definition 7 is violated. A correct mechanism requires a sensitivity bound or Lipschitz constant that is independent of the observed updates, followed by a composition argument across rounds; none is provided.
- [Section 6, noise formula] The text immediately after the noise formula states that 'if d(n) > 1, we are adding more noise with metric-privacy', but the formula for the standard deviation is ne*C/(nc*d(n)). For d(n)>1 this reduces the noise, not increases it. The comparison between global-DP and metric-privacy in Tables 4-7 depends directly on the actual noise level, so this contradiction must be resolved and the experimental numbers reconciled with the corrected interpretation.
- [Section 7.4, Tables 9-10 and D.13-D.16] The CIA evaluation cannot support the conclusion of 'similar protection' because all experiments include the target client in the training set. The statistic used is the relative difference between the loss on the target client's shadow dataset and the loss on the aggregated test set. Without a control condition in which the target client does not participate, this difference may reflect only the distributional mismatch between the target dataset and the aggregate data, not the fact of participation. The paper should report the same relative-loss statistic for a model trained without client 3, and ideally provide a decision rule or ROC analysis. As reported, the numbers only show that the noisy model has a different loss profile; they do not quantify CIA success.
- [Appendix F, final paragraph] The noise multiplier ne=0.01 was selected because it 'allows us to protect against the client inference attacks that we are trying to prevent'. This is circular: the same CIA experiments used to conclude protection are used to calibrate the privacy parameter. Moreover, the distance d(n) used to scale the metric-privacy noise is computed from the model updates being protected, making the privacy parameter data-dependent. The paper should provide an independent calibration rule, for example by fixing epsilon and delta and deriving the noise scale from a sensitivity bound, and then evaluate CIA under that rule.
minor comments (5)
- [Table 10] The caption of Table 10 identifies the strategy as FedYogi, while the text in Section 7.4 says the results are for FedOpt; please correct this inconsistency. Also, 'V anilla' appears in Tables 9-10 and D.13-D.16 due to a formatting artifact.
- [Section 6, d(n) formula] The formulas for d(n) and for the initial clipping-norm estimate contain a stray 'i,j' subscript under the max symbol; the intended domains of i and j should be written cleanly.
- [Section 3, FedAvgM] The displayed formula for Delta w^(r+1) is garbled: the denominator and the summation are not typeset correctly, and the expression ni(w^(r)-w_i^(r+1)) is missing the intended weighting or parentheses. Please rewrite the equation.
- [Appendix F] The paper describes the Flower code modification in words but provides no pseudocode or algorithm listing for the metric-privacy mechanism. An algorithm box with inputs (clipping norm, noise multiplier, d(n)) and outputs (perturbed aggregate) would make the method reproducible and easier to check.
- [Section 5, Definition 8] Definition 8 states the CIA as a posterior probability P(sx=1|Dx,W), but the experiments never compute this probability; they use a relative loss difference. The paper should either derive why this statistic estimates the posterior or redefine the attack in terms of the measured quantity.
Circularity Check
Metric-privacy guarantee is vacuous because d(n) is a data-dependent dispersion statistic used as the privacy metric, and the CIA protection result is a tuned outcome rather than a prediction.
-
self definitional
[Section 6, Definition 7 vs. the d(n) noise formula (pp. 6-7)]
"we propose a metric that depends on the distance between the model updates rather than the actual clients’ data... d(n) = max_{i,j in {1,...,nc}} (1/|L|) sum_{l in L} ||w_i^(n)(l) - w_j^(n)(l)||_F ... we have calculated the metric d(n) dynamically in each round n and use it by dividing the standard deviation of the Gaussian noise as follows: N(0, ne*C/(nc*d(n))) ... ensuring that metric-privacy in fulfilled in each FL round."
Definition 7 requires a fixed metric d on the input domain and a mechanism whose likelihood ratio is bounded by exp(eps*d(x,x')) for every pair of inputs. Here d(n) is a statistic of the current local updates, i.e., the very data being privatized, so the noise variance changes with the database. No metric between two possible input databases is ever defined, and no sensitivity bound or composition argument is supplied. Concretely, two input sets with the same FedAvg aggregate but different internal spreads (e.g., four near-identical updates vs. two at +C e1 and two at -C e1) get different noise variances; the ratio of the two output Gaussians is unbounded in the tail while the input distance under any finite metric is finite.
-
fitted input called prediction
[Appendix F (and reported in Section 7.4)]
"Usually a value greater than 1 is recommended for the noise multiplier, but in our use case we analyzed different values in order to select one that allow the model to converge. In particular, with the value 0.01 we achieved convergence and at the same time it allows us to protect against the client inference attacks that we are trying to prevent (as shown in Section 7.4)."
The noise multiplier is selected after inspecting convergence and the CIA protection outcome, and the same Section 7.4 results are then presented as evidence that metric-privacy offers 'similar protection against client inference attacks'. The CIA conclusion is therefore a fitted outcome, not a prediction of the metric-privacy mechanism. Since both global-DP and metric-privacy use the same tuned 0.01, the reported similarity in protection is partly forced by the tuning criterion rather than derived from the privacy definitions or from an independent attack experiment.
full rationale
The central privacy derivation in Section 6 is circular: d(n) is computed from the model updates being protected and then used as the metric that defines the privacy claim, so 'metric-privacy is fulfilled in each FL round' is true only by construction, not by Definition 7. This is the main score driver. A second, lesser circularity is the noise multiplier selection: 0.01 is chosen because it yields the desired CIA protection, and that same protection is then reported as a finding in Section 7.4. The accuracy comparisons (Tables 4-7) are a separate, self-contained empirical result and are not circular, which keeps the score below 8. I also note, without counting it as circularity, that the CIA evaluation in Section 7.4 lacks a non-participating control and uses a relative-loss-difference heuristic rather than the posterior probability in Definition 8; that is a serious correctness risk but not a reduction-by-construction. Overall: partial circularity in the central privacy claim and in the CIA conclusion, with independent utility evidence.
Assumptions & free parameters
free parameters (3)
- Clipping norm C =
5
- Noise multiplier ne =
0.01
- Distance d(n) =
data-dependent (max pairwise per-layer Frobenius distance)
assumptions (3)
- ad hoc to paper The Gaussian mechanism with standard deviation ne*C/(nc*d(n)) satisfies epsilon-metric-privacy for a fixed epsilon in each round.
- domain assumption Frobenius distance between local model weights is a meaningful metric for metric-privacy between clients.
- ad hoc to paper Relative loss difference between target shadow dataset and aggregated test set measures client inference attack success.
Cite this review
Pith. "Pith review of Metric Privacy in Federated Learning for Medical Imaging: Improving Convergence and Preventing Client Inference Attacks." pith.science (2026). https://pith.science/paper/4D35SJU6
@misc{pith2026250201352,
author = {Pith},
title = {Pith review of: Metric Privacy in Federated Learning for Medical Imaging: Improving Convergence and Preventing Client Inference Attacks},
year = {2026},
howpublished = {\url{https://pith.science/paper/4D35SJU6}},
note = {Machine review of arXiv:2502.01352}
}
read the original abstract
Federated learning is a distributed learning technique that allows training a global model with the participation of different data owners without the need to share raw data. This architecture is orchestrated by a central server that aggregates the local models from the clients. This server may be trusted, but not all nodes in the network. Then, differential privacy (DP) can be used to privatize the global model by adding noise. However, this may affect convergence across the rounds of the federated architecture, depending also on the aggregation strategy employed. In this work, we aim to introduce the notion of metric-privacy to mitigate the impact of classical server side global-DP on the convergence of the aggregated model. Metric-privacy is a relaxation of DP, suitable for domains provided with a notion of distance. We apply it from the server side by computing a distance for the difference between the local models. We compare our approach with standard DP by analyzing the impact on six classical aggregation strategies. The proposed methodology is applied to an example of medical imaging and different scenarios are simulated across homogeneous and non-i.i.d clients. Finally, we introduce a novel client inference attack, where a semi-honest client tries to find whether another client participated in the training and study how it can be mitigated using DP and metric-privacy. Our evaluation shows that metric-privacy can increase the performance of the model compared to standard DP, while offering similar protection against client inference attacks.
Figures
Forward citations
Cited by 1 Pith paper
-
AI4EOSC: a Federated Cloud Platform for Artificial Intelligence in Scientific Research
AI4EOSC is a federated cloud platform that integrates modular AI development, serverless AI-as-a-Service, and distributed orchestration with built-in FAIR metadata and provenance tracking for scientific AI workloads in EOSC.
Reference graph
Works this paper leans on
-
[29]
F. Galli, S. Biswas, K. Jung, T. Cucinotta, C. Palamidessi, Group privacy for personalized federated learning, in: Proceedings of the 9th Interna- tional Conference on Information Systems Security and Privacy-ICISSP, SCITEPRESS-Science and Technology Publications, 2023, pp. 252–263
work page 2023
-
[1]
L. Li, Y . Fan, M. Tse, K.-Y . Lin, A review of applications in federated learning, Computers & Industrial Engineering 149 (2020) 106854
work page 2020
-
[2]
McMahan, E
B. McMahan, E. Moore, D. Ramage, S. Hampson, B. A. y Arcas, Communication-efficient learning of deep networks from decentralized data, in: Artificial intelligence and statistics, PMLR, 2017, pp. 1273– 1282
2017
-
[3]
R. Shokri, M. Stronati, C. Song, V . Shmatikov, Membership Inference Attacks Against Machine Learning Models , in: 2017 IEEE Symposium on Security and Privacy (SP), IEEE Computer Society, Los Alamitos, CA, USA, 2017, pp. 3–18. doi:10.1109/SP.2017.41. URL https://doi.ieeecomputersociety.org/10.1109/SP. 2017.41
-
[4]
H. Hu, Z. Salcic, L. Sun, G. Dobbie, X. Zhang, Source inference attacks in federated learning, in: ICDM, IEEE, 2021
work page 2021
-
[5]
L. Zhu, Z. Liu, S. Han, Deep leakage from gradients, Advances in Neural Information Processing Systems 32 (2019)
work page 2019
-
[6]
C. Chen, L. Lyu, H. Yu, G. Chen, Practical attribute reconstruction attack against federated learning, IEEE Transactions on Big Data (2022)
work page 2022
- [7]
Show all 33 references
-
[8]
Chatzikokolakis, M
K. Chatzikokolakis, M. E. Andr ´es, N. E. Bordenabe, C. Palamidessi, Broadening the scope of di fferential privacy using metrics, in: E. De Cristofaro, M. Wright (Eds.), Privacy Enhancing Technologies, Springer Berlin Heidelberg, Berlin, Heidelberg, 2013, pp. 82–102
2013
-
[9]
M. S. Alvim, K. Chatzikokolakis, C. Palamidessi, A. Pazii, Metric-based local differential privacy for statistical applications, in: 31st Computer Security Foundations Symposium (CSF 2018), IEEE Computer Society, 2018, pp. 262–267
2018
-
[10]
Pan, Y .-S
K. Pan, Y .-S. Ong, M. Gong, H. Li, A. K. Qin, Y . Gao, Differential privacy in deep learning: A literature survey, Neurocomputing (2024) 127663
2024
-
[11]
Abadi, A
M. Abadi, A. Chu, I. Goodfellow, H. B. McMahan, I. Mironov, K. Tal- war, L. Zhang, Deep learning with di fferential privacy, in: Proceedings of the 2016 ACM SIGSAC conference on computer and communications security, 2016, pp. 308–318
2016
-
[12]
T. Ha, T. K. Dang, T. T. Dang, T. A. Truong, M. T. Nguyen, Di fferential privacy in deep learning: an overview, in: 2019 International Conference on Advanced Computing and Applications (ACOMP), IEEE, 2019, pp. 97–102
2019
-
[13]
Y . Xie, P. Li, C. Wu, Q. Wu, Di fferential privacy stochastic gradient descent with adaptive privacy budget allocation, in: 2021 IEEE inter- national conference on consumer electronics and computer engineering (ICCECE), IEEE, 2021, pp. 227–231
2021
-
[14]
Abadi, A
M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. Goodfellow, A. Harp, G. Irving, M. Isard, Y . Jia, R. Jozefowicz, L. Kaiser, M. Kud- lur, J. Levenberg, D. Man ´e, R. Monga, S. Moore, D. Murray, C. Olah,...
2015
-
[15]
Opacus PyTorch library, Available from opacus.ai
-
[16]
M. Nasr, R. Shokri, A. Houmansadr, Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against cen- tralized and federated learning, in: 2019 IEEE symposium on security and privacy (SP), IEEE, 2019, pp. 739–753
2019
-
[17]
Carlini, S
N. Carlini, S. Chien, M. Nasr, S. Song, A. Terzis, F. Tramer, Member- ship inference attacks from first principles, in: 2022 IEEE Symposium on Security and Privacy (SP), IEEE, 2022, pp. 1897–1914
2022
-
[18]
H. Hu, Z. Salcic, L. Sun, G. Dobbie, P. S. Yu, X. Zhang, Membership inference attacks on machine learning: A survey, ACM Comput. Surv. 54 (11s) (Sep. 2022). doi:10.1145/3523273. URL https://doi.org/10.1145/3523273
2022 doi
-
[19]
Pustozerova, J
A. Pustozerova, J. Baumbach, R. Mayer, Di fferentially private federated learning: Privacy and utility analysis of output perturbation and dp-sgd, in: 2023 IEEE International Conference on Big Data (BigData), 2023, pp. 5549–5558. doi:10.1109/BigData59044.2023.10386466
2023
-
[20]
Adnan, S
M. Adnan, S. Kalra, J. C. Cresswell, G. W. Taylor, H. R. Tizhoosh, Fed- erated learning and di fferential privacy for medical image analysis, Sci- entific reports 12 (1) (2022) 1953
2022
-
[21]
R. Hu, Y . Guo, H. Li, Q. Pei, Y . Gong, Personalized federated learning with differential privacy, IEEE Internet of Things Journal 7 (10) (2020) 9530–9539. doi:10.1109/JIOT.2020.2991416
2020
-
[22]
Naseri, J
M. Naseri, J. Hayes, E. De Cristofaro, Local and central di fferential privacy for robustness and privacy in federated learning, arXiv preprint arXiv:2009.03561 (2020)
2020 arXiv
-
[23]
T.-M. H. Hsu, H. Qi, M. Brown, Measuring the e ffects of non-identical data distribution for federated visual classification, arXiv preprint arXiv:1909.06335 (2019)
2019 arXiv
-
[24]
D. J. Beutel, T. Topal, A. Mathur, X. Qiu, J. Fernandez-Marques, Y . Gao, L. Sani, H. L. Kwing, T. Parcollet, P. P. d. Gusm ˜ao, N. D. Lane, Flower: A friendly federated learning research framework, arXiv preprint arXiv:2007.14390 (2020)
2020 arXiv
-
[25]
D. Yin, Y . Chen, R. Kannan, P. Bartlett, Byzantine-robust distributed learning: Towards optimal statistical rates, in: International conference on machine learning, Pmlr, 2018, pp. 5650–5659
2018
-
[26]
T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, V . Smith, Fed- erated optimization in heterogeneous networks, Proceedings of Machine learning and systems 2 (2020) 429–450
2020
-
[27]
Reddi, Z
S. Reddi, Z. Charles, M. Zaheer, Z. Garrett, K. Rush, J. Kone ˇcn`y, S. Ku- mar, H. B. McMahan, Adaptive federated optimization, arXiv preprint arXiv:2003.00295 (2020)
2020 arXiv
-
[28]
M. S. Alvim, N. Fernandes, A. McIver, C. Morgan, G. H. Nunes, A novel analysis of utility in privacy pipelines, using kronecker products and quantitative information flow, in: Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security, CCS ’23, Assoc...
2023
-
[30]
URL https://huggingface.co/datasets/Falah/Alzheimer_ MRI
Falah.G.Salieh, Alzheimer mri dataset (2023). URL https://huggingface.co/datasets/Falah/Alzheimer_ MRI
2023
-
[31]
Hatami, F
M. Hatami, F. Yaghmaee, R. Ebrahimpour, Investigating the potential of reinforcement learning and deep learning in improving alzheimer’s dis- ease classification, Neurocomputing 597 (2024) 128119. doi:https: //doi.org/10.1016/j.neucom.2024.128119. URL https://www.sciencedirect...
2024
-
[32]
R. C. Geyer, T. Klein, M. Nabi, Di fferentially private federated learning: A client level perspective, arXiv preprint arXiv:1712.07557 (2017)
2017 arXiv
-
[33]
H. B. McMahan, D. Ramage, K. Talwar, L. Zhang, Learning di fferen- tially private recurrent language models, arXiv preprint arXiv:1710.06963 (2017). 12 Appendix A. Analysis of the performance of the trained model in di fferent runs In this section we show the mean accuracy obt...
2017 arXiv
Reviewed August 9, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.