Pith. sign in

REVIEW 5 major objections 6 minor 109 references

Decoding FL Defenses: Systemization, Pitfalls, and Remedies

T0 review · 5 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Robustness evaluations of federated learning defenses are skewed by six common experimental pitfalls, and this paper re-evaluates three defenses to show how the pitfalls flip robustness conclusions.

desk verdict Useful evaluation-guidelines paper with a solid qualitative core, but the quantitative impact analysis needs controlled comparisons and error bars before the numbers are cited. read the letter →

arxiv 2502.05211 v1 pith:3FVMI7YU submitted 2025-02-03 cs.CR cs.AI

Momin Ahmad Khan , Virat Shejwalkar , Yasra Chandio , Amir Houmansadr , Fatima Muhammad Anwar This is my paper · ORCID
classification cs.CRcs.AI
keywords federatedlearningpoisoningattacksrobustaggregationdefenseevaluationpitfallsByzantinerobustnesscross-deviceadaptive
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Federated learning defenses are evaluated without shared guidelines, and this paper argues that the resulting evaluations often create a false sense of security. Reviewing 50 defense papers, it identifies six pitfalls in how experiments are set up: intrinsically robust datasets, homogeneous data distributions, slow-converging algorithms, limited cross-silo settings, naive attacks, and global-only metrics. Using three representative defenses, Trimmed Mean, FLDetector, and FedRecover, the paper re-runs evaluations under corrected settings and shows that robustness conclusions change, for example TrMean looks robust on MNIST but loses up to 50 percent accuracy on CIFAR10. The authors' claim is that following their six recommendations would make future robustness claims more trustworthy.

What carries the argument

The central machinery is a three-dimensional systemization of federated learning defenses, an organizing scheme mapping defenses along how client updates are processed (filtering, re-weighting, modification), what the server knows about data and updates, and at which phase the defense acts (pre-aggregation, post-aggregation, local training, or post-training). The systemization is used to select three representative defenses that occupy distinct cells of the scheme, and the evaluation pipeline is then varied one component at a time to isolate each pitfall. The quantitative engine is the re-evaluation itself: the authors rerun TrMean, FLDetector, and FedRecover across datasets, distributions, algorithms, scales, attacks, and metrics, so each pitfall can be tied to a measurable change in accuracy or detection error.

What would settle it

A reader could run the TrMean comparison again holding every setting fixed except the dataset: same FedAvg algorithm, same learning rate schedule, same number of clients, same attack, and same malicious fraction. If the accuracy gap between MNIST and the harder datasets shrinks or reverses, the claim that dataset simplicity causes the apparent robustness is weakened. Likewise, re-running the survey's prevalence statistics on a different sample of 50 defense papers would test whether the six pitfalls are as widespread as reported.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that six pitfalls in experimental setup, not the defenses themselves, frequently produce the robustness a defense appears to have. The authors demonstrate this with three case studies: TrMean is only robust with MNIST, not with FashionMNIST, FEMNIST, or CIFAR10; FLDetector detects attacks perfectly on MNIST and FashionMNIST but fails completely on FEMNIST; and FedRecover recovers nearly perfectly on simple datasets but leaves 11 to 19 percent accuracy gaps on FEMNIST and CIFAR10. The paper also shows that the plain mean aggregator, usually dismissed as non-robust, is unaffected by the Stat-Opt attack on the large-scale StackOverflow setting when fewer than 2 percent of clients are malicious, and that per-client and per-class accuracy reveal variation that global accuracy hides. The conclusion the authors draw is that these pitfalls lead to incorrect conclusions about robustness and that their recommendations constitute a necessary evaluation standard.

Load-bearing premise

The impact analysis assumes that when the authors compare settings, the named pitfall is the only meaningful difference; in several comparisons they also change the learning algorithm, learning rate, epoch count, or number of clients across datasets, so those confounds, rather than the pitfall itself, could drive the measured accuracy gaps.

Editorial extensions

If this is right

  • A defense shown robust only on MNIST should not be treated as robust; the paper's re-evaluation shows the dataset alone can account for the result.
  • Evaluations that use FedSGD instead of a fast-converging algorithm can make a defense look weaker or stronger than it is, because slow convergence gives the attacker more rounds to act.
  • History-based defenses like FLDetector and FedRecover are not compatible with cross-device FL, where clients appear in few rounds, so claims about them should be scoped to cross-silo settings.
  • Reporting global accuracy is misleading under heterogeneity and class imbalance; per-client and per-class accuracy should accompany global numbers.
  • Strong adaptive attacks, including the paper's perturbation-vector attack against FLDetector, can produce non-zero false positive and false negative rates that break downstream recovery.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the six pitfalls generalize as the paper suggests, many published robustness orderings among defenses become provisional until re-run under the recommended settings, which would affect comparison tables and meta-analyses built on those papers.
  • The finding that the mean aggregator is robust at very low malicious-client fractions on StackOverflow suggests that in real cross-device deployments, simple aggregation may be a stronger baseline than laboratory cross-silo experiments indicate.
  • The adaptive attack on FLDetector, which adds an averaged perturbation sized by the gap between actual and predicted updates, could transfer to other detectors that flag clients by deviation from an estimated update; testing that transfer is a direct extension.
  • The same pitfall lens likely applies to attack evaluations, since attacks tested only in cross-silo settings with high malicious fractions may also be over- or under-stated, because attacks and defenses sit in the same evaluation pipeline.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper argues that the federated-learning defense literature lacks evaluation guidelines, leading to a false sense of security. It contributes (i) a systemization of FL defenses along three dimensions (processing of client updates, server knowledge, defense phase), (ii) a survey of 50 top-tier defense papers that identifies six pitfalls (intrinsically robust datasets, homogeneous distributions, slow-converging algorithms, limited FL settings, naive attacks, unfair metrics) and their prevalence, and (iii) a case-study impact analysis of three defenses (TrMean, FLDetector, FedRecover) aimed at showing how these pitfalls change robustness conclusions. The central qualitative claim is that easy experimental setups—MNIST, i.i.d. data, FedSGD, simple attacks, global-only metrics—can make weak defenses look robust.

Significance. If the impact analysis were rigorously controlled, the paper would be a valuable corrective: the prevalence statistics (30% MNIST-only evaluations, 40% naive attacks, 50% i.i.d. distributions) are concrete, falsifiable observations, and the six recommendations would give practitioners a usable evaluation checklist. The systemization of defenses is a useful organizing contribution in its own right, and the paper is honest about its own scope in Section 8. The main risk is that the quantitative case studies, which are the load-bearing evidence for the 'pitfalls lead to incorrect conclusions' claim, currently mix multiple uncontrolled factors, so the specific attribution of robustness differences to individual pitfalls is not established. Because the qualitative conclusion is well-motivated and the quantitative gaps are fixable with additional controlled experiments, the paper is suitable for major revision rather than rejection.

major comments (5)
  1. [§6.1.1, Fig. 4] The claim that 'TrMean is only robust with MNIST, but not with other datasets' is not supported by the reported experiments because the datasets are compared under different algorithms, training budgets, and hyperparameters. MNIST and FashionMNIST use FedSGD over 2000 rounds, while CIFAR10 and FEMNIST use FedAvg over 100 and 200 rounds respectively (Sections 5.1–5.4); the MNIST learning rate is 0.03, while FashionMNIST uses 3e-3. Since the paper itself argues in Section 6.3 that FedSGD's slow convergence creates a larger attack window, the larger accuracy drops on CIFAR10/FEMNIST could be driven by the optimizer or the number of rounds rather than by dataset complexity. The authors should hold the FL algorithm, round budget, and client count fixed across datasets, or explicitly disentangle the factors, and should report multiple seeds or error bars before drawing the flagship Pitfall-1 conclusion.
  2. [§6.1, Fig. 4, Fig. 9, Table 2] The robustness evaluation never reports a no-defense (e.g., mean AGR) baseline for the cross-dataset and distribution experiments. Without the unprotected baseline accuracy under the same attack, one cannot tell whether the observed accuracy reflects the defense's contribution or simply the weakness of the attack on that dataset. This is especially important given the paper's own Section 6.4.1 result that the mean AGR is robust on StackOverflow at scale; that result shows 'robustness' without any defense is possible, so the defense evaluations in Figures 4 and 9 need the same baseline to support claims that TrMean is or is not providing protection.
  3. [§5.4, §6.1.2] The footnote in Section 5.4 states that the authors could not achieve the accuracy reported in [20] with the published FashionMNIST learning rate (3e-4) and therefore changed it to 3e-3. This undermines the statement in Section 6.1.2 that 'consistent results with [20] on MNIST and FashionMNIST validate our implementation,' and it exemplifies the very hyperparameter drift the paper warns about. Please quantify the effect of this learning-rate change on the reported accuracy and recovery numbers, or explicitly note that the FashionMNIST result is not a reproduction of [20].
  4. [Table 3, §6.5.2] The adaptive-attack evaluation of FLDetector is missing operational details that are necessary to interpret the FPR/FNR numbers. The paper does not state how FPR and FNR are aggregated over rounds, whether the full FLDetector pipeline (Gap statistics plus K-means clustering) is used, how many random seeds are averaged, or what the 'Baseline' column in Table 3 contains (it appears empty). Since the attack is the authors' own new construction, the paper should also compare it against a standard strong attack under the same budget (e.g., Stat-Opt or Dyn-Opt) so that the reader can separate the attack's strength from FLDetector's intrinsic performance limits.
  5. [§6.4.1, Fig. 10] The large-scale StackOverflow experiment uses FedAdam, whereas the rest of the paper compares FedSGD and FedAvg, and no error bars or repeated runs are reported. The conclusion that 'Mean AGR is robust on a large scale' and the quantitative contrast with [84] therefore conflates dataset, optimizer, and scale effects. The authors should either run the same attack under FedAvg/FedSGD on a comparable task or report confidence intervals over multiple runs, so that the scale claim is not attributable to the optimizer choice alone.
minor comments (6)
  1. [§4, Pitfall-4] There is a duplicated word in the sentence 'the attack showsshows no effect on the non-robust mean AGR' (Section 4, Pitfall-4, Prevalence and implications); please fix the typo.
  2. [§2.1, §3.2.1] The text contains several spacing and punctuation errors, e.g., 'FedA vg' instead of 'FedAvg' in Section 2.1, and the incomplete sentence 'Here, ∇𝑘𝑡−1. The server retains...' in the FLDetector description of Section 3.2.1; these should be corrected.
  3. [Fig. 4] The caption of Figure 4 says 'under trim attack,' but the text in Section 6.1.1 describes the attack as Stat-Opt; please make the terminology consistent.
  4. [§5.1, §5.3, §6.6.1] The units for training duration are inconsistent: Section 5.1 says '200 epochs,' Section 5.3 says '2000 epochs,' and Section 6.6.1 says '200 rounds' for FEMNIST. Please clarify whether these are FL rounds or local epochs and make the usage uniform.
  5. [§4, Table 4] The survey's inclusion criteria for the '50 top-tier defense papers' are not stated. Please provide the selection criteria (e.g., venues, time window, search process) so that the prevalence statistics in Figure 3 can be reproduced.
  6. [§7, Related Work] The paper's relationship to the authors' prior work [43] is described only as 'taking inspiration,' but the pitfall list and the FedRecover case study appear to overlap substantially with that workshop paper. Please state explicitly what is new in the pitfall analysis beyond [43] and cite the overlap more precisely.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the pitfalls and case-study results rest on measured outcomes over external datasets and published attacks; self-citations are attribution, not load-bearing evidence.

full rationale

The paper's central claims are empirical measurements rather than definitions in disguise. The claim that TrMean is only robust on MNIST (Section 6.1.1) is a comparison of measured accuracy drops under Stat-Opt across four external datasets; it is not obtained by plugging TrMean's definition into a robustness formula. The FedRecover and FLDetector case studies likewise report measured accuracies and FPR/FNR values, with baselines checked against the original defense papers (refs. [20] and [106]). The survey prevalence numbers (30% MNIST, 40% naive attacks) are counts over 50 cited papers, not derived from the authors' own prior work. The two self-citations do not carry the argument: [43] is explicitly credited as 'inspiration' and as a starting point that the paper extends, and Dyn-Opt from [83] is used as an externally published adaptive attack benchmark rather than as a theorem that forces a conclusion. The one internally constructed artifact, the adaptive attack against FLDetector (Section 6.5.2), is transparently engineered using FLDetector's own estimator so that malicious updates are 'close enough' to estimated updates; reporting that this attack evades detection is an adaptive-attack stress test, not a fitted parameter renamed as a prediction. Even if the across-dataset comparisons are confounded by different algorithms, learning rates, and round counts, that is a threat to internal validity, not circularity: no equation in the paper reduces to its own input and no benchmark value is fit then reported as a prediction. Accordingly, no circular step is exhibited.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim is empirical rather than derivational, so the ledger records hand-chosen experimental settings and domain assumptions rather than fitted mathematical constants. The main things the reader 'pays for' are the representativeness of the three case-study defenses, the validity of accuracy as a robustness measure, and the assumption that the compared settings differ only in the named pitfall.

free parameters (2)
  • Per-dataset learning rates and epoch counts = See Section 5: MNIST lr 0.03, FashionMNIST lr 3e-3, CIFAR10 lr 0.01, FEMNIST lr 0.05; epochs 2000, 2000, 100, 200
    The authors changed these values by hand, and even deviated from the original FedRecover learning rate (Section 5.4, footnote), so cross-dataset comparisons are not fully controlled.
  • FedRecover warmup and correction intervals (Tw, Tc) = Tw in {10, 20}, Tc in {5, 10}
    Chosen per dataset to make recovery work; the reported recovery accuracy depends on these values.
assumptions (4)
  • domain assumption Global test accuracy is a sufficient measure of robustness for poisoning defenses.
    All impact analysis in Section 6 uses accuracy as the primary outcome; per-client and per-class metrics are only added in Section 6.6, and no other security metric is considered.
  • domain assumption TrMean, FLDetector, and FedRecover are representative of the FL defense space.
    Section 3.2.2 justifies distinctness along the three systemization dimensions, but representativeness is asserted, not statistically established; the authors concede in Section 8 that other defenses could give different results.
  • domain assumption The 50-paper survey is a fair and accurate sample of top-tier FL defense evaluations.
    Table 4 lists the classification, but no sampling methodology, inter-rater reliability, or inclusion criteria are given, so prevalence percentages (30%, 40%, 50%) are approximate.
  • ad hoc to paper Observed performance differences across settings are caused by the named pitfall rather than by uncontrolled differences.
    Sections 5 and 6.1 vary algorithms, learning rates, epoch counts, and client counts together with the pitfall being studied, so the causal attribution is not fully controlled.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Decoding FL Defenses: Systemization, Pitfalls, and Remedies." pith.science (2026). https://pith.science/paper/3FVMI7YU

@misc{pith2026250205211,
  author       = {Pith},
  title        = {Pith review of: Decoding FL Defenses: Systemization, Pitfalls, and Remedies},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3FVMI7YU}},
  note         = {Machine review of arXiv:2502.05211}
}
read the original abstract

While the community has designed various defenses to counter the threat of poisoning attacks in Federated Learning (FL), there are no guidelines for evaluating these defenses. These defenses are prone to subtle pitfalls in their experimental setups that lead to a false sense of security, rendering them unsuitable for practical deployment. In this paper, we systematically understand, identify, and provide a better approach to address these challenges. First, we design a comprehensive systemization of FL defenses along three dimensions: i) how client updates are processed, ii) what the server knows, and iii) at what stage the defense is applied. Next, we thoroughly survey 50 top-tier defense papers and identify the commonly used components in their evaluation setups. Based on this survey, we uncover six distinct pitfalls and study their prevalence. For example, we discover that around 30% of these works solely use the intrinsically robust MNIST dataset, and 40% employ simplistic attacks, which may inadvertently portray their defense as robust. Using three representative defenses as case studies, we perform a critical reevaluation to study the impact of the identified pitfalls and show how they lead to incorrect conclusions about robustness. We provide actionable recommendations to help researchers overcome each pitfall.

Figures

Figures reproduced from arXiv: 2502.05211 by the authors.

Figure 1
Figure 1. FL defense evaluation pipeline. We display common choices for each stage in the pipeline, e.g., FedSGD or FedAvg as the FL [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Systemizing FL Defenses: Categorization of defenses based on their [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Frequency of choices of the six key components of robustness evaluation setup: dataset, distribution of clients’ data, FL [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: Comparative analysis of TrMean AGR with FedSGD and FedAvg under trim attack. TrMean is more susceptible to poisoning [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 5
Figure 5. Figure 5: FEMNIST histogram from leaf.cmu.edu letters. We use 300 randomly selected clients with their original data in a cross-silo fashion, as FedRecover uses the cross-silo setting in its implementation. We use the CNN used by [20] and use the Xavier weight initialization. Hy…
Figure 6
Figure 6. Figure 6: Performance of FedRecover on four datasets, both with and without trim attack in the original training. [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: Comparison of Sample and Class Distribution in FL Datasets: Histograms illustrating (a) the number of samples and (b) the [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: T-SNE projections of class frequency vectors of client datasets generated using FCJ (b) and Dir (c) distributions. From left to [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]
Figure 9
Figure 9. Figure 9: The effect of varying heterogeneity levels for FCJ and Dir distributions on the FashionMNIST dataset. [PITH_FULL_IMAGE:figures/full_fig_p019_9.png]
Figure 10
Figure 10. Figure 10: Stackoverflow in practical settings. Under both benign and adversarial conditions, FedAvg greatly surpasses FedSGD in performance, convergence, and communication for all datasets, as shown in [PITH_FULL_IMAGE:figures/full_fig_p020_10.png]
Figure 11
Figure 11. Figure 11: Communication-accuracy tradeoff for FedRecover. Despite the presence of four lines, their overlap is discernible as we can [PITH_FULL_IMAGE:figures/full_fig_p022_11.png]
Figure 12
Figure 12. Figure 12: Performance of FedRecover, represented by the difference between no-attack and post-recovery accuracies, under non-zero [PITH_FULL_IMAGE:figures/full_fig_p022_12.png]
Figure 13
Figure 13. Figure 13: Personalized evaluations, i.e., per-client accuracy for FEMNIST with TrMean and FedRecover. Note that the accuracy does [PITH_FULL_IMAGE:figures/full_fig_p025_13.png]
Figure 14
Figure 14. Figure 14: Overall and per-class accuracies for balanced CIFAR10 before and after Stat-Opt attack. [PITH_FULL_IMAGE:figures/full_fig_p025_14.png]
Figure 15
Figure 15. Figure 15: Overall, per-class, and mean per-class accuracies for imbalanced CIFAR10 before and after Stat-Opt attack. Class 0 has the [PITH_FULL_IMAGE:figures/full_fig_p026_15.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

109 extracted references · 50 canonical work pages

  1. [43]

    Momin Ahmad Khan, Virat Shejwalkar, Amir Houmansadr, and Fatima M Anwar. 2023. On the Pitfalls of Security Evaluation of Robust Federated Learning. In 2023 IEEE Security and Privacy Workshops (SPW) . IEEE, 57–68

    work page 2023

  2. [20]

    X. Cao, J. Jia, Z. Zhang, and N. Gong. 2023. FedRecover: Recovering from Poisoning Attacks in Federated Learning using Historical Information. In 2023 2023 IEEE Symposium on Security and Privacy (SP) (SP) . IEEE Computer Society, Los Alamitos, CA, USA, 326–343. https://doi.org/10.1109/ SP46215.2023.00019

    arXiv 2023

  3. [84]

    Shejwalkar, A

    V. Shejwalkar, A. Houmansadr, P. Kairouz, and D. Ramage. 2022. Back to the Drawing Board: A Critical Evaluation of Poisoning Attacks on Production Federated Learning. In 2022 2022 IEEE Symposium on Security and Privacy (SP) (SP) . IEEE Computer Society, Los Alamitos, CA, USA, 1117–1134. https://doi .org/10.1109/SP46214.2022.00065

    work page arXiv 2022

  4. [1]

    [n. d.]. How Apple personalizes Siri without hoovering up your data — technologyreview.com. https://www.technologyreview.com/2019/12/11/ 131629/apple-ai-personalizes-siri-federated-learning/

    2019

  5. [2]

    Federated Learning: Collaborative Machine Learning without Centralized Training Data

    2017. Federated Learning: Collaborative Machine Learning without Centralized Training Data. https://ai .googleblog.com/2017/04/federated- learning-collaborative.html

    2017

  6. [3]

    The Stack Overflow Data

    2019. The Stack Overflow Data. https://www .kaggle.com/datasets/stackoverflow/stackoverflow

    2019

  7. [4]

    Zeyuan Allen-Zhu, Faeze Ebrahimian, Jerry Li, and Dan Alistarh. 2020. Byzantine-resilient non-convex stochastic gradient descent. arXiv preprint arXiv:2012.14368 (2020)

    arXiv 2020

  8. [5]

    Sebastien Andreina, Giorgia Azzurra Marson, Helen Möllering, and Ghassan Karame. 2021. Baffle: Backdoor detection via feedback-based federated learning. In 2021 IEEE 41st International Conference on Distributed Computing Systems (ICDCS) . IEEE, 852–863

    2021

Show all 109 references
  1. [6]

    Daniel Arp, Erwin Quiring, Feargus Pendlebury, Alexander Warnecke, Fabio Pierazzi, Christian Wressnegger, Lorenzo Cavallaro, and Konrad Rieck

  2. [7]

    The TensorFlow Federated Authors. 2019. TensorFlow Federated Stack Overflow dataset

    2019

  3. [8]

    Sana Awan, Bo Luo, and Fengjun Li. 2021. Contra: Defending against poisoning attacks in federated learning. In Computer Security–ESORICS 2021: 26th European Symposium on Research in Computer Security, Darmstadt, Germany, October 4–8, 2021, Proceedings, Part I 26 . Springer, 455–475

    2021

  4. [9]

    Eugene Bagdasaryan, Andreas Veit, Yiqing Hua, Deborah Estrin, and Vitaly Shmatikov. 2020. How to backdoor federated learning. In AISTATS

    2020

  5. [10]

    Marco Barreno, Blaine Nelson, and Anthony D. Joseph. 2010. The security of machine learning. Machine Learning (2010)

    2010

  6. [11]

    Moran Baruch, Baruch Gilad, and Yoav Goldberg. 2019. A Little Is Enough: Circumventing Defenses For Distributed Learning. In NeurIPS

    2019

  7. [12]

    Arjun Nitin Bhagoji, Supriyo Chakraborty, Prateek Mittal, and Seraphin Calo. 2019. Analyzing federated learning through an adversarial lens. In ICML

    2019

  8. [13]

    Battista Biggio and Fabio Roli. 2018. Wild patterns: Ten years after the rise of adversarial machine learning. Pattern Recognition (2018)

    2018

  9. [14]

    Peva Blanchard, Rachid Guerraoui, Julien Stainer, et al. 2017. Machine learning with adversaries: Byzantine tolerant gradient descent. In NeurIPS

    2017

  10. [15]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901

    2020

  11. [16]

    Lukas Burkhalter, Hidde Lycklama, Alexander Viand, Nicolas Küchler, and Anwar Hithnawi. 2021. Rofl: Attestable robustness for secure federated learning. arXiv preprint arXiv:2107.03311 (2021)

    2021 arXiv

  12. [17]

    Sebastian Caldas, Sai Meher Karthik Duddu, Peter Wu, Tian Li, Jakub Konečn`y, H Brendan McMahan, Virginia Smith, and Ameet Talwalkar. 2018. Leaf: A benchmark for federated settings. arXiv preprint arXiv:1812.01097 (2018)

    2018 arXiv

  13. [18]

    Xiaoyu Cao, Minghong Fang, Jia Liu, and Neil Zhenqiang Gong. 2021. FLTrust: Byzantine-robust Federated Learning via Trust Bootstrapping. In NDSS

    2021

  14. [19]

    Xiaoyu Cao, Jinyuan Jia, and Neil Zhenqiang Gong. 2021. Provably Secure Federated Learning against Malicious Clients. In AAAI

    2021

  15. [21]

    Xiaoyu Cao, Zaixi Zhang, Jinyuan Jia, and Neil Zhenqiang Gong. 2022. FLCert: Provably Secure Federated Learning Against Poisoning Attacks. IEEE Transactions on Information Forensics and Security 17 (2022), 3691–3705

    2022

  16. [22]

    Nicholas Carlini, Anish Athalye, Nicolas Papernot, Wieland Brendel, Jonas Rauber, Dimitris Tsipras, Ian Goodfellow, Aleksander Madry, and Alexey Kurakin. 2019. On evaluating adversarial robustness. arXiv preprint arXiv:1902.06705 (2019)

    2019 arXiv

  17. [23]

    Hongyan Chang, Virat Shejwalkar, Reza Shokri, and Amir Houmansadr. 2019. Cronus: Robust and Heterogeneous Collaborative Learning with Black-Box Knowledge Transfer. arXiv:1912.11279 (2019)

    2019 arXiv

  18. [24]

    Gregory Cohen, Saeed Afshar, Jonathan Tapson, and Andre Van Schaik. 2017. EMNIST: Extending MNIST to handwritten letters. In IJCNN

    2017

  19. [25]

    Jean Ogier Du Terrail, Samy-Safwan Ayed, Edwige Cyffers, Felix Grimberg, Chaoyang He, Regis Loeb, Paul Mangold, Tanguy Marchand, Othmane Marfoq, Erum Mushtaq, et al. 2022. FLamby: Datasets and Benchmarks for Cross-Silo Federated Learning in Realistic Healthcare Settings. In Ne...

    2022

  20. [26]

    Sannara Ek, François Portet, Philippe Lalanda, and German Vega. 2020. Evaluation of federated learning aggregation algorithms: application to human activity recognition. In Adjunct proceedings of the 2020 ACM international joint conference on pervasive and ubiquitous computing...

    2020

  21. [27]

    El Mahdi El Mhamdi, Rachid Guerraoui, and Sébastien Louis Alexandre Rouault. 2021. Distributed momentum for byzantine-resilient stochastic gradient descent. In 9th International Conference on Learning Representations (ICLR)

    2021

  22. [28]

    Minghong Fang, Xiaoyu Cao, Jinyuan Jia, and Neil Zhenqiang Gong. 2020. Local Model Poisoning Attacks to Byzantine-Robust Federated Learning. In USENIX

    2020

  23. [29]

    Ines Feki, Sourour Ammar, Yousri Kessentini, and Khan Muhammad. 2021. Federated learning for COVID-19 screening from Chest X-ray images. Applied Soft Computing 106 (2021), 107330

    2021

  24. [30]

    Shuhao Fu, Chulin Xie, Bo Li, and Qifeng Chen. 2019. Attack-resistant federated learning with residual-based reweighting. arXiv:1912.11464 (2019)

    2019 arXiv

  25. [31]

    Clement Fung, Chris JM Yoon, and Ivan Beschastnikh. 2020. The limitations of federated learning in sybil settings. In RAID. Manuscript submitted to ACM Decoding FL Defenses: Systemization, Pitfalls, and Remedies 29

    2020

  26. [32]

    Micah Goldblum, Dimitris Tsipras, Chulin Xie, et al. 2020. Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses. arXiv:2012.10544 (2020)

    2020 arXiv

  27. [33]

    Eduard Gorbunov, Samuel Horváth, Peter Richtárik, and Gauthier Gidel. 2022. Variance Reduction is an Antidote to Byzantines: Better Rates, Weaker Assumptions and Communication Compression as a Cherry on the Top. arXiv preprint arXiv:2206.00529 (2022)

    2022 arXiv

  28. [34]

    Hanxi Guo, Hao Wang, Tao Song, Yang Hua, Zhangcheng Lv, Xiulang Jin, Zhengui Xue, Ruhui Ma, and Haibing Guan. 2021. Siren: Byzantine-robust federated learning via proactive alarming. In Proceedings of the ACM Symposium on Cloud Computing . 47–60

    2021

  29. [35]

    Filip Hanzely, Slavomír Hanzely, Samuel Horváth, and Peter Richtárik. 2020. Lower bounds and optimal algorithms for personalized federated learning. Advances in Neural Information Processing Systems 33 (2020), 2304–2315

    2020

  30. [36]

    Filip Hanzely and Peter Richtárik. 2020. Federated learning of a mixture of global and local models. arXiv preprint arXiv:2002.05516 (2020)

    2020 arXiv

  31. [37]

    Tatsunori Hashimoto, Megha Srivastava, Hongseok Namkoong, and Percy Liang. 2018. Fairness without demographics in repeated loss minimization. In International Conference on Machine Learning . PMLR, 1929–1938

    2018

  32. [38]

    Ling Huang, Anthony D Joseph, Blaine Nelson, Benjamin IP Rubinstein, and J Doug Tygar. 2011. Adversarial machine learning. In AISec

    2011

  33. [39]

    Najeeb Moharram Jebreel, Josep Domingo-Ferrer, David Sánchez, and Alberto Blanco-Justicia. 2022. Defending against the label-flipping attack in federated learning. arXiv preprint arXiv:2207.01982 (2022)

    2022 arXiv

  34. [40]

    Malhar S Jere, Tyler Farnan, and Farinaz Koushanfar. 2020. A taxonomy of attacks on federated learning. IEEE Security & Privacy (2020)

    2020

  35. [41]

    Peter Kairouz, H Brendan McMahan, Brendan Avent, et al. 2019. Advances and open problems in federated learning. arXiv:1912.04977 (2019)

    2019 arXiv

  36. [42]

    Sai Praneeth Karimireddy, Lie He, and Martin Jaggi. 2020. Byzantine-robust learning on heterogeneous datasets via bucketing. arXiv preprint arXiv:2006.09365 (2020)

    2020 arXiv

  37. [44]

    Jakub Konečn`y, H Brendan McMahan, Felix X Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon. 2016. Federated learning: Strategies for improving communication efficiency. NIPS Workshop on Private Multi-Party ML (2016)

    2016

  38. [45]

    Alex Krizhevsky. 2009. Learning multiple layers of features from tiny images . Technical Report. University of Toronto

    2009

  39. [46]

    Hanchao Ku, Willy Susilo, Yudi Zhang, Wenfen Liu, and Mingwu Zhang. 2022. Privacy-preserving federated learning in medical diagnosis with homomorphic re-encryption. Computer Standards & Interfaces 80 (2022), 103583

    2022

  40. [47]

    Yann LeCun and Corinna Cortes. 1998. The mnist database of handwritten digits. http://yann. lecun. com/exdb/mnist/ (1998)

    1998

  41. [48]

    Liping Li, Wei Xu, Tianyi Chen, Georgios B Giannakis, and Qing Ling. 2019. RSA: Byzantine-robust stochastic aggregation methods for distributed learning from heterogeneous datasets. In AAAI

    2019

  42. [49]

    Qinbin Li, Bingsheng He, and Dawn Song. 2021. Model-contrastive federated learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 10713–10722

    2021

  43. [50]

    Suyi Li, Yong Cheng, Yang Liu, Wei Wang, and Tianjian Chen. 2019. Abnormal client behavior detection in federated learning. arXiv preprint arXiv:1910.09933 (2019)

    2019 arXiv

  44. [51]

    Suyi Li, Yong Cheng, Wei Wang, Yang Liu, and Tianjian Chen. 2020. Learning to detect malicious clients for robust federated learning. arXiv preprint arXiv:2002.00211 (2020)

    2020 arXiv

  45. [52]

    Tian Li, Shengyuan Hu, Ahmad Beirami, and Virginia Smith. 2021. Ditto: Fair and robust federated learning through personalization. In ICML

    2021

  46. [53]

    Zhuohang Li, Luyang Liu, Jiaxin Zhang, and Jian Liu. 2021. Byzantine-robust federated learning through spatial-temporal analysis of local model updates. In 2021 IEEE 27th International Conference on Parallel and Distributed Systems (ICPADS) . IEEE, 372–379

    2021

  47. [54]

    Feng Lin, Weiyu Li, and Qing Ling. 2021. Stochastic alternating direction method of multipliers for byzantine-robust distributed learning. arXiv preprint arXiv:2106.06891 (2021)

    2021 arXiv

  48. [55]

    Yi Liu, Ruihui Zhao, Jiawen Kang, Abdulsalam Yassine, Dusit Niyato, and Jialiang Peng. 2021. Towards communication-efficient and attack-resistant federated edge learning for industrial internet of things. ACM Transactions on Internet Technology (TOIT) 22, 3 (2021), 1–22

    2021

  49. [56]

    Xindi Ma, Qi Jiang, Mohammad Shojafar, Mamoun Alazab, Sachin Kumar, and Saru Kumari. 2022. DisBezant: secure and robust federated learning against byzantine attack in IoT-enabled MTS. IEEE Transactions on Intelligent Transportation Systems (2022)

    2022

  50. [57]

    Saeed Mahloujifar, Mohammad Mahmoody, and Ameer Mohammed. 2019. Universal multi-party poisoning attacks. In ICML

    2019

  51. [58]

    Ranwa Al Mallah, David Lopez, Godwin Badu Marfo, and Bilal Farooq. 2021. Untargeted poisoning attack detection in federated learning via behavior attestation. arXiv preprint arXiv:2101.10904 (2021)

    2021 arXiv

  52. [59]

    H Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. 2017. Communication-efficient learning of deep networks from decentralized data. In AISTATS

    2017

  53. [60]

    El Mahdi El Mhamdi, Rachid Guerraoui, and Sébastien Rouault. 2018. The Hidden Vulnerability of Distributed Learning in Byzantium. In ICML

    2018

  54. [61]

    Thomas Minka. 2000. Estimating a Dirichlet distribution

    2000

  55. [62]

    Hamid Mozaffari, Virat Shejwalkar, and Amir Houmansadr. 2021. Frl: Federated rank learning. arXiv preprint arXiv:2110.04350 (2021)

    2021 arXiv

  56. [63]

    Luis Muñoz-González, Battista Biggio, Ambra Demontis, Andrea Paudice, Vasin Wongrassamee, Emil C Lupu, and Fabio Roli. 2017. Towards poisoning of deep learning algorithms with back-gradient optimization. In AISec

    2017

  57. [64]

    Lokesh Nagalapatti and Ramasuri Narayanam. 2021. Game of gradients: Mitigating irrelevant clients in federated learning. In Proceedings of the AAAI Conference on Artificial Intelligence , Vol. 35. 9046–9054. Manuscript submitted to ACM 30 Momin Ahmad Khan, Virat Shejwalkar, Ya...

    2021

  58. [65]

    Mohammad Naseri, Jamie Hayes, and Emiliano De Cristofaro. 2020. Local and central differential privacy for robustness and privacy in federated learning. arXiv preprint arXiv:2009.03561 (2020)

    2020 arXiv

  59. [66]

    Dinh C Nguyen, Quoc-Viet Pham, Pubudu N Pathirana, Ming Ding, Aruna Seneviratne, Zihuai Lin, Octavia Dobre, and Won-Joo Hwang. 2022. Federated learning for smart healthcare: A survey. ACM Computing Surveys (CSUR) 55, 3 (2022), 1–37

    2022

  60. [67]

    Jorge Nocedal. 1980. Updating quasi-Newton matrices with limited storage. Mathematics of computation 35, 151 (1980), 773–782

    1980

  61. [68]

    Xiaomin Ouyang, Zhiyuan Xie, Jiayu Zhou, Jianwei Huang, and Guoliang Xing. 2021. Clusterfl: a similarity-aware federated learning system for human activity recognition. In Proceedings of the 19th Annual International Conference on Mobile Systems, Applications, and Services . 54–66

    2021

  62. [69]

    Mustafa Safa Ozdayi, Murat Kantarcioglu, and Yulia R Gel. 2021. Defending against backdoors in federated learning with robust learning rate. In Proceedings of the AAAI Conference on Artificial Intelligence , Vol. 35. 9268–9276

    2021

  63. [70]

    Xudong Pan, Mi Zhang, Duocai Wu, Qifan Xiao, Shouling Ji, and Min Yang. 2020. Justinian’s gaavernor: Robust distributed learning with gradient aggregation agent. In Proceedings of the 29th USENIX Conference on Security Symposium . 1641–1658

    2020

  64. [71]

    Jungwuk Park, Dong-Jun Han, Minseok Choi, and Jaekyun Moon. 2021. Sageflow: Robust federated learning against both stragglers and adversaries. Advances in neural information processing systems 34 (2021), 840–851

    2021

  65. [72]

    Matthias Paulik, Matt Seigel, Henry Mason, et al . 2021. Federated Evaluation and Tuning for On-Device Personalization: System Design & Applications. arXiv:2102.08503 (2021)

    2021 arXiv

  66. [73]

    Krishna Pillutla, Sham M Kakade, and Zaid Harchaoui. 2019. Robust aggregation for federated learning. arXiv:1912.13445 (2019)

    2019 arXiv

  67. [74]

    Sai Praneeth Karimireddy, Lie He, and Martin Jaggi. 2020. Learning from History for Byzantine Robust Optimization. arXiv e-prints (2020), arXiv–2012

    2020

  68. [75]

    Adnan Qayyum, Kashif Ahmad, Muhammad Ahtazaz Ahsan, Ala Al-Fuqaha, and Junaid Qadir. 2022. Collaborative federated learning for healthcare: Multi-modal covid-19 diagnosis at the edge. IEEE Open Journal of the Computer Society 3 (2022), 172–184

    2022

  69. [76]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learni...

    2021

  70. [77]

    Priyesh Ranjan, Ashish Gupta, Federico Coro, and Sajal K Das. 2022. Securing Federated Learning against Overwhelming Collusive Attackers. In GLOBECOM 2022-2022 IEEE Global Communications Conference . IEEE, 1448–1453

    2022

  71. [78]

    Sashank J Reddi, Zachary Charles, Manzil Zaheer, Zachary Garrett, Keith Rush, Jakub Konečn`y, Sanjiv Kumar, and Hugh Brendan McMahan. 2020. Adaptive Federated Optimization. In ICLR

    2020

  72. [79]

    Jae Hun Ro, Ananda Theertha Suresh, and Ke Wu. 2021. FedJAX: Federated learning simulation with JAX. arXiv preprint arXiv:2108.02117 (2021)

    2021 arXiv

  73. [80]

    Nuria Rodríguez-Barroso, Daniel Jiménez-López, M Victoria Luzón, Francisco Herrera, and Eugenio Martínez-Cámara. 2023. Survey on federated learning threats: Concepts, taxonomy on attacks and defences, experimental study and challenges. Information Fusion 90 (2023), 148–173

    2023

  74. [81]

    Amrita Roy Chowdhury, Chuan Guo, Somesh Jha, and Laurens van der Maaten. 2022. EIFFeL: Ensuring Integrity for Federated Learning. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security . 2535–2549

    2022

  75. [82]

    Felix Sattler, Klaus-Robert Müller, Thomas Wiegand, and Wojciech Samek. 2020. On the byzantine robustness of clustered federated learning. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 8861–8865

    2020

  76. [83]

    Virat Shejwalkar and Amir Houmansadr. 2021. Manipulating the Byzantine: Optimizing Model Poisoning Attacks and Defenses for Federated Learning. In NDSS

    2021

  77. [85]

    Shiqui Shen, Shruti Tople, and Prateek Saxena. 2016. AUROR: Defending againsts poisoning attacks in collaborative deep learning systems. 2016 Annual Computer Security Applications Conference (2016)

    2016

  78. [86]

    Konstantin Sozinov, Vladimir Vlassov, and Sarunas Girdzijauskas. 2018. Human activity recognition using federated learning. In2018 IEEE Intl Conf on Parallel & Distributed Processing with Applications, Ubiquitous Computing & Communications, Big Data & Cloud Computing, Social C...

    2018

  79. [87]

    Jingwei Sun, Ang Li, Lin Duan, Samiul Alam, Xuliang Deng, Xin Guo, Haiming Wang, Maria Gorlatova, Mi Zhang, Hai Li, et al. 2022. FedSEA: A Semi-Asynchronous Federated Learning Framework for Extremely Heterogeneous Devices. InProceedings of the 20th ACM Conference on Embedded N...

    2022

  80. [88]

    Ziteng Sun, Peter Kairouz, Ananda Theertha Suresh, and H Brendan McMahan. 2019. Can you really backdoor federated learning? NeurIPS FL Workshop (2019)

    2019

  81. [89]

    Robert Tibshirani, Guenther Walther, and Trevor Hastie. 2001. Estimating the number of clusters in a data set via the gap statistic. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 63, 2 (2001), 411–423

    2001

  82. [90]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023)

    2023 arXiv

  83. [91]

    Hongyi Wang, Kartik Sreenivasan, Shashank Rajput, Harit Vishwakarma, Saurabh Agarwal, Jy-yong Sohn, Kangwook Lee, and Dimitris Papail- iopoulos. 2020. Attack of the tails: Yes, you really can backdoor federated learning. In NeurIPS. Manuscript submitted to ACM Decoding FL Defe...

    2020

  84. [92]

    Ning Wang, Yang Xiao, Yimin Chen, Yang Hu, Wenjing Lou, and Y Thomas Hou. 2022. Flare: defending federated learning against model poisoning attacks via latent space representations. In Proceedings of the 2022 ACM on Asia Conference on Computer and Communications Security . 946–958

    2022

  85. [93]

    Utilization of FATE in Risk Management of Credit in Small and Micro Enterprises

    webankcredit 2019. Utilization of FATE in Risk Management of Credit in Small and Micro Enterprises. https://www.fedai.org/cases/utilization-of- fate-in-risk-management-of-credit-in-small-and-micro-enterprises/

    2019

  86. [94]

    Chen Wu, Xian Yang, Sencun Zhu, and Prasenjit Mitra. 2020. Mitigating backdoor attacks in federated learning. arXiv:2011.01767 (2020)

    2020 arXiv

  87. [95]

    Zhaoxian Wu, Qing Ling, Tianyi Chen, and Georgios B Giannakis. 2020. Federated variance-reduced stochastic gradient descent with robustness to byzantine attacks. IEEE Transactions on Signal Processing 68 (2020), 4583–4596

    2020

  88. [96]

    Han Xiao, Kashif Rasul, and Roland Vollgraf. 2017. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747 (2017)

    2017 arXiv

  89. [97]

    Chulin Xie, Minghao Chen, Pin-Yu Chen, and Bo Li. 2021. CRFL: Certifiably Robust Federated Learning against Backdoor Attacks. In ICML

    2021

  90. [98]

    Cong Xie, Oluwasanmi Koyejo, and Indranil Gupta. 2018. Generalized byzantine-tolerant sgd. arXiv:1802.10116 (2018)

    2018 arXiv

  91. [99]

    Cong Xie, Sanmi Koyejo, and Indranil Gupta. 2019. Fall of empires: Breaking Byzantine-tolerant SGD by inner product manipulation.arXiv:1903.03936 (2019)

    2019 arXiv

  92. [100]

    Yueqi Xie, Weizhong Zhang, Renjie Pi, Fangzhao Wu, Qifeng Chen, Xing Xie, and Sunghun Kim. 2022. Robust federated learning against both data heterogeneity and poisoning attack via aggregation optimization. arXiv preprint (2022)

    2022

  93. [101]

    Chang Xu, Yu Jia, Liehuang Zhu, Chuan Zhang, Guoxie Jin, and Kashif Sharif. 2022. TDFL: Truth Discovery Based Byzantine Robust Federated Learning. IEEE Transactions on Parallel and Distributed Systems 33, 12 (2022), 4835–4848

    2022

  94. [102]

    Jian Xu, Shao-Lun Huang, Linqi Song, and Tian Lan. 2021. Signguard: Byzantine-robust federated learning through collaborative malicious gradient filtering. arXiv preprint arXiv:2109.05872 (2021)

    2021 arXiv

  95. [103]

    Dong Yin, Yudong Chen, Kannan Ramchandran, and Peter Bartlett. 2018. Byzantine-robust distributed learning: Towards optimal statistical rates. In ICML

    2018

  96. [104]

    Syed Zawad, Ahsan Ali, Pin-Yu Chen, Ali Anwar, Yi Zhou, Nathalie Baracaldo, Yuan Tian, and Feng Yan. 2021. Curse or redemption? how data heterogeneity affects the robustness of federated learning. In Proceedings of the AAAI conference on artificial intelligence , Vol. 35. 10807–10814

    2021

  97. [105]

    Kaiyuan Zhang, Guanhong Tao, Qiuling Xu, Siyuan Cheng, Shengwei An, Yingqi Liu, Shiwei Feng, Guangyu Shen, Pin-Yu Chen, Shiqing Ma, et al

  98. [106]

    Zaixi Zhang, Xiaoyu Cao, Jinyuan Jia, and Neil Zhenqiang Gong. 2022. FLDetector: Defending federated learning against model poisoning attacks via detecting malicious clients. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining . 2545–2555

    2022

  99. [107]

    arXiv preprint arXiv:2210.12873 (2022)

    FLIP: A Provable Defense Framework for Backdoor Mitigation in Federated Learning. arXiv preprint arXiv:2210.12873 (2022)

    2022 arXiv

  100. [109]

    Yuchen Zhao, Payam Barnaghi, and Hamed Haddadi. 2022. Multimodal federated learning on iot data. In 2022 IEEE/ACM Seventh International Conference on Internet-of-Things Design and Implementation (IoTDI) . IEEE, 43–54. A Appendix A.1 Our methodology to classify 50 defenses Tabl...

    2022

  101. [2022]

    In 31st USENIX Security Symposium (USENIX Security 22)

    Dos and don’ts of machine learning in computer security. In 31st USENIX Security Symposium (USENIX Security 22) . 3971–3988

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.