Efficient Client Selection in Federated Learning

Deepak K. Tosh; Shirley V. Moore; William Marfo

REVIEW 5 major objections 5 minor 1 cited by

Efficient Client Selection in Federated Learning

T0 review · 5 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that top-K utility-score client selection with differential privacy and checkpointing raises federated anomaly-detection accuracy by 7% and cuts training time by 25%.

desk verdict Plausible integration, but the central utility-score mechanism is undefined and Fig. 1 is missing, so the 7%/25% claims are unsupported. read the letter →

arxiv 2502.00036 v1 pith:L5RWMFNS submitted 2025-01-25 cs.LG cs.AIcs.DC

William Marfo , Deepak K. Tosh , Shirley V. Moore This is my paper · ORCID

classification cs.LGcs.AIcs.DC

keywords federatedlearningclientselectiondifferentialprivacyfaulttolerancecheckpointingnetworkanomalydetectionUNSW-NB15ROADdataset

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Federated learning trains a shared model from distributed data, but its success depends on which clients participate in each training round. This paper tries to establish that an adaptive selection scheme — ranking available clients by utility scores, adding Gaussian noise to updates for differential privacy, and checkpointing for fault tolerance — outperforms two published baselines on network anomaly detection. On the UNSW-NB15 and ROAD datasets, the reported gains are a 7% accuracy improvement and a 25% reduction in training time. The paper also claims that increasing the privacy budget $\epsilon$ improves accuracy (from 86% to 89% on UNSW-NB15 as $\epsilon$ rises from 10 to 100) and that fault tolerance costs about 2–3% accuracy while making training resilient to client dropout.

What carries the argument

The load-bearing mechanism is Algorithm 1, a per-round top-K client selection loop. Each round begins by computing a utility score for every available client — the paper names data quality and computational capacity as factors but gives no formula — then selects the highest-scoring K clients to train locally. Local training applies gradient clipping followed by Gaussian noise calibrated by the privacy budget $\epsilon$, so the server never sees raw updates, and a checkpointing interval $t_c^*$ lets failing clients resume from saved state. The $\epsilon$ sweep in Fig. 2 is the part of the machinery that is actually quantified: raising $\epsilon$ from 10 to 100 raises accuracy by 3 percentage points on UNSW-NB15 and 9 points on ROAD.

What would settle it

Rerun the UNSW-NB15 and ROAD tasks comparing top-K utility selection against random selection of K clients with identical Gaussian noise and checkpointing; if random selection matches the reported accuracy within noise, the utility ranking is not driving the 7% gain and the central claim collapses. As a second check, recompute the 7% accuracy and 25% training-time figures with the exact K, client-pool size, and baseline hyperparameters; if the margins disappear once those quantities are fixed, the headline numbers are not reproducible.

Watch

Extended reading notes

Core claim

The central claim of the paper is that integrating adaptive client selection with privacy protection and resilience improves federated learning on intrusion-detection data. Concretely, the proposed Algorithm 1 selects the top K available clients each round according to utility scores based on factors such as data quality and computational capacity, clips gradients and injects Gaussian noise controlled by the privacy budget $\epsilon$ before aggregation, and uses checkpointing at interval $t_c^*$ so clients can recover from failures. Evaluated against the ACFL and FedL2P baselines on UNSW-NB15 and ROAD, the authors report a 7% accuracy gain and a 25% training-time reduction, with the largest gains on the ROAD dataset. The privacy experiments show accuracy rising with $\epsilon$ (86% to 89% on UNSW-NB15, 73% to 82% on ROAD), and the fault-tolerance experiments show a 2–3% accuracy and AUC drop with a 5–10% training-time increase, accepted as the cost of resilience.

Load-bearing premise

The central result rests on the premise that clients can be meaningfully ranked by utility scores derived from data quality and computational capacity, and that selecting the top K by those scores improves the global model — but the paper gives no formula, features, or calibration for the scores, so the mechanism behind the claimed 7% accuracy gain is unstated.

Editorial extensions

If this is right

If the claims hold, federated intrusion-detection deployments can expect about 7% higher accuracy and 25% shorter training runs than the ACFL and FedL2P baselines by adopting utility-scored top-K selection with differential privacy.
The privacy budget $\epsilon$ becomes an explicit dial: raising it from 10 to 100 trades noise for accuracy, with the paper reporting 3-point and 9-point accuracy gains on the two datasets.
Checkpointing buys fault tolerance at a bounded cost of roughly 2–3% accuracy and 5–10% training time, so resilient federated systems do not have to sacrifice much performance.
The same recipe — top-K utility selection, gradient clipping, Gaussian noise, and checkpointing — is portable to other network-anomaly federated tasks of similar data shape, not just the two datasets tested.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper never states the utility-score formula or its calibration, so the 7% accuracy gain could come entirely from the noise schedule or the top-K truncation rather than from ranking by data quality and computational capacity; an ablation that replaces utility ranking with random selection of K clients would isolate the true cause.
The 25% training-time reduction should be read with care: if fewer clients are selected each round, wall-clock time falls mechanically, so the headline number only becomes meaningful when the exact K, client-pool size, and baseline settings are reported.
A testable extension is to run the same selection-plus-privacy-plus-checkpointing recipe on non-IID image or text federated benchmarks; if the accuracy gain does not transfer, the result is specific to anomaly-detection data rather than a general property of the selection scheme.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is a two-page extended abstract that might be a decent workshop poster, but the headline claims can't be checked. The integration of top-K utility-score selection with DP and checkpointing for network anomaly detection is a reasonable engineering idea, and the authors deserve credit for reporting the real cost of fault tolerance: Table I shows checkpointing costs 2-3% accuracy and 5-10% training time. That kind of honesty is rare.

The problem is that the entire claimed 7% accuracy gain and 25% time reduction are attributed to the utility-score ranking, and that score is never defined. Algorithm 1 says 'compute utility scores' based on 'factors such as data quality and computational capacity,' but there is no formula, feature vector, or calibration. Since DP noise is standard and Table I shows checkpointing hurts, top-K selection is the only candidate mechanism left, and it's a black box.

The evaluation is also not reproducible. Fig. 1, which is supposed to show the comparison against ACFL and FedL2P, is missing from the supplied text. Table I only compares the method with and without fault tolerance, not against the baselines. No code, data splits, or error bars. The baseline descriptions are confusing: ACFL is associated with reference [5], which is actually 'CriticalFL,' and FedL2P is a personalization method, not obviously a client-selection baseline. The conclusion says 'compared to FedL2P,' while the abstract says 'compared to baselines.'

There is nothing circular going on—the self-citation is just background—and the datasets are external. So the paper isn't dishonest; it's just too thin. If the authors produce a full version with Fig. 1, a precise utility-score definition, a proper comparison table, and code/data, the idea could be worth a look. As it stands, I would not send this to a serious peer-reviewed venue. It's a placeholder, not a paper. If it's for a workshop, it might get a poster slot, but only if the missing figure is included.

Referee Report

5 major / 5 minor

Summary. The paper proposes a federated learning client selection framework that combines top-K utility-score-based selection, Gaussian-noise differential privacy, and checkpoint-based fault tolerance. The authors report a 7% accuracy improvement and a 25% training time reduction over ACFL and FedL2P on the UNSW-NB15 and ROAD datasets, along with a privacy-budget study and a fault-tolerance trade-off table. However, the utility score is never defined, Figure 1 is missing from the supplied text, and no table reports any quantitative comparison against the two baselines; as a result, the headline claims cannot be verified from the manuscript as submitted.

Significance. If fully substantiated, the proposed combination of adaptive client selection, differential privacy, and fault tolerance would be a useful contribution to federated learning for network anomaly detection, where client heterogeneity and privacy are real concerns. The paper also uses standard public datasets and presents Algorithm 1 as an explicit procedural statement. However, the central selection mechanism is unspecified, the key comparison figure is absent, and the experimental setup lacks the details needed for reproduction. The current manuscript therefore does not establish the claimed gains, and its significance is limited until the missing definitions and evidence are supplied.

major comments (5)

[Section II, Algorithm 1] The central selection mechanism is underspecified. Algorithm 1 lines 4-5 compute utility scores and select the top K clients, but no formula, feature vector, normalization, or calibration procedure for the utility score is given. Since the abstract and introduction describe the selection as adaptive, the fact that Algorithm 1 takes K as a fixed input and never updates it is also inconsistent with that description. The claimed 7% accuracy gain and 25% training time reduction are attributed to this selection mechanism, so without an explicit definition of the utility score the method is untestable and its central result has no stated basis.
[Section III-B1, Fig. 1] The only quantitative support for the headline claims is Figure 1, which is not present in the supplied manuscript. No table reports accuracy, AUC-ROC, or training time for ACFL and FedL2P; Table I compares only the proposed method with and without fault tolerance. Consequently, the 7% accuracy improvement and 25% training time reduction cannot be checked against any reported numbers.
[Section III-A] The experimental setup omits crucial details needed for reproducibility. There is no description of the federated architecture, number of clients, data partitioning scheme, local training epochs, batch sizes, learning rates, or hyperparameter settings for the proposed method and the baselines. In addition, the text identifies ACFL as reference [5], but reference [5] is titled "CriticalFL"; either the acronym or the citation is incorrect. Without these details, the comparison to ACFL and FedL2P is not reproducible.
[Section III-B2, Fig. 2] The differential privacy evaluation reports only two points per dataset (epsilon = 10 and 100) and does not compare against a non-private version of the proposed method or against the same selection algorithm without DP noise. Figure 2, like Figure 1, is not present in the supplied text. This experiment therefore cannot isolate the effect of DP on the client selection mechanism, and it does not support the claimed accuracy gain.
[Section III-B3, Table I] The fault-tolerance results in Table I show that checkpointing decreases accuracy by 2.7 percentage points on UNSW-NB15 and 1.6 points on ROAD, while increasing training time by roughly 5%. These changes are in the opposite direction of the headline claims, so checkpointing cannot explain the 7% accuracy improvement or the 25% training time reduction. A controlled ablation that isolates the top-K utility-score selection from the DP and checkpointing components is missing; without it, the reported gains could arise from any component or from baseline choices.

minor comments (5)

[References] Reference [5] is cited for ACFL, but the listed title is "CriticalFL"; please correct the acronym or the reference.
[Abstract and Conclusion] The abstract says the method improves accuracy by 7% and reduces training time by 25% compared to baselines, while the conclusion says "compared to FedL2P"; clarify which comparison is actually reported.
[Section III-A] No error bars, standard deviations, or repeated-run information are reported for any metric; please add variance information for all measurements.
[Throughout] There are several typographical issues, including an extra space in the author name "Shirley V . Moore", stray spaces around commas in "Python 3.8 ," and "TensorFlow 2.6.0 , and PyTorch 0.5.0", and an unusually old PyTorch version that may be a typo.
[Figures] Figure 1 and Figure 2 are referenced in the text but are not included in the supplied manuscript; please provide the actual figures.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the evaluation is against external datasets and baselines, the only self-citation is background, and the unspecified utility score is a reproducibility gap rather than a circular reduction.

full rationale

The paper's claimed result is an empirical comparison against external baselines (ACFL and FedL2P) on external datasets (UNSW-NB15 and ROAD). The proposed Algorithm 1 combines utility-score top-K selection, Gaussian-noise differential privacy, and checkpointing, and the reported 7% accuracy gain and 25% training-time reduction are stated as observed evaluation outcomes, not derived from the assumptions by construction. No equation in the paper defines the headline metric in terms of a fitted parameter or a self-cited result. The only self-citation, [1] (MILCOM 2022 by the same authors), is used as background for federated learning and is not load-bearing for the claimed improvement. The serious weakness is that the utility score in Algorithm 1 lines 4-5 is never specified, so the mechanism behind the gains cannot be checked; however, missing specification is a reproducibility and correctness concern, not circularity, because there is no textual evidence that the score was tuned on the test sets or that the comparison collapses to an input by definition. Therefore no circular step is established.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests entirely on the experimental setup, and every mechanism that could make the method concrete is either a free input to Algorithm 1 or an unstated assumption. The utility-score formula, the DP noise calibration, the checkpoint recovery protocol, and the dataset-specific training configuration are all unspecified. Because these are the components that produce the 7% and 25% numbers, the ledger contains no external benchmark that independently pins them down. No invented physical or conceptual entities appear.

free parameters (5)

K (number of selected clients)
Algorithm 1 takes K as input; the experiments never state how many clients were selected per round, and accuracy and training time depend on it.
privacy budget epsilon = 10 to 100 in DP experiments
Section III-B2 reports accuracy varying from 86% to 89% (UNSW-NB15) as epsilon changes, so the headline result is conditional on an epsilon choice that is not part of a principled selection.
checkpoint interval t*_c
Algorithm 1 takes t*_c as input; the training-time cost of fault tolerance (5-10%) depends on this unstated value.
utility score weights
Line 4 of Algorithm 1 says 'Compute utility scores' without a formula; any weighted combination of data quality and computational capacity is a hand choice that controls which clients are selected.
gradient clipping threshold and Gaussian noise scale
The DP mechanism requires calibrated noise; neither the clipping norm nor the noise multiplier is reported, and the paper gives no proof that the stated epsilon is achieved.

assumptions (4)

domain assumption Adding Gaussian noise to clipped gradient updates provides differential privacy at the stated budget epsilon.
Invoked in Algorithm 1 lines 7-8; no sensitivity analysis or noise scale is given, so the privacy guarantee is assumed rather than shown.
ad hoc to paper Utility scores based on data quality and computational capacity can be computed and meaningfully rank clients for global model improvement.
Algorithm 1 line 4 depends on an unstated scoring function; this is the load-bearing modeling choice.
domain assumption Checkpointing at interval t*_c lets clients recover from failure without corrupting the global aggregation.
Algorithm 1 lines 10-13; no protocol for how recovered states are merged is described.
domain assumption UNSW-NB15 and ROAD datasets are representative enough to support a general claim about network anomaly detection FL.
Section III-A selects two datasets; the conclusion generalizes beyond them without a cross-domain argument.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Efficient Client Selection in Federated Learning." pith.science (2026). https://pith.science/paper/L5RWMFNS

@misc{pith2026250200036,
  author       = {Pith},
  title        = {Pith review of: Efficient Client Selection in Federated Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/L5RWMFNS}},
  note         = {Machine review of arXiv:2502.00036}
}

read the original abstract

Federated Learning (FL) enables decentralized machine learning while preserving data privacy. This paper proposes a novel client selection framework that integrates differential privacy and fault tolerance. The adaptive client selection adjusts the number of clients based on performance and system constraints, with noise added to protect privacy. Evaluated on the UNSW-NB15 and ROAD datasets for network anomaly detection, the method improves accuracy by 7% and reduces training time by 25% compared to baselines. Fault tolerance enhances robustness with minimal performance trade-offs.

Figures

Figures reproduced from arXiv: 2502.00036 by the authors.

**Figure 1.** Performance comparison of the proposed method, ACFL, and FedL2P [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Overcoming Challenges of Partial Client Participation in Federated Learning : A Comprehensive Review
cs.LG 2025-06 conditional novelty 4.0 of 10

A survey categorizing and comparing twelve federated learning methods for partial client participation, weakened by several citation mismatches and unsourced benchmark numbers.

Reference graph

Works this paper leans on

7 extracted references · 7 canonical work pages · cited by 1 Pith paper

[5]

G. Yan, H. Wang, X. Yuan, and J. Li, ``Criticalfl: A critical learning periods augmented client selection framework for efficient federated learning,'' in Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 1em plus 0.5em minus 0.4em ACM, 2023, pp. 2898--2907

work page 2023
[1]

Marfo, D

W. Marfo, D. K. Tosh, and S. V. Moore, ``Network anomaly detection using federated learning,'' in MILCOM 2022 - IEEE Military Communications Conference. 1em plus 0.5em minus 0.4em IEEE, 2022, pp. 484--489

work page 2022
[2]

Q. Li, X. Li, L. Zhou, and X. Yan, ``Adafl: Adaptive client selection and dynamic contribution evaluation for efficient federated learning,'' in ICASSP 2024 - IEEE International Conference on Acoustics, Speech and Signal Processing. 1em plus 0.5em minus 0.4em IEEE, 2024, pp. 6645--6649

work page 2024
[3]

Moustafa and J

N. Moustafa and J. Slay, ``Unsw-nb15: A comprehensive data set for network intrusion detection systems (unsw-nb15 network data set),'' in 2015 Military Communications and Information Systems Conference (MilCIS). 1em plus 0.5em minus 0.4em IEEE, 2015, pp. 1--6

work page 2015
[4]

M. E. Verma, R. A. Bridges, M. D. Iannacone, S. C. Hollifield, P. Moriano, S. C. Hespeler, B. Kay, and F. L. Combs, ``A comprehensive guide to can ids data and introduction of the road dataset,'' PLoS ONE, vol. 19, no. 1, p. e0296879, 2024

work page 2024
[6]

R. Lee, M. Kim, D. Li, X. Qiu, T. Hospedales, F. Husz \'a r, and N. Lane, ``Fedl2p: Federated learning to personalize,'' Advances in Neural Information Processing Systems, vol. 36, 2024

work page 2024
[7]

Ԅ; ˏ 9"j8 6 tzyy l6@ kp' 7l6 d2X Q q Gr P m6o&EQo>qD0 dY ٗ_ ڵk (Y ` ^ݻw ޽ xSSS:j|Ǐ|YUnKh0 `` 歷2 ap8 4M˲o 9r xUUj H / Q eCP4 T < _ \ Ȋ p8l0 X t:L&F oQw 7 h a&6c(ʲl27 D

11em plus .33em minus .07em 4000 4000 100 4000 4000 500 `\.=1000 = #1 \@IEEEnotcompsoconly \@IEEEcompsoconly #1 * [1] 0pt [0pt][0pt] #1 * [1] 0pt [0pt][0pt] #1 * \| ** #1 \@IEEEauthorblockNstyle \@IEEEcompsocnotconfonly \@IEEEauthorblockAstyle \@IEEEcompsocnotconfonly \@IEEEcompsocconfonly \@IEEEauthordefaulttextstyle \@IEEEcompsocnotconfonly \@IEEEauthor...

work page

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [5]

G. Yan, H. Wang, X. Yuan, and J. Li, ``Criticalfl: A critical learning periods augmented client selection framework for efficient federated learning,'' in Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 1em plus 0.5em minus 0.4em ACM, 2023, pp. 2898--2907

work page 2023

[2] [1]

Marfo, D

W. Marfo, D. K. Tosh, and S. V. Moore, ``Network anomaly detection using federated learning,'' in MILCOM 2022 - IEEE Military Communications Conference. 1em plus 0.5em minus 0.4em IEEE, 2022, pp. 484--489

work page 2022

[3] [2]

Q. Li, X. Li, L. Zhou, and X. Yan, ``Adafl: Adaptive client selection and dynamic contribution evaluation for efficient federated learning,'' in ICASSP 2024 - IEEE International Conference on Acoustics, Speech and Signal Processing. 1em plus 0.5em minus 0.4em IEEE, 2024, pp. 6645--6649

work page 2024

[4] [3]

Moustafa and J

N. Moustafa and J. Slay, ``Unsw-nb15: A comprehensive data set for network intrusion detection systems (unsw-nb15 network data set),'' in 2015 Military Communications and Information Systems Conference (MilCIS). 1em plus 0.5em minus 0.4em IEEE, 2015, pp. 1--6

work page 2015

[5] [4]

M. E. Verma, R. A. Bridges, M. D. Iannacone, S. C. Hollifield, P. Moriano, S. C. Hespeler, B. Kay, and F. L. Combs, ``A comprehensive guide to can ids data and introduction of the road dataset,'' PLoS ONE, vol. 19, no. 1, p. e0296879, 2024

work page 2024

[6] [6]

R. Lee, M. Kim, D. Li, X. Qiu, T. Hospedales, F. Husz \'a r, and N. Lane, ``Fedl2p: Federated learning to personalize,'' Advances in Neural Information Processing Systems, vol. 36, 2024

work page 2024

[7] [7]

Ԅ; ˏ 9"j8 6 tzyy l6@ kp' 7l6 d2X Q q Gr P m6o&EQo>qD0 dY ٗ_ ڵk (Y ` ^ݻw ޽ xSSS:j|Ǐ|YUnKh0 `` 歷2 ap8 4M˲o 9r xUUj H / Q eCP4 T < _ \ Ȋ p8l0 X t:L&F oQw 7 h a&6c(ʲl27 D

11em plus .33em minus .07em 4000 4000 100 4000 4000 500 `\.=1000 = #1 \@IEEEnotcompsoconly \@IEEEcompsoconly #1 * [1] 0pt [0pt][0pt] #1 * [1] 0pt [0pt][0pt] #1 * \| ** #1 \@IEEEauthorblockNstyle \@IEEEcompsocnotconfonly \@IEEEauthorblockAstyle \@IEEEcompsocnotconfonly \@IEEEcompsocconfonly \@IEEEauthordefaulttextstyle \@IEEEcompsocnotconfonly \@IEEEauthor...

work page