REVIEW 5 major objections 4 minor 17 references
Interaction-Aware Gaussian Weighting for Clustered Federated Learning
T0 review · 5 major / 4 minor · reviewed 2026-08-09 · deepseek-v4-flash
Pith's one-line read FedGWC claims that grouping federated clients by the similarity of their loss trajectories, via Gaussian rewards on the empirical loss process, yields homogeneous clusters and better classification accuracy than existing…
desk verdict A clean new clustering algorithm for FL with real domain-separation results, but the class-distribution homogeneity claim is under-supported because the evaluation metric is blind to label identity. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The engine is the Gaussian reward combined with a running-average interaction matrix. At each round, the server forms the mean loss $m_{t,s}$ and sample variance $(\sigma_{t,s})^2$ across sampled clients, rewards each client with a Gaussian kernel $r^{t,s}_k$, averages over the $S$ local steps to get $\omega^t_k$, and updates the client weight $\gamma^{t+1}_k = (1-\alpha_t)\gamma^t_k + \alpha_t \omega^t_k$ and the interaction matrix $P^{t+1}_{kj} = (1-\alpha_t)P^t_{kj} + \alpha_t\omega^t_k$ for co-sampled clients. The weight update is a Robbins-Monro stochastic approximation, which the paper uses to prove convergence of $\Gamma^k_t$ to $\mu_k$ (the expected reward) almost surely under an $\ell^2/\ell^1$ step-size condition and in expectation for constant $\alpha$, plus a variance-reduction bound. The interaction matrix is then transformed into an affinity matrix via unbiased perception vectors and an RBF kernel, and spectral clustering over this affinity matrix produces the client partition.
What would settle it
Run FedGWC on synthetic federations with known per-client class distributions and compare the within-cluster Wasserstein distance of the detected clusters to that of random same-size partitions: if the clustered distances are not clearly lower, loss trajectories are not tracking distribution similarity, and the central claim collapses.
Extended reading notes
Core claim
The central discovery is that pairwise client similarity can be estimated from the empirical loss process alone, without gradients or data: rewards defined as $r^{t,s}_k = \exp\left(-(l^{t,s}_k - m_{t,s})^2 / (2(\sigma_{t,s})^2)\right)$ measure how close client $k$'s loss at round $t$, local step $s$, stays to the mean loss of the sampled clients, and running averages of these rewards converge (as $t\to\infty$) to the client's theoretical expected reward, with strictly smaller variance. The paper then organizes these estimates in an interaction matrix $P$, symmetrizes it through unbiased perception vectors and an RBF kernel into an affinity matrix $W$, and applies spectral clustering; a Davies-Bouldin-based rule decides whether to split a cluster, so the number of clusters is detected rather than fixed. Experiments reported on Cifar100, Femnist, Google Landmarks, and iNaturalist show accuracy gains (for example, 53.4 vs. 41.6 balanced accuracy versus FedAvg on Cifar100) and better separation of clients by heterogeneity level and visual domain.
Load-bearing premise
The load-bearing premise is that clients with similar data distributions produce similar loss trajectories, so loss similarity can stand in for the unseen data-distribution similarity that clustering actually wants.
Editorial extensions
If this is right
- Automatic cluster detection: FedGWC decides the number of clusters from the data via the Davies-Bouldin score, removing the preset cluster count that IFCA, FeSEM, and similar methods require.
- Plug-in personalization: FedGWC sits on top of any existing FL aggregator; the reported tables show FedAvg, FedAvgM, and FedProx all gain accuracy in heterogeneous settings when run per cluster.
- Out-of-distribution detection: experiments that separate clean, noisy, and blurred domains with near-perfect Rand Index suggest the same machinery can flag anomalous or malicious clients by their loss behavior.
- A fairer clustering benchmark: the Wasserstein Adjusted Score gives class-imbalance-aware cluster-quality numbers, so future FL clustering methods can be compared on distributional cohesion rather than accuracy alone.
Reading between the lines
- Because the server only needs scalar loss traces, FedGWC's privacy story is stronger than gradient-based clustering; a natural next test is whether the loss trajectories themselves leak class information under membership inference, which the paper does not analyze.
- The same loss-reward mechanism could be reused outside federated learning, for example to partition tasks among agents with different difficulty regimes in decentralized multi-task learning.
- The convergence theorems assume the reward process is stationary, but during real training losses drift downward; a testable extension is to analyze whether the stationary assumption holds under the paper's 'single cluster initially, then split' schedule, or whether the weights need a forgetting factor.
- A direct empirical check of the core premise would be to measure the correlation between the learned interaction matrix $P$ and the true Wasserstein distances between clients' class distributions, a number the paper does not report directly.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes FedGWC, a clustered federated learning algorithm that groups clients using a Gaussian reward mechanism applied to scalar loss trajectories. At each round, a client's reward is computed from proximity of its loss process to the mean loss process of sampled clients; the reward is averaged over local iterations and updated with an exponential running average. These weights populate an interaction matrix from which an RBF affinity matrix is built and spectral clustering is applied recursively, with the cluster count selected by a Davies-Bouldin criterion. The paper also introduces a Wasserstein Adjusted Score for evaluating cluster cohesion under class imbalance. Experiments cover Cifar100, Femnist, Google Landmarks, and iNaturalist, comparing FedGWC against IFCA, FeSEM, CFL, FedAvg, FedAvgM, FedProx, and FairAvg, and claiming superior accuracy and cluster quality.
Significance. The design is attractive for cross-device FL: clients communicate only scalar losses, the number of clusters is selected automatically, and the method is orthogonal to the aggregation rule. The paper ships a reasonably complete experimental campaign spanning small, large-scale, class-imbalanced, and domain-shift settings. If the central claim about distributional homogeneity were established, the contribution would be useful to the clustered-FL community. However, the evidence as written does not support the strong claims made in the abstract and contributions: Table 1 contradicts the accuracy claim on Femnist, the proposed WAS metric is invariant to class-label permutations and therefore cannot certify distributional homogeneity, and the theoretical results reduce to convergence of a running average to its mean. These are load-bearing gaps rather than presentation issues.
major comments (5)
- [§7.1, Table 1 and Table 9] The central empirical claim that FedGWC outperforms existing FL algorithms is not supported on Femnist. In Table 1, FedGWC achieves 76.1±0.1 on Femnist, while FedAvgM reaches 83.3±0.3 and FedAvg reaches 76.6±0.1; on Cifar100 FedGWC ties FeSem at 53.4±0.4 vs 53.4±1.8 rather than clearly outperforming it. Table 9 further shows FedGWC+FedAvgM on Femnist gives 83.3±0.4, i.e. no improvement over no clustering. The text in §7.1 concedes that clustering-based methods perform worse on Femnist, which is in tension with the unqualified claim in the abstract and contributions. The claim should be scoped to the settings where the data support it, or additional evidence should be added.
- [§6, Eq. (7), and Appendix B] The Wasserstein Adjusted Score is computed on rank-ordered class frequency vectors, making it invariant under permutations of class labels. Consequently, a client holding only class A and a client holding only class B have identical ranked frequency vectors, so their WAS distance is exactly zero. The high WAS and low WADB values reported in Tables 1 and 3 therefore do not establish that FedGWC groups clients by shared class identity; they at most measure similarity of imbalance skewness. This is a load-bearing limitation for the paper's cluster-quality validation and should be addressed directly, for example by also reporting label-aware metrics or by testing on label-permuted synthetic partitions.
- [§4.1, Eq. (1)] The clustering signal is a proximity-to-mean measure on scalar loss trajectories, not a measure of class identity. Two clients with disjoint but equally learnable class distributions can produce nearly identical loss trajectories, receive similar Gaussian rewards, and be placed in the same cluster even when their data distributions are maximally different. The paper's premise in §1 that similar data distributions produce similar loss landscapes is an assumption, not an established fact, and the experiments do not test it separately from the reward construction. A concrete test would be a synthetic experiment in which class labels are permuted between two equal-size client groups; this would show whether FedGWC separates clients by data identity or only by loss-difficulty profile.
- [§5, Theorems 5.1 and 5.2, and Appendix A] The theoretical results are considerably weaker than the text suggests. Since Eq. (2) defines the Gaussian weight as an exponential running average of rewards whose expectation is µk, Theorems 5.1 and 5.2 essentially state that the average converges to its mean, which is the standard Robbins-Monro guarantee and does not depend on the specific Gaussian construction. The proofs do not analyze cluster recovery, identifiability of the interaction matrix, or consistency of the split decision. Calling this a rigorous framework that provides 'theoretical guarantees for our clustering approach' (§1, Contributions) overstates the scope; the theorems can at most support the claim that the weight estimates are consistent estimators of the expected reward.
- [§7.3, Table 4] The domain-detection success is only partial. In the (40, 30, 30) clean/noise/blur split of Cifar100, the ground truth has three domains, yet FedGWC detects four clusters and obtains a Rand Index of only 0.60, whereas the two-domain rows reach 1.0. The sentence 'Consistently FedGWC accurately distinguishes all visual domains' is contradicted by this row. Additionally, in Table 3, FedGWC reports C=5 on two-group Cifar100 splits (10,0,90) and (10,90,0), indicating that the automatic cluster-count selection can substantially over-split. These discrepancies should be discussed and either explained or used to temper the clustering-quality claims.
minor comments (4)
- [§4.2, Eq. (3)] The interaction-matrix update uses the same ωt_k for every co-sampled client j in Pt×Pt, so each row is constant across the sampled clients at a given round. This makes the raw update non-pairwise; the paper should clarify how the UPV and RBF stages convert this row-constant signal into a valid pairwise interaction, or justify why the simplification is harmless.
- [§7.1] The paragraph beginning 'In Table 1, we present a comparative analysis' is duplicated almost verbatim later in the same section; one copy should be removed.
- [§7.3] The word 'anaylsis' should be corrected to 'analysis', and the phrase 'Wasserstein's Adjusted Score' should be made consistent with the term 'Wasserstein Adjusted Score' used elsewhere.
- [Appendix D] The text refers to a baseline as 'e.g. FedA'; this appears to be a typo for 'FedAvg' and should be corrected.
Circularity Check
Convergence 'guarantee' tracks the estimator's own defined mean; WAS is a known identity renamed; benchmark results remain independent.
-
self definitional
[Section 5, Theorem 5.1 and the preceding paragraph (Eqs. 1, 2)]
"Due to the linearity of the expectation operator, the expected reward E[R^{t,s}_k] for the k-th client at round t, local iteration s equals the expected Gaussian reward E[Omega_t^k] that, to simplify the notation, we denote by mu_k. mu_k is the theoretical value that we aim to estimate by designing our Gaussian weights Gamma_t^k appropriately, as it encodes the ideal reward to quantify the closeness of the distribution of each client k to the main distribution. ... Theorem 5.1. ... Gamma_t^k -> mu_k a.s."
mu_k is defined as E[Omega_t^k], the expectation of the very reward sequence whose running average defines the weights in Eq. 2. Theorem 5.1's conclusion Gamma_t^k -> mu_k is therefore a Robbins-Monro consistency statement about the estimator converging to its own mean, not a result linking rewards to data distributions. The clause 'encodes the ideal reward to quantify the closeness of the distribution' imports the Section 1 hypothesis that similar loss processes imply similar distributions. The advertised 'theoretical guarantees for our clustering approach' thus reduce to the definition of mu_k.
-
renaming known result
[Section 6, Eq. 7; Appendix B, Theorem B.3]
"The class-adjusted clustering metric s-tilde is defined as the standard clustering metric s computed on the ranked frequency vectors x^k_(i). ... As we show in Appendix B, this adjustment is mathematically equivalent to assessing the dispersion between the empirical class probability distributions of different clients using the Wasserstein distance."
Eq. 7 defines the pairwise distance as the L2 distance between rank-ordered class-frequency vectors. For equal-weight empirical measures on the line, the Wasserstein distance is exactly the L2 distance between sorted samples, a classical identity. Theorem B.3 re-derives this identity and presents the result as a novel 'Wasserstein Adjusted Score.' The 'derivation' is the definition; the metric is a known distance renamed. It is also label-permutation invariant (only frequency magnitudes are ranked), which is a validity caveat for using it to 'prove' cluster homogeneity, though not itself a circular step.
full rationale
The central benchmark evidence is not circular: FedGWC is compared against external baselines on held-out test accuracy and, in Table 4, on Rand Index against ground-truth domain labels, and those results do not depend on the paper's own definitions. The theoretical section, however, contains one self-definitional step: the reward r^{t,s}_k (Eq. 1) is a Gaussian kernel on loss proximity to the sampled mean, mu_k is defined as the expectation of the averaged reward, and Theorems 5.1 and 5.2 prove that the running average Gamma_t^k converges to mu_k. This is a valid stochastic-approximation consistency result, but it is a statement about the estimator converging to its own mean; the paper then interprets mu_k as 'the ideal reward to quantify the closeness of the distribution' and lists 'theoretical guarantees for our clustering approach' as a contribution. That interpretation is the Section 1 hypothesis, not a consequence of the theorem. Separately, the Wasserstein Adjusted Score is introduced by defining the distance on ranked class-frequency vectors and then proved equivalent to the 1D Wasserstein distance; this is a known identity presented as a new metric. Neither issue affects the external benchmark comparisons, so the paper is only partially circular.
Assumptions & free parameters
free parameters (2)
- beta (RBF kernel spread) =
0.5 for Cifar100, Landmarks, iNaturalist; tuned over {0.1, 0.5, 1.0, 2.0, 4.0}
- epsilon (convergence threshold for clustering) =
1e-5 for Cifar10/100 and Femnist, 1e-4 for Landmarks, 1e-2 for iNaturalist
assumptions (4)
- standard math Robbins-Monro stochastic approximation convergence: for alpha_t in l2/l1, a running average converges in L2 and almost surely to the expectation of its observations.
- domain assumption The empirical loss process is stationary by construction, so the expected Gaussian reward mu_k does not depend on t.
- domain assumption Clients with similar data distributions exhibit similar loss landscapes.
- standard math The p-Wasserstein distance between empirical class-frequency measures is computed by sorting the frequencies.
Cite this review
Pith. "Pith review of Interaction-Aware Gaussian Weighting for Clustered Federated Learning." pith.science (2026). https://pith.science/paper/6A4GPJWO
@misc{pith2026250203340,
author = {Pith},
title = {Pith review of: Interaction-Aware Gaussian Weighting for Clustered Federated Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/6A4GPJWO}},
note = {Machine review of arXiv:2502.03340}
}
read the original abstract
Federated Learning (FL) emerged as a decentralized paradigm to train models while preserving privacy. However, conventional FL struggles with data heterogeneity and class imbalance, which degrade model performance. Clustered FL balances personalization and decentralized training by grouping clients with analogous data distributions, enabling improved accuracy while adhering to privacy constraints. This approach effectively mitigates the adverse impact of heterogeneity in FL. In this work, we propose a novel clustered FL method, FedGWC (Federated Gaussian Weighting Clustering), which groups clients based on their data distribution, allowing training of a more robust and personalized model on the identified clusters. FedGWC identifies homogeneous clusters by transforming individual empirical losses to model client interactions with a Gaussian reward mechanism. Additionally, we introduce the Wasserstein Adjusted Score, a new clustering metric for FL to evaluate cluster cohesion with respect to the individual class distribution. Our experiments on benchmark datasets show that FedGWC outperforms existing FL algorithms in cluster quality and classification accuracy, validating the efficacy of our approach.
Figures
Figures from the paper (5 more)
Reference graph
Works this paper leans on
-
[1]
Notice that, due to the linearity of the expectation operator, E[Ωt k] = µk, that is the true, unknown, expected reward. The observed value for the random variable is given by ωt k = 1/S PS s=1 rt,s k , which is sampled from a distribution centered on µk. Each client’s weight is updated according to γt+1 k = (1 − αt)γt + αtωt k . (9) Since such an estimat...
work page 1997
-
[2]
C. Privacy of FedGWC In the framework of FedGWC, clients are required to send only the empirical loss vectors lt,s k to the server (Cho et al., 2022). While concerns might arise regarding the potential leakage of sensitive information from sharing this data, it is important to clarify that the server only needs to access aggregated statistics, working on ...
work page 2022
-
[5]
A sensitivity analysis on the RBF kernel hyper-parameter β is conducted. We present the balanced accuracy for FedGWC on the Cifar10, Cifar100, and Femnist datasets for β ∈ {0.1, 0.5, 1.0, 2.0, 4.0}. It is noteworthy that FedGWC demonstrates robustness to variations in this hyperparameter. β Cifar100 Femnist Google Landmarks iNaturalist 0.1 49.9 76.0 55.0 ...
work page 2009
-
[7]
Hsu, T.-M. H., Qi, H., and Brown, M. Federated visual classification with real-world data distribution. In Com- puter Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part X 16, pp. 76–92. Springer,
work page 2020
-
[10]
Federated learning with non-iid data.arXiv preprint arXiv:1806.00582,
Zhao, Y ., Li, M., Lai, L., Suda, N., Civin, D., and Chandra, V . Federated learning with non-iid data.arXiv preprint arXiv:1806.00582,
-
[12]
We recall, from Eq.3, that for any couple of clients k, j∈ Pt the entries of the interaction matrix are updated according to P t+1 kj = (1 − α)P t kj + αωt k . (21) If we iterate backward until P 0 kj, we obtain the following update P t+1 kj = (1 − α)t+1P 0 kj + tX τ =0 α(1 − α)τ ωt−τ k . (22) We know that, by constructions, the Gaussian rewards ωt k < 1 ...
work page 1942
-
[14]
with K = 2714 clients, as partitioned in (Hsu et al., 2020). For Landmarks and iNaturalist, we always refer to the Landmark-Users-160K and iNaturalist-Users-120K partition, respectively. The classification model is MobileNetV2 architecture (Sandler et al.,
work page 2020
-
[15]
optimized with SGD having learning rate of 0.1. To mimic real world low client availability we employed 10 sampled clients per communication round, with a total training of 1000 and 2000 communication rounds, with 7 and 5 batch iterations respectively. ForFedGWC we tuned the hyper-parameter β ∈ {0.1, 0.5, 1, 2, 4}, i.e. the spread of the RBF kernel, and w...
work page 2000
Show all 17 references
-
[17]
It utilizes the Rand Index score (Rand, 1971), where a value close to 1 represents a perfect match between clustering and labels
Clustering performance of FedGWC is assessed on federations with clients from varied domains using clean, noisy, and blurred (Clean, Noise, Blur) images from Cifar10 dataset. It utilizes the Rand Index score (Rand, 1971), where a value close to 1 represents a perfect match bet...
1971
-
[1942]
P., Jaggi, M., Kale, S., Mohri, M., Reddi, S
Karimireddy, S. P., Jaggi, M., Kale, S., Mohri, M., Reddi, S. J., Stich, S. U., and Suresh, A. T. Mime: Mimicking centralized stochastic algorithms in federated learning. arXiv preprint arXiv:2008.03606, 2020a. Karimireddy, S. P., Kale, S., Mohri, M., Reddi, S., Stich, S., and...
2008 arXiv
-
[1979]
Imagenet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248–255. Ieee,
2009
-
[1998]
and Wang, J
Li, D. and Wang, J. Fedmd: Heterogenous feder- ated learning via model distillation. arXiv preprint arXiv:1910.03581,
1910 arXiv
-
[2009]
Fedgroup: Efficient federated learning via decom- posed similarity-based clustering
Duan, M., Liu, D., Ji, X., Liu, R., Liang, L., Chen, X., and Tan, Y . Fedgroup: Efficient federated learning via decom- posed similarity-based clustering. In 2021 IEEE Intl Conf on Parallel & Distributed Processing with Applications, Big Data & Cloud Computing, Sustainable Com...
2021
-
[2016]
Federated learning with autotuned communication-efficient secure aggregation
Bonawitz, K., Salehi, F., Kone ˇcn`y, J., McMahan, B., and Gruteser, M. Federated learning with autotuned communication-efficient secure aggregation. In 2019 53rd Asilomar Conference on Signals, Systems, and Com- puters, pp. 1222–1226. IEEE,
2019
-
[2018]
Cao, X., Fang, M., Liu, J., and Gong, N. Z. Fltrust: Byzantine-robust federated learning via trust bootstrap- ping. arXiv preprint arXiv:2012.13995,
2012 arXiv
-
[2020]
B., Patel, S., Ramage, D., Segal, A., and Seth, K
Bonawitz, K., Ivanov, V ., Kreuter, B., Marcedone, A., McMahan, H. B., Patel, S., Ramage, D., Segal, A., and Seth, K. Practical secure aggregation for federated learn- ing on user-held data. arXiv preprint arXiv:1611.04482,
-
[2022]
Caldas, S., Duddu, S. M. K., Wu, P., Li, T., Kone ˇcn`y, J., McMahan, H. B., Smith, V ., and Talwalkar, A. Leaf: A benchmark for federated settings. arXiv preprint arXiv:1812.01097,
Reviewed August 9, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.