{"id":"12277c7e-fbbf-4e4f-9c28-ad109955d55d","arxiv_id":"2505.02540","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"pFedLIA clusters federated learning clients using a cheap 'lazy influence' score, and on CIFAR10/100 and FashionMNIST it matches a perfect-clustering oracle while improving on existing personalized FL baselines by up to 17 percentage points.","lead":"This paper proposes pFedLIA, a federated learning method that groups clients by how much one client's partially trained model improves another client's validation loss, then trains one model per group. On image benchmarks it matches an oracle that knows the true client groups and beats several personalized learning baselines, with the largest gain on CIFAR100.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Cluster success hinges on the unvalidated warm-up operating point; LIA score is a gradient-alignment heuristic, not the exact influence claimed.","rationale":"The reader correctly identified Eq. (1) as the weakest assumption. I agree that the sign and ordering of the Lazy Influence score are the load-bearing condition for the central claim. However, I want to sharpen the concern: the score is not merely an unproven estimator of exact influence; in the actual pipeline it is not the exact influence at all, because theta0 is trained on all clients' data before clustering. The score is closer to a first-order gradient alignment between i's validation batch and j's local batch, and its informativeness depends on the warm-up model being far from converged. This makes the choice of clustering after round 20 and the choice of k=20 potentially decisive. The paper provides no ablation over these hyperparameters and no direct measurement of clustering quality, so the 'matches Oracle' result may be tied to a single favorable operating point. That said, the benchmark evidence in Table I is real and the method is clearly described; the concern does not warrant rejection, but it does justify the CONDITIONAL verdict. The proposed concrete test would settle whether the result is robust or hyperparameter-anchored.","tokens_in":12967,"tokens_out":10408,"duration_ms":139935,"concrete_test":"On the CIFAR10 pathological setting, where true cluster labels are available, compute the full I_LIA matrix and measure cluster recovery via adjusted Rand index and final test accuracy while varying the warm-up round in {5, 10, 20, 50, 100} and the number of local epochs k in {1, 5, 20, 50}, using three random seeds per configuration. If both ARI and final accuracy remain close to the Oracle across most settings, the round-20/k=20 choice is not load-bearing. If accuracy drops by more than a few points in any reasonable setting, the central claim should be narrowed to that operating point.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The weakest point is not just that Eq. (1) lacks a guarantee; it is that the quantity actually computed is not the influence defined in Section III-B. In the evaluation, theta0 is the FedAvg model after 20 rounds on all clients' data, so every client j's data is already present in theta0. The score I_LIA(i,j) = sum_{z in Zval_i} [L(z,theta0) - L(z,theta_j)] therefore measures the effect of a few extra local steps from a model that already contains j's contribution, not the effect of adding j's data to a model trained without it. By a first-order Taylor expansion, I_LIA(i,j) is approximately eta times the inner product of the gradient of i's validation loss and the gradient of j's training batch. That is a raw gradient-similarity heuristic. Its sign and ordering can be expected to be informative only while theta0 is far from a good fit for either client; as the warm-up model improves, the marginal effect of j's local update shrinks, so clustering success depends on the arbitrary choice of clustering after round 20 and on k=20. The paper runs no ablation over warm-up length or k and reports no cluster-purity measure, so the 'matches Oracle' result could be an artifact of that operating point rather than a robust property of LIA. This is load-bearing for the central claim, and it also limits transfer to a new heterogeneity regime.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes pFedLIA, a personalized federated learning method that clusters clients before model aggregation using a Lazy Influence Approximation (LIA). Each client j performs a small number of local epochs from a common reference model θ0 on a batch of its own data, sends the partially updated model to client i, and i scores j by the reduction in i's validation loss (Eq. 1). Clustering is performed once, either centrally with OPTICS or peer-to-peer with k-means, without requiring the number of clusters in advance; subsequent training uses any federated aggregator within each cluster. The method is evaluated on FashionMNIST, CIFAR10, CIFAR100 under pathological and noisy non-IID splits, and on a Scandinavian next-word prediction task with GPT-2. The headline claims are that pFedLIA matches the performance of an Oracle clustering and outperforms the best baseline by up to 17%.","tokens_in":13243,"tokens_out":4377,"duration_ms":54800,"significance":"If the central claim holds, the paper offers a simple and communication-efficient way to obtain personalized models in heterogeneous federated learning, with the practical advantage of a one-time clustering step and no need to know the number of clusters. The evaluation spans multiple datasets and modalities, includes a real-world language task, and compares against several established baselines with mean and standard deviation reported over four runs. The main weakness is that the core quantity, the Lazy Influence score, is not validated as a faithful proxy for the exact influence defined in Section III-B, and the headline Oracle-matching result is demonstrated only in the clean pathological setting. These issues are load-bearing for the stated contributions.","major_comments":[{"comment":"The quantity computed in the evaluation is not the influence defined in Section III-B. In the experiments, θ0 is the FedAvg model after 20 communication rounds on all clients' data, so every client's data already contributes to θ0. Thus I_LIA(i,j) measures the effect of a few extra local steps from a model that already contains j's contribution, not the marginal effect of adding j's data to a model trained without it. A first-order expansion shows I_LIA(i,j) is approximately a gradient-alignment score (an inner product of the validation gradient and the training-batch gradient). The paper provides no ablation over the warm-up length (round 20) or the number of local epochs k, and no cluster-purity or score-fidelity metric. The 'matches Oracle' result could therefore be an artifact of a specific operating point rather than a robust property of the proposed approximation. This directly affects the central claim and should be addressed with a calibration experiment, an ablation, or a theoretical condition under which the score preserves the sign and ordering of the exact influence.","section":"§III-D, Eq. (1), and §IV-D"},{"comment":"The claim that pFedLIA 'matches the performance of the Oracle clustering' is supported only for the pathological non-IID setting, where labels are exclusively assigned to clusters and perfect recovery is expected. The Oracle row is absent for the noisy non-IID setting, so the claim is not established there. In noisy FashionMNIST, pFedLIA is actually worse than FedFomo (85.11% vs. 82.10%) and FedProto (87.39% vs. 82.10%), which conflicts with the unqualified wording of the abstract and conclusion. The large standard deviations, e.g., 57.09% ± 13.32% on noisy CIFAR100, also make the 17% improvement fragile. Please qualify the Oracle-matching claim to the setting where an Oracle is defined and add the missing comparison or a clear statement of scope.","section":"Table I and Abstract"},{"comment":"The core assumption of the method is that a small number of noisy local updates from θ0, combined with a small validation set, preserves the sign and ordering of the true influence of one client's data on another. The paper states that 'we only need an accurate enough estimate of its direction' but provides no theoretical guarantee and no empirical validation of this property. Figure 3a compares LIA with exact influence only in computation time, not in the fidelity of the resulting scores. A calibration experiment on a small problem where exact influence is tractable, or a study of cluster purity under varying k and warm-up length, is necessary to establish that the clustering signal is actually the influence signal claimed.","section":"§III-D"}],"minor_comments":[{"comment":"The noisy non-IID description says 'we opted to only add 1' random label per client but then states that CIFAR100 assigns five random labels instead; please clarify the protocol used for each dataset.","section":"§IV-B"},{"comment":"The method description says the warm-up model M0 'can be trained, e.g., by the center C using a small set of warm-up data,' while the evaluation trains it via FedAvg on all clients' data for 20 rounds. Please specify which protocol the reported results use and discuss whether the warm-up model must exclude target data for the influence interpretation to hold.","section":"§III-D vs. §IV-D"},{"comment":"The exact-influence comparison uses three convergence thresholds with no explanation of how they were chosen; adding the threshold values and error bars would make the speedup claim more reproducible.","section":"Figure 3a"},{"comment":"Reporting four runs without statistical tests is insufficient given the large standard deviations in Table I; consider reporting confidence intervals or performing a significance test for the headline improvements.","section":"§IV-D"},{"comment":"The phrase 'an improvement of 17% on CIFAR100' refers to the noisy setting, while the pathological CIFAR100 improvement is 16.93% over FeSEM; please identify the exact comparison in the text to avoid ambiguity.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":"The paper relies on the authors' own prior LIA paper [43] for the load-bearing approximation component; the novelty overlap should be clarified in the revised version. The current evidence for the Oracle-matching claim is narrower than the abstract suggests, and the lack of any fidelity check on the LIA score is a substantive correctness risk rather than a presentation issue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know about arXiv:2505.02540. First, pFedLIA is a clean idea: compute a cheap influence score between every pair of clients after a short FedAvg warm-up, cluster with OPTICS or k-means, then train per cluster. The paper's benchmark results are decent, and it matches the Oracle in the pathological label-exclusive setting, which is exactly the setting where any gradient-similarity method should work. Second, the load-bearing assumption is that the LIA score, computed from a few local epochs on top of a global model that already contains every client's data, reliably orders clients by distributional similarity. The paper doesn't test that assumption, and the 'matches Oracle' result could be an artifact of the chosen warm-up length and k=20.\n\nThe novelty is moderate: LIA comes from the authors' prior work, and clustering-based PFL is not new. The extension to one-time clustering with no a priori cluster count is useful, and the method is clearly described. The evaluation on four datasets with two heterogeneity types and several baselines is solid, and the 17% gain on CIFAR100 in the noisy setting is real if you keep the variance in mind.\n\nThe soft spots are real. Eq. (1) is not the influence defined in Section III-B; it is, to first order, an inner product of the gradient of i's validation loss and the gradient of j's training batch. That is a reasonable heuristic, but the paper calls it an approximation to exact influence without noting that the warm-up model already includes j's contribution. No ablations over warm-up length, k, or learning rate; no cluster purity measure; no Oracle in the noisy setting. The communication story is also oversold: every client sends a partial model to every other client, so one-time clustering is O(N^2) in model transfers. That is fine for small N, but the paper claims communication efficiency without qualification. The Nordic experiment is only a loss curve; no quantitative comparison. No code or supplementary material is attached despite pointers to it.\n\nWho is this for? People working on personalized FL will find the method interesting, but they should treat the Oracle-matching claim as conditional. A serious referee should see this, but the paper needs major revision with ablations and a more honest discussion of what LIA measures. I'd bring it to our reading group to debate whether the heuristic can be trusted.","headline":"Plausible clustering method for personalized FL whose Oracle-matching claim rests on an unvalidated warm-up operating point.","tokens_in":13774,"tokens_out":5583,"would_cite":true,"duration_ms":65663,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A lazy influence score clusters federated clients with oracle-level accuracy.","keywords":["personalized federated learning","client clustering","lazy influence approximation","non-IID data","label heterogeneity","feature heterogeneity","peer-to-peer learning","next-word prediction"],"falsifier":"Take two clusters with disjoint label sets, compute the LIA matrix with the paper's recommended short run, and check whether every within-cluster score exceeds every between-cluster score. A single inversion of that ordering, visible as a positive $I_{LIA}$ between clients from different clusters, would break the clustering step and prevent the method from matching the oracle.","tokens_in":12739,"feed_emoji":"🧩","tokens_out":6337,"duration_ms":69910,"temperature":0.7,"pith_summary":"The paper tries to establish that a cheap, one-time estimate of how much each client's data helps every other client's validation loss is enough to partition federated-learning clients into groups that should train together, without knowing how many groups exist. This matters because a single global model trained over non-IID client data performs poorly for individual users, while per-client models lose the benefit of collaboration. pFedLIA computes a 'lazy influence' score for every pair of clients, clusters with OPTICS in the centralized case or with k-means in the peer-to-peer case, and then runs any federated aggregator inside each cluster. The paper reports that this matches the accuracy of an oracle cluster assignment and beats the best baseline by up to 17% on CIFAR100.","feed_headline":"Cheap influence scores cluster federated clients as well as an oracle","feed_subtitle":"A one-time clustering step recovers accuracy lost to skewed client data, even without knowing cluster count.","key_machinery":"The Lazy Influence Approximation (LIA) is the mechanism: a pairwise score between clients defined as the reduction in client $i$'s validation loss when the shared warm-up model is moved by a few local training steps on client $j$'s data. It is a deliberately inaccurate but cheap proxy for exact influence, meant to preserve only the direction and ordering of true influence. Clustering on this matrix, either centrally with OPTICS, which needs no target cluster count in advance, or per-client with a two-cluster k-means split, is what converts the scores into personalized training groups.","core_discovery":"The central claim is that the sign and ordering of $I_{LIA}(i,j) = \\sum_{z \\in Z^{\\mathrm{val}}_i} L(z,\\theta_0) - L(z,\\theta_j)$ carry enough distributional signal to recover the performance lost to data heterogeneity. Here $\\theta_j$ is obtained by a small number of local training epochs on client $j$'s batch starting from a shared warm-up model $\\theta_0$, and $Z^{\\mathrm{val}}_i$ is client $i$'s validation set. The paper argues that exact influence is too costly and that first-order influence approximations require Hessian operations that are impractical for modern models, whereas this lazy score needs only a few local updates and one validation pass. It then shows that clustering once on this score, before any cluster-specific training, matches a hypothetical Oracle clustering and outperforms existing clustering and personalization baselines across pathological and noisy non-IID splits of image benchmarks and a Nordic-language next-word prediction task.","pith_inferences":["An extension the authors leave implicit: if the LIA ordering remains stable under distribution shift, the same one-time clustering could be re-run periodically, turning pFedLIA into a lightweight adaptive personalization scheme.","Because the score is computed from a small validation set, clients with very few labeled validation samples are a risk case; a testable extension is to measure how cluster quality degrades as validation set size shrinks.","The paper demonstrates the method with FedAvg as the within-cluster aggregator; nothing in the argument prevents combining the same clusters with robust aggregation methods, which could additionally filter malicious or outlier clients.","Another inference: the lazy score only needs the sign and ordering of influence, so the method's failure mode is not miscalibrated loss values but inconsistent ranking across initializations; a stress test would randomize the warm-up model and check cluster stability."],"forward_implications":["Within a small number of communication rounds, pFedLIA can switch from a single global model to cluster-specific models, so personalization does not require iterative re-clustering.","Because the clustering step needs no prior knowledge of the number of clusters, the method applies to federations whose client distribution is unknown in structure.","The same lazy score can be computed in a peer-to-peer setting, so each client can choose its own aggregation partners rather than relying on a central server.","The reported 40x to 500x speedup over exact influence makes the method practical for large models while preserving most of the personalization benefit.","On the pathological and noisy non-IID benchmarks, the method matches Oracle accuracy and outperforms the strongest baseline by up to 17%, demonstrating that a one-time clustering step can recover much of the performance lost to heterogeneity."],"supporting_citations":[{"why":"Supplies the Lazy Influence Approximation that the method adapts into a pairwise client-clustering score.","marker":"[43]"},{"why":"Provides OPTICS, the density-based clustering algorithm used to form client clusters without knowing the cluster count in advance.","marker":"[1]"},{"why":"Defines FedAvg, the aggregation rule used for the warm-up phase and for per-cluster training, and the main single-global-model baseline.","marker":"[38]"},{"why":"Is the gradient-similarity clustering baseline (FeSEM) that the method outperforms, and it defines the multi-center clustering comparison.","marker":"[35]"},{"why":"Is the FedFomo personalization baseline that the method is compared against and beats on most settings.","marker":"[61]"},{"why":"Is the CFL iterative clustering baseline, which splits clusters dynamically and serves as the main iterative alternative.","marker":"[15]"},{"why":"Provides the OSCAR corpus used for the real-data next-word prediction motivating experiment.","marker":"[40]"},{"why":"Defines the approximate influence-function method whose Hessian-based cost the paper argues is ill-matched to federated learning, motivating the lazy alternative.","marker":"[27]"}],"fun_headline_variants":["Lazy influence score clusters clients as well as oracle","Cheap lazy-influence clustering recovers non-IID accuracy","One lazy pass clusters heterogeneous clients like an oracle","Lazy influence: efficient clustering for personalized federated learning","Clustering with lazy influence matches oracle, boosts CIFAR100 by 17%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the small loss difference observed after a few local training steps from a shared starting model ranks other clients by how much their data genuinely helps the client's task; if that ordering is an optimization artifact, the clusters will be wrong.","fun_headline_variants_meta":{"raw":{"variants":["Lazy influence score clusters clients as well as oracle","Cheap lazy-influence clustering recovers non-IID accuracy","One lazy pass clusters heterogeneous clients like an oracle","Lazy influence: efficient clustering for personalized federated learning","Clustering with lazy influence matches oracle, boosts CIFAR100 by 17%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000735,"raw_usage":{"total_tokens":3293,"prompt_tokens":958,"completion_tokens":2335,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":574,"completion_tokens_details":{"reasoning_tokens":2250}},"tokens_in":574,"tokens_out":2335,"duration_ms":16427,"temperature":1.0,"reasoning_tokens":2250,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T00:47:56.725351+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take two clusters with disjoint label sets, compute the LIA matrix with the paper's recommended short run, and check whether every within-cluster score exceeds every between-cluster score. A single inversion of that ordering, visible as a positive $I_{LIA}$ between clients from different clusters, would break the clustering step and prevent the method from matching the oracle.","supporting_citations":[{"cited_title":"Lia: Privacy-preserving data quality evaluation in fed- erated learning using a lazy influence approximation","cited_arxiv_id":null,"evidence_quote":"Supplies the Lazy Influence Approximation that the method adapts into a pairwise client-clustering score."},{"cited_title":"Optics: Ordering points to identify the clustering structure","cited_arxiv_id":null,"evidence_quote":"Provides OPTICS, the density-based clustering algorithm used to form client clusters without knowing the cluster count in advance."},{"cited_title":"Multi-center federated learning: clients clustering for better personalization","cited_arxiv_id":null,"evidence_quote":"Is the gradient-similarity clustering baseline (FeSEM) that the method outperforms, and it defines the multi-center clustering comparison."},{"cited_title":"A monolingual approach to contextualized word embeddings for mid- resource languages","cited_arxiv_id":null,"evidence_quote":"Provides the OSCAR corpus used for the real-data next-word prediction motivating experiment."},{"cited_title":"Understanding black-box predictions via influence functions","cited_arxiv_id":null,"evidence_quote":"Defines the approximate influence-function method whose Hessian-based cost the paper argues is ill-matched to federated learning, motivating the lazy alternative."}],"review_version":1}