{"id":"6436c799-ec98-4377-9085-43070b441c07","arxiv_id":"2412.18845","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":7,"one_line_summary":"FedGCF fuses clustered structural models and selected node-feature models with a bandit-tuned ratio, claiming accuracy and communication improvements in federated graph classification, though its test-set-based tuning undermines the evaluation.","lead":"This paper presents FedGCF, a federated graph learning method that combines structural and node-feature information from distributed graphs using an adaptively tuned ratio. The authors report accuracy gains of 4.94% to 7.24% and lower communication costs, but the evaluation relies on test-set access that may not exist in real federated settings.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"MAB reward in Eq. (6) uses the test set to choose the fusion ratio, and the same test set is then reported as final accuracy; the claimed gains may be test-set overfitting.","rationale":"The load-bearing condition for the central claim is that the reported accuracy gains reflect better learning, not extra access to labels. That condition fails as written. Section V-B (Performance Metrics) says the global model is evaluated on the test set every round, and Algorithm 2 plus Eq. (6) feed that test accuracy into the MAB reward that selects the fusion ratio. So the test set is not a held-out evaluation set; it is part of the training/selection loop. This makes the comparison to baselines (which do not use test feedback) unfair and the final Table II/III numbers untrustworthy as generalization estimates. I agree with the reader's weakest_assumption on this point. The check is straightforward: move the MAB reward to a validation split, report test accuracy only at the end. If the advantage persists, the core algorithm may still be sound; if it disappears, the paper's main numerical claims are artifacts. Secondary issues (unstated baseline architectures, no error bars) reinforce REJECT but are not needed for the verdict. The internal evaluation flaw is sufficient; no need to challenge the method's novelty or consistency beyond this.","tokens_in":21462,"tokens_out":5269,"duration_ms":48496,"concrete_test":"Re-run all experiments with the MAB reward in Eq. (6) computed from a validation split (or from client-side validation accuracy) instead of the test set; freeze the test set and evaluate each method only once after 200 rounds. Compare FedGCF to FedStar, GCFL, and FedAvg on the same held-out test set. If FedGCF retains a 4.94%-7.24% advantage, the test-leakage objection fails; if the advantage collapses toward the fixed-ratio EF variant (Table IV), the headline gains are test-set artifacts.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that FedGCF's adaptive fusion ratio improves accuracy without seeing test labels. Section V-B states: 'In each round, we will evaluate the global model on the test dataset and record the test accuracy.' Eq. (6) then defines the MAB reward using r_t, the test accuracy in the current round, and r_b, the best historical test accuracy. Algorithm 2 uses this reward to select the structural/node-feature fusion ratio. Thus the server either needs access to labeled test data each round (not stated, and inconsistent with the privacy-preserving FGL premise), or the same test set used for final evaluation is also used to tune the fusion ratio. Either way, the reported accuracies in Tables II and III are not an unbiased measure of generalization: FedGCF is effectively selected on the test set while the baselines are not. The gains of 4.94%-7.24% could therefore be an artifact of test-set feedback rather than of the proposed structural/node-feature fusion. The fix is to compute the reward from a validation split and hold out the test set until after training; if the gains persist under that protocol, the concern is resolved, but as written the evaluation does not support the headline claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FedGCF, a federated graph learning framework that extracts two kinds of graph characteristics separately—structural properties, via structural models clustered by client similarity, and node features, via a client topology and selection of common-feature clients—and then fuses the resulting structural and node models with an adaptively chosen ratio. The ratio is selected by a Multi-Armed Bandit algorithm (GCF). The authors report accuracy gains of 4.94%–7.24% over baselines on Small Molecules, Social Networks, and MIX datasets, and communication-cost reductions of 64.18%–81.25% to reach target accuracy, alongside ablations and scalability experiments.","tokens_in":21742,"tokens_out":5022,"duration_ms":50493,"significance":"The underlying idea—explicitly separating structural and node-feature information and adaptively fusing them under non-IID federated graph distributions—is well motivated and addresses a real limitation of existing FGL methods that emphasize only one type of characteristic. The component-wise ablation in Table IV and Fig. 8 is a useful contribution and suggests the framework's components each matter. However, the central evaluation is compromised by a methodological flaw: the MAB reward in Eq. (6) is computed from test accuracy, and the same test set is used to report the final accuracies in Tables II and III. This makes the headline accuracy and communication-cost claims optimistic and not an unbiased measure of generalization. The paper also does not report variances or specify several important hyperparameters and model-architecture details. If the evaluation is redone with a validation-based reward and a strictly held-out test set, the approach may well show value, but the current manuscript does not establish its central claim.","major_comments":[{"comment":"The reward function in Eq. (6) uses r_t, defined as the test accuracy in the current round, and r_b, the best historical test accuracy, and Algorithm 2 uses this reward to select the structural/node-feature fusion ratio. Section V-B states explicitly that \"in each round, we will evaluate the global model on the test dataset.\" Since the final accuracies in Tables II and III are also computed on the test set, the fusion ratio is effectively tuned on the test set, while the baselines receive no such feedback. The reported 4.94%–7.24% gains may therefore be an artifact of test-set feedback rather than of the proposed fusion mechanism. The evaluation must be changed so that the reward is computed on a validation split and the test set is used only once after training is complete; the communication-cost comparisons in Figs. 3–6 should be rerun under the same protocol.","section":"V-B and Eq. (6)"},{"comment":"The proposed algorithm assumes the parameter server has access to a labeled test set in every round. This is not stated in the problem formulation and is inconsistent with the paper's privacy-preserving federated learning premise, where clients do not share their local graph data. If a public labeled test set is assumed, the assumption must be stated explicitly and justified; otherwise, Algorithm 2 cannot compute the reward defined in Eq. (6), and the reported accuracies are not achievable in the claimed setting. A validation split available to the server would resolve both this issue and the circularity concern.","section":"V-B, Eq. (6), and Algorithm 2"},{"comment":"The paper reports that it averages over \"five replicate experiments,\" but Table II and Table III report only mean accuracy with no standard deviations, confidence intervals, or significance tests. Several comparisons are close (e.g., FedStar at 74.69% vs. FedGCF at 76.38% on Small Molecules), and without variance information it is impossible to judge whether the reported improvements are statistically reliable. Given that the fusion ratio is selected using test-set feedback, this lack of variance reporting makes the headline gains even harder to interpret.","section":"Table II and Section V-C"},{"comment":"The communication-cost comparison is not sufficiently specified. FedGCF trains and uploads two models per client (a local model and a structural model), while the baseline architectures are not fully described beyond a shared hidden size of 64. It is unclear whether the reported communication cost for FedGCF includes both uploaded models and whether the baselines use the same architecture (GCN, GIN, or both). Without this information, the claimed communication-cost reduction of 64.18%–81.25% cannot be fairly assessed.","section":"Section V-A and Fig. 3-6"}],"minor_comments":[{"comment":"The UCB-style score in Eq. (7) is written as sqrt(2 log t) / N_m^t, which differs from the standard UCB form sqrt(2 log t / N_m^t). Please clarify whether this is intentional or a typo, and define the base of the logarithm.","section":"Eq. (7)"},{"comment":"Several important hyperparameters are not given: the number of client clusters K, the number of selected paths P, the number of candidate combination ratios M, the distance sensitivity alpha in Eq. (5), and the reward amplification factor beta in Eq. (6). These are free parameters that directly affect the reported performance, and their values or selection procedures should be stated for reproducibility.","section":"Section V-B"},{"comment":"There are spacing artifacts in the author and affiliation lines (e.g., \"Liushen g Huang,\" \"University of Sc ience\") that should be corrected in the final version.","section":"Author affiliations"}],"recommendation":"major_revision","confidential_remarks":"The paper's core idea is reasonable and the ablation is informative, but the evaluation protocol is not acceptable as written. The test-set feedback issue is the main reason for the decision; it is fixable by rerunning the experiments with a validation-based reward and a strictly held-out test set. If the authors cannot provide such results, the paper should not be accepted. I would also ask the editor to ensure the communication-cost comparison is made with matched architectures and clear accounting of the two uploaded models."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea is plausible but the evaluation is compromised by a clear circularity: the MAB reward in Eq. (6) is computed from test accuracy, and the same test set is then used to report final accuracy. Section V-B says the global model is evaluated on the test dataset each round, and Algorithm 2 uses that reward to pick the fusion ratio. By the paper's own equations, the chosen ratio is a function of the test labels, so the reported numbers are partially a fit to the test set. The baselines do not get this benefit, which alone explains some of the 4.94–7.24% gains. This is not a minor issue; it undermines the central claim as stated.\n\nWhat is genuinely new: combining structural-model clustering (borrowed from GCFL) and node-feature path selection (borrowed from FedStar) with an MAB-tuned fusion ratio is a reasonable engineering contribution. The component ablation in Table IV is useful, and the communication-cost analysis is more thorough than most FGL papers. The writing is clear and the motivation is sensible.\n\nThe soft spots beyond the test-set leak: baseline model architectures are not fully specified (it says hidden size 64, but not which GNN backbones each baseline uses), there are no error bars or standard deviations despite claiming five replicates, and key hyperparameters (K, P, M, alpha, beta, reward decay) are never reported. The non-IID results showing accuracy increasing with skew are unusual but the authors offer an explanation; without error bars it is hard to tell whether this is real or an artifact of the test-set feedback.\n\nThe fix is straightforward: compute the MAB reward from a validation split and hold out the test set until after training. If the gains persist under that protocol, the paper becomes a solid within-subfield contribution. As written, the evaluation does not support the headline claim.\n\nThis paper deserves a serious referee because the idea is worth testing and the flaw is fixable. I would not cite the results as reported, but I would bring it to a reading group as a case study in evaluation leakage. A responsible editor should send it to review with a clear request to redo the experiments with a proper validation split.","headline":"Test-set feedback in the MAB reward makes the headline accuracy gains unreliable, though the fusion idea is worth a redo.","tokens_in":22265,"tokens_out":1368,"would_cite":false,"duration_ms":14105,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Federated graph learning gains 4.9%-7.2% accuracy by fusing structural properties with node features, and cuts communication cost by up to 81%.","keywords":["federated graph learning","non-IID graph data","structural properties","node features","adaptive fusion","multi-armed bandit","graph neural networks","client clustering"],"falsifier":"Run FedGCF in a setting where the server genuinely has no labeled test data (so Eq. (6) cannot be computed) and compare its accuracy against a fixed-ratio fusion; if the reported 4.94%-7.24% advantage disappears, the claim's load-bearing assumption is falsified.","tokens_in":21255,"feed_emoji":"🕸️","tokens_out":3476,"duration_ms":31464,"temperature":0.7,"pith_summary":"This paper proposes FedGCF, a federated graph learning framework that extracts both structural properties and node features from distributed graph data, then fuses them with an adaptively chosen ratio. The authors claim this combined view outperforms methods that emphasize only one aspect, improving test accuracy by 4.94%-7.24% under different data distributions and reducing communication cost by 64.18%-81.25% to reach the same accuracy as baselines. A sympathetic reader would care because non-IID graph data is common in practice, and existing FGL methods tend to be specialized for either structure-heavy or feature-heavy tasks.","feed_headline":"Two graph views beat one in federated learning","feed_subtitle":"A multi-armed bandit picks the fusion ratio per round, lifting accuracy 4.9-7.2% and cutting communication cost by up to 81%.","key_machinery":"The key machinery is the combination of Parallel Characteristic Extraction (PCE) and Graph Characteristics Fusion (GCF). PCE encodes structural properties using random-walk position embeddings and maximum-degree embeddings, trains a GIN structural model per client, clusters clients by these structural models, and builds a connected topology from pairwise model similarities (distance $d_{i,j}=e^{\\alpha(1-\\sigma_{i,j})/(1+\\sigma_{i,j})}-1$) to select the top-$P$ longest shortest paths; clients on those paths contribute to the common node model. GCF is a UCB-style multi-armed bandit that chooses the fusion ratio using a reward based on the relative change between current test accuracy $r_t$ and the best historical accuracy $r_b$ (Eq. 6).","core_discovery":"FedGCF is a framework that, in each round, clusters clients by structural similarity and aggregates their structural models within each cluster to form shared structural models, while separately selecting clients with common node features via a connected topology of model similarities and aggregating their local models into a common node model. These two models are then fused with a ratio chosen by a multi-armed bandit algorithm (GCF) that adapts to the training progress and data distribution. The central claim is that this simultaneous extraction and adaptive fusion yields higher accuracy and faster convergence than FedAvg, FedProx, GCFL, and FedStar on benchmark molecule and social-network datasets, including under non-IID skew.","pith_inferences":["A testable extension: applying the same PCE and GCF recipe to heterogeneous graph types (e.g., knowledge graphs) where the optimal emphasis between structure and features also varies per client could show whether the gains generalize beyond the three benchmark families.","The reward in Eq. (6) requires the server to compute test accuracy, which implies access to labeled test data; a variant that derives the reward from client-computable statistics (like training loss or validation metrics) would better match the stated privacy-preserving premise.","The bandit treats the fusion ratio as a discrete set of actions; a continuous-armed-bandit formulation could smooth the trade-off and possibly converge faster, though the paper's discrete grid is simpler and adequate for the reported experiments."],"forward_implications":["If FedGCF's claim holds, a single federated framework can handle both structure-heavy tasks (like molecule classification) and feature-heavy tasks (like social-network relationship prediction) without knowing the task type in advance.","The adaptive fusion ratio lets the server shift emphasis during training, so the same algorithm works under IID and non-IID data distributions without reconfiguration.","The reported communication cost reductions of 64%-81% to reach a target accuracy suggest the method is practical for bandwidth-limited edge deployments.","The accuracy gains over FedStar and GCFL imply that structural sharing and feature-based clustering are complementary, and that neither alone captures the information needed for heterogeneous graph data."],"supporting_citations":[{"why":"GCFL is the node-feature-clustering baseline that FedGCF must beat, and it supplies the idea of clustering clients to handle non-IID graph data.","marker":"[23]"},{"why":"FedStar is the structural-sharing baseline, providing the comparison that shows structural properties alone are insufficient.","marker":"[24]"},{"why":"FedAvg is the standard federated averaging baseline used throughout the experiments.","marker":"[29]"},{"why":"FedProx is the heterogeneity-robust baseline that directly motivates the non-IID challenge FedGCF addresses.","marker":"[48]"},{"why":"GCN is the model used to encode node features in the local models.","marker":"[27]"},{"why":"GIN is the model used to encode structural properties via the structural property vectors.","marker":"[47]"},{"why":"Random walk algorithms provide the position embeddings used to construct structural property vectors.","marker":"[38]"},{"why":"The UCB finite-time analysis of multi-armed bandits underpins the exploration-exploitation score in Eq. (7).","marker":"[44]"}],"fun_headline_variants":["Fusing structural and node features lifts federated graph learning","Adaptive bandit fusion of graph views wins in federated learning","Graph structure plus node features: FedGCF fuses for better federated learning","FedGCF combines structure and nodes to boost accuracy, cut communication","Bandit chosen fusion of graph views improves federated learning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The adaptive fusion ratio relies on the server computing the current test accuracy and comparing it with the best historical accuracy, which assumes the server has access to labeled test data that the privacy-preserving federated setting never explicitly provides.","fun_headline_variants_meta":{"raw":{"variants":["Fusing structural and node features lifts federated graph learning","Adaptive bandit fusion of graph views wins in federated learning","Graph structure plus node features: FedGCF fuses for better federated learning","FedGCF combines structure and nodes to boost accuracy, cut communication","Bandit chosen fusion of graph views improves federated learning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001226,"raw_usage":{"total_tokens":5051,"prompt_tokens":971,"completion_tokens":4080,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":587,"completion_tokens_details":{"reasoning_tokens":3992}},"tokens_in":587,"tokens_out":4080,"duration_ms":28284,"temperature":1.0,"reasoning_tokens":3992,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T04:24:34.709022+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run FedGCF in a setting where the server genuinely has no labeled test data (so Eq. (6) cannot be computed) and compare its accuracy against a fixed-ratio fusion; if the reported 4.94%-7.24% advantage disappears, the claim's load-bearing assumption is falsified.","supporting_citations":[{"cited_title":"Federated graph cla ssification over non-iid graphs,","cited_arxiv_id":null,"evidence_quote":"GCFL is the node-feature-clustering baseline that FedGCF must beat, and it supplies the idea of clustering clients to handle non-IID graph data."},{"cited_title":"Fed erated learn- ing on non-iid graphs via structural knowledge sharing,","cited_arxiv_id":null,"evidence_quote":"FedStar is the structural-sharing baseline, providing the comparison that shows structural properties alone are insufficient."},{"cited_title":"Communication-efficient learning of deep networks from de centralized data,","cited_arxiv_id":null,"evidence_quote":"FedAvg is the standard federated averaging baseline used throughout the experiments."},{"cited_title":"Federated optimization in heterogeneous networks,","cited_arxiv_id":null,"evidence_quote":"FedProx is the heterogeneity-robust baseline that directly motivates the non-IID challenge FedGCF addresses."},{"cited_title":"Random w alks: A review of algorithms and applications,","cited_arxiv_id":null,"evidence_quote":"Random walk algorithms provide the position embeddings used to construct structural property vectors."},{"cited_title":"Finite-time analysis of the multiarmed bandi t problem,","cited_arxiv_id":null,"evidence_quote":"The UCB finite-time analysis of multi-armed bandits underpins the exploration-exploitation score in Eq. (7)."}],"review_version":1}