{"id":"a0400e53-b2fc-42f5-8ad4-033bc828f068","arxiv_id":"2501.11054","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"Across eight federated models on MNIST, adversarial attacks active in the final or all rounds generally lower test accuracy more than middle-round attacks, though the pattern depends on attack type and model.","lead":"This paper tests how the timing of malicious client attacks affects eight federated learning models on MNIST, comparing attacks that run through all, middle, or final rounds. It reports that attacks in later or all rounds damage accuracy more, and that an outlier-detection defense helps against some attacks.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline temporal claim is unsupported because the model poisoning attack collapses CNN/RNN/LSTM to chance in every window, so 'later rounds more damaging' may be an artifact of attack strength, not timing.","rationale":"The reader's weakest assumption correctly identifies the key threat: model poisoning is so strong that it eliminates the temporal contrast for CNN, RNN, and LSTM. My analysis agrees and sharpens it: the missing λ and random-base-model specification are the concrete missing control. I also note that the absence of an EARLY-only window is a second indent, but it is secondary to the attack-strength confound. The conditional verdict stands: with re-scoped claims, reported attack-strength parameters, and either removal or relabeling of the supervised-defense results, the paper would be acceptable. My recommendation remains CONDITIONAL rather than REJECT because the raw tables appear internally consistent and the label flipping results do show some temporal variation; the fix is interpretive and experimental, not a wholesale failure.","tokens_in":18604,"tokens_out":2015,"duration_ms":15298,"concrete_test":"Re-run the model poisoning attack on CNN, RNN, and LSTM with a calibrated set of magnification factors (e.g., λ ∈ {1.1, 2, 5, 10}) and report λ, the poisoning ratio (25% of clients), and the base-model initialization seed. Then check whether there exists a λ such that FULL and END accuracy drop substantially while MID accuracy remains near baseline; if no such λ exists for these models, Table B.3's near-random MID results mean the temporal ordering claim cannot be separated from attack strength. Additionally, add an EARLY-only attack window (first 30% of rounds) to directly test the recovery claim.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's central claim is that temporal attacks matter most when adversaries are active throughout or in later rounds. The label flipping results partially support this (e.g., MLR FULL 0.6187, END 0.7446, MID 0.8529), but the model poisoning results in Table B.3 undermine it: CNN, RNN, and LSTM accuracies are 0.0980/0.1145/0.0892 in MID and 0.1420/0.1780/0.1525 in END, i.e., all near-random. If the attack is strong enough to destroy the model in the middle window too, the observed 'later rounds matter' pattern for these models does not follow; it collapses into 'the attack is catastrophic whenever active.' Section 2.4.2 leaves the magnification factor λ unreported and specifies only 'a randomly initialized base model,' so the attack strength is uncontrolled and unquantified. With λ large and a random base model, every attack window becomes overwhelming, making the temporal comparison vacuous for the deep models. The conclusion (Section 6) that 'attacks in the earlier rounds do not have a strong effect' and 'FL models can recover from attacks' is asserted without running an EARLY-only window; recovery is inferred from END and MID, not tested.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper experimentally studies how the timing of adversarial client attacks affects federated learning (FL) models on MNIST. It defines three temporal windows—FULL, MID, and END (last 30% of rounds)—and tests three attack types (label flipping, model poisoning via MPAF, and GAN reconstruction) across eight classifiers (MLR, SVC, MLP, CNN, RNN, LSTM, Random Forest, XGBoost). It also evaluates a supervised outlier-detection defense at the aggregation server. The central claim is that temporal attacks are most damaging when adversaries are active throughout or in later rounds, and that outlier detection mitigates simple attacks. Results are reported as accuracy, precision, recall, F1, and loss in bar charts and in detailed appendix tables.","tokens_in":18838,"tokens_out":4133,"duration_ms":38669,"significance":"If the temporal claim held, the paper would give FL practitioners a concrete reason to treat attack timing as a first-order variable and to prioritize defenses for late or full-duration attacks. The breadth of the model zoo and the reporting of the full metric set in Appendix B are useful for replication. However, the support is mixed: label flipping partially supports the claim, while model poisoning results in Table B.3 do not, and the defense evaluation in Section 5.2 uses ground-truth malicious-client labels to train the outlier detector, an unrealistic assumption. The significance is therefore currently qualified by these issues.","major_comments":[{"comment":"The claim that attacks matter 'especially when the adversaries are active throughout or during the later rounds' is contradicted by the model poisoning results for CNN, RNN, and LSTM. In the MID window, these models achieve accuracies of 0.1145, 0.1780, and 0.0892, respectively, which are at or near the random baseline of 0.10 and are virtually indistinguishable from the FULL and END results (e.g., CNN FULL 0.0980, END 0.0951). The temporal distinction collapses for these models; the observed pattern is that the attack is catastrophic whenever active, not that later rounds are worse. This undermines the abstract and the Section 6 conclusion.","section":"Section 5.4, Table B.3"},{"comment":"The model poisoning attack (MPAF) is specified only as using 'a randomly initialized base model' with a magnification factor λ > 1, but λ is never reported. Without controlling λ, the attack strength is unquantified; with a large λ and a random base, the deep models are overwhelmed in every window, making the temporal comparison an artifact of attack strength rather than timing. The paper should report λ, justify its value, and include a sensitivity analysis over λ, or at least an ablation demonstrating that the results are not driven by an arbitrarily strong attack.","section":"Section 2.4.2"},{"comment":"The outlier detector is trained using the ground-truth labels of which clients are malicious ('based on the client labels of honest or malicious'). In a realistic FL setting the server does not know which clients are malicious; this is precisely what the detector is supposed to infer. Training on these labels leaks privileged information, so the reported defense accuracies (e.g., One-Class SVM accuracy 0.97 in Table 2) overstate the benefit of outlier detection. The defense should be evaluated without exposing ground-truth adversary identities, for example using unsupervised anomaly detection or a held-out validation set that is not part of the attack scenario.","section":"Section 5.2, Tables B.2, B.4, B.6"},{"comment":"The conclusion states that 'attacks in the earlier rounds do not have a strong effect' and that 'FL models can recover from attacks,' but no EARLY-only attack window was run; the experiments only cover FULL, MID, and END. Recovery cannot be inferred from the absence of damage in MID relative to END; a direct EARLY-only condition, with later rounds clean, is needed to test the claim that early attacks are harmless and that models recover.","section":"Section 6"}],"minor_comments":[{"comment":"The heading 'Multinominal Logistic Regression' contains a typo; it should be 'Multinomial Logistic Regression'.","section":"Section 2.6.1"},{"comment":"The text refers to 'the Fl system' in one place; this should be 'FL system'.","section":"Section 3"},{"comment":"The hyperparameter 'Colsample by-tree' is likely a typo for 'colsample_bytree' in the XGBoost notation.","section":"Table A.7"},{"comment":"Figure 1 cites a Wikipedia image as reference [18]; a primary scholarly source for centralized versus decentralized FL would be more appropriate.","section":"Figure 1, reference [18]"},{"comment":"No link to source code or a specification of the exact λ value is provided; adding these would materially improve reproducibility.","section":"Reproducibility"}],"recommendation":"major_revision","confidential_remarks":"The defense evaluation issue is the most serious: training the outlier detector with ground-truth adversary labels is a form of label leakage that makes the defense results largely uninterpretable. The temporal claim is also only partially supported. I recommend major revision with a request for the λ value, an EARLY window, and a reworked defense evaluation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"I read this as an empirical sweep: three attack types (label flipping, model poisoning, GAN reconstruction), three temporal windows (FULL, MID, END), and eight model families on MNIST with FedAvg/bagging. The temporal comparison itself is the new part, and I don't think that specific combination appears in the cited literature. The raw tables in Appendix B look internally consistent, and the label-flipping results do show a timing effect for many models: END and FULL hurt more than MID for MLR, CNN, RF, and XGBoost, for instance. The hyperparameter appendix is a nice touch.\n\nThe soft spots are real, though. First, the model-poisoning attack collapses CNN, RNN, and LSTM to near-random accuracy in the MID window as well as FULL and END (Table B.3: 0.1145, 0.1780, 0.0892 in MID vs 0.0980, 0.1420, 0.1427 in FULL). For those models the attack is catastrophic whenever it is active, so the abstract's 'especially in later rounds' is not supported for that attack. Second, the conclusion says attacks in early rounds do not have a strong effect and that models can recover, but no early-only window was run; recovery is inferred from END and MID conditions. That is an overreach. Third, the attack-strength parameters are under-reported: the model-poisoning magnification factor lambda is not given, the label-flipping poisoning ratio is never stated, and the GAN details are thin. That makes the temporal comparison hard to interpret as a statement about timing rather than about attack strength.\n\nThe defense section is the weakest part. Section 5.2 explicitly trains the outlier detector using client labels of honest or malicious. That is a label leak: a real FL server does not know which clients are malicious, and the reported 97% detection accuracy is an upper bound, not a realistic defense result. The paper does describe the procedure as supervised, so it is not hidden, but the conclusion presents outlier detection as effective without that caveat.\n\nOverall: the experiments are competently organized and the raw measurements are probably reproducible, but the claims outrun the data. The paper would be acceptable after re-scoping the claims to the windows actually tested, reporting the attack parameters, and either removing the supervised-defense results or labeling them clearly as a leaked-label upper bound. I would send it to peer review rather than desk-reject it; a serious referee can push for the needed revisions. I would not cite it in my own work until those parameters are reported.","headline":"A competent but overstated temporal-attack benchmark on MNIST; the label-flipping results mostly support the timing claim, the model-poisoning results undercut it, and the defense numbers leak ground-truth labels.","tokens_in":19417,"tokens_out":2809,"would_cite":false,"duration_ms":26755,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper experimentally shows that the timing of adversarial attacks in federated learning matters as much as their type: attacks running in the final rounds or throughout training cause the largest accuracy drops, while a One-Class SVM…","keywords":["federated learning","temporal attacks","model poisoning","label flipping","GAN reconstruction","outlier detection","FedAvg","MPAF"],"falsifier":"Re-run the model-poisoning experiments with a magnification factor close to 1 or with a base model that already has moderate accuracy, and check whether the MID window still collapses CNN, RNN, and LSTM to near-random accuracy. If it does, the late-round conclusion is an artifact of attack strength; if MID stays high while END collapses, the timing claim holds. Separately, stop the attack after round 3 and measure whether accuracy returns to baseline by round 10, which the paper's recovery claim predicts.","tokens_in":18318,"feed_emoji":"⏰","tokens_out":6924,"duration_ms":66984,"temperature":0.7,"pith_summary":"This paper tries to establish that the timing of an adversarial client's attacks in federated learning is a first-order factor in how much damage the attack does. In experiments on eight model types and MNIST, the authors show that attacks running in the last 30% of federated rounds, or throughout all rounds, cut accuracy far more than attacks confined to the middle rounds. They also show that early attacks leave models able to recover, that a One-Class SVM outlier-detection defense restores much of the lost accuracy under model poisoning, and that the same timing pattern does not hold equally for every attack or model family. If true, this means FL system designers should treat when an attacker is active as a key variable and should prioritize defenses for late and full-duration attack windows.","feed_headline":"Late-round attacks do the real damage in federated learning","feed_subtitle":"Across 8 models, poisoning in the last 30% of rounds or every round collapses accuracy; outlier detection restores much of it.","key_machinery":"The experiments run ten federated rounds with 25% of clients adversarial and compare three attack windows: FULL (adversaries active every round), MID (active in the middle 30% of rounds), and END (active in the last 30%). Aggregation is FedAvg for neural and linear models and bagging for tree models. The model-poisoning attack follows MPAF: a malicious client takes the difference between a randomly initialized low-accuracy base model and the current global model, multiplies the difference by a factor $\\lambda>1$, and sends the magnified fake update for aggregation. Defense is a supervised One-Class SVM trained on client evaluation metrics (precision, recall, F1, loss) to flag and drop malicious updates. These pieces together make the temporal comparison possible by varying only when the malicious behavior occurs.","core_discovery":"The central discovery is that temporal attack windows are not interchangeable: for the label-flipping and model-poisoning attacks tested, the FULL and END windows degrade the global model substantially, while the MID window has little effect in label flipping and the strongest models recover from early attacks. The effect is most dramatic under MPAF model poisoning, where CNN, RNN, and LSTM accuracies collapse to near-random levels (for example, 0.098, 0.142, and 0.089 in the FULL, MID, and END cases for CNN) without defense; One-Class SVM outlier detection lifts these models back to about 0.81–0.84 accuracy. By contrast, the GAN reconstruction attack is weak against neural models and only affects the tree-based models. The paper concludes that models can recover from early attacks, that later or persistent attacks are the dangerous ones, and that model family and defense choice materially change the outcome.","pith_inferences":["Editorial: Because the MPAF attack's magnification factor and random base model are not reported, the quantitative ranking of windows may not transfer to weaker or stronger poisoning; sweeping $\\lambda$ would separate timing from attack magnitude.","Editorial: The recovery from early attacks suggests that defenders could treat persistence, not just anomaly size, as a signal, weighting recent-round behavior more heavily in client trust scores.","Editorial: The outlier-detection evaluation is supervised, trained on known honest-versus-malicious labels; an unsupervised deployment, where such labels are absent, is the stricter test of whether the defense works in practice.","Editorial: The GAN attack's weakness may be a consequence of the horizontal partition keeping seven of ten classes per client, so the synthetic missing-label data is easy for the model to ignore; a partition with fewer classes per client would test the attack more severely."],"forward_implications":["Defenses that cannot protect every round should concentrate on the final 30% of federated rounds and on clients that are anomalous in every round.","A transient attack early in training is survivable, so detection mechanisms can afford to be slower if the threat is known to be intermittent.","One-Class SVM filtering of client updates recovers most of the accuracy lost to model poisoning, restoring CNN, RNN, and LSTM from near-random to roughly 0.81–0.84 accuracy.","Model selection changes exposure: MLP is the most attack-resistant model tested, while LSTM, despite the best clean accuracy, collapses under model poisoning, and tree ensembles are the weak point for GAN reconstruction."],"supporting_citations":[{"why":"Supplies the FedAvg aggregation algorithm used to combine client updates for all non-tree models.","marker":"[35]"},{"why":"Supplies the federated learning implementation whose server-client-strategy abstraction supports the temporal attack injection and the outlier-detection defense.","marker":"[3]"},{"why":"Supplies the MPAF model-poisoning attack recipe (fake updates from a random base model scaled by a magnification factor) used in the model-poisoning experiments.","marker":"[8]"},{"why":"Supplies the MNIST handwritten-digit benchmark that all eight models are trained and attacked on.","marker":"[27]"},{"why":"Supplies the bagging aggregation strategy used to merge tree-based Random Forest and XGBoost client models.","marker":"[20]"},{"why":"Supplies the XGBoost ensemble model whose behavior under GAN reconstruction attacks is a reported result.","marker":"[9]"},{"why":"Supplies the One-Class SVM outlier detector that is selected as the defense and applied to client evaluation metrics.","marker":"[45]"},{"why":"Supplies the classic models (MLR, SVC) and the outlier-detection implementations used in the experiments.","marker":"[46]"}],"fun_headline_variants":["Late and full attacks crush federated models, early ones don't","When in the training a poisoning attack hits matters most","Federated learning most vulnerable to late or persistent attacks","Model poisoning hits hardest in later rounds, early attacks fade","Outlier detection restores federated models hit by late poisoning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The timing conclusion rests on the assumption that the model-poisoning attack is strong but not so overwhelming that it destroys the model in every window; if the random base model and the magnification factor $\\lambda$ make the attack catastrophic even in the middle rounds, then the observed 'later rounds damage more' pattern is an artifact of attack strength, not of timing.","fun_headline_variants_meta":{"raw":{"variants":["Late and full attacks crush federated models, early ones don't","When in the training a poisoning attack hits matters most","Federated learning most vulnerable to late or persistent attacks","Model poisoning hits hardest in later rounds, early attacks fade","Outlier detection restores federated models hit by late poisoning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000846,"raw_usage":{"total_tokens":3650,"prompt_tokens":880,"completion_tokens":2770,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":496,"completion_tokens_details":{"reasoning_tokens":2687}},"tokens_in":496,"tokens_out":2770,"duration_ms":18601,"temperature":1.0,"reasoning_tokens":2687,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T18:40:47.089338+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the model-poisoning experiments with a magnification factor close to 1 or with a base model that already has moderate accuracy, and check whether the MID window still collapses CNN, RNN, and LSTM to near-random accuracy. If it does, the late-round conclusion is an artifact of attack strength; if MID stays high while END collapses, the timing claim holds. Separately, stop the attack after round 3 and measure whether accuracy returns to baseline by round 10, which the paper's recovery claim predicts.","supporting_citations":[{"cited_title":"Communication-efficient learning of deep networks from decentralized data","cited_arxiv_id":null,"evidence_quote":"Supplies the FedAvg aggregation algorithm used to combine client updates for all non-tree models."},{"cited_title":"MPAF: Model poisoning attacks to federated learning based on fake clients","cited_arxiv_id":null,"evidence_quote":"Supplies the MPAF model-poisoning attack recipe (fake updates from a random base model scaled by a magnification factor) used in the model-poisoning experiments."},{"cited_title":"https://flower.ai/docs/framework/tutorial-quickstart- xgboost.html#tree-based-bagging-aggregation, 2023","cited_arxiv_id":null,"evidence_quote":"Supplies the bagging aggregation strategy used to merge tree-based Random Forest and XGBoost client models."},{"cited_title":"XGBoost: A scalable tree boosting sys- tem","cited_arxiv_id":null,"evidence_quote":"Supplies the XGBoost ensemble model whose behavior under GAN reconstruction attacks is a reported result."},{"cited_title":"Platt, John Shawe-Taylor, Alex J","cited_arxiv_id":null,"evidence_quote":"Supplies the One-Class SVM outlier detector that is selected as the defense and applied to client evaluation metrics."},{"cited_title":"https://scikit-learn.org/ stable/","cited_arxiv_id":null,"evidence_quote":"Supplies the classic models (MLR, SVC) and the outlier-detection implementations used in the experiments."}],"review_version":1}