{"id":"e77a1cc1-867a-4a62-a26f-5fdafc2f48db","arxiv_id":"2505.00487","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"Applying the FGSM poisoning attack to a simulated 5G pathloss regression task raises the reported MSE by about a third, and a gradient-boosted classifier detects the poisoned records, but the paper's restoration result is undermined by comparing metrics on different data subsets.","lead":"Researchers poisoned a machine-learning model that predicts 5G signal loss, using a known attack recipe, and then trained a second model to catch the poisoned data points. Their headline numbers come from one experiment and from comparing results on different subsets of data, so the claimed fix is not proven.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The restoration claim in Table 2 is unsupported: Secured Model metrics come from a 5,029-record filtered subset while baseline metrics use the full test set; lower MSE on a smaller subset can be pure selection.","rationale":"The reader's weakest assumption identifies exactly this issue, and I agree with it. The selection effect is sufficient to reject the central restoration claim: the paper's own Section VI wording confirms that the Secured Model is evaluated on a reduced 5,029-record set, which is roughly 24% of the test split, while the other two rows are evaluated on the full test set. Lower MSE and higher R2 on a smaller, cleaned subset are expected even without any defense if the removed records were the hardest to predict. I did not choose the epsilon-scale issue as the primary concern because raw features such as received power in watts can have very large regression coefficients, so an epsilon of 1e-10 is not by itself impossible; however, the text's inconsistency about which epsilon is effective remains a secondary concern. The paper's general observation that FGSM can degrade a linear regression model is plausible, but the headline restoration claim is not demonstrated. Since this comparison flaw invalidates the defense claim, the reader's REJECT verdict stands unchanged.","tokens_in":6366,"tokens_out":6861,"duration_ms":71450,"concrete_test":"Re-run the Secured Model scenario without shrinking the test set: use LightGBM to flag poisoned records in the training/poisoning sample, remove or correct those records, retrain the linear regression, and evaluate on the full original 21,168-record test set, the same set used for the Undefended and Attacked rows of Table 2. If MSE/R2 do not return to approximately 38.51/0.80 on the full set, the restoration claim is an artifact of subset selection.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that 'isolation ... allows us to restore the regression model metrics to their original values' rests on Table 2, but the Secured Model row is computed on a 5,029-record subset after LightGBM removes flagged records, while the Undefended and Attacked rows are computed on the full test split (about 21,168 records from the 40:40:20 split of 105,842). Section VI states: 'we will remove detected adversarial examples from the data set and obtain a reduced data set of 5029 records, on which we will re-evaluate the quality of the regression model.' The paper never states how 5,029 is derived, whether the linear regression is retrained, or whether the same test set is used. If the removed records are concentrated in hard-to-predict or poisoned regions, MSE falls and R2 rises on the subset even if the model is unchanged; this is a selection effect, not a demonstrated defense. This is the load-bearing flaw: without identical evaluation sets, the 37.55 MSE / 0.80 R2 restoration cannot be attributed to the defense, and the headline claim of restoring original metrics fails. A secondary internal inconsistency (Fig. 3 text says epsilon=1-7 and higher sharply increases MSE while the conclusion calls epsilon=1-10 'most vulnerable') further weakens confidence, but the comparison-set flaw alone is sufficient.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies adversarial attacks on a linear regression model that predicts pathloss in a simulated 5G/DeepMIMO scenario. The authors generate a dataset with the DeepMIMO emulator, apply a Fast Gradient Sign Method (FGSM) attack with varying perturbation sizes and poisoned fractions, train LightGBM, CatBoost, and XGBoost classifiers to detect poisoned records, and report that filtering detected records with LightGBM restores the regression metrics to their original values. The main claimed results are that an FGSM attack with parameter epsilon = 1e-10 and poisoned fraction 0.99999 increases MSE by about 33% and decreases R2 by about 10%, and that LightGBM detects adversarial anomalies with 98% accuracy, allowing the regression metrics to be restored (MSE 37.55, R2 0.80).","tokens_in":6419,"tokens_out":2929,"duration_ms":30197,"significance":"If the results were supported, the paper would provide a useful empirical data point on the vulnerability of regression models in wireless-network settings and a practical comparison of gradient-boosting detectors for tabular adversarial examples. Credit is due for using a public dataset (DeepMIMO), reporting detector hyperparameters, and comparing three classifiers. However, the central claim of metric restoration is not supported by the experimental protocol, and the internal inconsistency about the most vulnerable epsilon value further weakens confidence. The topic is relevant, but the contribution as presented is primarily an engineering report with a flawed evaluation design.","major_comments":[{"comment":"The Secured Model row in Table 2 is evaluated on a reduced 5,029-record subset, while the Undefended and Attacked rows are evaluated on the full test set. The paper states: 'we will remove detected adversarial examples from the data set and obtain a reduced data set of 5029 records, on which we will re-evaluate the quality of the regression model.' This makes the comparison invalid: lower MSE and higher R2 on a smaller subset can be a pure selection effect, not a property of the defense. The claim that the metrics are 'restored to their original values' is load-bearing and unsupported. All three scenarios must be evaluated on identical data (either the full test set or the same filtered subset) before any restoration claim can be made.","section":"Section VI, Table 2"},{"comment":"There is an internal inconsistency in the epsilon parameter claim. The text says 'a value of ε = 1-7 and higher leads to a sharp increase in the values of the MSE metric and a decrease in the values of the R2 metric, which is inappropriate,' but the conclusion states that the model is 'most vulnerable' to ε = 1-10 (i.e., 1e-10), the smallest perturbation listed. These statements contradict each other. Additionally, the notation '1-10', '1-9', '1-8', '1-7' is ambiguous; it should be expressed as powers of ten (e.g., 10^-10), and the axes of Fig. 3 should be labeled unambiguously.","section":"Section VI, Fig. 3 and Conclusion"},{"comment":"The attack is described as 'poisoning source data for training based on generative adversarial networks,' but the actual method is FGSM, and it is unclear whether the poisoned samples are injected into the training set or the test set. If this is a poisoning attack, the Secured Model scenario must retrain the linear regression on cleaned training data and evaluate on an untouched test set. If it is an evasion attack on the test set, then removing flagged records from the test set is not a countermeasure but a redefinition of the evaluation set. The manuscript never states whether the regression model was retrained, which is essential for interpreting the Secured Model result.","section":"Section VI, Scenario 2"}],"minor_comments":[{"comment":"The abstract contains the typo 'advertorial attack' instead of 'adversarial attack.'","section":"Abstract and Section I"},{"comment":"The paper refers to 'the data set obtained in Section 2' and describes the organization as 'second chapter,' 'fourth chapter,' 'sixth chapter,' and 'seven chapter'; these references should use the actual section numbers.","section":"Section IV and Section VI"},{"comment":"The CatBoost hyperparameter row contains the typo \"depth'=4\"; it should read 'depth=4'.","section":"Table I"},{"comment":"The paper claims that the trained neural network is 'a neural network with one linear layer and without an activation function,' but it is never explained why this is equivalent to the linear regression beyond the initialization step; the description of steps 1.1–1.4 should be clarified.","section":"Section VI"},{"comment":"The abstract and conclusion state that 'rapid analysis of network traffic' enables detecting malicious activity, but no timing or throughput experiments are reported; this claim goes beyond the presented evidence.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The central experimental flaw in Table 2 is correctable by re-evaluation on a common test set, so I am not recommending outright rejection. However, the authors must also resolve the epsilon inconsistency and clarify the attack protocol. If the re-analysis does not support the restoration claim, the paper should be rejected."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper's headline claim doesn't survive contact with its own Table 2. The Undefended and Attacked models are evaluated on the full test set, but the Secured Model is evaluated on a 5,029-record subset left after the LightGBM detector removes whatever it flags. Lower MSE on a smaller subset is a selection effect, not a demonstrated defense. The paper never says where 5,029 comes from, whether the regression model was retrained on cleaned data, or why removing records counts as defense rather than redefining the test set. This is the load-bearing flaw, and it is fatal to the restoration claim as stated.\n\nTo give credit where it's due: the paper does a reasonable job generating a DeepMIMO 5G pathloss dataset, running a standard FGSM poisoning, and comparing three off-the-shelf boosting classifiers for detecting poisoned records. The finding that FGSM degrades a linear regressor and that LightGBM can flag the perturbation is consistent with prior work cited in the paper itself, so this is a modest extension within an established program, not a new phenomenon.\n\nThere are additional soft spots, in proportion to how soft they are. The text contradicts itself about which epsilon matters: Section VI says epsilon=1e-7 and higher sharply increases MSE and is inappropriate, while the conclusion says epsilon=1e-10 is the most vulnerable. That's a direct internal inconsistency. Also, an additive perturbation of 1e-10 on a linear model should have negligible effect unless the feature scales are enormous, and the paper doesn't report them. \"On average\" appears with a single configuration and no error bars. No code, data, or seeds are provided, which makes independent verification difficult.\n\nWho should read this? It could serve as a cautionary example in a methodology discussion about evaluation sets in adversarial ML, but not as a reliable source for the empirical claims. The underlying idea is fixable: if the authors re-evaluate the defended model on the same test set that the baselines use, or report per-record errors and show the model is actually improved rather than the test set selectively emptied, the defense claim could become testable. As is, I would not trust the restoration numbers.\n\nRecommendation: this deserves a serious referee only in the sense that a referee could clearly identify the evaluation flaw and demand a revision. It is not a desk-reject-without-thought paper. But the current version should be rejected, and the authors should be told exactly what the Table 2 comparison does and why it is invalid.","headline":"The restoration claim is unsupported because the defended model is evaluated on a filtered 5,029-record subset while the baselines run on the full test set, making the central defense result a selection effect.","tokens_in":7179,"tokens_out":2386,"would_cite":false,"duration_ms":25623,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Tiny FGSM perturbations in nearly all test records raise a 5G pathloss regression model's mean-squared error by about a third, and a gradient-boosted binary classifier flags them with roughly 98% accuracy, restoring the original metrics…","keywords":["adversarial attacks","FGSM","regression models","5G networks","massive MIMO","pathloss prediction","anomaly detection","machine learning security"],"falsifier":"Re-run the identical dataset split, FGSM attack, and gradient-boosted filter, then evaluate the regression model on the full test set with only the flagged records removed, and compare against removing the same number of records at random. If the full-test-set MSE and $R^2$ do not return to approximately 38.51 and 0.80, or if random removal improves the metrics by the same amount, the restoration claim is refuted.","tokens_in":5941,"feed_emoji":"📡","tokens_out":8323,"duration_ms":73100,"temperature":0.7,"pith_summary":"The paper tests whether a regression model that predicts wireless signal loss in a 5G massive-MIMO setting can be sabotaged by adversarial poisoning of the data it sees, and whether a second, simpler model can catch the poisoned records. It reports that a Fast Gradient Sign Method (FGSM) attack with a tiny noise step ($\\epsilon=10^{-10}$) applied to almost all records raises the linear regression model's mean-squared error by about 33% and lowers its $R^2$ by about 10%. A gradient-boosted binary classifier detects the poisoned records with about 98% accuracy, and removing the flagged records restores the regression metrics to their original values. The study matters because pathloss predictions guide routing and access decisions in next-generation wireless networks, so a cheap, barely visible manipulation of the data could degrade service for many users.","feed_headline":"FGSM poisoning raises 5G pathloss-model MSE by 33%","feed_subtitle":"A gradient-boosted filter flags 98% of poisoned records and restores the model's original error.","key_machinery":"The attack engine is the Fast Gradient Sign Method (FGSM), which perturbs each input record by $\\epsilon$ times the sign of the gradient of the regression loss with respect to the features, so that the model's error grows in the chosen direction. The defense machinery is a gradient-boosted binary classifier trained to separate poisoned from benign records; the paper compares three boosted-tree variants and selects the best by cross-validated hyperparameter tuning. The regression target is the combined pathloss, and the features are user coordinates, arrival and departure angles, phase, power, and line-of-sight status. The vulnerability itself is carried by the linear regression model, which is re-expressed as a one-layer neural network without an activation function, giving a clean gradient for the attack to follow.","core_discovery":"On a tabular dataset generated by a 3D radio-propagation ray-tracing emulator for a street-level 5G base station, a linear regression model predicts combined signal pathloss with an MSE of 38.51 and an $R^2$ of 0.80. The paper's central claim is that an adversarial FGSM attack that maximizes the gradient sign, using $\\epsilon=10^{-10}$ and corrupting 99.999% of the poisoning sample, degrades the model to MSE 51.40 and $R^2$ 0.72 on the same test set, a rise of roughly 33% in error and a drop of 10% in explained variance. A gradient-boosted binary classifier trained on poisoned versus benign records detects the anomalous records with approximately 98% accuracy, and after these flagged records are removed, the remaining 5,029 records give MSE 37.55 and $R^2$ 0.80, matching the undefended values. The paper presents this as evidence that regression models in wireless networks are vulnerable to adversarial poisoning, but that the attack is detectable and its effect can be reversed by filtering the data before evaluation.","pith_inferences":["A testable extension the paper leaves implicit is what happens when the regression model is retrained on the cleaned training set rather than merely evaluated on a trimmed test set; that would distinguish a true defense from a data-filtering artifact.","Because the attack corrupts 99.999% of the poisoning sample, the detector's 98% accuracy may partly reflect how easy near-total corruption is to spot; a sparser attack (e.g., 10-20% poisoned records) would test detection limits more fairly.","The reduced 5,029-record subset is about 24% of the test portion, and the paper does not describe how that subset is distributed across feature space; comparing the feature distributions of the removed and remaining records would show whether the recovery is a selection effect."],"forward_implications":["A near-invisible perturbation (step size $10^{-10}$) applied to almost all records can raise a regression model's prediction error by a third, so even tiny data manipulation is a practical threat to pathloss-based decisions.","A gradient-boosted binary classifier can act as a data filter that brings MSE and $R^2$ back to their undefended values after an attack.","Because the attack's effect on the model is reversed by filtering alone, the model itself does not need to be retrained after this particular attack.","For 5G/6G network operations, monitoring the data entering pathloss predictors is a viable defensive measure."],"supporting_citations":[{"why":"Supplies the 3D ray-tracing massive-MIMO dataset generation method that produces the pathloss records used in all experiments.","marker":"[12]"},{"why":"Analyzes the robustness of overparameterized linear regression to adversarial perturbations, motivating the attack objective used here.","marker":"[2]"},{"why":"Studies the adversarial robustness of linear regression against poisoned training data, grounding the poisoning scenario.","marker":"[3]"},{"why":"Presents gradient-optimization attacks on regression systems, the family to which the FGSM variant belongs.","marker":"[5]"},{"why":"Derives adversarial examples from perturbation analysis of learning algorithms, providing the theoretical basis for the attack.","marker":"[8]"}],"fun_headline_variants":["FGSM attack spikes 5G pathloss error 33%, LightGBM recovers it","5G regression: FGSM poisoning +33% MSE, 98% detectable","Poisoned 5G data? LightGBM flags 98% and restores model","33% MSE jump from FGSM on 5G pathloss — filter restores","Adversarial FGSM hurts 5G regression, but filter rebuilds it"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim that the filter restores the model depends on the assumption that evaluating the regression model on the reduced 5,029-record subset left after removing flagged records is a fair comparison to the full test set; if the removed records are concentrated in hard-to-predict regions, the lower MSE and higher $R^2$ come from the subset itself rather than from any genuine defense.","fun_headline_variants_meta":{"raw":{"variants":["FGSM attack spikes 5G pathloss error 33%, LightGBM recovers it","5G regression: FGSM poisoning +33% MSE, 98% detectable","Poisoned 5G data? LightGBM flags 98% and restores model","33% MSE jump from FGSM on 5G pathloss — filter restores","Adversarial FGSM hurts 5G regression, but filter rebuilds it"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000202,"raw_usage":{"total_tokens":1386,"prompt_tokens":951,"completion_tokens":435,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":567,"completion_tokens_details":{"reasoning_tokens":318}},"tokens_in":567,"tokens_out":435,"duration_ms":4374,"temperature":1.0,"reasoning_tokens":318,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:43:31.142966+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the identical dataset split, FGSM attack, and gradient-boosted filter, then evaluate the regression model on the full test set with only the flagged records removed, and compare against removing the same number of records at random. If the full-test-set MSE and $R^2$ do not return to approximately 38.51 and 0.80, or if random removal improves the metrics by the same amount, the restoration claim is refuted.","supporting_citations":[{"cited_title":"Overparameterized linear regression under adversarial attacks,","cited_arxiv_id":null,"evidence_quote":"Analyzes the robustness of overparameterized linear regression to adversarial perturbations, motivating the attack objective used here."},{"cited_title":"On the adversarial robustness of linear regression,","cited_arxiv_id":null,"evidence_quote":"Studies the adversarial robustness of linear regression against poisoned training data, grounding the poisoning scenario."},{"cited_title":"Adversarial Attacks on Regression Systems via Gradient Optimization ,","cited_arxiv_id":null,"evidence_quote":"Presents gradient-optimization attacks on regression systems, the family to which the FGSM variant belongs."},{"cited_title":"Perturbation analysis of learning algorithms: Generation of adversarial examples from classification to regression,","cited_arxiv_id":null,"evidence_quote":"Derives adversarial examples from perturbation analysis of learning algorithms, providing the theoretical basis for the attack."}],"review_version":1}