{"id":"ba6172a0-db51-477e-8cc4-78c9275fc25f","arxiv_id":"2501.14745","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":2.0,"correctness_risk":"high","formal_verification":"none","parameter_count":1,"one_line_summary":"An XGBoost model with SHAP explanations is applied to edge node health classification, but the weak reported accuracy and missing experimental details do not support the paper's claims.","lead":"This paper applies XGBoost and SHAP to classify edge computing nodes as healthy or abnormal, reporting that XGBoost edges out five other classifiers. The reported accuracy is only 47.5 percent, which is below chance for a balanced binary task, and the dataset and code are not described, so the claimed superiority is not established.","discovery_kind":"incremental","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No experimental protocol is reported: Table 1's 47.5% accuracy cannot support XGBoost superiority without dataset size, class balance, split, and label source, so the chance-level result is uninterpretable.","rationale":"The paper's central claim is that XGBoost is the best-performing and appropriately interpretable model for edge-node health monitoring. For that claim to hold, the reported comparison must be a valid experiment on a trustworthy dataset. The manuscript provides no dataset description beyond a feature list and no evaluation protocol beyond a single accuracy/F1 table. The reported 47.5% accuracy is at chance level for a balanced binary task and cannot be interpreted at all without class balance information. This is not a disagreement with existing consensus or a question of style; it is a direct threat to the empirical foundation of the claim. The reader identified the same weak point: the privately collected dataset's labels and representativeness are assumed without evidence, and my analysis agrees that this is the load-bearing assumption. I do not see a need to change the REJECT verdict: the central empirical claim is unsupported as presented. I also note a secondary internal inconsistency in Section III-C, where the weight-based importance analysis names power consumption and disk IO as most critical while the SHAP analysis names response time as highest; the paper explains the distinction but does not reconcile it with the abstract's emphasis on response time and power consumption. This inconsistency is secondary to the missing experimental protocol, so I keep the attack focused on the dataset and evaluation gap.","tokens_in":6768,"tokens_out":2279,"duration_ms":24160,"concrete_test":"Obtain or reconstruct the dataset metadata and rerun the comparison under a specified protocol: report sample size, class balance, labeling procedure, collection window, train/test split, and feature definitions; then run all six models with stratified 5-fold cross-validation, tuned hyperparameters, a majority-class baseline, and 95% confidence intervals. If XGBoost's accuracy advantage over MLP or Naive Bayes is not significant, or if the majority baseline matches or exceeds 47.5% accuracy, the claimed superior performance is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that XGBoost is superior for edge-node health monitoring rests entirely on Table 1, where XGBoost achieves 47.5% accuracy and 50.39% F1 against five baselines. Section III-A describes the dataset only qualitatively: it lists eight indicators and says healthy and abnormal samples are present, but gives no sample size, class balance, labeling procedure, collection period, feature normalization, or train/test split. For a binary classification task, 47.5% accuracy is near chance under balanced classes; under strong class imbalance it could be either above or below a trivial majority-class baseline. Without class proportions, the headline accuracy is uninterpretable. The paper also reports no error bars, no repeated runs, no confusion matrix, no precision/recall, no majority-class baseline, and no code or data release. The comparison among the six models is therefore not a controlled experiment: the ranking could be produced by label noise, train/test leakage, or arbitrary hyperparameter choices. The paper itself acknowledges in the conclusion that feature selection depends on the existing dataset and that dynamic modeling is insufficient, but it never supplies the metadata needed to judge whether the dataset supports the claimed result. Because the entire argument for XGBoost's 'superior performance' reduces to one uncontextualized table, the load-bearing assumption is that the private dataset and evaluation protocol are sound; nothing in the manuscript establishes that.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an XGBoost-based binary classifier for edge-computing node health status and uses SHAP to interpret feature contributions. It reports an experiment comparing XGBoost against SVM, Random Forest, KNN, Naive Bayes, and MLP on a privately collected dataset, with XGBoost achieving the best accuracy of 47.5% and F1-score of 50.39%. The paper further presents weight-based and SHAP-based feature importance analyses and claims that response time, power consumption, and disk IO are key drivers of node health. The central claim is that XGBoost is superior for this task and that SHAP provides actionable interpretability for system optimization.","tokens_in":7021,"tokens_out":3445,"duration_ms":33761,"significance":"If the empirical claims were substantiated, the paper would offer a moderately useful application of standard machine learning and interpretability tools to edge-node monitoring. However, the manuscript provides no reproducible experimental protocol: the dataset is described only qualitatively, no data or code are made available, no statistical validation is reported, and the headline accuracy is at or near chance for a balanced binary problem. The SHAP analysis is descriptive of a fitted model whose predictive performance is not established, so the practical conclusions are unsupported. The paper's acknowledged limitations in the conclusion confirm, rather than repair, the absence of evidence for dynamic and generalizable monitoring. I find no machine-checked proofs, reproducible code, or parameter-free derivations to credit; the contribution is currently an unvalidated case study.","major_comments":[{"comment":"The central claim that XGBoost is superior rests entirely on Table 1, where XGBoost achieves 47.5% accuracy and 50.39% F1. For a binary classification task this is at or near chance if the classes are balanced, and if the classes are imbalanced the value could be below or above a trivial majority-class baseline. The paper reports no dataset size, class distribution, train/test split, cross-validation procedure, repeated runs, error bars, confusion matrix, or majority-class baseline, so the ranking in Table 1 is uninterpretable and the comparison is not a controlled experiment.","section":"III-B, Table 1"},{"comment":"Section III-A describes the dataset only qualitatively: it lists eight indicators and states that healthy and abnormal samples are present, but gives no sample size, class balance, labeling procedure, collection period, feature preprocessing, normalization, or division into training and test sets. Without this metadata the experiment cannot be reproduced or assessed for leakage or label noise. The conclusion's statement that feature selection depends on the existing dataset does not supply the missing information; it merely acknowledges that the analysis is dataset-specific.","section":"III-A and IV"},{"comment":"The SHAP analysis is computed from the same fitted XGBoost model whose accuracy is not demonstrated to be above chance, so the feature-importance narrative cannot validate the paper's practical recommendations. In addition, the directional claims appear internally inconsistent: Figure 3's text states that high response times are primarily associated with healthy nodes while low response times align with abnormalities, and Figure 4 states that low CPU usage is associated with abnormal nodes, whereas the abstract and introduction identify response time and power consumption as indicators of abnormal status. The label encoding and the direction of effects are not defined, so these interpretation claims are unsupported.","section":"III-C"},{"comment":"The paper does not provide any quantitative evaluation of the SHAP or weight-based importance results, such as feature importance values, rankings with confidence, or validation against independent outcomes. The figures appear to be screenshots without axis labels or numeric scales in the manuscript text. Since the model's predictive performance is near chance, the feature importance rankings cannot be taken as evidence about the actual drivers of edge-node health.","section":"III-C, Figures 1-4"}],"minor_comments":[{"comment":"The equations in Section II contain notation errors and OCR artifacts: the second-order gradient is incorrectly also denoted g_i, and in the SHAP formula the symbol A is used for the feature-subset size without definition, where the standard notation is |S|. The displayed formulas should be rewritten cleanly.","section":"II"},{"comment":"Figures 1-4 lack axis labels, units, and numeric scales, making it impossible to read the claimed importance values or SHAP magnitudes from the text.","section":"III-C, Figures 1-4"},{"comment":"Several citations do not support the claims at the positions where they appear; for example, [19] is cited for the GBDT basis and [20] for XGBoost, but the listed titles are about few-shot learning and graph neural networks, respectively. The reference list should be checked and corrected throughout.","section":"References"},{"comment":"The manuscript contains formatting artifacts in the keyword line and in some equations, and the Introduction and Conclusion discuss 'traditional methods' without identifying or citing any such methods for edge-node health monitoring. These issues should be corrected in a revision.","section":"General"}],"recommendation":"reject","confidential_remarks":"The citation pattern, in which several cited papers are unrelated to the claim being supported, suggests the reference list was assembled without careful vetting. I did not base the verdict on that alone, but the editor may wish to screen the manuscript for citation integrity before any resubmission."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Honest take: this is a routine XGBoost+SHAP application to a private edge-node health dataset, and the reported results don't support the title. The one thing a reader should know: the best accuracy in Table 1 is 47.5%, which is at chance for a balanced binary problem, and the paper never reports class balance, sample size, split, or error bars. So the headline claim that XGBoost is superior collapses.\n\nWhat is actually new: nothing in method. The contribution is the private dataset, which is not described beyond a list of eight indicators. To the paper's credit, the methodology section correctly states the XGBoost objective and SHAP equations, and the writing is clear about what weight-based importance vs SHAP importance mean. The SHAP discussion does distinguish feature usage frequency from actual contribution. That is honest. The conclusion also admits feature selection depends on the existing dataset and dynamic modeling is insufficient; those are real limitations.\n\nSoft spots are large. The experiment is uninterpretable. No majority baseline, no confusion matrix, no precision/recall, no repeated runs. The dataset section is qualitative; no labeling procedure or collection period. The references are mostly irrelevant to edge computing health monitoring; many are about LLMs, GNNs, and medical text. The SHAP narrative is hard to reconcile: the abstract says response time and power consumption are the key features, but Figure 1 ranks power and disk IO highest by weight, and Figure 2 says response time is highest by SHAP. That difference is not necessarily contradictory, but the paper doesn't explain how the two rankings align with the claimed captured features.\n\nProportionate verdict: this is not a fraudulent paper, it is an under-developed workshop-level report. It might have been acceptable as a short paper if the full experimental protocol, data description, and code were included, and if the accuracy had been put next to a majority baseline. As it stands, the central claim is unsupported. I would not send this to peer review in its current form; a serious editor would desk reject and invite a resubmission with real experiments, data release, and a comparison against trivial baselines. If the authors can show the dataset is balanced and the split is clean, 47.5% might still be at chance, so the method needs a lot more work before it supports the conclusions.","headline":"Routine XGBoost+SHAP application whose only table is uninterpretable: 47.5% accuracy is chance-level and no protocol or data is reported.","tokens_in":7564,"tokens_out":2217,"would_cite":false,"duration_ms":19670,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that an XGBoost classifier, with SHAP explanations, is the most accurate of six models for judging edge-computing node health and that response time, power consumption, and disk I/O drive the judgments.","keywords":["edge computing","health monitoring","XGBoost","SHAP","interpretability","distributed systems","node health status"],"falsifier":"Re-run the six models on the same dataset with a documented train/test split, then shuffle the health labels and repeat; if shuffled labels yield a similar ranking or if all models sit near the majority-class baseline, the claimed superiority of XGBoost would be refuted.","tokens_in":6565,"feed_emoji":"🖥️","tokens_out":9118,"duration_ms":74159,"temperature":0.7,"pith_summary":"This paper tries to establish that an XGBoost classifier, explained with SHAP values, gives the best health-status judgments for edge-computing nodes among six common models on its collected dataset, while also making the reasoning behind each judgment visible. XGBoost reaches 47.5% accuracy and a 50.39% F1 score, ahead of support vector machines, random forests, k-nearest neighbors, naive Bayes, and a multilayer perceptron. SHAP analysis identifies response time, power consumption, and disk I/O as the features that most influence predictions, offering a concrete list of indicators for operators to monitor. If these claims hold, edge-computing systems could move from opaque anomaly scoring to interpretable, indicator-driven health monitoring.","feed_headline":"XGBoost beats five rivals at edge-node health checks","feed_subtitle":"SHAP analysis says response time, power, and disk I/O drive the model, telling operators what to watch.","key_machinery":"The machinery is XGBoost, a regularized gradient-boosted decision-tree ensemble that uses first- and second-order gradients to grow trees, combined with SHAP, which computes each feature's contribution as a weighted average of marginal effects over all feature subsets. XGBoost supplies the classifier; SHAP supplies global and local interpretability. The paper's argument depends on the difference between XGBoost's split-frequency weight importance and SHAP's prediction-contribution importance, using the contrast to claim that response time is the decisive operational indicator even though power consumption and disk I/O dominate split counts.","core_discovery":"The central claim is that gradient boosting with XGBoost handles the nonlinear, multi-feature patterns of edge-computing node data better than the five baselines, and that SHAP values provide a trustworthy account of which features matter. The paper reports two complementary importance rankings: XGBoost's built-in weight metric, which counts how often a feature is used at tree splits, puts power consumption and disk I/O first, while SHAP's contribution-based ranking puts response time first, followed by power consumption and disk I/O. The paper interprets this contrast as a feature, not a flaw: weight reflects construction, SHAP reflects actual contribution to predictions, and together they guide monitoring priorities and anomaly diagnosis.","pith_inferences":["Editorial inference: the accuracy gap between XGBoost (47.5%) and MLP (46.8%) is small, so the practical benefit on real workloads may be modest even if the ranking is stable.","Editorial inference: the SHAP color-direction claims, such as high response times aligning with healthy nodes, could reflect a dataset-specific pattern and should be validated against operational logs before being used to set alarms.","Editorial inference: the reference markers attached to GBDT, XGBoost, and SHAP in the method section point to unrelated work, so the paper's central claim rests on its own experiment rather than on the cited sources."],"forward_implications":["If the central claim is correct, monitoring dashboards for edge nodes should lead with response time, power consumption, and disk I/O.","If the central claim is correct, SHAP explanations can turn each health alert into an attributable cause, such as elevated response time, which supports faster remediation.","If the central claim is correct, the same XGBoost-plus-SHAP pipeline can be applied to other distributed-system health monitoring tasks with comparable tabular telemetry.","If the central claim is correct, operators can use the weight-versus-SHAP distinction to avoid over-trusting split-based feature rankings."],"supporting_citations":[{"why":"Cited as evidence that XGBoost has significant performance advantages on data tasks, motivating its choice.","marker":"[11]"},{"why":"Cited as the source for gradient boosting decision trees, the base algorithm of the proposed method.","marker":"[19]"},{"why":"Cited as the source for XGBoost's regularization and optimized splitting strategy.","marker":"[20]"},{"why":"Cited as the theoretical basis for SHAP's Shapley-value feature contribution framework.","marker":"[21]"}],"fun_headline_variants":["XGBoost outperforms 5 rivals in edge node health checks, SHAP explains why","Edge node AI: XGBoost beats baselines, SHAP pinpoints key signals","SHAP reveals response time and power as top edge node health indicators","XGBoost + SHAP: precise and explainable edge computing health monitoring","Edge health monitoring: XGBoost's edge over rivals, decoded by SHAP"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported comparison rests on the assumption that the privately collected dataset is correctly labeled and representative of edge-node health, and that the train/test split is sound; if labels are noisy or the split leaks, the performance ranking is meaningless.","fun_headline_variants_meta":{"raw":{"variants":["XGBoost outperforms 5 rivals in edge node health checks, SHAP explains why","Edge node AI: XGBoost beats baselines, SHAP pinpoints key signals","SHAP reveals response time and power as top edge node health indicators","XGBoost + SHAP: precise and explainable edge computing health monitoring","Edge health monitoring: XGBoost's edge over rivals, decoded by SHAP"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000222,"raw_usage":{"total_tokens":1449,"prompt_tokens":933,"completion_tokens":516,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":549,"completion_tokens_details":{"reasoning_tokens":407}},"tokens_in":549,"tokens_out":516,"duration_ms":4483,"temperature":1.0,"reasoning_tokens":407,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T14:52:51.852390+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the six models on the same dataset with a documented train/test split, then shuffle the health labels and repeat; if shuffled labels yield a similar ranking or if all models sit near the majority-class baseline, the claimed superiority of XGBoost would be refuted.","supporting_citations":[{"cited_title":"A Hybrid Model for Predicting Missing Records in Data Using XGBoost,","cited_arxiv_id":null,"evidence_quote":"Cited as evidence that XGBoost has significant performance advantages on data tasks, motivating its choice."}],"review_version":1}