{"id":"fb1ad9a5-ae53-4be8-8e8a-c48c45e621af","arxiv_id":"2507.05132","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"An extreme learning machine classifier obtained roughly 95% accuracy detecting DDoS attacks in the CICIoMT2024 IoMT dataset, but the evaluation may leak test information during feature selection and the low-cost claim is unmeasured.","lead":"This paper tests a lightweight machine learning model, the extreme learning machine, for detecting denial-of-service attacks on internet-connected medical devices. It reports about 95% accuracy on a public IoMT dataset, and claims the model is cheap enough to run on low-power hospital hardware.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 94.87% accuracy claim may be optimistically biased because feature selection and z-score normalization are applied before the train/test split, letting test labels influence training.","rationale":"The reader's weakest assumption identifies the same load-bearing concern: the evaluation protocol may be leaky because feature selection is performed before the train/test split. This is the most direct threat to the paper's central claim because the 94.87% accuracy is the main evidence for the proposed ELM approach. I agree with the reader's conditional assessment. I add two closely related points that reinforce the concern: z-score normalization also appears to be fit on the full data before the split, and the 'low implementation budget' portion of the claim is not backed by any measured compute metrics. However, the primary issue is the pre-split feature selection, and the recommended remedy is straightforward: move feature selection and scaler fitting inside the training folds, then re-measure. The paper has no formal verification and no shipped code, but the proposed method is a standard ELM on a public dataset, so the empirical claim is plausible if the protocol is fixed. Given the leak risk and the absence of reproducibility details, CONDITIONAL remains the appropriate verdict; no change from the reader's recommendation is needed.","tokens_in":10325,"tokens_out":2853,"duration_ms":37665,"concrete_test":"Re-run the full pipeline with feature selection and StandardScaler fitting performed only on the training portion inside each cross-validation fold, then evaluate on the untouched test split. Specifically: (1) split the data 80/20 once; (2) on the 80% training set only, compute absolute correlation with the target, discard features below 0.02, and fit the z-score scaler; (3) apply the same selected features and scaler to the test set; (4) fix the ELM random seed and report the selected feature count, hidden-layer size, activation function, and test-set accuracy, precision, recall, and AUC. If the held-out accuracy drops by more than about 2 percentage points from 94.87%, or if precision/recall drop materially, the reported performance is leak-inflated and the central claim is not supported as written.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim, that the ELM achieves roughly 95% accuracy on CICIoMT2024 while needing minimal resources, depends on a leak-free evaluation protocol. In Sec. IV-B, the authors state: 'the correlation of each feature with the target class was determined. Features with an absolute correlation value of less than 0.02 were discarded.' This feature selection is described before the sentence 'The dataset was split into 80% training and 20% test sets.' If the correlation with the target is computed on the full 476,150-sample dataset, then test labels have already influenced which features are retained. Correlation-based selection with a low 0.02 threshold on a dataset with many candidate features is particularly prone to selection bias: features whose sample correlations with the target are nonzero by chance on the test partition will be preferentially kept, inflating held-out accuracy. The same section also applies z-score normalization via StandardScaler before describing the split, which leaks test-set means and standard deviations into the training pipeline. These are not merely procedural details; they directly affect the validity of the 94.87% accuracy, 0.95 precision, 0.95 recall, and 0.945 AUC-ROC figures that support the paper's main conclusion. The 'low implementation budget' claim is additionally unmeasured: no CPU time, memory usage, hidden-layer size, selected feature count, random seed, or final ELM hyperparameters are reported. The proposed approach may still work, but as written the evaluation protocol does not establish the headline result.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an extreme learning machine (ELM) classifier for detecting DDoS attacks on Internet of Medical Things (IoMT) traffic. Using the CICIoMT2024 benchmark (476,150 samples), the authors apply cleaning, correlation-based feature selection (threshold 0.02), z-score normalization, an 80/20 split, and an ELM with a randomly initialized hidden layer whose output weights are obtained via the Moore–Penrose pseudoinverse. They report 94.87% accuracy, 0.95 precision, 0.95 recall, 0.95 F1, and 0.945 AUC-ROC, and claim the approach requires a minimal implementation budget, making it suitable for fog-level IoMT deployment. The paper includes a comparison with several prior DDoS detection models.","tokens_in":10604,"tokens_out":4683,"duration_ms":47203,"significance":"The potential value of the paper is real: an ELM-based detector with single-pass training would be a plausible fit for resource-constrained IoMT/fog environments, and the paper uses a recent public IoMT-specific benchmark. The strengths are the use of an externally available dataset and a clear application motivation. However, the significance of the empirical claim depends entirely on a leakage-free evaluation; the current protocol undermines the headline numbers. The 'low implementation budget' claim, which is part of the abstract and conclusion, is supported only by qualitative assertions and not by measurements. If the authors re-run the experiments with a leakage-free pipeline and provide resource measurements, the work could be a useful empirical baseline.","major_comments":[{"comment":"The text states that 'the correlation of each feature with the target class was determined' and features with absolute correlation below 0.02 were discarded, and this appears before the sentence describing the 80/20 split. If the correlation is computed on the full 476,150-sample dataset, the test labels can influence which features are retained, which is a label leak. Because Table III reports 94.87% accuracy, 0.95 precision/recall/F1, and 0.945 AUC-ROC on that test set, this leak can directly inflate every headline metric. Please repeat feature selection using only the training portion (ideally inside each cross-validation fold) and report the resulting metrics.","section":"Section IV-B"},{"comment":"The same paragraph applies z-score normalization with StandardScaler before the split is described. Fitting the scaler on the full dataset transfers test-set means and standard deviations into the training pipeline; while less severe than label leakage, it still invalidates the strict holdout protocol. Fit the scaler on the training split only and report whether the metrics change.","section":"Section IV-B"},{"comment":"The central claim that the approach requires 'minimal training time and implementation resources' is not measured. The manuscript reports no wall-clock training time, inference latency, memory usage, CPU usage, number of hidden neurons, selected feature count, or random seed. It also does not report the final hyperparameters chosen by GridSearchCV (number of neurons, activation function, correlation threshold). Without these measurements, the low-budget claim is an unsupported assertion rather than an empirical finding; please add a resource evaluation and report the hyperparameters.","section":"Section IV-C and Conclusion"},{"comment":"There are material inconsistencies in the results and comparison. Table III gives 94.87% accuracy and 0.945 AUC, while Table IV lists '95.0', 0.94 precision, 0.95 recall, and 0.95 F1 for the proposed work; the text also says precision, recall, and F1 are all 0.95. In addition, the reference labels do not match the bibliography: 'Neto et al. [50]' and 'Fernández Maimó et al. [56]' correspond to reference entries [50] (Tascı) and [56] (Urooj et al.), so the comparison table cannot be verified as written. Please correct the table, the references, and the unresolved 'Table ??' placeholder.","section":"Table IV and Section IV-C"}],"minor_comments":[{"comment":"The sentence 'The proposed methodology compromises the availability and reliability of IoMT systems' appears to be a wording error; the intended meaning is likely 'comprises' or 'addresses'.","section":"Section III"},{"comment":"The heading 'DDOS' in the Conclusion should be 'DDoS' for consistency, and 'electrocardiogram (ECK/EKG)' in Section II should be 'ECG'.","section":"Sections II and V"},{"comment":"Figure 6 is referenced only as 'the confusion matrix 6,' and the caption lacks the actual matrix values; either add the values to the figure or replace the reference with a properly labeled 'Figure 6'.","section":"Section IV-C"},{"comment":"The dataset name is abbreviated inconsistently as both 'CICIoMT2024' and 'CICIoMT24', and reference [49] has a malformed author field; standardize the abbreviation and fix the citation.","section":"Throughout"},{"comment":"No code, random seeds, software versions, or hardware details are provided, so the experiments are not reproducible as reported; please include these details or a link to a public repository.","section":"Section IV-C"}],"recommendation":"major_revision","confidential_remarks":"This is a borderline major-revision case: the label leak is likely fixable by re-running feature selection and normalization inside the training folds, and an ELM on this dataset may still perform reasonably, but the reported numbers cannot be trusted as-is. The journal should require a leakage-free re-evaluation and concrete resource measurements before any acceptance decision. The comparison table also needs to be corrected because several cited references do not match the stated authors."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a textbook ELM application to the new CICIoMT2024 benchmark, and the headline 95% accuracy is probably optimistic because the feature-selection step peeks at the test set. The resource claim is asserted, not measured. The paper is otherwise straightforward and readable, and the topic is timely, but the evaluation needs to be redone before the numbers can be trusted.\n\nWhat's actually there: the authors train an extreme learning machine for binary DDoS/normal classification on 476k samples from CICIoMT2024. ELM is a well-established fast classifier, and the paper correctly cites prior ELM-based IDS work (including ELM for IoMT), so the novelty is mainly 'new dataset.' That's acceptable for a benchmark study, and the comparison table with other published results on CICIoMT2024 is useful, even if cross-paper comparisons are loose.\n\nThe main soft spot is the pipeline order in Section IV-B. Correlation-based feature selection is described before the 80/20 split. If the correlation with the target is computed on all 476,150 samples, test labels get used to pick features, which can inflate held-out accuracy — especially with a weak 0.02 threshold and many candidate features. The normalization step is also ambiguous: the text places the split before StandardScaler, but that should be explicit. This is not a fatal flaw, but the 94.87% figure isn't established as written.\n\nSecond issue: 'minimal training time and implementation resources' is asserted, never measured. No CPU time, memory, model size, or selected feature count. For a paper whose selling point is low-budget deployment, that's a missing key result.\n\nThird: reproducibility. No hidden-layer size, activation function, or random seed are reported. Minor language and formatting slips (e.g., 'compromises' in Section III, 'Table ??' in Section IV-C) need copyediting.\n\nThe citations are fine; self-citations are for prior ELM work and don't carry the argument.\n\nWho this is for: readers wanting a quick sanity check on whether ELM competes on a recent IoMT IDS benchmark. They'll get a plausible yes, but the numbers are provisional. The paper deserves a serious referee; my recommendation is major revision — move feature selection and normalization inside the training folds, report actual resource measurements, and give the hyperparameters. Without that, the result is only suggestive.","headline":"Routine ELM benchmark with likely test-set leakage in feature selection; the headline accuracy is not yet trustworthy, but the paper is a legitimate candidate for revision.","tokens_in":11145,"tokens_out":2937,"would_cite":false,"duration_ms":34723,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A single-hidden-layer network trained by one pseudoinverse pass can detect DDoS attacks on IoMT traffic with about 95% accuracy, offering a low-resource detection option for constrained medical networks.","keywords":["extreme learning machine","DDoS detection","Internet of Medical Things","IoMT security","intrusion detection","CICIoMT2024","fog computing","low-resource machine learning"],"falsifier":"Re-run the identical ELM pipeline on the same dataset with feature selection applied only to the training portion, or moved inside the cross-validation folds, and compare held-out accuracy; if the 94.87% figure drops substantially, the central claim is inflated by using test information during feature selection.","tokens_in":10143,"feed_emoji":"🩺","tokens_out":9914,"duration_ms":106358,"temperature":0.7,"pith_summary":"This paper sets out to establish that an extreme learning machine—a single-hidden-layer neural network trained in one analytical pass—can reliably separate benign from attack traffic in Internet of Medical Things (IoMT) networks, reporting 94.87% accuracy on the CICIoMT2024 benchmark. The motivation is practical: medical devices are resource-constrained, life-critical, and increasingly hit by DDoS attacks, so detection must be both accurate and cheap to run. The paper argues that because the ELM is non-iterative, it can be trained quickly and deployed at the fog layer, the intermediate computing tier between devices and the cloud, where heavier deep-learning models are not feasible. If the result holds, healthcare networks gain a lightweight, real-time DDoS detector that costs little to implement.","feed_headline":"Single-pass neural net catches IoMT DDoS attacks at 95%","feed_subtitle":"One-pass training and small memory needs could put real-time DDoS detection on fog-level medical networks.","key_machinery":"The load-bearing mechanism is the extreme learning machine (ELM): a single-hidden-layer feedforward network whose input weights and biases are fixed randomly, and whose output weights are solved in closed form as $\\beta = H^{\\dagger} T$, where $H$ is the hidden-layer output matrix and $H^{\\dagger}$ is its pseudoinverse. This makes training non-iterative—a single matrix computation replaces backpropagation—which is what produces the claimed training speed and small implementation budget. The rest of the pipeline uses correlation-based feature selection, dropping features with absolute correlation below 0.02, followed by z-score normalization before the ELM classifies traffic as benign or attack.","core_discovery":"On the CICIoMT2024 benchmark—a realistic IoMT testbed with 40 devices and 18 attack types grouped into five families—an ELM classifier with correlation-based feature selection and z-score normalization reaches 94.87% accuracy, with precision, recall, and F1-score all near 0.95 and an AUC-ROC of 0.945. The paper presents this as evidence that a non-iterative, single-hidden-layer classifier can match the practical utility of far heavier deep models on IoMT attack detection while needing much less training time, memory, and tuning, making it suitable for execution at the fog level. In the comparative analysis, the model is positioned against baselines whose overall accuracy is high but per-class precision and recall are poor, or whose accuracy is only a few points higher but whose resource demands are substantially larger.","pith_inferences":["A leak-free replication would likely lower the reported accuracy; the size of the drop would show how much of the 94.87% is due to feature-selection leakage versus genuine signal.","The single-pass learning rule could be extended to online or sequential updates so the detector adapts to new attack types without full retraining, a direction the paper does not explore.","The paper reports aggregate metrics; per-category results across the five DDoS, DoS, MQTT, Recon, and Spoofing families would reveal whether accuracy is uniform or driven by the most frequent attack types.","Deployment evidence on actual constrained hardware—measuring inference latency, RAM, and power—would test the claimed fog-level suitability more directly than accuracy alone."],"forward_implications":["An ELM-based detector can be trained and retrained in a single pass, making it practical for fog-level and edge deployment on IoMT networks.","At 0.95 recall, the model is claimed to catch most attack traffic while keeping false alarms near 5%, which matters in clinical settings where missed attacks can be life-threatening.","The model provides a lightweight accuracy baseline on CICIoMT2024 that future IoMT intrusion-detection work can compare against.","Because ELM requires fewer tunable hyperparameters than deep networks, the reported approach can be re-implemented and validated on other IoMT traffic datasets at low cost.","The comparison implies ELM trades a few accuracy points (94.87% versus 97–99% for deep hybrids) for a large reduction in training and memory requirements, positioning it for resource-constrained devices."],"supporting_citations":[{"why":"Defines the ELM architecture and its closed-form learning rule, which the proposed classifier directly implements.","marker":"[30]"},{"why":"Supplies the pseudoinverse theory used to compute the output weights in one pass.","marker":"[35]"},{"why":"Provides the CICIoMT2024 benchmark dataset with 18 attack types across 40 IoMT devices that the model is trained and evaluated on.","marker":"[49]"},{"why":"A comparison baseline that reports high accuracy but weak precision and recall, used to motivate the ELM's balanced per-class performance.","marker":"[50]"},{"why":"A security-at-the-edge model with high accuracy but very low precision and recall, used as evidence that high accuracy alone is insufficient for IoMT detection.","marker":"[52]"},{"why":"A deep hybrid model on CICIoMT24 with slightly higher accuracy, used to frame the ELM as a lighter alternative.","marker":"[53]"},{"why":"An ensemble AI model on CICIoMT2024 whose higher accuracy is compared against the ELM's lower resource demands.","marker":"[54]"},{"why":"A CNN-LSTM baseline on CICIoT2023 and CICIoMT24 with higher accuracy but heavier resource use, used to position the ELM for constrained devices.","marker":"[55]"}],"fun_headline_variants":["ELM knocks out DDoS on medical IoT at 95%","One-pass AI blocks IoMT DDoS with 95% accuracy","Low-cost ELM detects IoMT DDoS near 95%","Fog-ready ELM catches DDoS on medical devices","Efficient ELM for IoMT DDoS hits 95%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported 94.87% accuracy assumes the evaluation is leak-free, but the paper describes choosing features by their correlation with the target before dividing the data into training and testing sets, so the test labels could have influenced which features were kept.","fun_headline_variants_meta":{"raw":{"variants":["ELM knocks out DDoS on medical IoT at 95%","One-pass AI blocks IoMT DDoS with 95% accuracy","Low-cost ELM detects IoMT DDoS near 95%","Fog-ready ELM catches DDoS on medical devices","Efficient ELM for IoMT DDoS hits 95%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000142,"raw_usage":{"total_tokens":1120,"prompt_tokens":852,"completion_tokens":268,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":468,"completion_tokens_details":{"reasoning_tokens":178}},"tokens_in":468,"tokens_out":268,"duration_ms":3303,"temperature":1.0,"reasoning_tokens":178,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T19:31:12.723550+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the identical ELM pipeline on the same dataset with feature selection applied only to the training portion, or moved inside the cross-validation folds, and compare held-out accuracy; if the 94.87% figure drops substantially, the central claim is inflated by using test information during feature selection.","supporting_citations":[{"cited_title":"Extreme learning machine: theory and applications,","cited_arxiv_id":null,"evidence_quote":"Defines the ELM architecture and its closed-form learning rule, which the proposed classifier directly implements."},{"cited_title":"The moore–penrose pseu- doinverse: A tutorial review of the theory,","cited_arxiv_id":null,"evidence_quote":"Supplies the pseudoinverse theory used to compute the output weights in one pass."},{"cited_title":"Ciciomt2024: Attack vectors in healthcare devices-a multi-protocol dataset for assessing iomt device security,","cited_arxiv_id":null,"evidence_quote":"Provides the CICIoMT2024 benchmark dataset with 18 attack types across 40 IoMT devices that the model is trained and evaluated on."},{"cited_title":"Deep-learning-based approach for iot attack and mal- ware detection.,","cited_arxiv_id":null,"evidence_quote":"A comparison baseline that reports high accuracy but weak precision and recall, used to motivate the ELM's balanced per-class performance."},{"cited_title":"Security at the edge for resource-limited iot devices,","cited_arxiv_id":null,"evidence_quote":"A security-at-the-edge model with high accuracy but very low precision and recall, used as evidence that high accuracy alone is insufficient for IoMT detection."},{"cited_title":"A Cutting-Edge Deep Learning Method For Enhancing IoT Security","cited_arxiv_id":"2406.12400","evidence_quote":"A deep hybrid model on CICIoMT24 with slightly higher accuracy, used to frame the ELM as a lighter alternative."},{"cited_title":"Enhanced Anomaly Detection in IoMT Networks using Ensemble AI Models on the CICIoMT2024 Dataset","cited_arxiv_id":"2502.11854","evidence_quote":"An ensemble AI model on CICIoMT2024 whose higher accuracy is compared against the ELM's lower resource demands."},{"cited_title":"Enhancing iot security with cnn and lstm-based intrusion detection systems,","cited_arxiv_id":null,"evidence_quote":"A CNN-LSTM baseline on CICIoT2023 and CICIoMT24 with higher accuracy but heavier resource use, used to position the ELM for constrained devices."}],"review_version":1}