{"id":"a12543b0-225d-4120-977a-524759a04960","arxiv_id":"2505.17972","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"MR-EEGWaveNet, a multiresolution extension of EEGWaveNet, reports improved seizure detection F1 scores on Siena and Juntendo datasets, but the post-processing rule as written replaces the classifier output and is inconsistent with the reported results.","lead":"MR-EEGWaveNet extends an existing EEG seizure detection model with multi-resolution feature extraction and reports higher F1 scores on two datasets. The reported gains depend on an anomaly-score post-processing step whose written rule contradicts the observed results, so the headline improvement requires verification.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The written post-classification rule (Eq. 2) cannot produce the reported model-dependent results: an ECOD-only threshold would give identical outputs for same-window models, yet Table 6 shows post-processed F1 = 0.177 vs 0.336.","rationale":"The reader's verdict is REJECT, and this stress-test supports that verdict rather than moving it. The central claim is that MR-EEGWaveNet significantly improves seizure detection, with F1 rising from 0.177 to 0.336 on Siena and 0.327 to 0.488 on Juntendo. Those exact numbers are the post-processed rows of Tables 6 and 7. The load-bearing condition for that claim is therefore that the post-classification rule is exactly as described and that its threshold does not leak test-set information. That condition fails on two independent grounds. First, if Eq. 2 were applied literally using ECOD on raw segments, all models evaluated on the same 10-s segments would produce identical post-processed labels; the reported 0.177 vs 0.336 difference for EEGWaveNet-3 and MR-EEGWaveNet-2 in Table 6 is impossible under the written rule. Some undocumented interaction between the classifier and the anomaly score, or an undocumented difference in how anomaly scores are computed for the multiresolution model, must be responsible. Second, the threshold mu_a is the mean anomaly score of the entire test recording, so the decision rule uses information from the test recording itself, and no held-out threshold selection or error bars are reported. The raw, non-post-processed comparisons in Tables 4 and 5 are more credible and do show a plausible architectural benefit, but the abstract does not cite those numbers; it cites post-processed numbers. Until the post-processing mechanism is disclosed or reproduced by the provided code, the headline claim should not be accepted at face value. This is exactly the same load-bearing concern identified by the reader, so the verdict should remain REJECT.","tokens_in":25613,"tokens_out":5216,"duration_ms":49616,"concrete_test":"Run the literal rule from Eqs. 1-2 on the same LOSO test splits: for each recording, flatten and z-score the 10-s EEG segments, fit ECOD from PyOD, label each segment by whether its anomaly score exceeds the recording mean, and compute per-patient F1. Check whether EEGWaveNet-3 and MR-EEGWaveNet-2 post-processed F1 are 0.177 and 0.336; if not, the published rule is not the actual mechanism. Independently inspect the post-processing function in the GitHub repository and trace whether classifier predictions enter the final label; if they do, the paper must disclose that combination explicitly.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline improvement in the abstract is taken from the post-processed rows of Tables 6 and 7. The only post-classification rule described in Section 4.2, Eq. 2, labels a segment as seizure iff its ECOD anomaly score exceeds the recording mean anomaly score, and Section 5 states ECOD is applied directly to raw flattened, z-scored EEG segments. If this rule is literal, the post-processed labels depend only on the raw EEG and on the recording-level threshold, not on which trained network produced the segment. Then for the same 10-s test segments, EEGWaveNet-3 and MR-EEGWaveNet-2 would receive identical post-processed labels, so their post-processed F1 scores could not differ as reported (0.177 vs 0.336 in Table 6). This internal inconsistency means the actual post-processing mechanism must be different from the published rule, e.g., it may combine classifier outputs with anomaly scores, or compute anomaly scores on different sub-segment representations, neither of which is specified. Because the abstract's central claim attributes the large F1 and precision gains to the multiresolution architecture, and because those gains come from the undocumented post-processing rows, the central claim is not reproducible from the text. A second, related weakness is that the threshold mu_a in Eq. 1 is computed on the full test recording used for evaluation, so test-set information enters the decision rule and no independent threshold validation is reported. The availability of the GitHub repository is a positive feature, but the manuscript itself must state the post-processing rule consistently or supply code that reproduces Tables 6 and 7 exactly.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MR-EEGWaveNet, a multiresolution extension of EEGWaveNet for seizure detection from long multichannel EEG recordings. The architecture adds sub-segment feature extraction at multiple resolutions, and the authors introduce an anomaly-score-based post-classification rule intended to reduce false positives. The model is evaluated with leave-one-subject-out cross-validation on the public Siena dataset and a private Juntendo dataset, with comparisons to EEGWaveNet, ablated variants, and a pretrained foundation model. The abstract's headline improvement is the post-processed F1 increase from 0.177 to 0.336 on Siena and 0.327 to 0.488 on Juntendo.","tokens_in":25876,"tokens_out":4953,"duration_ms":58141,"significance":"If the results were reproducible, the contribution would be a useful incremental improvement: the raw comparisons in Tables 4 and 5 do show that the multiresolution architecture improves precision and F1 over EEGWaveNet, and the use of LOSO evaluation on two datasets, including a private clinical dataset, is a strength. The paper also provides ablation studies and a code repository. However, the central post-classification mechanism, which supplies the abstract's headline numbers, is internally inconsistent as written, and the threshold choice uses test-recording information without independent validation. These issues currently prevent the reported post-processed gains from being evaluated or reproduced.","major_comments":[{"comment":"The post-classification rule as written cannot produce the reported results. In Eq. (2), the final label Li depends only on the ECOD anomaly score ai and the recording-level mean threshold, and Section 5 states that ECOD is applied directly to raw z-scored flattened EEG segments, not to classifier outputs. Therefore, for a fixed test segmentation and a fixed window length, every model would receive identical post-processed labels. Yet Tables 6 and 7 report model-dependent post-processed performance for models sharing the same window length; for example, on Siena 10-s segments, EEGWaveNet-3 has post-processed F1 = 0.177 while MR-EEGWaveNet-2 has post-processed F1 = 0.336. Because the abstract's central quantitative claim is taken from these post-processed rows, the actual post-classification mechanism must be disclosed and the experiments re-reported with a self-consistent rule.","section":"Section 4.2, Eqs. (1)-(2); Section 5; Tables 6-7"},{"comment":"The threshold µa is computed on the full test recording used for evaluation, and Section 7.2 explains that the mean was chosen because experimental results supported it. This means test-set information enters the decision rule and no independent threshold validation is reported. The resulting post-processed precision and F1 numbers are therefore not a clean evaluation of the proposed method; the threshold selection should be performed on training or validation data, or an explicit offline-analysis justification with a separate threshold-fixing procedure should be given.","section":"Section 4.2, Eq. (1); Section 7.2"},{"comment":"The event-based comparison with Sigsgaard et al. reports only recall and specificity, and the event-detection definition is introduced only later in the same section. The claim that the proposed method outperforms the prior work by approximately 13% in seizure detection is not supported by a matched evaluation protocol, since the compared method uses a different evaluation scheme and different event criteria. The comparison should be made on identical segment- or event-level metrics, or clearly labeled as not directly comparable.","section":"Section 7.7, Table 10"}],"minor_comments":[{"comment":"The term 'post-classification processing' is misleading if ECOD is applied directly to raw EEG segments rather than to classifier outputs; the terminology should be aligned with the actual processing chain, especially given the inconsistency flagged in the major comments.","section":"Section 5"},{"comment":"AUC values are omitted for all post-processed rows, making it difficult to assess whether the post-processing step changes ranking quality or only threshold-based metrics; reporting AUC for these rows would strengthen the analysis.","section":"Tables 6 and 7"},{"comment":"The Wilcoxon signed rank test is described only by the resulting p-value; the manuscript should state the test statistic, the number of paired patients, and which patient-wise metric (F1, precision, or AUC) was used for the comparison.","section":"Section 7.5"},{"comment":"The figure labels contain formatting artifacts, such as 'C) Predictor Module' and 'B) Feature Extraction Module§Fully Connected...', which should be cleaned up for readability.","section":"Figures 2a and 2b"}],"recommendation":"major_revision","confidential_remarks":"The internal inconsistency in the post-classification rule is the decisive issue: as written, Eq. (2) is incompatible with the model-dependent post-processed results in Tables 6 and 7. I recommend major revision rather than immediate rejection because the raw architecture comparisons (Tables 4-5) appear to support the core contribution, and the authors may be able to document the actual post-classification procedure and re-validate the threshold. If the actual mechanism cannot be fully specified and validated, the headline post-processed claims should be withdrawn or rejected."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague — the stress-test is right, and it hits the load-bearing claim. Equation (2) labels a segment as seizure iff its ECOD anomaly score exceeds the recording mean, and ECOD is applied to raw flattened z-scored segments, independent of which network produced the segment. So for the same 10-s test windows, EEGWaveNet-3 and MR-EEGWaveNet-2 would receive identical post-processed labels, and their post-processed F1 scores could not be 0.177 vs 0.336. The abstract's headline improvements are taken from those post-processed rows, so as written the architecture is not responsible for them. That is an internal contradiction, not a matter of taste.\n\nWhat is actually new and good: the multiresolution feature fusion — feeding a long segment and its sub-segments through the same convolution and feature-extraction modules, then concatenating — is a sensible extension of EEGWaveNet, and the raw (un-post-processed) comparisons in Tables 4 and 5 show consistent gains (Siena F1 0.161→0.319; Juntendo 0.274→0.474). The LOSO evaluation is appropriate, the ablations are informative, and code is available. Those parts deserve credit.\n\nThe soft spots, in order: (1) the post-processing inconsistency above; (2) the anomaly threshold is computed on the same test recording used for evaluation, so the decision rule leaks test-set information and no error bars are reported; (3) the CBraMod comparison uses frozen features with a stock LightGBM, not a strong baseline, and the Sigsgaard comparison mixes segment-based and event-based metrics. These are secondary.\n\nWho is this for? Someone working on EEG seizure detection who wants to try a multiresolution segment architecture. The raw results are worth a look. But the paper cannot be published with the abstract's numbers attributed to the architecture. The authors need to either correct Eq. 2 to describe the actual post-processing combination (e.g., classifier output plus anomaly score) or supply code that reproduces Tables 6 and 7 exactly. This is fixable in a major revision.\n\nMy recommendation: send it to a knowledgeable referee, but the referee should require the post-processing pipeline to be clarified and the headline claims re-derived from the raw or properly described pipeline. I wouldn't accept it as is.","headline":"The multiresolution idea is real, but the abstract's headline gains come from a post-processing rule that, as written, cannot depend on the architecture.","tokens_in":26513,"tokens_out":5294,"would_cite":false,"duration_ms":40789,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MR-EEGWaveNet roughly doubles seizure-detection F1 on two long scalp-EEG datasets by pooling features from whole segments and sub-segments and filtering outputs with an anomaly-score threshold.","keywords":["seizure detection","electroencephalogram","multiresolutional deep learning","EEGWaveNet","anomaly score post-classification","ECOD","leave-one-subject-out validation","scalp EEG artifacts"],"falsifier":"Re-run the LOSO experiments with the ECOD threshold computed from training-fold segments only (or fixed at a percentile), and check whether the F1 gains from 0.177 to 0.336 on Siena and from 0.327 to 0.488 on Juntendo survive; if they shrink or vanish, the reported advantage comes from the per-recording threshold rather than from multiresolution features.","tokens_in":25366,"feed_emoji":"🧠","tokens_out":6196,"duration_ms":47782,"temperature":0.7,"pith_summary":"The paper claims that a single-resolution seizure detector can be improved by analyzing each EEG segment at several time resolutions at once. Its model, MR-EEGWaveNet, feeds a long segment and its shorter sub-segments through the same convolutional feature extractor, concatenates the resulting features, and then applies an anomaly-score threshold to cut false alarms. On the public Siena dataset the F1 score rises from 0.177 to 0.336, and on the private Juntendo dataset from 0.327 to 0.488, with precision gains of about 16 and 21 percentage points. The improvements matter because scalp EEG is noisy and imbalanced, and a detector that reduces false positives without losing many true seizure segments is the kind of system clinicians could actually use.","feed_headline":"Multiresolution EEG model doubles seizure-detection F1","feed_subtitle":"Lifts F1 to 0.488 on a private dataset while cutting false alarms.","key_machinery":"The central object is the multiresolutional feature vector formed by concatenating the 32-dimensional outputs of the shared convolution-plus-feature-extraction modules for a segment of length $W$ and for each of its sub-segments of lengths in $D$, giving $K = \\sum_{d \\in D} F \\lfloor W/d \\rfloor$ features that the predictor module maps to a class. The second mechanism is the post-classification rule: each segment's ECOD anomaly score is compared with the mean anomaly score of all segments in the recording, and segments below the mean are relabeled nonseizure. Together they let the model keep the statistical context of a long window while retaining the fine temporal detail of short windows, and then suppress scattered false positives.","core_discovery":"The central claim is that multiresolutional feature extraction plus anomaly-score post-processing makes EEGWaveNet markedly better at separating seizures from background EEG and artifacts in long recordings. For a 10-second segment, the model also looks at five 2-second sub-segments, computes a 32-dimensional feature vector for each, concatenates them, and classifies with a small fully connected network. The mean anomaly score of the test recording, computed by ECOD on the raw segments, is then used as a threshold to relabel low-scoring positives as nonseizure. The stated outcome is F1 rising from 0.177 to 0.336 on Siena and from 0.327 to 0.488 on Juntendo, with specificity rising to about 95 percent on both datasets while recall stays near 78-80 percent. Ablations attribute part of the gain to the feature extraction module and part to the 10-second context stream, and the full model also beats a frozen foundation-model baseline on the same leave-one-subject-out protocol.","pith_inferences":["The mean-anomaly threshold is computed per test recording; in a prospective clinical deployment the full recording would not be available at decision time, so a threshold trained on the training distribution or fixed at a quantile would be the honest online variant.","Because the multiresolution gain is measured against EEGWaveNet with different segment lengths but not against a single-resolution model matched in parameter count or compute, part of the improvement could in principle come from added capacity; a parameter-matched ablation would isolate the multiresolution mechanism.","The same ECOD-based post-classification rule could be applied to other segment-level EEG classifiers; testing whether it generalizes across datasets and montages would clarify whether the reported gains are architecture-specific."],"forward_implications":["Window length alone moves performance: longer windows improve capture of seizure regularity but lose temporal detail, and the multiresolution design combines both effects.","The anomaly-score post-processing step improves precision and specificity across all tested single- and multi-resolution models, at a small recall cost.","In leave-one-subject-out evaluation, MR-EEGWaveNet-2 [10 s, 2 s] gives the best balance, and removing either the feature-extraction module or the 10-second stream degrades F1 and AUC.","On the Siena dataset the model detects about 94 percent of annotated seizure events, outperforming the closest LOSO baseline in event-based detection by about 13 points while trailing in specificity by about 4 points.","A task-specific end-to-end model can outperform a frozen large pretrained EEG foundation model plus gradient-boosted classifier on this seizure-detection benchmark."],"supporting_citations":[{"why":"Supplies the base EEGWaveNet architecture and the non-multiresolution baseline that the new model extends.","marker":"[16]"},{"why":"Provides ECOD, the unsupervised outlier detector whose anomaly scores drive the post-classification threshold.","marker":"[46]"},{"why":"Justifies leave-one-subject-out validation, the evaluation protocol used to avoid subject overlap between training and test sets.","marker":"[13]"},{"why":"Provides the public Siena dataset, including patient recordings and seizure labels used in the first evaluation.","marker":"[44]"},{"why":"Serves as the closest LOSO baseline on Siena and the basis for the event-based detection comparison.","marker":"[25]"},{"why":"Supplies the frozen CBraMod foundation model combined with LightGBM as the comparison for the claim that task-specific end-to-end models win.","marker":"[39]"}],"fun_headline_variants":["Multiresolution EEG model doubles seizure-detection F1","Seizure detection F1 more than doubles with MR-EEGWaveNet","MR-EEGWaveNet lifts seizure F1 to 0.488 on private data","Artifact-aware multiresolution EEGWaveNet doubles F1","Long EEG recordings see seizure F1 double via multiresolution"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The post-classification threshold is the mean anomaly score of the very test recording being evaluated, and the paper assumes this per-recording threshold is a valid, unbiased part of the evaluation rather than test-set leakage or an artifact of threshold tuning on the same data.","fun_headline_variants_meta":{"raw":{"variants":["Multiresolution EEG model doubles seizure-detection F1","Seizure detection F1 more than doubles with MR-EEGWaveNet","MR-EEGWaveNet lifts seizure F1 to 0.488 on private data","Artifact-aware multiresolution EEGWaveNet doubles F1","Long EEG recordings see seizure F1 double via multiresolution"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000237,"raw_usage":{"total_tokens":1543,"prompt_tokens":1019,"completion_tokens":524,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":635,"completion_tokens_details":{"reasoning_tokens":429}},"tokens_in":635,"tokens_out":524,"duration_ms":5546,"temperature":1.0,"reasoning_tokens":429,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:36:59.832053+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the LOSO experiments with the ECOD threshold computed from training-fold segments only (or fixed at a percentile), and check whether the F1 gains from 0.177 to 0.336 on Siena and from 0.327 to 0.488 on Juntendo survive; if they shrink or vanish, the reported advantage comes from the per-recording threshold rather than from multiresolution features.","supporting_citations":[{"cited_title":"EEGWaveNet: Multiscale CNN-based spatiotemporal feature extraction for EEG seizure detection,","cited_arxiv_id":null,"evidence_quote":"Supplies the base EEGWaveNet architecture and the non-multiresolution baseline that the new model extends."},{"cited_title":"ECOD: Unsupervised outlier detection using empirical cumulative distribution functions,","cited_arxiv_id":null,"evidence_quote":"Provides ECOD, the unsupervised outlier detector whose anomaly scores drive the post-classification threshold."},{"cited_title":"The necessity of Leave One Subject Out 30 (LOSO) cross validation for EEG disease diagnosis,","cited_arxiv_id":null,"evidence_quote":"Justifies leave-one-subject-out validation, the evaluation protocol used to avoid subject overlap between training and test sets."},{"cited_title":"EEG synchronization analysis for seizure prediction: A study on data of noninvasive recordings,","cited_arxiv_id":null,"evidence_quote":"Provides the public Siena dataset, including patient recordings and seizure labels used in the first evaluation."},{"cited_title":"Improving the generalization of patient non-specific model for epileptic seizure detection,","cited_arxiv_id":null,"evidence_quote":"Serves as the closest LOSO baseline on Siena and the basis for the event-based detection comparison."},{"cited_title":"CBraMod: A criss-cross brain foundation model for EEG decoding,","cited_arxiv_id":null,"evidence_quote":"Supplies the frozen CBraMod foundation model combined with LightGBM as the comparison for the claim that task-specific end-to-end models win."}],"review_version":1}