{"id":"6b7b72e0-a9a6-4a7a-95ec-862d90f78975","arxiv_id":"2412.00378","paper_version":3,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"Bi-Band ECoGNet uses dual-length learnable convolution kernels and a 2D electrode-grid encoder to replace a hand-crafted time-frequency transform, reporting +1.24% accuracy and 6x faster training over MST-ECoGNet on a private macaque ECoG dataset.","lead":"A new deep learning model for classifying visual brain signals (ECoG) from macaque monkeys is reported, claiming a small accuracy gain over the author's prior model at about six times the training speed. The practical interest is faster, smaller decoders for brain-computer interfaces, but the accuracy gap sits within the reported noise.","discovery_kind":"incremental","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central accuracy claim (1.24% improvement over MST-ECoGNet) is not independently supported: kernel sizes and TCN count were selected on the same 5-fold CV used for final reporting, so the reported gain may be selection bias.","rationale":"The reader's weakest assumption points to the same place I would: in-sample hyperparameter selection. I agree. The paper's Section IV-A explicitly describes tuning the TCN count and kernel lengths using the accuracy obtained on the dataset that is later used in Table II for the headline comparison. This is a classic selection-bias setup. The 5-fold CV described in Section III-C averages test-fold accuracy, but because the hyperparameters were chosen using the full dataset (or at least using the same data in an unspecified way), the mean is optimistically biased. The effect can easily exceed 1.24 percent in small neuroimaging datasets. The absence of code, data, or configuration makes it impossible to quantify the bias post hoc. The rest of the paper—module design, frequency-importance experiments, channel contribution maps—is reasonable exploratory analysis, but the central accuracy claim is the one that matters for the abstract and conclusion. I therefore agree with rejection; a nested-CV re-analysis, or a fully held-out test set, would be needed to change that.","tokens_in":12356,"tokens_out":3049,"duration_ms":30841,"concrete_test":"Re-run the comparison with a nested 5-fold cross-validation. Split each subject's data once into 5 outer folds. For each outer training fold, carry out an inner cross-validation to select the TCN count and the two kernel lengths (including single-length baselines), then train Bi-Band ECoGNet and MST-ECoGNet on that outer fold and evaluate on the outer test fold. Report mean and per-fold difference of outer test accuracy for both subjects, with multiple random seeds for confidence intervals. If the outer-test difference is not consistently positive, or is within noise, the claimed 1.24% improvement is not established.","verdict_should_be":"REJECT","load_bearing_attack":"The load-bearing concern is selection bias in the headline accuracy gain. Section IV-A reports that the number of TCNs and the two kernel lengths were chosen after comparing test accuracy on the same visual-ECoG dataset: 'the more kernel (TCN), the better performance' and 'Finally 32 & 512 kernel size are selected, base on the performance' (Figures 5-6). The final 5-fold cross-validated numbers in Table II are then computed on that same dataset, so the reported 54.15% and 35.98% are not out-of-sample accuracies in the required sense; they have already been optimized over the hyperparameters. The resulting difference versus MST-ECoGNet (0.72 points on MonC, 1.76 points on MonJ, 1.24 average) is within the overlap of the reported std values, and no significance test is provided. Without a nested or fully held-out evaluation, the true out-of-sample gain could be zero or negative. The speed and size claims are more robust—replacing the costly MST transform with learned convolutions plausibly explains the 6x speedup—but the paper's central 'higher performance' claim rests on the unvalidated accuracy comparison.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Bi-Band ECoGNet for visual ECoG classification, replacing the computationally expensive Modified Stockwell Transform (MST) preprocessing with an in-network Bi-Band Channel-Wise Transform (Bi-BCWT) composed of 64 TCN modules with two kernel lengths (512 and 32). The architecture adds a 2D spatial-temporal encoder that exploits the 8x16 electrode layout, followed by an EEGNet-style fusion/classification head. On two monkey subjects (MonC and MonJ), the authors report 54.15% and 35.98% accuracy, an average 1.24% improvement over the MST-ECoGNet baseline, a 6x training speedup, and a smaller model. Ablation studies examine the number of TCNs, kernel length, spatial encoder type, and individual channel contributions.","tokens_in":12590,"tokens_out":8606,"duration_ms":72025,"significance":"If the accuracy gain is real, Bi-Band ECoGNet is a useful lightweight alternative to MST-based ECoG decoding: it replaces a storage-heavy, slow preprocessing step with learned convolutions, and the 2D spatial encoder is a sensible way to exploit the electrode array geometry. The channel-importance and frequency-analysis experiments are interesting exploratory contributions. However, the central quantitative claim is not currently supported. Hyperparameters were selected on the evaluation set, the reported error bars overlap, no significance tests are given, and the ablation tables contain internal inconsistencies. The core idea is promising, but the validation must be redone before the performance claims can be accepted.","major_comments":[{"comment":"The final hyperparameters (64 TCN modules and kernel lengths 32 and 512) were selected by comparing test accuracy on the same dataset used for the reported final results (Figures 5 and 6, including the statement 'Finally 32 & 512 kernel size are selected, base on the performance'). Consequently, the 5-fold cross-validated accuracies in Table II have already been optimized over these hyperparameters and are not independent out-of-sample estimates; the reported 1.24% average improvement over MST-ECoGNet may be inflated by selection bias. A nested cross-validation or a fully held-out test set must be used, with hyperparameters chosen using training folds only.","section":"Section IV-A and Table II"},{"comment":"For each subject, the reported mean accuracies overlap within one standard deviation (MonC: 54.15±1.15 vs 53.43±0.55; MonJ: 35.98±1.15 vs 34.22±0.78). No statistical significance test is provided, and with only five folds the variance estimates are noisy. The authors should report per-fold results and apply a paired test across folds (or across repeated runs) to support the claim that the accuracy difference is not due to chance.","section":"Table II"},{"comment":"Table III does not support the claims made in the text. With the 2D encoder, the bi-band 512+32 model achieves 46.04%, which is lower than both the 512-only model (46.13%) and the 32-only model (46.27%), contradicting the statement in Section IV-B that mixed TCNs improve performance. The gain to 46.77% in the last row is confounded by simultaneously switching to the 3D encoder. Furthermore, Section IV-C claims the 3D encoder improves accuracy by 1.04%, but the last two rows of Table III differ by only 0.73 percentage points, and the conclusion cites 0.5%. These numbers must be reconciled, and the bi-band versus solo-band comparison must be conducted with the encoder type held fixed.","section":"Table III and Section IV-C"},{"comment":"The frequency importance test applies 5 Hz band-pass filters to the input ECoG and feeds the filtered signals to a pretrained model. Because the model was trained on full-spectrum data, lower accuracy on filtered inputs reflects distribution shift rather than a causal measure of the frequency bands' contribution to the decision. The strong conclusions in Section V (item 1) — that distinguishing information comes only from the low-frequency region and that the high-frequency domain contains almost no effective information — are therefore not established. These results should be treated as exploratory, or validated by training a model on filtered data.","section":"Algorithm 1 and Figure 7"}],"minor_comments":[{"comment":"Reference '[0]' is undefined; the citations to the baseline architecture should point to [1] or another appropriate reference.","section":"Sections I and IV-C"},{"comment":"'nn.Cov3d' should be 'nn.Conv3d' throughout the table.","section":"Table I"},{"comment":"References [2] and [11] are the same paper, as are [13] and [15]; please deduplicate.","section":"Section III-C"},{"comment":"The Adam optimizer is cited as [32], which is Gross's 'Inferior temporal cortex' entry; the original Adam paper (Kingma and Ba, 2015) or another standard source should be cited instead.","section":"Section III-C"},{"comment":"The table header is garbled, with repeated 'MonJ' labels and unclear column grouping; please reformat the table so that columns for subject, model size, accuracy, and speed are clearly labeled.","section":"Table II"},{"comment":"There are typos in Algorithm 1: 'f ron' and 'f rof f' should be 'from' and 'off'.","section":"Algorithm 1"}],"recommendation":"major_revision","confidential_remarks":"The paper is a deep-learning application manuscript rather than a numerical analysis contribution, which may be a scope mismatch for the journal. More importantly, the central accuracy claim is undermined by in-sample hyperparameter selection and by numerical inconsistencies in the ablation table; the speed and size advantages are more credible. If the authors can rerun the evaluation with a proper held-out protocol, fix the table inconsistencies, and temper the frequency-importance interpretation, the work could be acceptable in an applied ML/BCI venue. The citation problems (undefined [0], duplicated references, wrong Adam citation) also need correction."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know. The architecture is a sensible incremental extension of the author's own MST-ECoGNet: replace the expensive Modified Stockwell Transform with a bank of learnable 1D convolutions (two kernel lengths, 512 and 32) and add a 2D spatial encoder that respects the 8×16 electrode grid. The reported 6x training speedup and the 0.0396M parameter count are credible, because dropping the offline MST feature computation is exactly the kind of thing that would remove a bottleneck. The ablation and frequency-importance plots are genuinely useful exploratory work.\n\nThe problem is the headline accuracy claim. The kernel lengths and TCN count were selected after comparing test accuracy on the same dataset that later appears in the final comparison (Section IV-A: 'Finally 32 & 512 kernel size are selected, base on the performance'). That makes the 54.15% and 35.98% numbers in-sample selections, not out-of-sample results. The 1.24% average gain over MST-ECoGNet is also inside the overlap of the reported standard deviations (e.g., MonC 54.15±1.15 vs 53.43±0.55), and there is no significance test. So the central 'higher performance' claim is not supported. The speedup and size claims are more robust, but the paper markets the accuracy gain as the main contribution.\n\nSoft spots beyond that: no code or data released, so the numbers can't be checked. The frequency-importance test uses the trained model itself, so it is interpretability rather than independent evidence. Table III mixes both subjects and gives only point estimates with no error bars. Minor: the writing has typos and the abstract has duplicate '3)' numbering.\n\nWho is this for? If you work on this exact visual ECoG dataset or on replacing hand-crafted time-frequency features with learned convolutions for BCI, you would want to know about the architecture. The speedup idea is worth copying. But the evaluation needs to be redone: a nested cross-validation or a fully held-out test set, with significance testing and ideally released code. I would send it to peer review because the architectural idea and the speedup are worth referee time, but I would expect major revision and an honest reframing of the accuracy claim.\n\nRecommendation: treat it as a promising but unvalidated engineering note. If the author fixes the evaluation, the speedup alone might justify publication.","headline":"Incremental ECoG architecture with a credible speedup but an unsupported accuracy gain due to in-sample hyperparameter selection.","tokens_in":13167,"tokens_out":3386,"would_cite":false,"duration_ms":30914,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Bi-Band ECoGNet replaces the costly MST transform with learned convolutions to decode visual ECoG faster and slightly more accurately.","keywords":["ECoG decoding","brain-computer interface","visual ECoG classification","multi-class classification","temporal-frequency feature extraction","Bi-Band ECoGNet","2D spatial feature encoder","convolutional neural network"],"falsifier":"Train Bi-Band ECoGNet and MST-ECoGNet on the same two subjects but choose all hyperparameters (kernel lengths, TCN count) on a separate validation split or via nested cross-validation, then compare accuracies on a held-out test set; if the difference is not at least 1.24% on average, or if a model with only one kernel length matches the two-band version on out-of-sample data, the central claim of a frequency-complementarity advantage would be refuted.","tokens_in":12076,"feed_emoji":"🧠","tokens_out":6596,"duration_ms":51964,"temperature":0.7,"pith_summary":"The paper proposes Bi-Band ECoGNet, a compact neural network for decoding visual categories from electrocorticography (ECoG) signals recorded with an 8-by-16 electrode array. It replaces the slow Modified Stockwell Transform (MST) used in prior work with a learnable Bi-Band Channel-Wise Transform made of 64 temporal convolutions, half with kernel length 512 and half with length 32, so the network extracts both low- and high-frequency features directly from raw signals. It also reshapes each frequency feature map back into the electrode-grid layout and applies 2D convolutions to capture spatial structure that a 1D spatial filter would miss. On the two macaque visual ECoG datasets, the model reaches 54.15% and 35.98% accuracy respectively, about 1.24% higher on average than MST-ECoGNet while keeping the same parameter count and training about six times faster. The reason a sympathetic reader would care is that the result suggests the hand-designed time-frequency transform can be replaced by a small learned module without losing decoding power, which makes invasive BCI decoding more practical.","feed_headline":"Visual ECoG decoding gains 1.24% accuracy and 6x speed","feed_subtitle":"A two-band convolution module captures low and high frequencies; a 2D encoder exploits the 8-by-16 electrode grid.","key_machinery":"The central machinery is the Bi-Band Channel-Wise Transform (Bi-BCWT), a bank of 64 channel-wise one-dimensional temporal convolutions with two kernel lengths, 512 and 32, whose learned parameters substitute for the window function of the Modified Stockwell Transform. Half the kernels cover low frequencies, half cover high frequencies, and their outputs are concatenated into a 3D frequency-spatial-temporal feature space. The companion mechanism is the Spatial-Temporal Feature Encoder, which reshapes each feature map into the physical 8-by-16 electrode layout and applies two 2D convolutions, so the network uses the electrode array's geometry as an inductive bias. Together they replace the two most expensive or rigid parts of the prior MST-ECoGNet: the hand-tuned MST time-frequency analysis and the 1D spatial filter.","core_discovery":"The central claim is that a fully learned, two-band temporal-convolution front end can replace the expensive MST time-frequency analysis in ECoG decoding while preserving or improving accuracy. The Bi-Band ECoGNet's Bi-BCWT module uses 64 channel-wise 1D convolutions, 32 with kernel length 512 that attend to low-frequency content and 32 with kernel length 32 that attend to higher-frequency content; an FIR band-pass probing experiment confirms these frequency preferences. Each resulting feature map is reshaped from 128 channels into an 8-by-16 grid matching the electrode array, then passed through two 2D convolutional layers (8-by-8-by-1 and 1-by-2-by-1 kernels) that collapse the spatial dimensions into a 1-by-1-by-T vector, extracting the 2D spatial pattern that the 1D spatial filter of MST-ECoGNet ignores. The paper reports that on subject MonC accuracy rises from 53.43% to 54.15% and on MonJ from 34.22% to 35.98%, at equal or smaller model size (0.0396M parameters), with training time per epoch dropping from 189s to 33s (MonC) and 139s to 21.2s (MonJ). It also reports ablation results showing mixed kernel lengths outperform single lengths, and the 2D encoder outperforms the 1D alternative by 1.04%.","pith_inferences":["The in-sample selection of kernel lengths (32 and 512) from the same evaluation dataset may inflate the reported gains; a nested cross-validation or a fully held-out subject would give a more conservative estimate of the 1.24% improvement.","The FIR band-pass probing method could be turned into a diagnostic tool for other ECoG decoding models to map which frequency bands drive each model's predictions, allowing principled kernel design beyond two bands.","If the low-frequency concentration is general, then downsampling or low-pass filtering ECoG before training might reduce computation further without hurting accuracy, and simpler non-deep baselines might close part of the gap.","The channel heat maps suggest automated channel selection could shrink the 128-channel input to a smaller subset, potentially enabling lighter hardware for clinical BCI."],"forward_implications":["Hand-designed time-frequency transforms can be replaced by learned convolutions in ECoG decoding, removing the need to precompute and store large 3D feature tensors.","A model of 0.0396M parameters with training at roughly 33 seconds per epoch becomes feasible for real-time or repeated BCI retraining on laboratory hardware.","The 2D spatial encoder's gain shows that preserving the physical electrode grid layout during feature extraction improves classification, a design choice that transfers to other grid-based neural recordings.","The frequency-importance ablation indicates that discriminative visual ECoG information concentrates in low-frequency bands, with high-frequency bands near random-guess accuracy, which can guide where to allocate model capacity.","Subject-cross results suggest models trained on one animal do not transfer to another, implicating subject-specific spatial patterns; this motivates channel selection or adaptation as the next step."],"supporting_citations":[{"why":"The prior MST-ECoGNet baseline; its accuracy and per-epoch training times are the comparison numbers.","marker":"[1]"},{"why":"The Modified Stockwell Transform that Bi-BCWT is designed to replace; the paper's speedup claim is defined against it.","marker":"[2]"},{"why":"Source of the visual ECoG dataset and the earlier deep-learning decoder whose accuracy the proposed model surpasses.","marker":"[25]"},{"why":"EEGNet, whose compact fusion and fully-connected design is reused in the feature fusion module of the proposed model.","marker":"[30]"}],"fun_headline_variants":["ECoG decoding gains 1.24% accuracy and 6x speed","Two-band ECoGNet improves accuracy and trains 6x faster","1.24% accuracy boost, 6x speedup for ECoG","Replace MST with learned dual-band filters in ECoG","Bi-Band ECoGNet: +1.24% accuracy, 6x faster training"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that choosing the two kernel lengths (32 and 512) and the 64 TCNs based on the same dataset's test accuracy did not bias the cross-validated results upward; if that selection used the same folds as the final evaluation, the reported 1.24% advantage over MST-ECoGNet could shrink or disappear outside this dataset.","fun_headline_variants_meta":{"raw":{"variants":["ECoG decoding gains 1.24% accuracy and 6x speed","Two-band ECoGNet improves accuracy and trains 6x faster","1.24% accuracy boost, 6x speedup for ECoG","Replace MST with learned dual-band filters in ECoG","Bi-Band ECoGNet: +1.24% accuracy, 6x faster training"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000621,"raw_usage":{"total_tokens":3004,"prompt_tokens":1196,"completion_tokens":1808,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":812,"completion_tokens_details":{"reasoning_tokens":1705}},"tokens_in":812,"tokens_out":1808,"duration_ms":11761,"temperature":1.0,"reasoning_tokens":1705,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T05:26:32.364557+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train Bi-Band ECoGNet and MST-ECoGNet on the same two subjects but choose all hyperparameters (kernel lengths, TCN count) on a separate validation split or via nested cross-validation, then compare accuracies on a held-out test set; if the difference is not at least 1.24% on average, or if a model with only one kernel length matches the two-band version on out-of-sample data, the central claim of a frequency-complementarity advantage would be refuted.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Source of the visual ECoG dataset and the earlier deep-learning decoder whose accuracy the proposed model surpasses."}],"review_version":1}