{"id":"14052bf1-2eef-4b43-af43-9985f1a407d0","arxiv_id":"2506.00587","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"An ST-GCN on hybrid structural-functional EEG graphs claims 69% accuracy for stress detection, but the evaluation appears to leak subject information and tune hyperparameters on the test set.","lead":"This paper builds a graph from EEG electrode positions and signal correlations, then uses a spatio-temporal graph neural network to classify stressed versus relaxed brain states. The method reportedly beats standard baselines on a 40-subject stress dataset, but the evaluation has several methodological problems that make the headline numbers unreliable.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central accuracy claim rests on a trial-level split that almost certainly leaks subject identity; a subject-independent re-evaluation is needed.","rationale":"The reader's weakest assumption correctly identifies the most load-bearing concern: the trial-level split allows subject identity to leak across train and test. The central claim is a comparative performance claim, and any comparison on overlapping subjects cannot establish that the model detects stress rather than memorizes individuals. The paper's own reported numbers support this concern: with 40 subjects and only 12 trials each, a random stratified split almost certainly creates overlap, and Section 4.3 gives no indication that this was avoided. The proposed grouped cross-validation check is the minimal experiment that would settle whether the reported margin is real. I agree with the REJECT verdict because the current evaluation is not credible, though a corrected subject-independent evaluation could make a future version of the claim testable.","tokens_in":8374,"tokens_out":3055,"duration_ms":33011,"concrete_test":"Re-run the full comparison using grouped cross-validation, e.g., leave-one-subject-out or 5-fold where all 12 trials of a subject are kept in the same fold, and report accuracy/F1 for ST-GCN and the Transformer baseline under identical settings. If the 8-point accuracy and 7-point F1 margins do not persist (or become statistically insignificant), the central claim is an artifact of trial-level subject leakage.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's headline result (Table 3: ST-GCN 69.06% accuracy vs. Transformer 61.25%, F1 78.08% vs. 71.04%) is only meaningful if trials are independent samples. Section 4.3 states only that data was split using 'stratified random sampling', with no subject partitioning. SAM-40 contains 40 subjects with 12 trials each; a random trial-level split places trials from the same subject in both train and test with high probability, letting the model memorize individual EEG signatures instead of learning stress-related structure. The claimed 8-point accuracy and 7-point F1 improvements over the Transformer baseline are therefore not evidence of stress detection unless the same margin survives subject-independent evaluation. This is a structural flaw in the evaluation protocol, independent of model architecture choices, and it directly undermines the central empirical claim. The additional discrepancy between Table 2 (78% for k=2, tau=0.5) and Table 3 (69.06% mean accuracy) reinforces that the reported numbers are protocol-dependent, but subject leakage alone is sufficient to invalidate the comparison as stated.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a spatio-temporal graph convolutional network (ST-GCN) for binary stress classification from raw EEG on the SAM-40 dataset. Each trial is converted into a graph by averaging a structural adjacency (inverse inter-electrode distance with k-nearest neighbors) and a functional adjacency (thresholded Pearson correlations), and the resulting graphs are processed by a temporal Conv1D stage followed by one GCN layer. The authors report that ST-GCN outperforms MLP, LSTM, and Transformer baselines by about 8 accuracy points and 7 F1 points, and they include ablation studies of channels, brain regions, and time segments. The main empirical claim is that integrating electrode geometry with temporal processing improves stress detection on raw EEG.","tokens_in":8496,"tokens_out":5301,"duration_ms":51898,"significance":"If the central claim were supported, the work would be a useful demonstration that geometric deep learning can exploit EEG electrode structure for stress classification, and the ablation studies could offer interpretability insights. The graph-construction method and model architecture are clearly described and easy to reproduce in principle. However, the evaluation as presented does not support the claim: the split protocol is not subject-independent, the reported accuracy is below the trivial majority-class baseline, and the hyperparameter sensitivity analysis appears to select parameters on the test set. These are load-bearing flaws that invalidate the headline comparison, so the current manuscript does not meet the bar for publication. A corrected subject-independent evaluation with proper baselines would be needed before the contribution can be assessed.","major_comments":[{"comment":"The evaluation protocol is described only as 'stratified random sampling' with a 10% validation split and no subject-level grouping. SAM-40 contains 40 subjects with 12 trials each, so a random trial-level split places trials from the same subject in both training and test with high probability. The model can then memorize individual EEG signatures rather than learn stress-related structure, and the reported 8-point accuracy and 7-point F1 improvements over the Transformer are not evidence of stress detection. The authors must re-evaluate with a subject-independent protocol such as leave-one-subject-out or subject-stratified grouping and report performance per subject.","section":"§4.3, Table 3"},{"comment":"With 360 stressed and 120 relaxed trials, a trivial always-stressed classifier achieves 75% accuracy and 85.7% F1. The reported ST-GCN accuracy (69.06%) and F1 score (78.08%) are below these trivial baselines, and the MLP, LSTM, and Transformer baselines are even lower. The paper does not report a majority-class baseline, so the claim that ST-GCN 'outperforms' the baselines is not meaningful for stress detection. The authors should report balanced accuracy, Matthews correlation coefficient, confusion matrices, and comparisons against the majority-class and chance baselines, or use class-balanced training/evaluation.","section":"§5, Table 3"},{"comment":"Table 2 reports 78% accuracy for the chosen configuration k=2, tau=0.5 with a fixed random seed, while Table 3 reports 69.06% ± 8.27 mean accuracy for the same configuration. This discrepancy is unexplained and suggests protocol instability. Moreover, Table 2 is used to select k and tau by comparing test accuracy across nine configurations, which is test-set tuning and inflates the reported effect size. Hyperparameters should be selected on a validation fold, and the selection procedure should be described explicitly.","section":"§3.1, Tables 2 and 3"}],"minor_comments":[{"comment":"There are formatting typos such as 'seperate' and missing spaces in 'wegroupedtherelaxationtrialsintoa'.","section":"§4.1"},{"comment":"The reported average shortest path of 0.55 is impossible for a connected graph with 32 nodes and k=2 nearest neighbors; the minimum possible average shortest path is 1. This suggests the metric is mislabeled or incorrectly computed.","section":"Table 1"},{"comment":"The single-channel ablation reports accuracies of 25–27%, which are below the 50% chance level for binary classification. The text interprets this as meaning no single electrode suffices, but anti-chance performance is not explained and may indicate an implementation issue in the ablation setup.","section":"§5.1"},{"comment":"The conclusion acknowledges the single-dataset limitation and the frontal-lobe anomaly, but it does not acknowledge the more serious evaluation-protocol issues of subject leakage, test-set hyperparameter selection, or below-majority accuracy, which should be addressed before any performance claims are made.","section":"§6"},{"comment":"No code repository, data availability statement, or full hyperparameter details for the baseline models are provided; these would be needed for reproducibility.","section":"§4.3"}],"recommendation":"reject","confidential_remarks":"The central empirical claim is invalidated by the combination of a trial-level split that almost certainly leaks subject identity, test-set hyperparameter tuning, and accuracy below the majority-class baseline. These problems require a substantially different evaluation design and new experiments, not just local corrections, so I cannot recommend major revision on the current manuscript. If the authors perform a proper subject-independent evaluation and report balanced metrics, a resubmission could be considered."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know before you read this: the headline result is almost certainly inflated by trial-level leakage, and the reported 69% accuracy is below the 75% always-stressed baseline. Together, those mean the central claim—that an ST-GCN beats a Transformer for EEG stress detection—is not supported as stated.\n\nWhat's actually new: this is the first application of an ST-GCN with structural-plus-functional graph fusion to stress detection on SAM-40. The graph construction is standard (distance-based k-NN plus correlation threshold), and the architecture is a straightforward spatio-temporal GCN. The writing is clear, and the ablation studies, while flawed, are an honest attempt to probe channel and region importance. They also explicitly flag the frontal-lobe contradiction with prior literature as a limitation, which is more than many papers do.\n\nThe evaluation protocol is the load-bearing flaw. Section 4.3 mentions only \"stratified random sampling\"—no subject-level split. With 12 trials per subject, random trial splitting puts the same subject in train and test with high probability, letting the model memorize per-subject EEG signatures. The 8-point accuracy gain over the Transformer is therefore not trustworthy. Second, all reported accuracies are below the 75% baseline you'd get by always predicting \"Stressed\" (the dataset is 360 stressed vs. 120 relaxed). The paper never mentions this, and it suggests the model may be worse than a trivial predictor. Third, Table 2 reports 78% for k=2, tau=0.5 with a fixed seed, while Table 3 reports 69.06% ± 8.27 for the same configuration across 10 runs. That discrepancy needs explanation, and the hyperparameters were selected using test accuracy, so the main result is partly tuned on the test set. These are not minor issues; together they invalidate the empirical claim.\n\nThe interpretability results are also confusing: single-channel accuracies of 25–27% are below chance, and frontal-region ablations score below chance while the text initially claims consistency with frontal-lobe stress literature. The authors later walk this back, which is honest, but it leaves the interpretability story weak.\n\nWho this is for: someone working on EEG-based affective computing might read it as a cautionary example of evaluation pitfalls. As a scientific claim, it needs a subject-independent split, a majority-class baseline, pre-registered hyperparameters, and ideally significance testing. Without those, the result is not credible.\n\nMy recommendation: I would not publish this in current form, and as an editor I would likely desk reject. If the authors fix the evaluation, the method might be worth another look, but the evidence here does not justify referee time.","headline":"The ST-GCN on raw EEG is a reasonable idea, but the reported gain over baselines is not credible: the evaluation leaks subject identity and the accuracy sits below the 75% majority-class baseline.","tokens_in":9140,"tokens_out":2957,"would_cite":false,"duration_ms":30637,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a spatio-temporal graph convolutional network on a hybrid structural-functional EEG graph detects stress from raw recordings better than MLP, LSTM, and Transformer baselines, gaining 8 accuracy points and 7 F1…","keywords":["Stress Diagnosis","EEG","Graph Neural Networks","Mental Health","Spatio-temporal graph convolutional network","Functional connectivity","Structural connectivity","Interpretability"],"falsifier":"Re-run the same model comparison with a subject-disjoint split (for example, assign all twelve trials of each subject to the same fold) and check whether the ST-GCN's 8-point accuracy advantage over the Transformer survives; if it does not, the reported gain is explained by subject leakage rather than by graph-based stress detection.","tokens_in":8058,"feed_emoji":"🧠","tokens_out":10227,"duration_ms":83249,"temperature":0.7,"pith_summary":"The paper sets out to show that stress is visible in a raw EEG trial as a spatial pattern across the scalp, not only as a temporal waveform, and that a graph neural network built on electrode geometry plus signal correlations is a better reader of that pattern than models that see the time series alone. On the SAM-40 dataset, the proposed spatio-temporal graph convolutional network labels each trial relaxed or stressed with 69.06% mean accuracy and 0.67 AUC-ROC, beating MLP, LSTM, and Transformer baselines by 8 percentage points in accuracy and 7 in F1 score. Because each graph node is a named 10-20 electrode, the same model can be taken apart channel by channel, region by region, and time segment by time segment; those ablations point to central and parietal electrodes and early-to-mid trial windows as the most informative. The broader interest is that an objective EEG-based stress readout could replace subjective questionnaires, and the graph formulation makes the readout interpretable enough to suggest candidate biomarkers.","feed_headline":"Graph network lifts EEG stress accuracy 8 points over Transformer","feed_subtitle":"A graph net fusing electrode geometry with signal correlations spots stress in raw brain waves, channel by channel.","key_machinery":"The load-bearing object is the hybrid adjacency matrix $A=(A_{\\mathrm{struct}}+A_{\\mathrm{func}})/2$. $A_{\\mathrm{struct}}$ marks each electrode's $k=2$ nearest neighbours with inverse-Euclidean-distance weights, giving the graph its anatomical skeleton; $A_{\\mathrm{func}}$ is the indicator that the Pearson correlation between two raw 3200-step signals exceeds $\\tau=0.5$, giving the graph its functional connections. A time-distributed 1D convolution with ReLU and global average pooling first produces per-electrode features, and one graph-convolution layer using the symmetric normalized adjacency with self-loops propagates those features across $A$; a final global pooling and dense sigmoid head make the binary prediction. The fusion is what carries the argument: it turns a set of independent time series into a spatial object whose nodes are named electrodes, which is also what makes the channel- and region-level ablation studies meaningful.","core_discovery":"The central claim is that the hybrid adjacency $A=(A_{\\mathrm{struct}}+A_{\\mathrm{func}})/2$ — an element-wise average of an inverse-distance $k$-nearest-neighbour matrix ($k=2$) and a thresholded Pearson-correlation matrix ($\\tau=0.5$) over the 32 electrodes — gives a graph convolution the right geometry to separate stressed from relaxed EEG. Feeding this graph into a single-layer GCN after per-electrode temporal convolution yields a mean accuracy of $69.06\\% \\pm 8.27$, F1 of $78.08\\% \\pm 8.99$, and AUC-ROC of $0.67$ across ten runs on SAM-40, against $61.25\\%$, $71.04\\%$, and $0.54$ for the best baseline, the Transformer. The paper frames the result as the first ST-GCN for stress detection on raw, unfiltered EEG and supports it with sensitivity analyses over graph-construction parameters, plus ablations showing no single electrode suffices while central and parietal regions and early/mid trial segments carry most of the signal.","pith_inferences":["If the model's advantage survives a subject-disjoint evaluation split, the hybrid structural-functional graph is likely a transferable inductive bias: the same construction should apply to other EEG affective states such as emotion, workload, or fatigue, and to other electrode montages, simply by recomputing the adjacency from electrode positions.","The frontal-lobe anomaly the paper reports — frontal regions scoring below chance despite prior literature — could be tested directly by ablating the functional edge threshold: a pure correlation graph may preserve frontal information that the distance-weighted average dilutes.","The early-segment result suggests a testable extension: training on the first ten seconds or so of each trial might match full-trial accuracy, enabling faster stress screening and wearable deployment.","A control graph with randomly rewired or fully connected edges would isolate whether the gain comes from the specific electrode geometry or from the extra capacity and regularization of the graph convolution itself."],"forward_implications":["If the reported comparison holds, stress classification from raw EEG improves by 8 accuracy points and 7 F1 points over a Transformer baseline without band-pass filtering or manual spectral features.","Because the graph nodes are standard 10-20 electrodes, the same trained model can be interrogated channel by channel and region by region, turning a classifier into a source of hypotheses about where stress appears in the scalp signal.","The stability under graph-construction choices (accuracy varies by less than 6% across $k \\in \\{2,3,4\\}$ and $\\tau \\in \\{0.4,0.5,0.6\\}$) suggests the method does not depend on a finely tuned adjacency.","The temporal ablations imply that the first half of a 25-second trial carries most of the discriminative signal, which matters for any real-time stress monitoring use."],"supporting_citations":[{"why":"Supplies the SAM-40 EEG recordings and the relaxed/stressed trial labels used in every experiment and comparison.","marker":"[7]"},{"why":"Defines structural connectivity as inverse Euclidean distance over k-nearest-neighbour electrodes, the basis of $A_{\\mathrm{struct}}$.","marker":"[2]"},{"why":"Defines functional connectivity by thresholded pairwise correlation, the basis of $A_{\\mathrm{func}}$.","marker":"[20]"},{"why":"Provides the graph-convolutional propagation rule used for spatial message passing.","marker":"[12]"},{"why":"Identifies optimal EEG channels for stress, used to contextualise the single-channel ablation findings.","marker":"[9]"},{"why":"Documents frontal-lobe EEG correlates of mental stress, the expectation against which the paper's frontal ablation result is measured.","marker":"[1]"},{"why":"Supplies the 10-20 electrode placement standard that fixes node positions and therefore the structural graph.","marker":"[10]"},{"why":"Reviews Transformer architectures in EEG analysis, the class of model used as the strongest baseline.","marker":"[25]"}],"fun_headline_variants":["Hybrid graph beats Transformer by 8 on EEG stress","ST-GCN with fused connectomes beats Transformer on EEG stress","Graph blending space and signal spots stress in raw EEG","Geometric graph nets lift EEG stress accuracy over Transformer","Fused connectomes power graph net to top Transformer on stress"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluation assumes that the stratified random split of 480 trials never mixes trials from the same subject into both training and test; if a subject appears on both sides, the model can memorize that person's EEG signature rather than learning stress in general.","fun_headline_variants_meta":{"raw":{"variants":["Hybrid graph beats Transformer by 8 on EEG stress","ST-GCN with fused connectomes beats Transformer on EEG stress","Graph blending space and signal spots stress in raw EEG","Geometric graph nets lift EEG stress accuracy over Transformer","Fused connectomes power graph net to top Transformer on stress"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001542,"raw_usage":{"total_tokens":6138,"prompt_tokens":882,"completion_tokens":5256,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":498,"completion_tokens_details":{"reasoning_tokens":5175}},"tokens_in":498,"tokens_out":5256,"duration_ms":34142,"temperature":1.0,"reasoning_tokens":5175,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T12:02:08.295157+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the same model comparison with a subject-disjoint split (for example, assign all twelve trials of each subject to the same fold) and check whether the ST-GCN's 8-point accuracy advantage over the Transformer survives; if it does not, the reported gain is explained by subject leakage rather than by graph-based stress detection.","supporting_citations":[{"cited_title":"Data in Brief40, 107772 (2022)","cited_arxiv_id":null,"evidence_quote":"Supplies the SAM-40 EEG recordings and the relaxed/stressed trial labels used in every experiment and comparison."},{"cited_title":"Nature Reviews Neuroscience10(3), 186–198 (2009)","cited_arxiv_id":null,"evidence_quote":"Defines structural connectivity as inverse Euclidean distance over k-nearest-neighbour electrodes, the basis of $A_{\\mathrm{struct}}$."},{"cited_title":"NeuroImage52(3), 1059–1069 (2010)","cited_arxiv_id":null,"evidence_quote":"Defines functional connectivity by thresholded pairwise correlation, the basis of $A_{\\mathrm{func}}$."},{"cited_title":"Brain Sciences13(9), 1340 (2023)","cited_arxiv_id":null,"evidence_quote":"Identifies optimal EEG channels for stress, used to contextualise the single-channel ablation findings."},{"cited_title":"Journal of integrative neuroscience21(1), 20 (2022)","cited_arxiv_id":null,"evidence_quote":"Documents frontal-lobe EEG correlates of mental stress, the expectation against which the paper's frontal ablation result is measured."},{"cited_title":"Elec- troencephalogr","cited_arxiv_id":null,"evidence_quote":"Supplies the 10-20 electrode placement standard that fixes node positions and therefore the structural graph."},{"cited_title":"Sensors25(5), 1293 (2025)","cited_arxiv_id":null,"evidence_quote":"Reviews Transformer architectures in EEG analysis, the class of model used as the strongest baseline."}],"review_version":1}