Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

Mentality: A Mamba-based Approach towards Foundation Models for EEG

T0 review · 4 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A Mamba-based EEG model reaches 0.72 AUROC on seizure detection after self-supervised pretraining.

desk verdict A modest Mamba-on-EEG paper with one useful observation (spectral loss helps reconstruction) and a central pretraining claim that is confounded by unstated training budgets. read the letter →

arxiv 2509.02746 v1 pith:ZY5DQ4RJ submitted 2025-09-02 cs.LG cs.AIq-bio.NC

Saarang Panchavati , Corey Arnold , William Speier This is my paper · ORCID
classification cs.LGcs.AIq-bio.NC
keywords EEGMambaselectivestatespacemodelsseizuredetectionself-supervisedpretrainingfoundationTUSZspectralloss
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a Mamba-based selective state-space model can act as a foundation model for EEG: pretrain an encoder-decoder to reconstruct raw seizure and non-seizure windows, then fine-tune the latent representation for seizure detection. On a held-out set of 43 patients from the TUSZ corpus, the pretrained model reaches a detection AUROC of 0.72, compared with 0.64 when trained from scratch. The paper also shows that adding a spectral (Fourier-domain) loss to plain mean-squared error improves reconstruction, lowering MSE from 0.025 to 0.0063. A reader should care because EEG is noisy, high-dimensional, and labor-intensive to read, and a transferable pretrained model could support diverse neurological diagnoses and recording setups.

What carries the argument

The load-bearing object is the stacked Mamba block—a selective state-space layer with layer normalization and a residual connection—embedded in a SaShiMi/U-Net-style encoder-decoder. Mamba is a sequence model that selectively decides what to retain at each time step, allowing it to handle long-range temporal dependencies in EEG at linear cost. The block does two jobs: during pretraining it must reconstruct the full signal through symmetric downsampling and upsampling, and after fine-tuning its latent representation feeds the seizure classifier. The initial CNN and channel-mixing layer translate raw channels into frequency-based features, and the spectral loss keeps the reconstruction faithfu

What would settle it

Train the same model from scratch with the same number of epochs, same optimizer and learning-rate schedule, and same compute budget as the pretrained-then-fine-tuned model; if the from-scratch AUROC rises to about 0.72, the reported pretraining gain is explained by training conditions rather than the reconstruction objective.

Watch

Extended reading notes

Core claim

The central claim is that self-supervised reconstruction pretraining is what makes a Mamba EEG model useful for seizure detection. The authors build a U-Net-like encoder-decoder whose core layers are Mamba blocks, preceded by a per-channel convolutional layer that learns frequency filters up to 50 Hz and a channel-mixing layer. After pretraining on 10-second TUSZ windows with an MSE-plus-spectral loss, they attach two linear layers to the downsampled latent and train a seizure classifier, reaching AUROC 0.72 on a held-out test set of 43 patients; the same architecture trained from scratch reaches only 0.64. Channel-wise saliency on a seizure sample highlights channels T4 and P4, which the au

Load-bearing premise

The paper credits self-supervised pretraining with improving the model's seizure-detection score from 0.64 to 0.72, but it does not report matching training budgets, number of passes over the data, or optimizer schedules for the two runs, so the improvement may come from confounded training conditions rather than the pretraining task itself.

Editorial extensions

If this is right

  • If the 0.72 AUROC holds, self-supervised reconstruction on unlabeled EEG is enough to pretrain useful representations for seizure detection.
  • The fourfold MSE change implies that frequency-domain supervision should be retained in follow-up work; without it reconstruction degrades substantially.
  • Because downstream classification needs only two linear layers on the latent, the same pretrained backbone could be fine-tuned cheaply for other EEG tasks.
  • The channel-mixing architecture suggests possible adaptation to different montages, though the paper only tests the standard 19-channel setup.
  • Expanded to larger corpora spanning other conditions, the model could become a general EEG foundation model rather than a seizure detector.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A matched-budget comparison—same number of epochs, same optimizer schedule, and same compute for from-scratch and pretrained runs—would settle whether pretraining itself, rather than longer or different training, causes the 0.08 AUROC gain; the paper does not provide that control.
  • Mamba's linear-time inference should make continuous hour-scale EEG processing feasible, but the paper only tests 10-second windows.
  • Explicitly using the learned state-space dynamics, rather than only the final hidden latent, could improve both performance and interpretability; the paper itself notes those dynamics are not currently used.
  • The proposed channel-masked pretraining could be tested directly by random channel dropout during training and measuring AUROC on partial-channel inputs, mimicking wearable EEG setups.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes 'Mentality,' a Mamba-based encoder-decoder for EEG, trained in two stages: self-supervised reconstruction on the Temple University Hospital EEG Seizure Corpus (TUSZ) v2.0.1, followed by supervised seizure detection. The authors report a held-out AUROC of 0.72 with pretraining and 0.64 when training from scratch, and claim that self-supervised pretraining significantly improves downstream detection. They also describe a spectral loss that reduces reconstruction MSE, and provide qualitative interpretability analyses via channel saliency and first-layer weights.

Significance. If the claims hold, the paper would be a useful positive data point for Mamba-style state-space models applied to EEG, and for self-supervised pretraining on physiological signals. The use of a large public corpus with patient-disjoint train/test splits is a genuine strength, and the held-out AUROC is a plausible quantitative signal. However, the central comparison between pretrained and from-scratch models is currently under-controlled, and the absence of uncertainty estimates, strong baselines, and full experimental details weakens the contribution. The work is preliminary but the core idea is worth pursuing; the manuscript needs additional experiments and reporting before the claims can be accepted.

major comments (4)
  1. [Section 3.2 and Discussion] The central claim that self-supervised pretraining improves AUROC from 0.64 to 0.72 is confounded by training budget. The paper does not report the number of epochs, optimization steps, learning rate schedules, or total compute for either the pretrained-plus-fine-tuned model or the from-scratch model. If the from-scratch model was trained for fewer iterations, the gain could be due to additional training budget rather than the representations learned by the reconstruction task. Please report matched training conditions, including total steps and wall-clock time, or run an ablation where the from-scratch model receives the same total number of optimization steps as pretraining plus fine-tuning.
  2. [Section 3.2] Only a single AUROC value is reported for each condition, with no confidence intervals, no standard deviations across seeds, and no repeated runs. Because the difference is only 0.08 AUROC, it is not possible to assess whether this is statistically meaningful. Please provide at least 3–5 independent runs with reported mean and standard deviation, and ideally a bootstrap confidence interval on the held-out test set.
  3. [Section 2.2] The spectral loss is described only in words ('computes the loss in the Fourier domain') and no equation or weighting hyperparameter is provided. Since the paper explicitly credits the spectral loss for a four-fold MSE reduction, this term is load-bearing for the pretraining result. Please define the loss precisely, including the relative weight of MSE and spectral components, and report the hyperparameter value used.
  4. [Section 2.2 and Table-free results] The architecture description omits several details needed to reproduce the model: number of Mamba blocks, hidden dimensions at each downsampling level, downsampling factors, kernel sizes for the double convolution blocks, and the size of the final hidden representation. For a paper proposing a new architecture, these choices are not presentation trivia; they affect the validity of the comparison and the usefulness of the results. Please include a table of architectural hyperparameters and training details.
minor comments (4)
  1. [Section 2.1] The data section states the test subset contains '43 patients, with 469 seizure events' but does not report the number of seizure versus non-seizure windows in the train or test sets. Reporting the window-level class balance is important for interpreting a single AUROC value, especially if the test set is highly imbalanced.
  2. [Section 3.1] The reconstruction MSE values (0.0063 and 0.025) are reported without units or normalization. Since raw EEG amplitude scales with preprocessing, it would be clearer to also report a normalized metric such as relative MSE or signal-to-noise ratio.
  3. [Section 3.3] The interpretability analysis is qualitative. Figure 3 shows a saliency map for one seizure sample; it would strengthen the paper to quantify saliency consistency across samples or to validate the highlighted channels against clinical annotations.
  4. [General] The paper is formatted as a conference submission and contains no code or data availability statement. Given the emphasis on reproducibility in deep learning research, please add a statement about releasing code and model checkpoints.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the AUROC result is an independent held-out measurement, and no claim reduces to its inputs by construction or self-citation.

full rationale

The paper's main result is an empirical AUROC of 0.72 on a held-out test set (Section 3.2), obtained by training a Mamba-based model on TUSZ. There is no fitted parameter that is subsequently renamed as a prediction; the model is evaluated on a disjoint test set of 43 patients. Pretraining uses an MSE plus spectral loss, and downstream detection uses linear layers; neither is defined in terms of the target AUROC. The comparison to a from-scratch model (0.64) is a genuine experimental comparison, although the paper does not report matching training budgets or multiple seeds; that is a validity/confounding concern, not a circularity. The architecture borrows from EEGNet, SaShiMi, and U-Net, but those are standard building blocks, not self-citations carrying the argument. No uniqueness theorem or prior work by the same authors is invoked to force a choice. Therefore no circular step can be identified.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new mathematical or physical postulates. It relies on the TUSZ corpus, standard deep learning assumptions, and the untested premise that reconstruction pretraining generalizes across EEG tasks. The free parameters are mostly unreported training and architecture choices.

free parameters (3)
  • Model hyperparameters (number of Mamba blocks, hidden dimensions, downsampling factors)
    Not reported in the paper; needed for replication and to interpret the pretraining effect.
  • Spectral loss weight
    The spectral loss is said to improve reconstruction (MSE 0.0063 vs 0.025) but its weighting is not reported.
  • 10s window length and 200 Hz resampling = 10 s, 200 Hz
    Design choices from Section 2.1 that shape the input representation.
assumptions (4)
  • domain assumption TUSZ v2.0.1 seizure annotations are correct.
    Labels from the corpus are used without external validation (Section 2.1).
  • domain assumption A window is labeled seizure if any seizure event overlaps it.
    Simplifies detection to binary windows (Section 2.1).
  • domain assumption Disjoint patient split prevents data leakage.
    Training and test patients are disjoint, but preprocessing steps are not fully specified.
  • ad hoc to paper Self-supervised reconstruction on raw EEG is a useful pretext for seizure detection.
    This is the central method premise; only one comparison is offered (Section 3.2).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Mentality: A Mamba-based Approach towards Foundation Models for EEG." pith.science (2026). https://pith.science/paper/ZY5DQ4RJ

@misc{pith2026250902746,
  author       = {Pith},
  title        = {Pith review of: Mentality: A Mamba-based Approach towards Foundation Models for EEG},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZY5DQ4RJ}},
  note         = {Machine review of arXiv:2509.02746}
}
read the original abstract

This work explores the potential of foundation models, specifically a Mamba-based selective state space model, for enhancing EEG analysis in neurological disorder diagnosis. EEG, crucial for diagnosing conditions like epilepsy, presents significant challenges due to its noisy, high-dimensional, and nonlinear nature. Traditional machine learning methods have made advances in automating EEG analysis but often fail to capture its complex spatio-temporal dynamics. Recent advances in deep learning, particularly in sequence modeling, offer new avenues for creating more generalized and expressive models capable of handling such complexities. By training a Mamba-based model on a large dataset containing seizure and non-seizure EEG recordings through a self-supervised reconstruction task followed by a seizure detection task, we demonstrate the model's effectiveness, achieving an AUROC of 0.72 on a held-out test set. This approach marks a significant step toward developing large-scale, clinically applicable foundation models for EEG data analysis.

Figures

Figures reproduced from arXiv: 2509.02746 by the authors.

Figure 1
Figure 1. Proposed Model Architecture. Inspired by SaShiMi, U-Net, and EEGNet, the model learns [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Example reconstruction of input sequence on a test sample. Predicted time series are [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Sample seizure window with channel-wise saliency. Channel opacity is a function of its [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. EEG-FM-Compass: Progress, Benchmarking, and Future Directions for EEG Foundation Models

    cs.LG 2026-01 conditional novelty 6.0 of 10

    A unified benchmark of 12 EEG foundation models across 13 datasets finds specialists remain competitive and larger pre-trained models do not consistently improve downstream decoding.

Reference graph

Works this paper leans on

16 extracted references · 9 canonical work pages · cited by 1 Pith paper

  1. [1]

    Rajendra Acharya, S

    U. Rajendra Acharya, S. Vinitha Sree , G. Swapna, Roshan Joy Martis, and Jasjit S. Suri. Automated eeg analysis of epilepsy: A review. Knowledge-Based Systems, 45: 0 147--165, 2013. ISSN 0950-7051. doi:https://doi.org/10.1016/j.knosys.2013.02.014. URL https://www.sciencedirect.com/science/article/pii/S0950705113000798

    work page doi:10.1016/j.knosys.2013.02.014 2013

  2. [2]

    Cavanagh, Arthur Napolitano, Christopher Wu, and Abdullah Mueen

    James F. Cavanagh, Arthur Napolitano, Christopher Wu, and Abdullah Mueen. The patient repository for eeg data + computational tools (pred+ct). Frontiers in Neuroinformatics, 11, 2017. ISSN 1662-5196. doi:10.3389/fninf.2017.00067. URL https://www.frontiersin.org/articles/10.3389/fninf.2017.00067

    work page arXiv 2017

  3. [3]

    It's raw! audio generation with state-space models, 2022

    Karan Goel, Albert Gu, Chris Donahue, and Christopher Ré. It's raw! audio generation with state-space models, 2022

    work page 2022

  4. [4]

    Mamba: Linear-time sequence modeling with selective state spaces, 2023

    Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces, 2023

    2023

  5. [5]

    A review on machine learning for eeg signal processing in bioengineering

    Mohammad-Parsa Hosseini, Amin Hosseini, and Kiarash Ahi. A review on machine learning for eeg signal processing in bioengineering. IEEE Reviews in Biomedical Engineering, 14: 0 204--218, 2021. doi:10.1109/RBME.2020.2969915

    arXiv 2021

  6. [6]

    Eegnet: a compact convolutional neural network for eeg-based brain--computer interfaces

    Vernon J Lawhern, Amelia J Solon, Nicholas R Waytowich, Stephen M Gordon, Chou P Hung, and Brent J Lance. Eegnet: a compact convolutional neural network for eeg-based brain--computer interfaces. Journal of neural engineering, 15 0 (5): 0 056013, 2018

    work page 2018

  7. [7]

    Foundation models for generalist medical artificial intelligence

    Michael Moor, Oishi Banerjee, Zahra Shakeri Hossein Abad, Harlan M Krumholz, Jure Leskovec, Eric J Topol, and Pranav Rajpurkar. Foundation models for generalist medical artificial intelligence. Nature, 616 0 (7956): 0 259--265, 2023

    work page 2023

  8. [8]

    The temple university hospital eeg data corpus

    Iyad Obeid and Joseph Picone. The temple university hospital eeg data corpus. Frontiers in neuroscience, 10: 0 196, 2016

    work page 2016

Show all 16 references
  1. [9]

    U-net: Convolutional networks for biomedical image segmentation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention--MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part...

    2015

  2. [10]

    Learnable latent embeddings for joint behavioural and neural analysis

    Steffen Schneider, Jin Hwa Lee, and Mackenzie Weygandt Mathis. Learnable latent embeddings for joint behavioural and neural analysis. Nature, pp.\ 1--9, 2023

    2023

  3. [11]

    The temple university hospital seizure detection corpus

    Vinit Shah, Eva von Weltin, Silvia Lopez, James Riley McHugh, Lillian Veloso, Meysam Golmohammadi, Iyad Obeid, and Joseph Picone. The temple university hospital seizure detection corpus. Frontiers in Neuroinformatics, 12, 2018. ISSN 1662-5196. doi:10.3389/fninf.2018.00083. URL...

    2018

  4. [12]

    Modeling multivariate biosignals with graph neural networks and structured state space models

    Siyi Tang, Jared A Dunnmon, Qu Liangqiong, Khaled K Saab, Tina Baykaner, Christopher Lee-Messer, and Daniel L Rubin. Modeling multivariate biosignals with graph neural networks and structured state space models. In Bobak J. Mortazavi, Tasmie Sarker, Andrew Beam, and Joyce C. H...

    2023

  5. [13]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  6. [14]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  7. [15]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  8. [16]

    ߊ0L)e>Ȳ EQZr ZFy ^zZ K ۇ\.׈ X j

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.