REVIEW 4 major objections 5 minor 43 references
Understanding and Correcting Low-Frequency Bias in EEG Foundation Model
T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read This paper claims that EEG foundation models are biased toward low frequencies and that FAME, a masked autoencoder that reconstructs per-band standardized log-power time-frequency targets, corrects the bias and reaches state-of-the-art on…
desk verdict A genuinely useful diagnosis of low-frequency bias in EEG foundation models, plus a plausible fix—but the paper's central causal claim is not yet isolated because FAME changes supervision density along with band weighting. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
FAME (Frequency-balanced Autoencoding for Masked EEG) is the central method: from masked channel-time patches it predicts dense time-frequency activity per predefined band, sets the targets as averaged log power within each band, standardizes each band with corpus-level mean and variance, and weights every band's loss equally. The load-bearing mathematical identity is the Parseval expansion of Eq. 1, which shows that pointwise mean-squared error in the time domain equals a power-weighted sum of squared relative spectral errors; this is what turns EEG's $1/f^\alpha$ spectrum into a low-frequency supervision bias. Supporting diagnostics include the Wasserstein distance between broadband and band-limited embeddings and per-band recoverability losses from frozen embeddings, summarized by low-frequency bias, frequency imbalance, and signed frequency slope.
What would settle it
Pretrain a masked autoencoder and FAME on spectrally whitened EEG (per-band variance equalized), then recompute the embedding-distribution distances and per-band recoverability losses; if the low-frequency bias and the FAME downstream gap disappear, the input spectrum was the cause rather than the model's learning dynamics.
Extended reading notes
Core claim
Existing pretrained EEG encoders systematically retain more low-frequency than high-frequency information, a preference that persists across pretraining objectives, dataset scales, and model capacities. The paper explains this by a power-weighted reconstruction argument: by Parseval's theorem, $\|x-\hat{x}\|_2^2 \propto \sum_f |X_f|^2 |r_f|^2$, so under EEG's $1/f^\alpha$ spectrum, equal relative errors at low frequencies contribute far more to the loss than at high frequencies. FAME replaces raw-signal reconstruction with the prediction of band-averaged log power, where each band's target is standardized with corpus-level statistics and band losses are averaged equally, so no band dominates supervision. On the 41-task benchmark, FAME with 1B parameters achieves the best linear-probing balanced accuracy on 24 tasks, and its 50M version improves on the standard masked-autoencoder baseline in 27 of 42 tasks, supporting the claim that balanced spectral supervision, not scale alone, drives transfer.
Load-bearing premise
The core assumption is that the measured low-frequency preference belongs to the learned encoder; if the metrics merely mirror the input power spectrum, the explanation and the FAME advantage are not as clean as claimed.
Editorial extensions
If this is right
- If FAME's central claim holds, a 1B-parameter transformer trained with frequency-balanced targets converts additional capacity into downstream gains, whereas the same scaling in conventional masked autoencoders does not consistently help.
- The paper's bias metrics (low-frequency bias, frequency imbalance, signed slope) can serve as model-level diagnostics: on tasks like TUEV, lower measured bias is associated with higher linear-probing accuracy, so reporting these numbers alongside task accuracy would make representation quality visible.
- The benefit of frequency balancing is task-dependent; tasks requiring information across many frequency bands gain most, while tasks dominated by low-frequency activity may see little benefit, so equal band weighting should be viewed as a default to adjust per task.
- FAME's advantage extends outside linear probing: under full fine-tuning on 13 independent tasks, the 50M model attains the best average balanced accuracy among the compared models, indicating the representation change is not an artifact of probe calibration.
Reading between the lines
- Beyond the paper's claims: a band-normalized reconstruction target is a natural candidate for any 1/f-like neural signal, such as ECoG or local field potentials, where the same power-weighted-loss argument applies.
- Beyond the paper's claims: the bias-performance correlation is based on a small number of pretrained models; a denser sweep over seeds, checkpoints, and architectures would reveal whether the correlation is robust when model size is controlled.
- Beyond the paper's claims: whitening the input spectrum before pretraining is a direct test of the causal story — if low-frequency preference persists under whitened inputs, the network's spectral bias is doing the work; if not, the input spectrum alone explains it.
- Beyond the paper's claims: an adaptive version that learns band weights or band boundaries per task could help tasks with concentrated discriminative bands, but the paper does not explore adaptive partitions.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper argues that existing EEG foundation models exhibit a persistent low-frequency representational bias, explains this bias through the interaction between EEG's 1/f^alpha spectral structure and neural networks' low-frequency learning preference, and proposes FAME, a masked autoencoder that reconstructs independently standardized log-power targets within predefined EEG bands with equal band-wise weighting. The authors evaluate FAME on 41 OmniEEG-Bench tasks with linear probing and report state-of-the-art results on 24 tasks, and further report an advantage under full fine-tuning on a subset of NeuralBench tasks. They also propose diagnostic metrics for frequency bias and correlate them with downstream performance.
Significance. If the causal claim is established, FAME is a simple and broadly applicable pretraining objective, and the proposed frequency-bias diagnostics would be a useful tool for EEG representation analysis. The paper has clear strengths: Eq. (1) is a correct Parseval-based identity that motivates the objective-level imbalance; the controlled small-scale comparison in Fig. 4 is a sensible experimental design; the evaluation covers many datasets; and the Limitations section is honest about the exploratory nature of the bias-performance correlations. However, the central attribution of the gains to band-wise weighting is not yet isolated from supervision density, and several evaluation choices need additional controls before the state-of-the-art claim is fully supported.
major comments (4)
- [Controlled Comparison between Traditional MAE and FAME; Eq. (17)] The controlled comparison is confounded because the FAME loss in Eq. (17) is defined over Omega_b containing 'both masked and unmasked positions', i.e. dense supervision, while the traditional MAE baseline reconstructs only masked positions. The text states that the two models differ only in their reconstruction heads and targets, but the objectives differ in three ways: targets, band weighting, and supervision density. Any of these could flatten the band-wise loss trajectories in Fig. 4(B), reduce the Wasserstein distances in Fig. 4(A), and improve linear probing by providing more gradient signal at high frequencies. The causal claim that 'frequency-balanced supervision' is the active ingredient requires an ablation that fixes supervision density while toggling band standardization and equal weighting. The supplementary FAME-vs-MAE comparison inherits the same confound.
- [Selection of checkpoints and representation layers; Details of Linear Probe of 1B Model] The representation layer and checkpoint were selected using ADHD validation accuracy, and the main text states that the selected 80%-depth layer was applied consistently to all model variants, including the 50M and 1B models. However, the supplementary linear-probing protocol for the 1B model uses the 15th Transformer block of a 20-block network, which is 75% depth, not 80%. This inconsistency changes the representation being probed across model sizes and can affect the scaling comparison and the reported SOTA numbers. In addition, selecting a layer and checkpoint on ADHD and then reporting ADHD as one of the 24 SOTA tasks risks selection bias; the paper should report sensitivity to layer and checkpoint choice across tasks.
- [Frequency Preference of Pretrained Representations; Eqs. (2)-(3)] The Wasserstein distance in Eq. (3) is computed between embeddings of broadband EEG and embeddings of 5-Hz band-limited versions of the same signals. Because raw EEG power decays as roughly 1/f^alpha, high-frequency band-limited inputs have much smaller variance than low-frequency ones, so a larger W1 at high frequencies may reflect input statistics rather than a deficiency of the learned encoder. The recoverability analysis in Eqs. (4)-(5) standardizes targets and is less affected, but the W1 metric in Figs. 3(A) and 4(A) needs a control that normalizes each band-limited input to equal variance, or a comparison against a trivial or linear encoder, before the result is interpreted as a learned representational bias. This is load-bearing for the cross-model persistence claim and for the explanatory narrative.
- [Generalization Performance; Table 1] The state-of-the-art claim on 24 of 41 tasks rests on OmniEEG-Bench without significance tests, and several entries have overlapping standard deviations across models. The benchmark allows model-specific input adapters, which is reasonable but further complicates direct comparability. The paper should report paired significance tests across seeds for the highlighted SOTA differences, and should disclose the authors' role in building OmniEEG-Bench in the main text. As written, the SOTA claim is not statistically grounded.
minor comments (5)
- [Eq. (1)] The displayed equality is approximate because r_f uses epsilon in the denominator; the approximation should be stated explicitly rather than presented as an exact Parseval identity.
- [Supplementary Figure 6] The supplementary text says the MAE comparison covers 42 tasks with FAME better on 27 tasks, while the main text and Table 1 report 41 tasks and 24 SOTA results; the counts should be unified.
- [Table 1, BCIC4-1 row] The entry '56.87±6.2654.50±2.02' is missing a separator between the FAME-50M and FAME-1B values.
- [Architecture descriptions] The 1B model is described with N=24 Transformer layers in the architecture section but with 20 Transformer blocks in the linear-probing supplement; these descriptions should be aligned.
- [Figure 7 and Limitations] The correlation analyses use a small number of models and are not corrected for multiple comparisons; the Limitations paragraph acknowledges low power, but the main-text phrase 'consistent relationships' should be softened to match the exploratory nature of the analysis.
Circularity Check
No significant circularity: the core Parseval-based derivation and FAME objective are self-contained; identified issues are experimental confounds, not derivation-by-construction.
full rationale
The paper's derivation chain is not circular. Eq. (1) uses Parseval's theorem to express the time-domain reconstruction loss as a power-weighted sum of relative spectral errors, which is a mathematical identity and does not presuppose the low-frequency-bias conclusion. The FAME objective (Eqs. 12-17) defines band-wise standardized log-power targets with equal weighting; its downstream evaluation on OmniEEG-Bench and NeuralBench is an empirical measurement, not a consequence of fitting a parameter to those benchmarks. Two concerns raised in the text are real but are not circularity under the strict definition. First, the W1 embedding-similarity metric (Eqs. 2-3) feeds band-limited signals F_f(x_i) into the encoder without power normalization, so, given EEG's 1/f^alpha spectrum, the distance naturally grows with frequency even for an unbiased encoder; this undermines the diagnostic interpretation but is not an equation that reduces to its own input. Second, the controlled comparison (Fig. 4) changes supervision density as well as targets: Eq. (17) says FAME supervises 'both masked and unmasked positions,' whereas standard MAE reconstructs masked positions only; this confounds the active ingredient but is an experimental design issue, not a circular step. The limitations section appropriately labels the bias-generalization correlation as 'exploratory rather than conclusive.' Self-citations (OmniEEG-Bench, Liu et al. 2026) are used for motivation and evaluation, but the external NeuralBench result and the supplementary correlations computed without FAME provide independent support, so the self-citations are not load-bearing. No fitted parameter is renamed as a prediction, and no uniqueness theorem is imported from the authors' prior work.
Assumptions & free parameters
free parameters (4)
- Masking ratio =
0.50 (50M model), 0.75 (1B model)
- Band partition boundaries =
delta, theta, alpha, beta, gamma (conventional; exact cutoffs not stated in main text)
- B_low frequency ranges =
F_low = 1-15 Hz, F_high = 30-45 Hz
- Representation layer and checkpoint =
Layer 9 of 12 (50M), layer 15 of 20 (1B); fifth-epoch checkpoint
assumptions (5)
- domain assumption EEG power spectral density follows an approximate 1/f^alpha law with higher power at low frequencies.
- standard math Neural networks learn low-frequency components earlier and more easily than high-frequency components.
- standard math Parseval's theorem equates temporal MSE with summed squared spectral errors.
- domain assumption A three-layer MLP decoder's per-band recovery loss measures how much frequency-specific information the representation retains.
- ad hoc to paper Equal weighting of standard EEG bands is a beneficial task-agnostic spectral prior.
Cite this review
Pith. "Pith review of Understanding and Correcting Low-Frequency Bias in EEG Foundation Model." pith.science (2026). https://pith.science/paper/POTK32GV
@misc{pith2026260801898,
author = {Pith},
title = {Pith review of: Understanding and Correcting Low-Frequency Bias in EEG Foundation Model},
year = {2026},
howpublished = {\url{https://pith.science/paper/POTK32GV}},
note = {Machine review of arXiv:2608.01898}
}
abstract
Increasing EEG pretraining data scale or model capacity does not consistently improve downstream performance. We identify a persistent low-frequency bias in representations learned by diverse EEG foundation models, which remains across dataset scales, model capacities, and pretraining objectives. Our analysis links this bias to the interaction between EEG's $1/f^\alpha$-like spectral structure and neural networks' tendency to preferentially learn low-frequency components. In masked autoencoders, the $\ell_2$ reconstruction objective further amplifies this imbalance: under comparable relative reconstruction errors, high-power low-frequency components contribute disproportionately to the loss. To address this issue, we introduce FAME, a frequency-balanced masked autoencoding framework that reconstructs time--frequency activity in predefined EEG bands from masked EEG inputs. FAME independently standardizes the reconstruction targets within each band and assigns equal weight to all band-specific losses, thereby balancing supervision across the EEG spectrum. Evaluated on 41 downstream tasks in OmniEEG-Bench, FAME learns more spectrally balanced representations and achieves state-of-the-art performance on 24 of them. These results underscore the importance of balanced spectral supervision for learning transferable EEG representations.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
Advances in Neural Information Processing Systems , volume=
Biot: Biosignal transformer for cross-data learning in the wild , author=. Advances in Neural Information Processing Systems , volume=
-
[2]
arXiv preprint arXiv:2407.20254 , year=
EEGMamba: Bidirectional state space model with mixture of experts for EEG multi-task classification , author=. arXiv preprint arXiv:2407.20254 , year=
-
[3]
2024 IEEE International Symposium on Biomedical Imaging (ISBI) , pages=
Neuro-gpt: Towards a foundation model for eeg , author=. 2024 IEEE International Symposium on Biomedical Imaging (ISBI) , pages=. 2024 , organization=
2024
-
[4]
The Fourteenth International Conference on Learning Representations , year=
Are EEG foundation models worth it? comparative evaluation with traditional decoders in diverse BCI tasks , author=. The Fourteenth International Conference on Learning Representations , year=
-
[5]
arXiv preprint arXiv:2601.17883 , year=
EEG Foundation Models: Progresses, Benchmarking, and Open Problems , author=. arXiv preprint arXiv:2601.17883 , year=
-
[6]
International Journal of Neuroscience , volume=
The brain in fractal time: 1/f-like power spectrum scaling of the human electroencephalogram , author=. International Journal of Neuroscience , volume=. 1992 , publisher=
work page 1992
-
[7]
International conference on machine learning , pages=
On the spectral bias of neural networks , author=. International conference on machine learning , pages=. 2019 , organization=
2019
-
[8]
Aperiodic and Low-Frequency Spectral Bias in Reconstruction based EEG Foundation Models
Aperiodic and Low-Frequency Spectral Bias in Reconstruction based EEG Foundation Models , author=. arXiv preprint arXiv:2605.26434 , year=
Show all 43 references
-
[9]
Frontiers in Human Neuroscience , volume=
BENDR: Using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data , author=. Frontiers in Human Neuroscience , volume=. 2021 , publisher=
2021
-
[10]
Advances in Neural Information Processing Systems , volume=
Eegpt: Pretrained transformer for universal and reliable representation of eeg signals , author=. Advances in Neural Information Processing Systems , volume=
-
[11]
arXiv preprint arXiv:2211.02625 , year=
Maeeg: Masked auto-encoder for eeg representation learning , author=. arXiv preprint arXiv:2211.02625 , year=
-
[12]
International Conference on Learning Representations , volume=
Large brain model for learning generic representations with tremendous EEG data in BCI , author=. International Conference on Learning Representations , volume=
-
[13]
arXiv preprint arXiv:1901.06523 , year=
Frequency principle: Fourier analysis sheds light on deep neural networks , author=. arXiv preprint arXiv:1901.06523 , year=
1901 arXiv
-
[14]
Nature neuroscience , volume=
Parameterizing neural power spectra into periodic and aperiodic components , author=. Nature neuroscience , volume=. 2020 , publisher=
2020
-
[15]
Brain topography , volume=
Separating fractal and oscillatory components in the power spectrum of neurophysiological signal , author=. Brain topography , volume=. 2016 , publisher=
2016
-
[16]
Clinical neurophysiology , volume=
Event-related EEG/MEG synchronization and desynchronization: basic principles , author=. Clinical neurophysiology , volume=. 1999 , publisher=
1999
-
[17]
Proceedings of the IEEE , volume=
Motor imagery and direct brain-computer communication , author=. Proceedings of the IEEE , volume=. 2001 , publisher=
2001
-
[18]
NeuroImage , volume=
Mu rhythm (de) synchronization and EEG single-trial classification of different motor imagery tasks , author=. NeuroImage , volume=. 2006 , publisher=
2006
-
[19]
Journal of clinical sleep medicine , volume=
The visual scoring of sleep in adults , author=. Journal of clinical sleep medicine , volume=
-
[20]
Rules, Terminology and Technical Specifications, Darien, Illinois, American Academy of Sleep Medicine , volume=
The AASM manual for the scoring of sleep and associated events , author=. Rules, Terminology and Technical Specifications, Darien, Illinois, American Academy of Sleep Medicine , volume=
-
[21]
Journal of Clinical Neurophysiology , volume=
High-frequency EEG activity at the start of seizures , author=. Journal of Clinical Neurophysiology , volume=. 1992 , publisher=
1992
-
[22]
Epilepsia , volume=
Hippocampal and entorhinal cortex high-frequency oscillations (100--500 Hz) in human epileptic brain and in kainic acid-treated rats with chronic seizures , author=. Epilepsia , volume=. 1999 , publisher=
1999
-
[23]
Annals of neurology , volume=
High-frequency oscillations as a new biomarker in epilepsy , author=. Annals of neurology , volume=. 2012 , publisher=
2012
-
[24]
arXiv preprint arXiv:2302.14367 , year=
BrainBERT: Self-supervised representation learning for intracranial recordings , author=. arXiv preprint arXiv:2302.14367 , year=
-
[25]
Advances in neural information processing systems , volume=
Self-supervised contrastive pre-training for time series via time-frequency consistency , author=. Advances in neural information processing systems , volume=
-
[26]
Journal of neural engineering , volume=
Uncovering the structure of clinical EEG signals with self-supervised learning , author=. Journal of neural engineering , volume=. 2021 , publisher=
2021
-
[27]
Advances in Neural Information Processing Systems , volume=
Spectral bias in practice: The role of function frequency in generalization , author=. Advances in Neural Information Processing Systems , volume=
-
[28]
arXiv preprint arXiv:2606.00815 , year=
OmniEEG-Bench: A Standardized Evaluation Benchmark for EEG Foundation Models , author=. arXiv preprint arXiv:2606.00815 , year=
-
[29]
arXiv preprint arXiv:2605.08495 , year=
NeuralBench: A Unifying Framework to Benchmark NeuroAI Models , author=. arXiv preprint arXiv:2605.08495 , year=
-
[30]
International conference on learning representations , volume=
NeuroLM: A universal multi-task foundation model for bridging the gap between language and EEG signals , author=. International conference on learning representations , volume=
-
[31]
Advances in Neural Information Processing Systems , volume=
Brant: Foundation model for intracranial neural signal , author=. Advances in Neural Information Processing Systems , volume=
-
[32]
Scientific Data , volume=
Human EEG recordings for 1,854 concepts presented in rapid serial visual presentation streams , author=. Scientific Data , volume=. 2022 , publisher=
2022
-
[33]
2022 , doi =
James F Cavanagh AND Trevor C J Jackson , title =. 2022 , doi =
2022
-
[34]
AND Heald, S
Veillette, J. AND Heald, S. AND Wittenbrink, B. AND Nusbaum, H. , title =. 2022 , doi =
2022
-
[35]
2022 , doi =
Yulin Wang AND Wei Duan AND Debo Dong AND Lihong Ding AND Xu Lei , title =. 2022 , doi =
2022
-
[36]
Hassall AND Yan Yan AND Laurence T
Cameron D. Hassall AND Yan Yan AND Laurence T. Hunt , title =. 2022 , doi =
2022
-
[37]
2025 , doi =
Lukas Gehrke AND Sezen Akman AND Albert Chen AND Pedro Lopes AND Klaus Gramann , title =. 2025 , doi =
2025
-
[38]
Cappelloni AND Ross K
Tong Shan AND Madeline S. Cappelloni AND Ross K. Maddox , title =. 2024 , doi =
2024
-
[39]
Ribeiro AND Miguel Castelo-Branco , title =
Maria J. Ribeiro AND Miguel Castelo-Branco , title =. 2021 , doi =
2021
-
[40]
2024 , doi =
Grootswagers, Tijl AND Robinson, Amanda AND Shatek, Sofia AND Carlson, Thomas , title =. 2024 , doi =
2024
-
[41]
2022 , doi =
Moerel, Denise AND Grootswagers, Tijl AND Robinson, Amanda AND Shatek, Sophia AND Woolgar, Alexandra AND Carlson, Thomas AND Rich, Anina , title =. 2022 , doi =
2022
-
[42]
AND Gobbini, M.I
Haxby, J.V. AND Gobbini, M.I. AND Furey, M.L. AND Ishai, A. AND Schouten, J.L. AND Pietrini, P. , title =. 2023 , doi =
2023
-
[43]
AND Robinson, Amanda K
Shatek, Sophia M. AND Robinson, Amanda K. AND Grootswagers, Tijl AND Carlson, Thomas A. , title =. 2023 , doi =
2023
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.