Pith. sign in

REVIEW 4 major objections 6 minor 34 references

One self-supervised EEG transformer pretrained on 47,703 sessions becomes a foundation model that predicts age to 3.06 years MAE and ranks first on three public downstream tasks from a shared backbone.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-11 01:06 UTC pith:M5PZE3GL

load-bearing objection Solid multi-task EEG foundation-model paper; the multi-site pretraining and NeuralBench transfer are real, while the 3.06 yr age headline is softer than the abstract sells. the 4 major comments →

arxiv 2607.06629 v2 pith:M5PZE3GL submitted 2026-07-07 cs.LG q-bio.NC

STST-JEPA: Shallow-Target Spatio-Temporal Joint Embedding Prediction Architecture For EEG Self-Supervised Learning

Roy Segal , Yoni Svechinsky , Tomer Fekete This is my paper
classification cs.LG q-bio.NC
keywords EEGself-supervised learningbrain agefoundation modelJEPAtransformerspatiotemporal maskingmontage heterogeneity
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that EEG can support a practical brain-age biomarker if a single pretrained encoder can absorb montage differences, limited labels, and subject-level non-stationarity across childhood through older adulthood. STST-JEPA is a 24-layer transformer trained with a joint objective: predict latent representations of masked spatiotemporal patches against a shallow EMA tokenizer target, while a down-weighted reconstruction term keeps those latents faithful to the waveform. On frozen embeddings, a lightweight attentive probe reaches a best held-out validation age MAE of 3.06 years (r = 0.924) on 3,367 sessions, far below a predict-the-mean baseline near 10 years. With light final-layer fine-tuning, the same checkpoint places rank 1 on the public NeuralBench × brain.space leaderboard for sex, age, and a psychopathology composite using the model’s native 30-second windows. The age residual, after bias correction, also trends negatively with cognitive efficiency across several tasks, tying the representation to behavioral performance beyond chronological age alone.

Core claim

A single STST-JEPA encoder pretrained without age labels on 47,703 multi-site EEG sessions can serve as an EEG foundation model: frozen embeddings support competitive age regression across ages 5–81, and light task-specific fine-tuning of final layers yields rank-1 leaderboard performance on sex classification, age prediction, and psychopathology regression from one shared backbone.

What carries the argument

Shallow-target spatio-temporal JEPA: masked-token latent prediction against an EMA-of-tokenizer target (not a deep-encoder twin), plus a lighter per-patch signal reconstruction loss, with coordinate-aware pooled multihead attention that collapses arbitrary channel montages into a fixed 128-channel token grid.

Load-bearing premise

That the headline age result fairly proves foundation-model quality when it is the best validation score over training—not a fixed internal test protocol—on a pediatric-heavy corpus whose narrower age spread lowers MAE, while the leaderboard age win uses longer 30-second windows than the standard 2-second protocol.

What would settle it

Retrain or re-evaluate under a single fixed subject-disjoint internal test protocol and under the leaderboard’s standard 2-second windows for all three tasks; if age MAE and rank collapse while sex/psychopathology do not, the foundation claim rests mainly on evaluation choices and window length rather than the joint objective and backbone.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • One frozen EEG backbone can supply competitive probes for age, sex, and paradigm identity without retraining the encoder.
  • Light final-layer fine-tuning of that backbone can transfer to multiple public leaderboard tasks, including a long-horizon psychopathology composite.
  • Bias-corrected brain-age residuals can carry small but directionally consistent links to speeded cognitive efficiency beyond chronological age.
  • Coordinate-aware channel pooling plus a unified channel budget can absorb cross-site montage mismatch without forcing a single physical electrode layout.
  • A deliberately down-weighted reconstruction term can act as a soft floor that keeps latent prediction from drifting away from the waveform.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If corpus scaling and light age-aware pretraining auxiliaries deliver the projected gains the authors sketch, EEG brain-age models may become practical screening tools long before clinical biomarker claims are warranted.
  • The same shallow-target-plus-reconstruction recipe may transfer to other sparse, montage-heterogeneous biosignals (e.g., multi-site MEG or wearable ECG) where pure reconstruction overfits artifact energy.
  • A resting-only BAG analysis would cleanly separate trait age structure from task-evoked dynamics that currently mix into the residual–efficiency correlations.
  • Rank-1 multitask transfer from one checkpoint is stronger evidence of foundation status than any single age MAE, so future work should prioritize fixed multitask protocols over chasing absolute age error alone.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces STST-JEPA, a 24-layer self-supervised EEG transformer pretrained on 47,703 sessions (ages 5–81) from brain.space and HBN. It combines a shallow-target latent prediction loss (predictor outputs matched to an EMA-of-tokenizer target under spatiotemporal block masks) with a down-weighted per-patch smooth-L1 reconstruction term, using coordinate-aware PMA channel pooling over a unified 128-channel budget. A frozen attentive probe reaches best held-out validation age MAE 3.06 yr (r=0.924) on 3,367 sessions; light final-layer fine-tuning yields rank-1 NeuralBench×brain.space placements for sex (bal. acc. 0.911), age (r=0.749), and psychopathology (r=0.215) with native 30 s windows. An exploratory bias-corrected brain-age-gap analysis shows small negative correlations with several cognitive-efficiency targets.

Significance. If the multi-task transfer results hold under matched protocols, this is a useful empirical contribution to EEG foundation models: one backbone, heterogeneous montages, pediatric-to-adult span, and public leaderboard rank-1 on three diverse labels. Strengths include subject-level stratified splits, detailed preprocessing and hyperparameter disclosure (Table 1), transparent caveats (pediatric-heavy MAE, 30 s vs 2 s windows, exploratory BAG, no clinical claim), and an external fixed-protocol benchmark. The shallow-target JEPA-plus-reconstruction design is a clear architectural choice relative to BENDR, LaBraM, EEG2Rep, and related work. The work is significant as an engineering and transfer study more than as a definitive brain-age biomarker paper.

major comments (4)
  1. [Results §Validation Age Regression; Table 2] Results §Validation Age Regression and Table 2: the headline internal age result is the best validation MAE across the training trajectory (3.06 yr combined; 4.82 yr brain.space-only), not a fixed internal test-protocol evaluation. The internal test partition is reserved and only used for exploratory BAG. For a foundation-model claim that leads with competitive age regression across 5–81 years, a single fixed-protocol subject-disjoint test number (or at least reporting the MAE at a pre-specified checkpoint) is load-bearing. Please report fixed-protocol test metrics for age on the reserved internal test set, or reframe the internal age result as a monitoring/stress-test metric rather than a primary endpoint.
  2. [Results §External Benchmark Performance; Table 4; Abstract] Results §External Benchmark Performance and Table 4: NeuralBench age rank-1 (r=0.749) uses unmatched 30 s windows against prior entries evaluated under the standard 2 s protocol; under 2 s the same checkpoint falls to rank 4 (r=0.691) while sex and psychopathology remain rank 1. The abstract and contributions list lead with rank-1 age without foregrounding this mismatch. The multi-task foundation claim is still supported by sex and psychopathology under matched 2 s windows, but age should not be presented as rank-1 without equal prominence of the 2 s result. Please restructure Table 4 / abstract so matched-protocol ranks are primary and 30 s results are secondary.
  3. [Introduction; Methods §Self-Supervised Objective; Discussion] Introduction and Methods §Self-Supervised Objective: the joint latent-plus-reconstruction objective is the paper’s central design choice, yet the manuscript states that controlled ablations were beyond compute budget and are not claimed empirically. Without at least a minimal ablation (latent-only vs reconstruction-only vs joint at fixed compute/steps) on a held-out probe, the claim that the joint formulation is the right inductive bias remains architectural intuition. A small-scale ablation on a subset of the corpus, or an honest demotion of the joint objective from a validated contribution to a design hypothesis, is needed for the Methods framing to match the evidence.
  4. [Abstract; Introduction; Table 2] Abstract and Introduction frame competitive age regression “across the full pediatric to older adult range” as a gap the work fills, but Results explicitly note that the combined 3.06 yr MAE is mechanically lowered by HBN’s pediatric mass and is not comparable to adult clinical benchmarks (e.g., TUAB 7–8 yr MAE). Brain.space-only MAE is 4.82 yr. The abstract should report brain.space-only (or age-stratified) MAE alongside the combined figure, and avoid implying adult-clinical comparability that the body correctly disclaims.
minor comments (6)
  1. [Results figures] Figure numbering is inconsistent in the text: “Figure 1b” is described as the age distribution, then “Figure 2” and “Figure 3” captions appear to reuse “Figure 1b” / “Figure 2” labels in the manuscript dump. Please renumber figures and cross-references consistently.
  2. [Methods §Datasets] Table 0 is numbered before Table 1; consider renumbering cohort summary as Table 1 and shifting the hyperparameter table, or using a non-table summary box.
  3. [Abstract; Introduction] Typographical variants of the model name (STST-JEP A / STST-JEPA) appear in the abstract and introduction; standardize.
  4. [Results §Brain Age Gap] BAG analysis (Table 5, Figure 3/4): effect sizes are small (|r|<0.10) and subject-level n is inflated; the paper already flags this. Consider moving the full 21-target panel to a supplement and keeping only FDR-surviving targets plus the n-inflation caveat in the main text.
  5. [Discussion] Discussion projects falsifiable MAE ranges under 3× corpus scaling and fine-tuning; these are useful but should be clearly boxed as hypotheses so they are not read as secondary claims of the present work.
  6. [Introduction] Cite EEG2Rep more carefully when positioning shallow-target JEPA: the related-work paragraph already mentions it; a short sentence on how shallow EMA-tokenizer targets differ from EEG2Rep’s latent targets would help novelty placement.

Circularity Check

1 steps flagged

Empirical SSL transfer paper with no derivation-by-construction; only mild best-validation selection on the development trajectory.

specific steps
  1. fitted input called prediction [Results §Validation Age Regression; Table 2; Abstract]
    "A lightweight attentive probe trained on frozen pretrained embeddings achieves a best held-out-validation mean absolute error of 3.06 years (r = 0.924) for age regression on 3,367 sessions... The 3.06 year figure is the best validation MAE across the model’s training trajectory; it is not a fixed protocol test set value."

    The headline internal age number is the minimum MAE observed over the pretraining trajectory on the validation partition used for monitoring, not a single fixed-protocol draw on a never-touched internal test set. That is ordinary early-stopping / best-checkpoint selection, not a parameter fit that algebraically forces the reported error; the probe is still trained on frozen embeddings and scored on subject-disjoint sessions. Mild selection bias only; does not make the MAE equal to its inputs by construction.

full rationale

STST-JEPA is an empirical machine-learning paper: a self-supervised pretraining objective (latent prediction against an EMA tokenizer target plus a down-weighted reconstruction term) is trained without age or other downstream labels, then evaluated by frozen attentive probes and light final-layer fine-tuning on held-out sessions and an external NeuralBench leaderboard. There is no first-principles derivation, uniqueness theorem, or fitted constant that is later re-presented as a prediction. Age labels never enter pretraining, so the age MAE and NeuralBench scores are not forced by construction from the SSL loss. The only mild circularity-adjacent practice is reporting the best held-out validation MAE across the training trajectory rather than a single fixed internal test protocol (authors flag this and reserve the internal test partition for exploratory BAG analysis). Hyperparameter and checkpoint selection on the development trajectory is ordinary ML practice and does not reduce the reported metrics to their inputs by definition. Self-citations are absent as load-bearing uniqueness claims; external leaderboard comparisons and subject-disjoint splits provide independent content. Score 1 reflects that minor best-validation selection only.

Axiom & Free-Parameter Ledger

8 free parameters · 6 axioms · 2 invented entities

This is an empirical SSL paper. The load-bearing content is architectural choices, loss weights, data construction, and evaluation protocol rather than mathematical axioms. Free parameters are the usual deep-learning knobs that were selected during development; invented entities are model components, not physical objects. Domain assumptions about EEG preprocessing and subject-level non-leakage are standard but material to the claims.

free parameters (8)
  • lambda_rec (reconstruction weight)
    Set to 0.35 against lambda_lat=1.0; authors say it is a soft floor, not co-equal supervision, and provide no ablation of this weight.
  • lambda_lat (latent prediction weight)
    Fixed at 1.0 as the primary objective scale; jointly determines the training geometry with lambda_rec.
  • EMA momentum schedule (0.9996→0.9999 over 12k steps)
    Hand-chosen JEPA-style target stability schedule; target quality depends on this schedule.
  • Spatiotemporal block mask fractions (~24% realized)
    4 blocks with channel fraction U[0.2,0.5] and time fraction U[0.1,0.3]; defines what the model must predict.
  • Architecture width/depth (d=768, 24 layers, PMA Q=16)
    Capacity and channel-pooling design chosen by authors; central to reported transfer.
  • Window length 30 s / patch 16 samples @ 256 Hz
    Native input scale; also the unmatched advantage versus NeuralBench’s standard 2 s protocol for age rank-1.
  • Artifact amplitude threshold 150 and max invalid fraction 0.45
    Preprocessing rejection rules that shape the training distribution of windows.
  • Probe training caps and early-stopping (5 windows/session, patience 3)
    Affects the frozen-embedding age and auxiliary metrics used as headline evidence.
axioms (6)
  • domain assumption Subject-level stratified splits prevent identity leakage across train/val/test for age and probes.
    Methods §Datasets and Cohort Partitioning; required for interpreting age MAE as subject-general rather than person memorization.
  • domain assumption ICA/ASR-cleaned multi-site EEG still retains age- and trait-discriminative structure after robust per-window normalization.
    Upstream conditioning is heterogeneous across brain.space and HBN; SSL assumes residual signal is useful, not destroyed.
  • domain assumption EMA stop-gradient tokenizer targets plus predictor projection prevent latent collapse under masked prediction.
    Borrowed from BYOL/JEPA literature; not proved here, but assumed by the training recipe.
  • domain assumption NeuralBench held-out test subjects were never used in pretraining despite HBN train overlap.
    External Benchmark Protocol; foundation-model leaderboard claim depends on this disjointness.
  • ad hoc to paper Bias-corrected BAG is a meaningful residual for cognitive efficiency after linear age correction.
    Exploratory BAG analysis; authors note task windows may contaminate residuals and effects are small.
  • standard math Standard transformer/attention math and optimization (AdamW, RoPE, prenorm) behave as in prior literature.
    Background machinery; not re-derived.
invented entities (2)
  • STST-JEPA architecture (shallow-target JEPA + reconstruction + PMA channel pooling) no independent evidence
    purpose: Unify montage-heterogeneous EEG under one self-supervised encoder for multi-task transfer and brain-age decoding.
    Composite model design introduced by the paper; components are known, combination and shallow-target choice are paper-specific.
  • Attentive probe protocol on frozen embeddings no independent evidence
    purpose: Evaluate representation quality for age and auxiliary labels without retraining the backbone.
    Evaluation apparatus rather than a physical entity; results depend on this probe family.

pith-pipeline@v1.1.0-grok45 · 24929 in / 4059 out tokens · 43385 ms · 2026-07-11T01:06:11.502394+00:00 · methodology

0 comments
read the original abstract

Brain age - the age inferred from a physiological recording - is an emerging biomarker whose deviation from chronological age tracks neurological and psychiatric burden, and EEG is an attractive substrate for it because it is cheap, portable, and temporally rich. Yet EEG brain-age models must contend with cross-site montage heterogeneity, small labelled cohorts, and dominant subject-level non-stationarity, and few EEG foundation models have been shown to deliver competitive age regression across the full pediatric to older adult range in which such a biomarker would actually be deployed. We introduce STST-JEPA, a self-supervised transformer for resting-state and task EEG, pretrained on 47,703 sessions spanning ages 5-81 from the brain.space and Healthy Brain Network (HBN) corpora. The model combines a latent-prediction objective - predicting masked-token representations against an EMA-of-tokenizer target - with an auxiliary signal-reconstruction term, applied to 30-second multi-channel windows under spatiotemporal block masks. A lightweight attentive probe trained on frozen pretrained embeddings achieves a best held-out-validation mean absolute error of 3.06 years (r = 0.924) for age regression on 3,367 sessions, against a predict-the-mean baseline of approximately 10 years MAE. With light task-specific finetuning of the model's final layers, the same pretrained encoder achieves rank-1 placements - with the model's native 30-second windows - on the public NeuralBench x brain.space EEG leaderboard for sex classification (balanced accuracy 0.911), age prediction (r = 0.749), and psychopathology composite regression (r = 0.215). We further show that the model's age-prediction residual is negatively correlated with cognitive efficiency over several tasks we examined.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

34 extracted references · 34 canonical work pages · 19 internal anchors

  1. [1]

    A Path Towards Autonomous Machine Intelligence,

    Y. LeCun and Courant, “A Path Towards Autonomous Machine Intelligence,” 2022. Accessed: Jul. 06, 2026. [Online]. A vailable: https://www.semanticscholar.org/paper/A-Path-Towards- Autonomous-Machine-Intelligence-LeCun-Courant/775f42ed458b8c5b0f2094ea4ff5b64c557b1a34

    work page 2022

  2. [2]

    A reusable benchmark of brain-age prediction from M/EEG resting- state signals,

    D. A. Engemann et al., “A reusable benchmark of brain-age prediction from M/EEG resting- state signals,” NeuroImage, vol. 262, p. 119521, Nov. 2022, doi: 10.1016/j.neuroimage.2022.119521

    work page doi:10.1016/j.neuroimage.2022.119521 2022

  3. [3]

    A Simple Framework for Contrastive Learning of Visual Representations

    T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A Simple Framework for Con- trastive Learning of Visual Representations,” Jul. 01, 2020, arXiv: arXiv:2002.05709. doi: 10.48550/arXiv.2002.05709

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2002.05709 2020

  4. [4]

    An open resource for transdiagnostic research in pediatric men- tal health and learning disorders,

    L. M. Alexander et al., “An open resource for transdiagnostic research in pediatric men- tal health and learning disorders,” Sci Data, vol. 4, no. 1, p. 170181, Dec. 2017, doi: 10.1038/sdata.2017.181

    work page doi:10.1038/sdata.2017.181 2017

  5. [5]

    Attention Is All You Need

    A. Vaswani et al., “Attention Is All You Need,” Aug. 02, 2023, arXiv: arXiv:1706.03762. doi: 10.48550/arXiv.1706.03762

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.1706.03762 2023

  6. [6]

    Auto-Encoding Variational Bayes

    D. P. Kingma and M. Welling, “Auto-Encoding Variational Bayes,” Dec. 10, 2022, arXiv: arXiv:1312.6114. doi: 10.48550/arXiv.1312.6114

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.1312.6114 2022

  7. [7]

    BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data

    D. Kostas, S. Aroca-Ouellette, and F. Rudzicz, “BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data,” Jan. 28, 2021, arXiv: arXiv:2101.12037. doi: 10.48550/arXiv.2101.12037

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2101.12037 2021

  8. [8]

    BIOT: Biosignal Transformer for Cross-data Learning in the Wild,

    C. Yang, M. Westover, and J. Sun, “BIOT: Biosignal Transformer for Cross-data Learning in the Wild,” in Advances in Neural Information Processing Systems, Cur- ran Associates, Inc., 2023, pp. 78240–78260. Accessed: Jul. 06, 2026. [Online]. A vailable: https://proceedings.neurips.cc/paper_files/paper/2023/hash/f6b30f3e2dd9cb53bbf2024402d02295- Abstract-Con...

    work page 2023

  9. [9]

    Bootstrap your own latent: A new approach to self-supervised Learning

    J.-B. Grill et al., “Bootstrap your own latent: A new approach to self-supervised Learning,” Sep. 10, 2020, arXiv: arXiv:2006.07733. doi: 10.48550/arXiv.2006.07733

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2006.07733 2020

  10. [10]

    Brain age from the electroencephalogram of sleep,

    H. Sun et al., “Brain age from the electroencephalogram of sleep,” Neurobiol Aging, vol. 74, pp. 112–120, Feb. 2019, doi: 10.1016/j.neurobiolaging.2018.10.016

    work page doi:10.1016/j.neurobiolaging.2018.10.016 2019

  11. [11]

    BrainBERT: Self-supervised representation learning for intracranial recordings

    C. Wang et al., “BrainBERT: Self-supervised representation learning for intracranial record- ings,” Feb. 28, 2023, arXiv: arXiv:2302.14367. doi: 10.48550/arXiv.2302.14367

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2302.14367 2023

  12. [12]

    CBraMod: A Criss-Cross Brain Foundation Model for EEG Decoding,

    J. Wang et al., “CBraMod: A Criss-Cross Brain Foundation Model for EEG Decoding,” Nov. 06, 2025, arXiv: arXiv:2412.07236. doi: 10.48550/arXiv.2412.07236

    work page doi:10.48550/arxiv.2412.07236 2025

  13. [13]

    data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language

    A. Baevski, W.-N. Hsu, Q. Xu, A. Babu, J. Gu, and M. Auli, “data2vec: A General Frame- work for Self-supervised Learning in Speech, Vision and Language,” Oct. 25, 2022, arXiv: arXiv:2202.03555. doi: 10.48550/arXiv.2202.03555

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2202.03555 2022

  14. [14]

    Deep learning with convolutional neural networks for EEG de- coding and visualization,

    R. T. Schirrmeister et al., “Deep learning with convolutional neural networks for EEG de- coding and visualization,” Hum Brain Mapp, vol. 38, no. 11, pp. 5391–5420, Nov. 2017, doi: 10.1002/hbm.23730. 22

    work page doi:10.1002/hbm.23730 2017

  15. [15]

    EEG2Rep: Enhancing Self-supervised EEG Representation Through Informative Masked Inputs

    N. M. Foumani, G. Mackellar, S. Ghane, S. Irtza, N. Nguyen, and M. Salehi, “EEG2Rep: En- hancing Self-supervised EEG Representation Through Informative Masked Inputs,” Jun. 18, 2024, arXiv: arXiv:2402.17772. doi: 10.48550/arXiv.2402.17772

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2402.17772 2024

  16. [16]

    EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces,

    V. J. Lawhern, A. J. Solon, N. R. Waytowich, S. M. Gordon, C. P. Hung, and B. J. Lance, “EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces,” J Neural Eng, vol. 15, no. 5, p. 056013, Oct. 2018, doi: 10.1088/1741-2552/aace8c

    work page doi:10.1088/1741-2552/aace8c 2018

  17. [17]

    iBOT: Image BERT Pre-Training with Online Tokenizer

    J. Zhou et al., “iBOT: Image BERT Pre-Training with Online Tokenizer,” Jan. 27, 2022, arXiv: arXiv:2111.07832. doi: 10.48550/arXiv.2111.07832

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2111.07832 2022

  18. [18]

    Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCI

    W.-B. Jiang, L.-M. Zhao, and B.-L. Lu, “Large Brain Model for Learning Generic Represen- tations with Tremendous EEG Data in BCI,” May 29, 2024, arXiv: arXiv:2405.18765. doi: 10.48550/arXiv.2405.18765

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2405.18765 2024

  19. [19]

    LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics

    R. Balestriero and Y. LeCun, “LeJEPA: Provable and Scalable Self-Supervised Learning With- out the Heuristics,” Nov. 14, 2025, arXiv: arXiv:2511.08544. doi: 10.48550/arXiv.2511.08544

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2511.08544 2025

  20. [20]

    Masked Autoencoders Are Scalable Vision Learners

    K. He, X. Chen, S. Xie, Y. Li, P. Dollár, and R. Girshick, “Masked Autoencoders Are Scalable Vision Learners,” Dec. 19, 2021, arXiv: arXiv:2111.06377. doi: 10.48550/arXiv.2111.06377

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2111.06377 2021

  21. [21]

    Neuro-GPT: Towards A Foundation Model for EEG

    W. Cui et al., “Neuro-GPT: Towards A Foundation Model for EEG,” Mar. 02, 2024, arXiv: arXiv:2311.03764. doi: 10.48550/arXiv.2311.03764

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2311.03764 2024

  22. [22]

    On Layer Normalization in the Transformer Architecture

    R. Xiong et al., “On Layer Normalization in the Transformer Architecture,” Jun. 29, 2020, arXiv: arXiv:2002.04745. doi: 10.48550/arXiv.2002.04745

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2002.04745 2020

  23. [23]

    Predicting Age Using Neuroimaging: Innovative Brain Ageing Biomarkers,

    J. H. Cole and K. Franke, “Predicting Age Using Neuroimaging: Innovative Brain Ageing Biomarkers,” Trends Neurosci, vol. 40, no. 12, pp. 681–690, Dec. 2017, doi: 10.1016/j.tins.2017.10.001

    work page doi:10.1016/j.tins.2017.10.001 2017

  24. [24]

    Predictive coding in the visual cortex: a functional interpre- tation of some extra-classical receptive-field effects,

    R. P. N. Rao and D. H. Ballard, “Predictive coding in the visual cortex: a functional interpre- tation of some extra-classical receptive-field effects,” Nat Neurosci, vol. 2, no. 1, pp. 79–87, Jan. 1999, doi: 10.1038/4580

    work page doi:10.1038/4580 1999

  25. [25]

    REVE: A Foundation Model for EEG – Adapting to Any Setup with Large-Scale Pretraining on 25,000 Subjects,

    Y. E. Ouahidi et al., “REVE: A Foundation Model for EEG – Adapting to Any Setup with Large-Scale Pretraining on 25,000 Subjects,” Oct. 24, 2025, arXiv: arXiv:2510.21585. doi: 10.48550/arXiv.2510.21585

    work page doi:10.48550/arxiv.2510.21585 2025

  26. [26]

    Revisiting Feature Prediction for Learning Visual Representations from Video

    A. Bardes et al., “Revisiting Feature Prediction for Learning Visual Representations from Video,” Feb. 15, 2024, arXiv: arXiv:2404.08471. doi: 10.48550/arXiv.2404.08471

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2404.08471 2024

  27. [27]

    RoFormer: Enhanced Transformer with Rotary Position Embedding

    J. Su, Y. Lu, S. Pan, A. Murtadha, B. Wen, and Y. Liu, “RoFormer: Enhanced Trans- former with Rotary Position Embedding,” Nov. 08, 2023, arXiv: arXiv:2104.09864. doi: 10.48550/arXiv.2104.09864

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2104.09864 2023

  28. [28]

    Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture

    M. Assran et al., “Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture,” Apr. 13, 2023, arXiv: arXiv:2301.08243. doi: 10.48550/arXiv.2301.08243

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2301.08243 2023

  29. [29]

    Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks

    J. Lee, Y. Lee, J. Kim, A. R. Kosiorek, S. Choi, and Y. W. Teh, “Set Transformer: A Frame- work for Attention-based Permutation-Invariant Neural Networks,” May 26, 2019, arXiv: arXiv:1810.00825. doi: 10.48550/arXiv.1810.00825

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.1810.00825 2019

  30. [30]

    Ten Years of BrainAGE as a Neuroimaging Biomarker of Brain Aging: What Insights Have We Gained?,

    K. Franke and C. Gaser, “Ten Years of BrainAGE as a Neuroimaging Biomarker of Brain Aging: What Insights Have We Gained?,” Front Neurol, vol. 10, p. 789, 2019, doi: 10.3389/fneur.2019.00789

    work page doi:10.3389/fneur.2019.00789 2019

  31. [31]

    The free-energy principle: a unified brain theory?

    K. Friston, “The free-energy principle: a unified brain theory?,” Nat Rev Neurosci, vol. 11, no. 2, pp. 127–138, Feb. 2010, doi: 10.1038/nrn2787

    work page doi:10.1038/nrn2787 2010

  32. [32]

    Tracking multiple independent targets: evidence for a parallel tracking mechanism,

    Z. W. Pylyshyn and R. W. Storm, “Tracking multiple independent targets: evidence for a parallel tracking mechanism,” Spat Vis, vol. 3, no. 3, pp. 179–197, 1988, doi: 10.1163/156856888x00122

    work page doi:10.1163/156856888x00122 1988

  33. [33]

    VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning

    A. Bardes, J. Ponce, and Y. LeCun, “VICReg: Variance-Invariance-Covariance Regu- larization for Self-Supervised Learning,” Jan. 28, 2022, arXiv: arXiv:2105.04906. doi: 23 10.48550/arXiv.2105.04906

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2105.04906 2022

  34. [34]

    Whatever next? Predictive brains, situated agents, and the future of cognitive science,

    A. Clark, “Whatever next? Predictive brains, situated agents, and the future of cognitive science,” Behav Brain Sci, vol. 36, no. 3, pp. 181–204, Jun. 2013, doi: 10.1017/S0140525X12000477. 24

    work page doi:10.1017/s0140525x12000477 2013