Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

BrainStratify: Coarse-to-Fine Disentanglement of Intracranial Neural Dynamics

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read BrainStratify claims that a coarse-to-fine pipeline—functional channel clustering followed by decoupled product quantization—outperforms prior self-supervised intracranial speech decoders on sEEG and ECoG.

desk verdict A well-executed recipe that beats Du-IN consistently, but the 'disentanglement' story is not backed by the paper's own ablation; the empirical gains are real, the interpretation is not. read the letter →

arxiv 2505.20480 v1 pith:L2472NG5 submitted 2025-05-26 eess.SP cs.CLq-bio.NC

Hui Zheng , Hai-Teng Wang , Yi-Tao Jing , Pei-Yang Lin , Han-Qing Zhao , Wei Chen , Peng-Hu Wei , Yong-Zhi Shan
show 2 more authors
Guo-Guang Zhao Yun-Zhe Liu
This is my paper · ORCID
classification eess.SPcs.CLq-bio.NC
keywords intracranialneuraldecodingspeechbrain-computerinterfaceself-supervisedlearningdisentangledrepresentationproductquantizationsEEGECoGchannelclustering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that intracranial speech decoding fails in part because electrodes are sparse and the signals they record mix many neural processes at once. BrainStratify attacks both problems in sequence: first it uses a self-supervised spatial-context objective and spectral clustering to group electrodes into functional clusters; then it replaces the single vector quantizer with several parallel sub-quantizers, penalized toward independence, so that each one can capture a different neural dynamic. Across two public sEEG datasets and a new epidural ECoG dataset, the authors report that this two-stage pipeline outperforms existing self-supervised intracranial decoders on word, syllable, and onset-detection tasks. A sympathetic reader would care because the result suggests that explicitly modeling brain modularity, rather than treating all channels as one pooled signal, is a practical path toward more accurate speech brain-computer interfaces.

What carries the argument

The load-bearing mechanism is Decoupled Product Quantization (DPQ), a vector quantizer made of $G$ parallel sub-quantizers. Each sub-quantizer linearly projects neural-token embeddings into its own low-dimensional code space, looks up the nearest code in its own codebook, and contributes a code to a concatenated vector that is mapped back to the model dimension. The training objective adds a partial-correlation penalty $L_{pc}=\sum_{j<k} z_q^{[j]}\cdot z_q^{[k]}$ so that the sub-codexes become group-wise independent along the channel dimension. DPQ's discrete codes then supervise a mask-modeling pretraining stage, which is what lets the encoder learn contextual representations for fine-grained states. The coarse stage feeds DPQ: a spatial-context pretraining task trains the transformer whose attention graph, aggregated across layers, patches, and samples, is spectrally clustered into functional channel groups.

What would settle it

Train BrainStratify-Fine with a single codebook of matched total capacity and without the partial-correlation penalty; if decoding accuracy stays the same, the disentanglement mechanism is not what produces the reported gains. Separately, check whether each sub-codex used alone decodes the task or whether sub-codex choices vary by anatomical site or trial phase—if they do not, the codes are not separating distinct neural dynamics.

Watch

Extended reading notes

Core claim

On its own terms, BrainStratify establishes that intracranial speech recordings contain multiple distinct neural dynamics that can be separated in two stages. The coarse stage learns a spatial-context objective on sEEG, aggregates the spatial transformer's inter-channel attention, and spectrally clusters channels into functional groups. The fine stage then applies Decoupled Product Quantization (DPQ)—$G$ parallel sub-quantizers, each with its own projection and codebook, trained with a partial-correlation penalty that pushes the sub-codexes toward independence—and uses the discrete DPQ codes as supervision for mask modeling. Empirically, the paper reports that this pipeline outperforms all compared baselines on every task: 66.44$\pm$3.65% versus 62.70$\pm$4.69% on the Du-IN 61-word task, 75.36$\pm$3.17% versus 70.66$\pm$3.74% on the 49-syllable CTC task, 58.50$\pm$1.51% versus 52.63$\pm$1.68% word accuracy on the authors' new epidural ECoG dataset, and top ROC-AUCs of 0.79/0.91/0.95/0.98 on the Brain Treebank tasks. The authors conclude that combining data-driven channel stratification with neuroscience-inspired modularity is a robust way to decode speech from intracranial signals.

Load-bearing premise

The load-bearing premise is that the partial-correlation penalty and parallel sub-codebooks force the quantizer to learn independent components that correspond to distinct neural dynamics, rather than simply adding capacity through more codebooks and trainable projections.

Editorial extensions

If this is right

  • Channel selection no longer needs large labeled sets: the coarse stage clusters channels using self-supervision and matches supervised multi-channel selection on the 61-word task.
  • Decoupling the codebook into parallel sub-quantizers improves both word classification and syllable CTC decoding, with the largest absolute gains on epidural ECoG, where re-referencing is unavailable.
  • Mask modeling over DPQ codes transfers to speech-perception tasks, as shown by improved ROC-AUC on sentence-onset and word-onset detection in the Brain Treebank dataset.
  • Because the framework is trained per subject with its own codebooks, it can be applied to new recording modalities and channel layouts without retuning the architecture.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The authors' ablation shows that removing the partial-correlation penalty causes only minor performance drops because separate initialization of each sub-quantizer projection mimics the regularization effect; this implies the reported gains may come largely from added codebook capacity and initialization, not from verified disentanglement.
  • No independent evidence yet shows that individual sub-codexes track distinct neural processes; a direct check would be to decode using each sub-codex alone or to correlate sub-codex usage with anatomical locations or trial phases. If the sub-codexes behave interchangeably, the disentanglement claim is not supported beyond accuracy numbers.
  • The single-subject epidural ECoG result suggests clinical viability for an ALS user, but cross-subject replication is an untested extension before the approach can be claimed as a general neuroprosthetic solution.
  • A plausible extension, not tested here, is to apply the same coarse-to-fine pipeline to other high-dimensional neural recordings such as micro-electrode arrays or high-density EEG, where channel sparsity and signal mixing are similarly problematic.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes BrainStratify, a two-stage self-supervised framework for decoding speech from intracranial signals. In the coarse stage, a temporal-spatial transformer is pre-trained with a spatial-context task and spectral clustering is applied to the resulting inter-channel attention graph to identify functional channel groups. In the fine stage, a VQ-VAE with Decoupled Product Quantization (DPQ) — several parallel sub-codexes with an orthogonality penalty between their outputs — learns discrete neural codes, followed by mask modeling (MAE) pre-training and supervised fine-tuning for word/syllable classification and CTC tasks. The framework is evaluated on the Du-IN sEEG dataset (12 subjects), the Brain Treebank sEEG dataset (10 subjects), and a newly collected single-subject epidural ECoG dataset, reporting accuracy or ROC-AUC improvements over several baselines, including the authors' prior Du-IN method.

Significance. The empirical effort is substantial: three datasets, six seeds, subject-wise tables in the appendix, and a clean differential ablation (Figure 10) showing that removing the DPQ module reduces performance to the Du-IN baseline. If the disentanglement claim were supported, the paper would offer a useful step toward interpretable, modular neural decoding. As it stands, the reported accuracy gains are real but the central conceptual claim — that DPQ disentangles distinct neural dynamics — is not demonstrated by the evidence presented. The unsupervised coarse channel selection matching supervised MC selection is a valuable and defensible contribution in its own right. The careful reporting of subject-wise results and ablations deserves explicit credit.

major comments (3)
  1. [Section 3.3, Eq. (3) and Appendix I] The 'partial-correlation constraint' Lpc is defined as a sum of pairwise inner products zq[j]_i · zq[k]_i between quantized outputs of different sub-quantizers; this is an orthogonality penalty, not a partial correlation. More importantly, Appendix I states that removing Lpc 'causes only minor performance drops' because separate initialization of the embedding-to-codex projections 'mimics the regularization effect of Lpc.' Thus the accuracy gains of BrainStratify-Fine over Du-IN in Table 3 are not evidence that distinct neural dynamics are being decoupled; they are consistent with added codebook capacity and projection diversity from G parallel codexes. The central conceptual claim in Section 4.4 ('DPQ's effectiveness in identifying fine-grained states') is therefore unsupported. Please provide direct evidence of sub-codex specificity, such as codebook usage statistics, mutual information between each sub-codex assignment and phonetic/articulatory or temporal variables, or leave-one-sub-codex-out decoding; if the sub-codexes are redundant, the disentanglement claim should be removed or reframed as codebook expansion.
  2. [Table 3, Du-IN rows] The coarse stage does not actually improve downstream decoding in the final pipeline: BrainStra.-Fine with BrainStra.-Coarse selection reaches 66.44±3.65 word accuracy and 75.36±3.17 syllable accuracy, versus 66.35±3.86 and 75.54±3.19 with supervised MC selection. The paper's statement that BrainStratify-Coarse 'enhanc[es] downstream channel aggregation' (Section 4.4) is not supported by these numbers. The defensible and still valuable claim is that unsupervised coarse selection matches supervised MC selection; this should be stated explicitly and supported by a comparison against random or other unsupervised selections in the final pre-trained pipeline, since Table 2 evaluates channel selection strategies only with a non-pre-trained backbone.
  3. [Abstract and Tables 3-4] The claim that BrainStratify 'significantly outperforms' all baselines is not backed by any statistical test. The headline comparisons show overlapping standard errors (e.g., Table 3, Du-IN word: 66.44±3.65 vs Du-IN 62.70±4.69; Du-IN syllable: 75.36±3.17 vs 70.66±3.74). Because six seeds and subject-wise results are available in Appendix K, a paired permutation test across subjects and seeds should be reported for the main comparisons; otherwise the abstract's 'significantly' is unsupported.
minor comments (5)
  1. [Reproducibility Statement] The code link is given as 'TODO'; a working repository or supplementary file is needed before publication.
  2. [Table 4] The header contains a typo: 'V olumn' should be 'Volume'.
  3. [Appendix B.2] The Brain Treebank analysis window is narrowed from PopT's [-2.5,2.5]s to [-2.0,2.0]s; this protocol deviation is disclosed only in the appendix and should be highlighted in the main text because it changes the exact task definition and affects comparability with prior published PopT numbers.
  4. [Figure 4] The caption says 'codex groups (from 0 to 8)', but G=0 is undefined; the leftmost point presumably corresponds to the Du-IN model with G=1 and a larger codex, and this should be clarified.
  5. [Eq. (2)] The notation z_g^i for the selected code index is used before it is defined; please define z_g^i = arg min_j ... before writing c^g_{z_g^i}.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: DPQ gains come from held-out test comparisons against a clean ablation (Du-IN minus DPQ), and the inert-Lpc admission weakens interpretability but not the empirical derivation.

full rationale

The closest circular benchmark is Du-IN [55], a prior method by overlapping authors, but the differential test is clean: the paper states that 'removing DPQ entirely reduces BrainStratify-Fine to Du-IN [55]' (Appendix I), and all accuracies are measured on held-out test splits with fixed train/validation/test partitions shared across methods. The DPQ contribution is therefore measured, not fitted. The only load-bearing self-citation is the adoption of Du-IN's Neural Encoder and two-stage VQ-VAE/MAE pipeline, which is explicitly disclosed in Section 3.3 and does not import a uniqueness result or forbid alternatives. No equation reduces to its own input: the codebook assignment in Eq. (2) and the MAE objective in Eq. (6) are standard and independently optimized. However, the paper's own Appendix I seriously weakens the conceptual claim: Lpc is a pairwise inner-product penalty, and its removal 'causes only minor performance drops' because separate initialization 'mimics the regularization effect of Lpc,' meaning the reported accuracy gains do not demonstrate learned disentanglement of distinct neural dynamics. That is a missing-support/correctness concern, not circularity, because the accuracy numbers themselves are not constructed from the labels used to evaluate them. Section 4.3's statement that clusters are selected 'based on their performance in specific downstream tasks' is also a potential leakage risk if the selection used test labels, but the same procedure is applied to all clustering baselines and no by-construction equivalence is established. Score 2 reflects the minor self-citation exposure; the empirical core is not circular.

Assumptions & free parameters 7 free parameters · 6 assumptions · 1 invented entities

The framework imports most of its machinery from cited prior work (Du-IN pipeline, PopT spatial context task, product quantization, partial-correlation penalties) and the new elements (G, Ncodex, dcodex, k) are tuned on validation sets of the same datasets used for the headline results. The central interpretive claim, that DPQ sub-codexes correspond to distinct neural dynamics, rests on a domain assumption that the paper's own ablation weakens.

free parameters (7)
  • Number of channel clusters k = 10
    Hard clustering with k=10 applied to the attention connectivity matrix (Section 4.3, Table 2); no criterion for choosing k is given.
  • DPQ codex groups G = 4
    Selected by validation search on the Du-IN 61-word task (Figure 4a); G=1 with Ncodex=2048 is the Du-IN-equivalent baseline.
  • Codex size per group Ncodex = 256
    Selected by validation search (Figure 4b), with capacity kept comparable to the G=1 setting.
  • Codex embedding dimension dcodex = 64
    Selected by validation search (Figure 4c); described as providing more effective regularization.
  • VQ commitment loss weight beta = not reported
    Appears in Eq. 3 but its value is never stated, leaving the reconstruction/commitment balance unspecified.
  • Spatial context corruption and positive-label rates = 10% replaced; 10% of unreplaced channels labeled positive
    Set by hand in Section 3.2 and Appendix D.1; the labeling rule is ambiguous because positives should be the replaced channels.
  • MAE mask ratio = ~50%
    Set by hand in Eq. 4 (Section 3.3).
assumptions (6)
  • domain assumption The spatial context task makes the Spatial Transformer's attention reflect functional inter-channel connectivity.
    This is the foundation of the coarse stage (Eq. 1, Algorithm 1); the paper provides no lesion test or physiological verification that the connectivity matrix P has functional meaning.
  • domain assumption Population intracranial signals are mixtures of discrete neural dynamics that can be separated by orthogonal sub-codebooks.
    Underpins DPQ (Eq. 3); the paper's own ablation (Appendix I) finds the orthogonality penalty contributes little.
  • domain assumption Channels in a functional group are complementary, so spectral clustering of the attention graph yields task-relevant groups.
    Borrowed from the modularity literature [2,4,28,36] and used to justify group selection in Section 3.2.
  • domain assumption The two-stage VQ-VAE plus MAE pre-training transfers to downstream decoding.
    Inherited without re-derivation from Du-IN [55] (Section 3.3, Figure 6).
  • ad hoc to paper Narrowing the Brain Treebank window to [-2.0,2.0]s preserves comparability with PopT's protocol.
    Appendix B.2 changes PopT's [-2.5,2.5]s window; baselines are re-run at the same narrowed window, so internal comparisons are fair, but the deviation is a paper-specific choice.
  • standard math Standard background machinery (transformers, spectral clustering, product quantization) is correct.
    No proofs are needed for these well-established components.
invented entities (1)
  • DPQ sub-quantizer neural codexes claimed to carry distinct neural dynamics
    purpose: G=4 parallel codebooks with a decorrelation penalty intended to isolate functional modules within a target channel group (Section 3.3, Eq. 3).
    No falsifiable handle outside the paper: no predicted electrode-level effect, no anatomical or behavioral validation, and the paper's own ablation (Appendix I) shows the decorrelation term has minor effect.

how reviews work

0 comments
Cite this review

Pith. "Pith review of BrainStratify: Coarse-to-Fine Disentanglement of Intracranial Neural Dynamics." pith.science (2026). https://pith.science/paper/L2472NG5

@misc{pith2026250520480,
  author       = {Pith},
  title        = {Pith review of: BrainStratify: Coarse-to-Fine Disentanglement of Intracranial Neural Dynamics},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/L2472NG5}},
  note         = {Machine review of arXiv:2505.20480}
}
read the original abstract

Decoding speech directly from neural activity is a central goal in brain-computer interface (BCI) research. In recent years, exciting advances have been made through the growing use of intracranial field potential recordings, such as stereo-ElectroEncephaloGraphy (sEEG) and ElectroCorticoGraphy (ECoG). These neural signals capture rich population-level activity but present key challenges: (i) task-relevant neural signals are sparsely distributed across sEEG electrodes, and (ii) they are often entangled with task-irrelevant neural signals in both sEEG and ECoG. To address these challenges, we introduce a unified Coarse-to-Fine neural disentanglement framework, BrainStratify, which includes (i) identifying functional groups through spatial-context-guided temporal-spatial modeling, and (ii) disentangling distinct neural dynamics within the target functional group using Decoupled Product Quantization (DPQ). We evaluate BrainStratify on two open-source sEEG datasets and one (epidural) ECoG dataset, spanning tasks like vocal production and speech perception. Extensive experiments show that BrainStratify, as a unified framework for decoding speech from intracranial neural signals, significantly outperforms previous decoding methods. Overall, by combining data-driven stratification with neuroscience-inspired modularity, BrainStratify offers a robust and interpretable solution for speech decoding from intracranial recordings.

Figures

Figures reproduced from arXiv: 2505.20480 by the authors.

Figure 1
Figure 1. The 61-word performance on Du-IN [55] dataset using top-10 chan￾nels selected via the MC strategy across varying numbers of labeled samples. To tackle these issues, we propose BrainStratify, a gen￾eral framework for decoding speech from intracranial neu￾ral signals. This framework comprises two complemen￾tary stages: (1) Coarse Disentanglement Learning and (2) Fine Disentanglement Learning. In the Coarse-DL stage, w… view at source ↗
Figure 2
Figure 2. Overview of BrainStratify framework. (a). Coarse Disentanglement Learning Stage (BrainStratify-Coarse). (b). Fine Disentanglement Learning Stage (BrainStratify-Fine). Spatial Transformer, and (3) Channel Cluster Module. Following brain signal foundation models [54, 18, 45], we segment the sEEG signals into patches, enabling flexible input processing. For each sample X , we use a Wc-length window (0.25s) without over… view at source ↗
Figure 3
Figure 3. The channel connectivity from different methods. Since DUET relies heavily on inter-channel correlations, we pre-train it using raw sEEG signals (without re-referencing), while using re-referenced signals instead even degrades clustering perfor￾mance. As shown in [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: shows BrainStratify-Fine’s 61-word classification performance on the Du-IN dataset across different initialization strategies (details in Appendix D). We evaluate performance against varying codex groups (from 0 to 8) to ascertain if the number of codex groups affects …
Figure 5
Figure 5. Figure 5: Overview of ECoG configuration. (a). The implant configuration. Our developed (epidural) ECoG is placed above vSMC, which is involved in vocal production [36]. (b). The channel resistance. Electrodes at the four corners are excluded for downstream analysis. In the word…
Figure 6
Figure 6. Figure 6: An overview of the three-stage training pipeline of BrainStratify-Fine. (a). Learning discrete neural codex in the BrainStratify-Fine VQ-VAE by reconstructing the original neural signals. (b). Mask modeling pre-training of Neural Encoder in the BrainStratify-Fine MAE. …
Figure 7
Figure 7. Figure 7: The loss curve and accuracy curve during the training process of BrainStratify-Coarse. [PITH_FULL_IMAGE:figures/full_fig_p026_7.png]
Figure 8
Figure 8. Figure 8: The visualization of Vector-Quantized Neural Reconstruction. (a). The reconstruction loss curve during the training process of BrainStratify-Fine VQ-VAE. (b). The visualization of reconstructed neural signals. 26 [PITH_FULL_IMAGE:figures/full_fig_p026_8.png]
Figure 9
Figure 9. Figure 9: The loss curve and accuracy curve during the training process of BrainStratify-Fine MAE. [PITH_FULL_IMAGE:figures/full_fig_p027_9.png]
Figure 10
Figure 10. Figure 10: BrainStratify-Fine ablation study on two word-reading intracranial neural datasets (sEEG [PITH_FULL_IMAGE:figures/full_fig_p027_10.png]
Figure 11
Figure 11. Figure 11: Channel Connectivity from subjects (01-04). [PITH_FULL_IMAGE:figures/full_fig_p034_11.png]
Figure 12
Figure 12. Figure 12: Channel Connectivity from subjects (05-08). [PITH_FULL_IMAGE:figures/full_fig_p035_12.png]
Figure 13
Figure 13. Figure 13: Channel Connectivity from subjects (09-12). [PITH_FULL_IMAGE:figures/full_fig_p036_13.png]
Figure 14
Figure 14. Figure 14: Channel Connectivity from subjects (01-04). [PITH_FULL_IMAGE:figures/full_fig_p037_14.png]
Figure 15
Figure 15. Figure 15: Channel Connectivity from subjects (05,06,10). [PITH_FULL_IMAGE:figures/full_fig_p038_15.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Large-Scale AI and Foundation Models for Neuroscience: A Comprehensive Review

    cs.AI 2025-10 conditional novelty 1.0 of 10

    This paper is a survey: it organizes existing foundation-model work in neuroscience into five application domains and lists public datasets, without presenting new experiments.

Reference graph

Works this paper leans on

56 extracted references · 35 canonical work pages · cited by 1 Pith paper

  1. [1]

    Nine decades of electrocorticography: A comparison between epidural and subdural recordings.European Journal of Neuroscience, 57(8):1260–1288, 2023

    Mariana P Branco, Simon H Geukes, Erik J Aarnoutse, Nick F Ramsey, and Mariska J Vansteensel. Nine decades of electrocorticography: A comparison between epidural and subdural recordings.European Journal of Neuroscience, 57(8):1260–1288, 2023

    work page 2023

  2. [2]

    Oxford university press, 2006

    György Buzsáki.Rhythms of the Brain. Oxford university press, 2006

    2006

  3. [3]

    Brainlm: A foundation model for brain activity recordings.bioRxiv, pages 2023–09, 2023

    Josue Ortega Caro, Antonio H de O Fonseca, Christopher Averill, Syed A Rizvi, Matteo Rosati, James L Cross, Prateek Mittal, Emanuele Zappala, Daniel Levine, Rahul M Dhodapkar, et al. Brainlm: A foundation model for brain activity recordings.bioRxiv, pages 2023–09, 2023. 10

    work page 2023

  4. [4]

    Micro-scale functional modules in the human temporal lobe.Nature communications, 13(1):6263, 2022

    Julio I Chapeton, John H Wittig Jr, Sara K Inati, and Kareem A Zaghloul. Micro-scale functional modules in the human temporal lobe.Nature communications, 13(1):6263, 2022

    work page 2022

  5. [5]

    Population transformer: Learning population- level representations of neural activity.ArXiv, pages arXiv–2406, 2024

    Geeling Chau, Christopher Wang, Sabera Talukder, Vighnesh Subramaniam, Saraswati Soedar- madji, Yisong Yue, Boris Katz, and Andrei Barbu. Population transformer: Learning population- level representations of neural activity.ArXiv, pages arXiv–2406, 2024

    work page 2024

  6. [6]

    From similarity to superiority: Channel clustering for time series forecasting.Advances in Neural Information Processing Systems, 37:130635–130663, 2025

    Jialin Chen, Jan Eric Lenssen, Aosong Feng, Weihua Hu, Matthias Fey, Leandros Tassiulas, Jure Leskovec, and Rex Ying. From similarity to superiority: Channel clustering for time series forecasting.Advances in Neural Information Processing Systems, 37:130635–130663, 2025

    work page 2025

  7. [7]

    Neural mechanisms resolving exploitation-exploration dilemmas in the medial prefrontal cortex.Science, 369(6507):eabb0184, 2020

    Philippe Domenech, Sylvain Rheims, and Etienne Koechlin. Neural mechanisms resolving exploitation-exploration dilemmas in the medial prefrontal cortex.Science, 369(6507):eabb0184, 2020

    work page 2020

  8. [8]

    Brain-jepa: Brain dynamics foundation model with gradient positioning and spatiotemporal masking.Advances in Neural Information Processing Systems, 37:86048–86073, 2024

    Zijian Dong, Ruilin Li, Yilei Wu, Thuan Tinh Nguyen, Joanna Chong, Fang Ji, Nathanael Tong, Christopher Chen, and Juan Helen Zhou. Brain-jepa: Brain dynamics foundation model with gradient positioning and spatiotemporal masking.Advances in Neural Information Processing Systems, 37:86048–86073, 2024

    2024

Show all 56 references
  1. [9]

    Plug-and-play stability for intracortical brain-computer interfaces: a one-year demonstration of seamless brain-to-text communication

    Chaofei Fan, Nick Hahn, Foram Kamdar, Donald Avansino, Guy Wilson, Leigh Hochberg, Kr- ishna V Shenoy, Jaimie Henderson, and Francis Willett. Plug-and-play stability for intracortical brain-computer interfaces: a one-year demonstration of seamless brain-to-text communication. ...

    2023

  2. [10]

    Acoustic inspired brain-to-sentence decoder for logosyllabic language

    Chen Feng, Lu Cao, Di Wu, En Zhang, Ting Wang, Xiaowei Jiang, Jinbo Chen, Hui Wu, Siyu Lin, Qiming Hou, et al. Acoustic inspired brain-to-sentence decoder for logosyllabic language. Cyborg and Bionic Systems, 2023

    2023

  3. [11]

    Misa: Modality-invariant and-specific representations for multimodal sentiment analysis

    Devamanyu Hazarika, Roger Zimmermann, and Soujanya Poria. Misa: Modality-invariant and-specific representations for multimodal sentiment analysis. InProceedings of the 28th ACM international conference on multimedia, pages 1122–1131, 2020

    2020

  4. [12]

    Gaussian error linear units (gelus).arXiv preprint arXiv:1606.08415, 2016

    Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus).arXiv preprint arXiv:1606.08415, 2016

    2016 arXiv

  5. [13]

    beta-vae: Learning basic visual concepts with a constrained variational framework

    Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-vae: Learning basic visual concepts with a constrained variational framework. InInternational conference on learning representations, 2017

    2017

  6. [14]

    Disentanglement via latent quantization.Advances in Neural Information Processing Systems, 36:45463–45488, 2023

    Kyle Hsu, William Dorrell, James Whittington, Jiajun Wu, and Chelsea Finn. Disentanglement via latent quantization.Advances in Neural Information Processing Systems, 36:45463–45488, 2023

    2023

  7. [15]

    Tripod: Three complementary inductive biases for disentangled representation learning.arXiv preprint arXiv:2404.10282, 2024

    Kyle Hsu, Jubayer Ibn Hamid, Kaylee Burns, Chelsea Finn, and Jiajun Wu. Tripod: Three complementary inductive biases for disentangled representation learning.arXiv preprint arXiv:2404.10282, 2024

    2024 arXiv

  8. [16]

    Crossgnn: Confronting noisy multivariate time series via cross interaction refine- ment.Advances in Neural Information Processing Systems, 36:46885–46902, 2023

    Qihe Huang, Lei Shen, Ruixin Zhang, Shouhong Ding, Binwu Wang, Zhengyang Zhou, and Yang Wang. Crossgnn: Confronting noisy multivariate time series via cross interaction refine- ment.Advances in Neural Information Processing Systems, 36:46885–46902, 2023

    2023

  9. [17]

    Product quantization for nearest neighbor search.IEEE transactions on pattern analysis and machine intelligence, 33(1):117–128, 2010

    Herve Jegou, Matthijs Douze, and Cordelia Schmid. Product quantization for nearest neighbor search.IEEE transactions on pattern analysis and machine intelligence, 33(1):117–128, 2010

    2010

  10. [18]

    Large brain model for learning generic representations with tremendous eeg data in bci.arXiv preprint arXiv:2405.18765, 2024

    Wei-Bang Jiang, Li-Ming Zhao, and Bao-Liang Lu. Large brain model for learning generic representations with tremendous eeg data in bci.arXiv preprint arXiv:2405.18765, 2024

    2024 arXiv

  11. [19]

    A revisit of total correlation in disentangled variational auto-encoder with partial disentanglement.arXiv preprint arXiv:2502.02279, 2025

    Chengrui Li, Yunmiao Wang, Yule Wang, Weihan Li, Dieter Jaeger, and Anqi Wu. A revisit of total correlation in disentangled variational auto-encoder with partial disentanglement.arXiv preprint arXiv:2502.02279, 2025. 11

    2025 arXiv

  12. [20]

    Optimal referencing for stereo- electroencephalographic (seeg) recordings.NeuroImage, 183:327–335, 2018

    Guangye Li, Shize Jiang, Sivylla E Paraskevopoulou, Meng Wang, Yang Xu, Zehan Wu, Liang Chen, Dingguo Zhang, and Gerwin Schalk. Optimal referencing for stereo- electroencephalographic (seeg) recordings.NeuroImage, 183:327–335, 2018

    2018

  13. [21]

    Deep learning-powered electrical brain signals analysis: Advancing neurological diagnostics.arXiv preprint arXiv:2502.17213, 2025

    Jiahe Li, Xin Chen, Fanqi Shen, Junru Chen, Yuxin Liu, Daoze Zhang, Zhizhang Yuan, Fang Zhao, Meng Li, and Yang Yang. Deep learning-powered electrical brain signals analysis: Advancing neurological diagnostics.arXiv preprint arXiv:2502.17213, 2025

    2025

  14. [22]

    Reclaiming hand functions after complete spinal cord injury with epidural brain-computer interface.medRxiv, pages 2024–09, 2024

    Dingkun Liu, Yongzhi Shan, Penghu Wei, Wenzheng Li, Honglai Xu, Fangshuo Liang, Tao Liu, Guoguang Zhao, and Bo Hong. Reclaiming hand functions after complete spinal cord injury with epidural brain-computer interface.medRxiv, pages 2024–09, 2024

    2024

  15. [23]

    Challenging common assumptions in the unsupervised learning of disentangled representations

    Francesco Locatello, Stefan Bauer, Mario Lucic, Gunnar Raetsch, Sylvain Gelly, Bernhard Schölkopf, and Olivier Bachem. Challenging common assumptions in the unsupervised learning of disentangled representations. Ininternational conference on machine learning, pages 4114–

  16. [24]

    Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017

    2017 arXiv

  17. [25]

    Neural decoding from stereotactic eeg: accounting for electrode variability across subjects.Advances in Neural Information Processing Systems, 37:108600–108624, 2024

    Georgios Mentzelopoulos, Evangelos Chatzipantazis, Ashwin Ramayya, Michelle Hedlund, Vivek Buch, Kostas Daniilidis, Konrad Kording, and Flavia Vitale. Neural decoding from stereotactic eeg: accounting for electrode variability across subjects.Advances in Neural Information Pro...

    2024

  18. [26]

    A high- performance neuroprosthesis for speech decoding and avatar control.Nature, 620(7976):1037– 1046, 2023

    Sean L Metzger, Kaylo T Littlejohn, Alexander B Silva, David A Moses, Margaret P Seaton, Ran Wang, Maximilian E Dougherty, Jessie R Liu, Peter Wu, Michael A Berger, et al. A high- performance neuroprosthesis for speech decoding and avatar control.Nature, 620(7976):1037– 1046, 2023

    2023

  19. [27]

    Neuroprosthesis for decoding speech in a paralyzed person with anarthria.New England Journal of Medicine, 385(3):217–227, 2021

    David A Moses, Sean L Metzger, Jessie R Liu, Gopala K Anumanchipalli, Joseph G Makin, Pengfei F Sun, Josh Chartier, Maximilian E Dougherty, Patricia M Liu, Gary M Abrams, et al. Neuroprosthesis for decoding speech in a paralyzed person with anarthria.New England Journal of Med...

    2021

  20. [28]

    Creimbo: Cross-regional ensemble interactions in multi-view brain observations

    Noga Mudrik, Ryan Ly, Oliver Ruebel, and Adam Shabti Charles. Creimbo: Cross-regional ensemble interactions in multi-view brain observations. InThe Thirteenth International Confer- ence on Learning Representations, 2025

    2025

  21. [29]

    Sibblings: similarity-driven building-block inference using graphs across states

    Noga Mudrik, Gal Mishne, and Adam S Charles. Sibblings: similarity-driven building-block inference using graphs across states. InProceedings of the 41st International Conference on Machine Learning, pages 36504–36530, 2024

    2024

  22. [30]

    Sampling representational plasticity of simple imagined movements across days enables long-term neuroprosthetic control.Cell, 188(5):1208–1225, 2025

    Nikhilesh Natraj, Sarah Seko, Reza Abiri, Runfeng Miao, Hongyi Yan, Yasmin Graham, Adelyn Tu-Chan, Edward F Chang, and Karunesh Ganguly. Sampling representational plasticity of simple imagined movements across days enables long-term neuroprosthetic control.Cell, 188(5):1208–1225, 2025

    2025

  23. [31]

    On spectral clustering: Analysis and an algorithm

    Andrew Ng, Michael Jordan, and Yair Weiss. On spectral clustering: Analysis and an algorithm. Advances in neural information processing systems, 14, 2001

    2001

  24. [32]

    Hippocampal sharp-wave ripples linked to visual episodic recollection in humans

    Yitzhak Norman, Erin M Yeagle, Simon Khuvis, Michal Harel, Ashesh D Mehta, and Rafael Malach. Hippocampal sharp-wave ripples linked to visual episodic recollection in humans. Science, 365(6454):eaax1030, 2019

    2019

  25. [33]

    Disentangling time series representations via contrastive independence-of-support on l-variational inference

    Khalid Oublal, Said Ladjal, David Benhaiem, Emmanuel LE BORGNE, and François Roueff. Disentangling time series representations via contrastive independence-of-support on l-variational inference. InThe Twelfth International Conference on Learning Representa- tions, 2024

    2024

  26. [34]

    Scikit- learn: Machine learning in python.the Journal of machine Learning research, 12:2825–2830, 2011

    Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. Scikit- learn: Machine learning in python.the Journal of machine Learning research, 12:2825–2830, 2011. 12

    2011

  27. [35]

    Duet: Dual clustering enhanced multivariate time series forecasting.arXiv preprint arXiv:2412.10859, 2024

    Xiangfei Qiu, Xingjian Wu, Yan Lin, Chenjuan Guo, Jilin Hu, and Bin Yang. Duet: Dual clustering enhanced multivariate time series forecasting.arXiv preprint arXiv:2412.10859, 2024

    2024 arXiv

  28. [36]

    The speech neuroprosthesis.Nature Reviews Neuroscience, 25(7):473–492, 2024

    Alexander B Silva, Kaylo T Littlejohn, Jessie R Liu, David A Moses, and Edward F Chang. The speech neuroprosthesis.Nature Reviews Neuroscience, 25(7):473–492, 2024

    2024

  29. [37]

    Plug-and-play control of a brain–computer interface through neural map stabilization.Nature biotechnology, 39(3):326–335, 2021

    Daniel B Silversmith, Reza Abiri, Nicholas F Hardy, Nikhilesh Natraj, Adelyn Tu-Chan, Edward F Chang, and Karunesh Ganguly. Plug-and-play control of a brain–computer interface through neural map stabilization.Nature biotechnology, 39(3):326–335, 2021

    2021

  30. [38]

    Eeg conformer: Convolu- tional transformer for eeg decoding and visualization.IEEE Transactions on Neural Systems and Rehabilitation Engineering, 31:710–719, 2022

    Yonghao Song, Qingqing Zheng, Bingchuan Liu, and Xiaorong Gao. Eeg conformer: Convolu- tional transformer for eeg decoding and visualization.IEEE Transactions on Neural Systems and Rehabilitation Engineering, 31:710–719, 2022

    2022

  31. [39]

    Revealing vision-language integration in the brain with multimodal networks.ArXiv, pages arXiv–2406, 2024

    Vighnesh Subramaniam, Colin Conwell, Christopher Wang, Gabriel Kreiman, Boris Katz, Ignacio Cases, and Andrei Barbu. Revealing vision-language integration in the brain with multimodal networks.ArXiv, pages arXiv–2406, 2024

    2024

  32. [40]

    Neural discrete representation learning.Advances in neural information processing systems, 30, 2017

    Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning.Advances in neural information processing systems, 30, 2017

    2017

  33. [41]

    Attention is all you need.Advances in neural information processing systems, 30, 2017

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need.Advances in neural information processing systems, 30, 2017

    2017

  34. [42]

    Brainbert: Self-supervised representation learning for intracranial recordings.arXiv preprint arXiv:2302.14367, 2023

    Christopher Wang, Vighnesh Subramaniam, Adam Uri Yaari, Gabriel Kreiman, Boris Katz, Ignacio Cases, and Andrei Barbu. Brainbert: Self-supervised representation learning for intracranial recordings.arXiv preprint arXiv:2302.14367, 2023

    2023 arXiv

  35. [43]

    Brain treebank: Large-scale intracranial recordings from naturalistic language stimuli

    Christopher Wang, Adam Uri Yaari, Aaditya K Singh, Vighnesh Subramaniam, Dana Rosenfarb, Jan DeWitt, Pranav Misra, Joseph R Madsen, Scellig Stone, Gabriel Kreiman, et al. Brain treebank: Large-scale intracranial recordings from naturalistic language stimuli. InThe Thirty- eigh...

    2024

  36. [44]

    Eegpt: Pretrained transformer for universal and reliable representation of eeg signals.Advances in Neural Information Processing Systems, 37:39249–39280, 2024

    Guangyu Wang, Wenchao Liu, Yuhong He, Cong Xu, Lin Ma, and Haifeng Li. Eegpt: Pretrained transformer for universal and reliable representation of eeg signals.Advances in Neural Information Processing Systems, 37:39249–39280, 2024

    2024

  37. [45]

    Cbramod: A criss-cross brain foundation model for eeg decoding.arXiv preprint arXiv:2412.07236, 2024

    Jiquan Wang, Sha Zhao, Zhiling Luo, Yangxuan Zhou, Haiteng Jiang, Shijian Li, Tao Li, and Gang Pan. Cbramod: A criss-cross brain foundation model for eeg decoding.arXiv preprint arXiv:2412.07236, 2024

    2024

  38. [46]

    The chinese language.Scientific American, 228(2):50–63, 1973

    William SY Wang. The chinese language.Scientific American, 228(2):50–63, 1973

    1973

  39. [47]

    Disentangled representation learning.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    Xin Wang, Hong Chen, Si’ao Tang, Zihao Wu, and Wenwu Zhu. Disentangled representation learning.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    2024

  40. [48]

    Exploring behavior-relevant and disen- tangled neural dynamics with generative diffusion models.Advances in Neural Information Processing Systems, 37:34712–34736, 2024

    Yule Wang, Chengrui Li, Weihan Li, and Anqi Wu. Exploring behavior-relevant and disen- tangled neural dynamics with generative diffusion models.Advances in Neural Information Processing Systems, 37:34712–34736, 2024

    2024

  41. [49]

    A high-performance speech neuroprosthesis.Nature, 620(7976):1031–1036, 2023

    Francis R Willett, Erin M Kunz, Chaofei Fan, Donald T Avansino, Guy H Wilson, Eun Young Choi, Foram Kamdar, Matthew F Glasser, Leigh R Hochberg, Shaul Druckmann, et al. A high-performance speech neuroprosthesis.Nature, 620(7976):1031–1036, 2023

    2023

  42. [50]

    Cost: Contrastive learning of disentangled seasonal-trend representations for time series forecasting.arXiv preprint arXiv:2202.01575, 2022

    Gerald Woo, Chenghao Liu, Doyen Sahoo, Akshat Kumar, and Steven Hoi. Cost: Contrastive learning of disentangled seasonal-trend representations for time series forecasting.arXiv preprint arXiv:2202.01575, 2022

    2022 arXiv

  43. [51]

    Towards homogeneous lexical tone decoding from heterogeneous intracranial recordings.arXiv preprint arXiv:2410.12866, 2024

    Di Wu, Siyuan Li, Chen Feng, Lu Cao, Yue Zhang, Jie Yang, and Mohamad Sawan. Towards homogeneous lexical tone decoding from heterogeneous intracranial recordings.arXiv preprint arXiv:2410.12866, 2024. 13

    2024 arXiv

  44. [52]

    Group normalization

    Yuxin Wu and Kaiming He. Group normalization. InProceedings of the European conference on computer vision (ECCV), pages 3–19, 2018

    2018

  45. [53]

    Ppi: Pretraining brain signal model for patient-independent seizure detection.Advances in Neural Information Processing Systems, 36:69586–69604, 2023

    Zhizhang Yuan, Daoze Zhang, Yang Yang, Junru Chen, and Yafeng Li. Ppi: Pretraining brain signal model for patient-independent seizure detection.Advances in Neural Information Processing Systems, 36:69586–69604, 2023

    2023

  46. [54]

    Brant: Foundation model for intracranial neural signal.Advances in Neural Information Processing Systems, 36:26304–26321, 2023

    Daoze Zhang, Zhizhang Yuan, Yang Yang, Junru Chen, Jingjing Wang, and Yafeng Li. Brant: Foundation model for intracranial neural signal.Advances in Neural Information Processing Systems, 36:26304–26321, 2023

    2023

  47. [55]

    Du-in: Discrete units-guided mask modeling for decoding speech from intracranial neural signals.Advances in Neural Information Processing Systems, 37:79996–80033, 2025

    Hui Zheng, Haiteng Wang, Weibang Jiang, Zhongtao Chen, Li He, Peiyang Lin, Penghu Wei, Guoguang Zhao, and Yunzhe Liu. Du-in: Discrete units-guided mask modeling for decoding speech from intracranial neural signals.Advances in Neural Information Processing Systems, 37:79996–80033, 2025

    2025

  48. [56]

    -"), • 1 silence token (i.e.,

    Ding Zhou and Xue-Xin Wei. Learning identifiable and interpretable latent models of high- dimensional neural activity using pi-vae.Advances in neural information processing systems, 33:7234–7247, 2020. 14 A Experiment Design Due to the lack of open-source intracranial neural d...

    2020

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.