Pith. sign in

REVIEW 5 major objections 8 minor 1 cited by

Learning Interpretable Representations Leads to Semantically Faithful EEG-to-Text Generation

T0 review · 5 major / 8 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read EEG-to-text generation can be grounded in brain signals instead of hallucinated by the language model.

desk verdict A genuinely useful reframing and evaluation protocol for EEG-to-text, but the generation-grounding claim needs a stronger control than the noise test. read the letter →

arxiv 2505.17099 v1 pith:IBNNAN2U submitted 2025-05-21 cs.CL

Xiaozhao Liu , Dinggang Shen , Xihui Liu This is my paper · ORCID
classification cs.CL
keywords EEG-to-textdecodingposteriorcollapsehallucinationingenerativemodelssemanticsummarizationcontrastiverepresentationlearningzero-shotclassificationnaturalreadingEEGbrain
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that hallucination in EEG-to-text decoding is a case of posterior collapse: the frozen language model generates fluent sentences from its own linguistic prior, effectively ignoring the noisy EEG input. The proposed remedy, GLIM, reframes the task from verbatim stimulus reconstruction to semantic summarization, and trains the EEG encoder to align its representations with the latent space of a frozen instruction-tuned language model through a contrastive-generative objective. On the ZuCo natural-reading dataset, the authors report that GLIM generates fluent, EEG-grounded sentences without teacher forcing, and that its EEG representations support EEG-text retrieval and zero-shot classification across sentiment, relation type, and corpus topic. If these claims hold, the practical upshot is that the bottleneck in EEG-to-text is representation quality rather than decoder power, and that semantic evaluation protocols, not surface text similarity, are the right way to measure progress.

What carries the argument

The central machinery is GLIM's joint contrastive-generative objective, which ties a Q-former-style EEG encoder to a frozen Flan-T5 language model through a 'querying aligner' (a linear projection plus cross-attention). The EEG encoder uses learnable queries to downsample the time series and produces two representations: a token-level sequence used to condition the decoder, and a global embedding used for retrieval and classification. Domain prompts for subject, dataset, and task are injected through scale-shift adapters with prompt dropout, and each stimulus is expanded into multiple paraphrased text variants so the decoder learns core semantics rather than surface forms. The contrastive loss is the piece specifically aimed at preventing posterior collapse, by forcing the EEG embedding to be distinguishable among unrelated text embeddings in each batch.

What would settle it

Run the noise-input test with the dataset and task prompts shuffled or replaced by [UNKNOWN] tokens, keeping Gaussian EEG inputs, and recompute corpus-classification and sentiment scores. If these scores stay near the noise-condition levels, the prompts alone explain the gap and the EEG-grounding claim is unsupported; if they collapse, the prompts are not the confounder and the claim survives.

Watch

Extended reading notes

Core claim

The paper's central claim is that informative, interpretable EEG representations, not the strength of the generative decoder, are what make EEG-to-text outputs semantically faithful. The authors diagnose earlier EEG-to-text systems as suffering from posterior collapse, citing reproduced results where the same model produces fluent sentences from Gaussian noise and collapses to repetitive defaults when teacher forcing is disabled. GLIM addresses this by combining an autoregressive language-model loss over multiple paraphrased variants of each stimulus with a CLIP-style contrastive loss that forces EEG embeddings to be recognizable among their text counterparts in a frozen Flan-T5 latent space. The authors show that removing the contrastive loss nearly eliminates the gap between real-EEG and noise-input performance, and they claim this gap is evidence of genuine EEG grounding. They also present retrieval and zero-shot classification results as the first demonstration that abstract semantic categories, such as sentiment, relation type, and corpus source, can be decoded from EEG embeddings, framing these protocols as a more robust evaluation than BLEU-style text similarity.

Load-bearing premise

The load-bearing premise is that the noise-input test isolates what the model decodes from the EEG; in fact, the domain prompts are kept when EEG is replaced by Gaussian noise, and since the dataset and task prompts together determine the corpus, a substantial share of the reported grounding gap could come from prompt-driven priors rather than from the brain signal.

Editorial extensions

If this is right

  • If GLIM is correct, the standard for EEG-to-text evaluation should shift from surface text similarity to retrieval and zero-shot classification over EEG representations, since those protocols test whether the semantic content actually resides in the brain-derived embeddings.
  • The noise-input gap becomes a necessary sanity check for any generative brain-decoding model: a system that produces fluent output from Gaussian noise is not decoding the brain signal, even if its text looks plausible.
  • Framing decoding as semantic summarization implies that future EEG-language datasets should be built with semantic annotations and comprehension questions, because passive reading alone appears to yield weaker and more variable semantic encoding.
  • Adding a contrastive alignment component to generative decoding pipelines that pair powerful autoregressive decoders with noisy inputs may prevent collapse in other modalities, not just EEG.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Read more strictly than the paper does, the noise-input test as reported is confounded: with the dataset and task prompts held fixed, corpus classification is a deterministic function of the prompts, so the noise-condition corpus accuracy mostly reflects prompt priors rather than residual EEG information, and a shuffled-prompt noise test would settle whether the EEG itself carries the corpus signa
  • The same architecture-and-evaluation recipe transfers to other non-invasive decoding pairs, such as fMRI-to-image or MEG-to-speech, as a general antidote to hallucination: align the neural encoder to a frozen multimodal latent space, supervise semantics through retrieval and zero-shot classification, and reserve the generation metric for fluency.
  • A testable extension of the paper's logic is that retrieval accuracy between EEG and text should degrade monotonically as the evaluation batch grows, since the contrastive signal is batch-limited; measuring that curve would give a finer estimate of how much semantic information the EEG embeddings actually carry.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 8 minor

Summary. The paper proposes GLIM, a framework for EEG-to-text decoding that reframes the task as semantic summarization rather than verbatim reconstruction. GLIM aligns EEG representations with a frozen Flan-T5 encoder via a contrastive loss, uses multiple paraphrased variants as training targets, and injects domain prompts (task, dataset, subject) via adapters. The authors evaluate on ZuCo with a text-based split, reporting generation metrics (BLEU, ROUGE), EEG-text retrieval, and zero-shot classification of sentiment, relation, and corpus topics. They claim that GLIM produces fluent, EEG-grounded sentences without teacher forcing and that its representations support semantic evaluation beyond text similarity. They also claim to be the first to demonstrate zero-shot semantic evaluation in EEG-based generative decoding.

Significance. The task is important: hallucination in EEG-to-text is a known problem, and the paper's diagnosis of posterior collapse and its proposal to align with a frozen LM's latent space are sensible. The text-based split is a methodological improvement over prior subject-stratified splits that allowed train-test text overlap. The modular architecture with prompt dropout and the public code are assets. However, the current evaluation does not establish the central claim of EEG-grounded generation: the noise-input test retains domain prompts, the zero-shot classification is confounded by label-conditioned paraphrase generation and by evaluating with the same LM used for supervision, and no statistical significance is reported. The claimed novelty of being first to perform zero-shot semantic evaluation is contradicted by the cited EEG2Text work. If the confounds are addressed with proper controls, the approach has potential.

major comments (5)
  1. [Section 4.1, Table 1] The noise-input test does not isolate EEG information because domain prompts (task, dataset, subject) are retained. Since all TSR and ZuCo2 test samples are from the Wiki corpus and all NR-SST samples are from ZuCo1, the task and dataset prompts uniquely determine the corpus for every test sample; hence the corpus classification accuracy of 0.9348 (and even the noise-input accuracy of 0.6273) largely reflects prompt leakage rather than EEG-derived semantics. The paper's statement that the test eliminates 'any other confounder, such as the domain prompt inputs' is therefore incorrect. To support the grounding claim, the authors should compare against a control with EEG inputs removed or permuted while keeping prompts, and ideally with prompts ablated.
  2. [Appendix B.1 and Section 3.5] The multiple-text-variant construction uses supplementary label information: Llama3.1 is prompted with sentiment categories for NR-SST and candidate/true relation types for NR-Wiki/TSR-Wiki (Appendix B.1). These variants are used as targets in the autoregressive loss (Eq. 1) and as positives in the contrastive loss (Eq. 2). Thus semantic labels are implicitly provided during training, which contradicts the claim of zero-shot classification 'without requiring semantic labels during training' (Section 1). This label leakage could explain the high classification accuracies, as the model may learn label-specific textual patterns rather than decoding semantics from EEG. The authors should either remove label information from the paraphrase generation prompts and retrain, or provide an ablation demonstrating that the label-conditioned variants do not affect classification results.
  3. [Section 4.1, Table 2] The zero-shot classification evaluation is circular: the EEG embeddings are trained to align with the frozen Flan-T5 encoder's latent space, and classification computes cosine similarity between these EEG embeddings and label embeddings from the same encoder. The fact that EEG embeddings achieve sentiment/relation accuracies comparable to or higher than raw-text embeddings (0.4269 vs. 0.4556; 0.3245 vs. 0.2530) is suspicious, since the training objective directly optimizes this alignment. The raw-text baselines do not fully control for this, because the EEG embeddings are pulled toward the LM's label-sensitive representations by construction. An independent evaluation (e.g., linear probes on raw EEG features, or human judgment of generated text) is needed to establish that the representations reflect neural semantic content rather than simply matching the supervision target.
  4. [Section 4.2, Tables 1 and 3] No variance or statistical significance is reported for any of the main results, yet the text repeatedly claims that GLIM 'significantly outperforms' baselines and that ablations cause 'consistent drops'. With a test set of 2,227 sentences and evaluation groups of size 24, the differences could be within noise. The authors should report mean and standard deviation over at least 3-5 random seeds (or bootstrap confidence intervals) and perform paired tests for the real-vs-noise and ablation comparisons.
  5. [Section 4.2] The claim that this is 'the first demonstration of such zero-shot evaluation in EEG-based generative decoding' is contradicted by the authors' own citation: EEG2Text [10] is titled 'Open vocabulary electroencephalography-to-text decoding and zero-shot sentiment classification' and already performs zero-shot sentiment classification. The novelty appears to be limited to zero-shot relation and corpus classification, and the claim should be qualified accordingly.
minor comments (8)
  1. [Section 3.1] The number of text variants K is not specified in the main text; Appendix D gives eight variants, so K=8 should be stated in Section 3.1 or 3.3.
  2. [Section 3.5] The value of λ in Eq. (3) is never reported; a hyperparameter value (or a sensitivity analysis) should be given.
  3. [Section 3.2] The reference to DiT [47] for the scale-shift adapter design is opaque, since DiT is not primarily known for adapters; please describe the adapter equations or cite the original adapter work.
  4. [Section 4.2] The sentence 'GLIM significantly outperforms these baselines' overstates the evidence, as no significance tests are provided.
  5. [Section 4.3] The statement that GLIM 'even surpassing these baselines' is incorrect for relation ACC-3: both EEG embeddings (0.5714) and generated text (0.5633) are lower than raw text LLM-assisted (0.7551) in Table 2.
  6. [Appendix A.2] Corpus-level annotations are described as 'manually added' for all test samples; please specify the annotation criteria and whether annotators were blind to the model outputs.
  7. [Section 3.1] There is a typo: 'enabling robust joint training across heterogeneous.' should read 'across heterogeneous datasets.' Similar wording appears in the Introduction.
  8. [Fig. 4] The figure caption says 'dark bars show absolute improvements over the averaged Nin scores (i.e., Score − ScoreNin)', but the y-axis labels are simply 'BLEU1' and 'ROUGE1'; please clarify the units and the baseline in the axes.

Circularity Check

1 steps flagged · score 6.0 of 10

Zero-shot semantic classification reduces to label-conditioned training targets and the same frozen-LM space; the central EEG-grounding evaluation is partially circular.

  1. fitted input called prediction [Section 4.1 (Zero-shot classification) and Appendix B.1 (MTV construction with label-conditioned paraphrase prompts)]
    "Zero-shot classification. To evaluate the core semantic capturing in EEG representations, we perform zero-shot classifications on sentiment, relation type, and corpus source... The implementation follows CLIP [52], where we directly use the integrated LM’s encoder (and Q-aligner) to obtain label embeddings and compute their cosine similarities with each EEG embedding... To ensure the preservation of core semantics, we provide each variant generation prompt with supplementary label information (e.g., sentiment categories for NR-SST; candidate/true relation types for NR-Wiki/TSR-Wiki)."

    The zero-shot classifier is constructed from the same frozen Flan-T5 encoder and trained Q-aligner that define the training targets: Eq. (2) aligns EEG embeddings to text embeddings in this LM space, and Eq. (1) trains language modeling on MTVs generated with explicit label information. The classification labels are therefore embedded in the very representation space the EEG encoder is optimized to match, and the reported accuracies measure cosine proximity to those label embeddings. The paper's claim that this evaluation is performed 'without requiring semantic labels during training' is contradicted by Appendix B.1, which injects true labels into the paraphrase prompts used as supervision.

full rationale

The main circularity is in the evaluation protocol rather than in the generation architecture itself. GLIM's zero-shot classification is presented as label-free, but Appendix B.1 shows the paraphrase targets used in Eq. (1) are generated using the true semantic labels, and Section 4.1 embeds classification labels with the same trained LM encoder/Q-aligner used for the contrastive loss Eq. (2). Consequently, the sentiment/relation/corpus accuracies largely measure how well the model reproduces its own label-conditioned target distribution in Flan-T5 space. The noise-input test is additionally confounded because domain prompts and the LM prior remain when EEG is replaced with Gaussian noise, but that is an experimental-design weakness rather than a definitional circularity. Likewise, using the same LLM for variant generation and LLM-assisted text evaluation introduces bias but is not a by-construction reduction. No load-bearing self-citation was found: ref. [43] is a self-citation for prompt injection but is not central to the claims. The existence of imperfect raw-text baselines (sentiment 0.4556, relation 0.2530) prevents a score of 8, because the LM space is not trivially saturated; however, the central 'zero-shot semantic evaluation' claim partially reduces to the training setup, giving a score of 6.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claim depends on three hand-set modeling choices, lambda, prompt dropout, and number of variants, and on three domain assumptions about the LM semantic space, the fidelity of paraphrased variants, and the neutrality of domain prompts. The last assumption is violated for corpus classification, which is a key evaluation metric.

free parameters (3)
  • Loss weight lambda
    Weight of language modeling loss versus contrastive loss in Eq. 3; chosen by the authors but no value or tuning procedure is reported.
  • Prompt dropout probabilities = 0, 0.1, 0.1 for task, dataset, subject
    Hand-set in Section 4.4 to support unseen domains; affects how much the model relies on prompts.
  • Number of text variants K = 8 (6 from Llama3.1, 2 from Flan-T5)
    Chosen in Appendix B; more variants assumed to improve semantic robustness; affects training data size and objective.
assumptions (3)
  • domain assumption Flan-T5-Large encoder outputs provide a suitable semantic space for aligning EEG signals and for zero-shot classification of sentiment, relation, and corpus categories.
    Invoked in Sections 3.3 and 3.4; if this space does not track human semantic categories, both supervision and evaluation lose validity.
  • ad hoc to paper The eight paraphrased variants preserve the core semantics of each stimulus and do not inject spurious label information.
    Introduced in Section 3.3 and Appendix B; labels were given to the rewriting LLM to preserve semantics, but this is not validated beyond the model's own metrics.
  • ad hoc to paper Domain prompts carry only spatiotemporal adaptation information, not the semantic labels used for evaluation.
    Assumed in Sections 3.2 and 4.1; contradicted for corpus labels, which are determined by the task and dataset prompts.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learning Interpretable Representations Leads to Semantically Faithful EEG-to-Text Generation." pith.science (2026). https://pith.science/paper/IBNNAN2U

@misc{pith2026250517099,
  author       = {Pith},
  title        = {Pith review of: Learning Interpretable Representations Leads to Semantically Faithful EEG-to-Text Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IBNNAN2U}},
  note         = {Machine review of arXiv:2505.17099}
}
read the original abstract

Pretrained generative models have opened new frontiers in brain decoding by enabling the synthesis of realistic texts and images from non-invasive brain recordings. However, the reliability of such outputs remains questionable--whether they truly reflect semantic activation in the brain, or are merely hallucinated by the powerful generative models. In this paper, we focus on EEG-to-text decoding and address its hallucination issue through the lens of posterior collapse. Acknowledging the underlying mismatch in information capacity between EEG and text, we reframe the decoding task as semantic summarization of core meanings rather than previously verbatim reconstruction of stimulus texts. To this end, we propose the Generative Language Inspection Model (GLIM), which emphasizes learning informative and interpretable EEG representations to improve semantic grounding under heterogeneous and small-scale data conditions. Experiments on the public ZuCo dataset demonstrate that GLIM consistently generates fluent, EEG-grounded sentences without teacher forcing. Moreover, it supports more robust evaluation beyond text similarity, through EEG-text retrieval and zero-shot semantic classification across sentiment categories, relation types, and corpus topics. Together, our architecture and evaluation protocols lay the foundation for reliable and scalable benchmarking in generative brain decoding.

Figures

Figures reproduced from arXiv: 2505.17099 by the authors.

Figure 1
Figure 1. Overview of GLIM. (A) One typical experimental session in natural reading dataset [19] involves a task-specific instruction followed by sentence stimulus blocks and comprehension queries. Participants read at their own speed and the simultaneously recorded EEG signals are segmented to aligned with each sentence, forming the EEG-text pairs for downstream decoding studies. We consider that several factors—including ta… view at source ↗
Figure 2
Figure 2. Architecture and training objective of GLIM. It consists of three modules: a domain￾adaptive EEG encoder, a pretrained encoder-decoder language model (LM), and a cross-modal querying aligner. We train the EEG encoder and the querying aligner to align EEG representations with the latent space of the frozen LM. There are two forms of EEG representations: (1) a token-level sequence representation Zi , used to generate … view at source ↗
Figure 3
Figure 3. Representative examples of generated texts. The three groups correspond to NR-SST, NR-Wiki, and TSR-Wiki, each showing the raw stimulus and two generated texts from different subjects. We observe: (1) corpus distinctions are regularly captured (movie reviews in SST vs. personal bios in Wiki); (2) relation types are expressed diversely, especially in the TSR group (e.g., the “education” label is paraphrased as “educa… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: (A) Average generation scores per variant type. Light bars denote average BLEU-1 and ROUGE-1 scores under noise input tests (Nin); while dark bars show absolute improvements over the averaged Nin scores (i.e., Score − ScoreNin ). (a) Six LLM-generated variant types. (b…
Figure 5
Figure 5. Figure 5: Performance comparison across different groups. The five groups correspond to various [PITH_FULL_IMAGE:figures/full_fig_p017_5.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Escaping the BLEU Trap: A Signal-Grounded Framework with Decoupled Semantic Guidance for EEG-to-Text Decoding

    cs.CL 2026-02 unverdicted novelty 6.0 of 10

    SemKey predicts four semantic attributes from EEG and conditions a frozen LLM on them, beating prior decoders on new semantic-alignment metrics while leaving true word-level accuracy low (2.7% content recall).

Reference graph

Works this paper leans on

62 extracted references · 51 canonical work pages · cited by 1 Pith paper

  1. [10]

    Open vocabulary electroencephalography-to-text decoding and zero-shot sentiment classification

    Zhenhailong Wang and Heng Ji. Open vocabulary electroencephalography-to-text decoding and zero-shot sentiment classification. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 5350–5358, 2022. 1, 2, 3, 15

    work page 2022

  2. [1]

    Decoding mental states from brain activity in humans

    John-Dylan Haynes and Geraint Rees. Decoding mental states from brain activity in humans. Nature reviews neuroscience, 7(7):523–534, 2006. 1

    work page 2006

  3. [2]

    Decoding the brain: From neural representations to mechanistic models

    Mackenzie Weygandt Mathis, Adriana Perez Rotondo, Edward F Chang, Andreas S Tolias, and Alexander Mathis. Decoding the brain: From neural representations to mechanistic models. Cell, 187(21):5814–5832, 2024. 1

    work page 2024

  4. [3]

    Performance-optimized hierarchical models predict neural responses in higher visual cortex

    Daniel LK Yamins, Ha Hong, Charles F Cadieu, Ethan A Solomon, Darren Seibert, and James J DiCarlo. Performance-optimized hierarchical models predict neural responses in higher visual cortex. Proceedings of the national academy of sciences, 111(23):8619–8624, 2014. 1, 3

    work page 2014

  5. [4]

    Decoding speech perception from non-invasive brain recordings

    Alexandre Défossez, Charlotte Caucheteux, Jérémy Rapin, Ori Kabeli, and Jean-Rémi King. Decoding speech perception from non-invasive brain recordings. Nature Machine Intelligence, 5(10):1097–1107, 2023. 1, 3

    work page 2023

  6. [5]

    Brain diffusion for visual exploration: Cortical discovery using large scale generative models

    Andrew Luo, Maggie Henderson, Leila Wehbe, and Michael Tarr. Brain diffusion for visual exploration: Cortical discovery using large scale generative models. Advances in Neural Information Processing Systems, 36:75740–75781, 2023. 1

    work page 2023

  7. [6]

    Brain decoding: toward real-time reconstruction of visual perception

    Yohann Benchetrit, Hubert Banville, and Jean-Remi King. Brain decoding: toward real-time reconstruction of visual perception. In The Twelfth International Conference on Learning Representations, 2024. 1, 3

    work page 2024

  8. [7]

    Seeing beyond the brain: Conditional diffusion model with sparse masked modeling for vision decoding

    Zijiao Chen, Jiaxin Qing, Tiange Xiang, Wan Lin Yue, and Juan Helen Zhou. Seeing beyond the brain: Conditional diffusion model with sparse masked modeling for vision decoding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22710–22720, 2023. 1

    work page 2023

Show all 62 references
  1. [8]

    High-resolution image reconstruction with latent diffusion models from human brain activity

    Yu Takagi and Shinji Nishimoto. High-resolution image reconstruction with latent diffusion models from human brain activity. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14453–14463, 2023. 1

    2023

  2. [9]

    Semantic reconstruction of continuous language from non-invasive brain recordings

    Jerry Tang, Amanda LeBel, Shailee Jain, and Alexander G Huth. Semantic reconstruction of continuous language from non-invasive brain recordings. Nature Neuroscience, 26(5):858–866,

  3. [11]

    Non-invasive brain-computer interfaces: state of the art and trends

    Bradley J Edelman, Shuailei Zhang, Gerwin Schalk, Peter Brunner, Gernot Müller-Putz, Cuntai Guan, and Bin He. Non-invasive brain-computer interfaces: state of the art and trends. IEEE Reviews in Biomedical Engineering, 2024. 1

    2024

  4. [12]

    Dissociating language and thought in large language models

    Kyle Mahowald, Anna A Ivanova, Idan A Blank, Nancy Kanwisher, Joshua B Tenenbaum, and Evelina Fedorenko. Dissociating language and thought in large language models. Trends in cognitive sciences, 2024. 1

    2024

  5. [13]

    Onellm: One framework to align all modalities with language

    Jiaming Han, Kaixiong Gong, Yiyuan Zhang, Jiaqi Wang, Kaipeng Zhang, Dahua Lin, Yu Qiao, Peng Gao, and Xiangyu Yue. Onellm: One framework to align all modalities with language. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26584–26...

    2024

  6. [14]

    Unveiling thoughts: A review of advancements in eeg brain signal decoding into text

    Saydul Akbar Murad and Nick Rahimi. Unveiling thoughts: A review of advancements in eeg brain signal decoding into text. IEEE Transactions on Cognitive and Developmental Systems,

  7. [15]

    Dewave: Discrete encoding of EEG waves for EEG to text translation

    Yiqun Duan, Charles Zhou, Zhen Wang, Yu-Kai Wang, and Chin teng Lin. Dewave: Discrete encoding of EEG waves for EEG to text translation. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. 2, 3 10

    2023

  8. [16]

    Enhancing eeg-to-text decoding through transferable representations from pre-trained contrastive eeg- text masked autoencoder

    Jiaqi Wang, Zhenxi Song, Zhengyu Ma, Xipeng Qiu, Min Zhang, and Zhiguo Zhang. Enhancing eeg-to-text decoding through transferable representations from pre-trained contrastive eeg- text masked autoencoder. In Proceedings of the 62nd Annual Meeting of the Association for Computa...

    2024

  9. [17]

    Are eeg-to-text models working? arXiv preprint arXiv:2405.06459, 2024

    Hyejeong Jo, Yiqian Yang, Juhyeok Han, Yiqun Duan, Hui Xiong, and Won Hee Lee. Are eeg-to-text models working? arXiv preprint arXiv:2405.06459, 2024. 2, 3, 6, 15

    2024 arXiv

  10. [18]

    Neural discrete representation learning

    Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Advances in neural information processing systems, 30, 2017. 2, 3

    2017

  11. [19]

    Zuco, a simultaneous eeg and eye-tracking resource for natural sentence reading

    Nora Hollenstein, Jonathan Rotsztejn, Marius Troendle, Andreas Pedroni, Ce Zhang, and Nicolas Langer. Zuco, a simultaneous eeg and eye-tracking resource for natural sentence reading. Scientific data, 5(1):1–13, 2018. 2, 3, 14

    2018

  12. [20]

    Zuco 2.0: A dataset of phys- iological recordings during natural reading and annotation

    Nora Hollenstein, Marius Troendle, Ce Zhang, and Nicolas Langer. Zuco 2.0: A dataset of phys- iological recordings during natural reading and annotation. arXiv preprint arXiv:1912.00903,

    1912 arXiv

  13. [21]

    A survey of hallucination in large foundation models

    Vipula Rawte, Amit Sheth, and Amitava Das. A survey of hallucination in large foundation models. arXiv preprint arXiv:2309.05922, 2023. 3

    2023 arXiv

  14. [22]

    Survey of hallucination in natural language generation

    Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. Survey of hallucination in natural language generation. ACM computing surveys, 55(12):1–38, 2023. 3

    2023

  15. [23]

    Evaluating object hallucination in large vision-language models

    Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. Evaluating object hallucination in large vision-language models. arXiv preprint arXiv:2305.10355, 2023. 3

    2023 arXiv

  16. [24]

    Hallucination of multimodal large language models: A survey

    Zechen Bai, Pichao Wang, Tianjun Xiao, Tong He, Zongbo Han, Zheng Zhang, and Mike Zheng Shou. Hallucination of multimodal large language models: A survey. arXiv preprint arXiv:2404.18930, 2024. 3

    2024 arXiv

  17. [25]

    Estimating the hallucination rate of generative ai

    Andrew Jesson, Nicolas Beltran Velez, Quentin Chu, Sweta Karlekar, Jannik Kossen, Yarin Gal, John P Cunningham, and David Blei. Estimating the hallucination rate of generative ai. Advances in Neural Information Processing Systems, 37:31154–31201, 2024. 3

    2024

  18. [26]

    Brainbits: How much of the brain are generative reconstruction methods using? Advances in Neural Information Processing Systems, 37:54396–54420, 2024

    David Mayo, Christopher Wang, Asa Harbin, Abdulrahman Alabdulkareem, Albert Shaw, Boris Katz, and Andrei Barbu. Brainbits: How much of the brain are generative reconstruction methods using? Advances in Neural Information Processing Systems, 37:54396–54420, 2024. 3

    2024

  19. [27]

    Spurious reconstruction from brain activity

    Ken Shirakawa, Yoshihiro Nagano, Misato Tanaka, Shuntaro C Aoki, Kei Majima, Yusuke Muraki, and Yukiyasu Kamitani. Spurious reconstruction from brain activity. arXiv preprint arXiv:2405.10078, 2024. 3

    2024 arXiv

  20. [28]

    Z-forcing: Training stochastic recurrent networks

    Anirudh Goyal ALIAS PARTH GOY AL, Alessandro Sordoni, Marc-Alexandre Côté, Nan Rose- mary Ke, and Yoshua Bengio. Z-forcing: Training stochastic recurrent networks. Advances in neural information processing systems, 30, 2017. 3, 5

    2017

  21. [29]

    A survey on multimodal large language models

    Shukang Yin, Chaoyou Fu, Sirui Zhao, Ke Li, Xing Sun, Tong Xu, and Enhong Chen. A survey on multimodal large language models. National Science Review, 11, 2023. 3

    2023

  22. [30]

    Deep neural networks rival the representation of primate it cortex for core visual object recognition

    Charles F Cadieu, Ha Hong, Daniel LK Yamins, Nicolas Pinto, Diego Ardila, Ethan A Solomon, Najib J Majaj, and James J DiCarlo. Deep neural networks rival the representation of primate it cortex for core visual object recognition. PLoS computational biology, 10(12):e1003963, 2014. 3

    2014

  23. [31]

    Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence

    Radoslaw Martin Cichy, Aditya Khosla, Dimitrios Pantazis, Antonio Torralba, and Aude Oliva. Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence. Scientific reports, 6(1):27755, 2016. 3

    2016

  24. [32]

    Convolutional neural network-based encoding and decoding of visual object recognition in space and time

    Katja Seeliger, Matthias Fritsche, Umut Güçlü, Sanne Schoenmakers, J-M Schoffelen, Sander E Bosch, and MAJ Van Gerven. Convolutional neural network-based encoding and decoding of visual object recognition in space and time. NeuroImage, 180:253–266, 2018. 3 11

    2018

  25. [33]

    Evidence of a predictive coding hierarchy in the human brain listening to speech.Nature human behaviour, 7(3):430–441,

    Charlotte Caucheteux, Alexandre Gramfort, and Jean-Rémi King. Evidence of a predictive coding hierarchy in the human brain listening to speech.Nature human behaviour, 7(3):430–441,

  26. [34]

    The neural architecture of language: Integrative modeling converges on predictive processing

    Martin Schrimpf, Idan Asher Blank, Greta Tuckute, Carina Kauf, Eghbal A Hosseini, Nancy Kanwisher, Joshua B Tenenbaum, and Evelina Fedorenko. The neural architecture of language: Integrative modeling converges on predictive processing. Proceedings of the National Academy of Sc...

    2021

  27. [35]

    Low-dimensional structure in the space of language representations is reflected in brain responses

    Richard Antonello, Javier S Turek, Vy V o, and Alexander Huth. Low-dimensional structure in the space of language representations is reflected in brain responses. Advances in neural information processing systems, 34:8332–8344, 2021. 3

    2021

  28. [36]

    Incorporating context into language encoding models for fmri

    Shailee Jain and Alexander Huth. Incorporating context into language encoding models for fmri. Advances in neural information processing systems, 31, 2018. 3

    2018

  29. [37]

    Interpreting and improving natural-language processing (in machines) with natural language-processing (in the brain)

    Mariya Toneva and Leila Wehbe. Interpreting and improving natural-language processing (in machines) with natural language-processing (in the brain). Advances in neural information processing systems, 32, 2019. 3

    2019

  30. [38]

    Predictive coding or just feature discovery? an alternative account of why language models fit brain data

    Richard Antonello and Alexander Huth. Predictive coding or just feature discovery? an alternative account of why language models fit brain data. Neurobiology of Language, 5(1): 64–79, 2024. 3

    2024

  31. [39]

    Layer by layer: Uncovering hidden representations in language models

    Oscar Skean, Md Rifat Arefin, Dan Zhao, Niket Patel, Jalal Naghiyev, Yann LeCun, and Ravid Shwartz-Ziv. Layer by layer: Uncovering hidden representations in language models. arXiv preprint arXiv:2502.02013, 2025. 3

    2025 arXiv

  32. [40]

    Recursive deep models for semantic compositionality over a sentiment treebank

    Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language proce...

    2013

  33. [41]

    Integrating probabilistic extraction models and data mining to discover relations and patterns in text

    Aron Culotta, Andrew McCallum, and Jonathan Betz. Integrating probabilistic extraction models and data mining to discover relations and patterns in text. In Proceedings of the Human Language Technology Conference of the NAACL, Main Conference, pages 296–303, 2006. 4

    2006

  34. [42]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning, pages 19730–19742. PMLR, 2023. 4

    2023

  35. [43]

    Towards large-scale 3d representation learning with multi-dataset point prompt training

    Xiaoyang Wu, Zhuotao Tian, Xin Wen, Bohao Peng, Xihui Liu, Kaicheng Yu, and Hengshuang Zhao. Towards large-scale 3d representation learning with multi-dataset point prompt training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 19...

    2024

  36. [44]

    Transfer learning in brain-computer interfaces

    Vinay Jayaram, Morteza Alamgir, Yasemin Altun, Bernhard Scholkopf, and Moritz Grosse- Wentrup. Transfer learning in brain-computer interfaces. IEEE Computational Intelligence Magazine, 11(1):20–31, 2016. 4

    2016

  37. [45]

    Inter-subject deep transfer learning for motor imagery eeg decoding

    Xiaoxi Wei, Pablo Ortega, and A Aldo Faisal. Inter-subject deep transfer learning for motor imagery eeg decoding. In2021 10th international IEEE/EMBS conference on neural engineering (NER), pages 21–24. IEEE, 2021. 4

    2021

  38. [46]

    Brief segments of neurophysiological activity enable individual differentiation

    Jason da Silva Castanheira, Hector Domingo Orozco Perez, Bratislav Misic, and Sylvain Baillet. Brief segments of neurophysiological activity enable individual differentiation. Nature communications, 12(1):5713, 2021. 4

    2021

  39. [47]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4195–4205, 2023. 5

    2023

  40. [48]

    Scaling instruction-finetuned language models

    Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, and et al. Scaling instruction-finetuned language models. Journal of Machine Learning Research, 25(1),

  41. [49]

    Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension

    M Lewis. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461, 2019. 5

    1910 arXiv

  42. [50]

    Exploring the limits of transfer learning with a unified text-to-text transformer

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1–67, 2020. 5

    2020

  43. [51]

    Extraphrase: Efficient data augmentation for abstractive summarization

    Mengsay Loem, Sho Takase, Masahiro Kaneko, and Naoaki Okazaki. Extraphrase: Efficient data augmentation for abstractive summarization. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technolog...

    2022

  44. [52]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pa...

    2021

  45. [53]

    Recent progress in wearable brain–computer interface (bci) devices based on electroencephalogram (eeg) for medical applications: a review

    Jiayan Zhang, Junshi Li, Zhe Huang, Dong Huang, Huaiqiang Yu, and Zhihong Li. Recent progress in wearable brain–computer interface (bci) devices based on electroencephalogram (eeg) for medical applications: a review. Health Data Science, 3:0096, 2023. 15

    2023

  46. [54]

    Brain-to-text decoding: A non-invasive approach via typing.arXiv preprint arXiv:2502.17480, 2025

    Jarod Lévy, Mingfang Zhang, Svetlana Pinet, Jérémy Rapin, Hubert Banville, Stéphane d’Ascoli, and Jean-Rémi King. Brain-to-text decoding: A non-invasive approach via typing.arXiv preprint arXiv:2502.17480, 2025. 15

    2025 arXiv

  47. [55]

    Thinking out loud, an open-access eeg-based bci dataset for inner speech recognition

    Nicolás Nieto, Victoria Peterson, Hugo Leonardo Rufiner, Juan Esteban Kamienkowski, and Ruben Spies. Thinking out loud, an open-access eeg-based bci dataset for inner speech recognition. Scientific data, 9(1):52, 2022. 15

    2022

  48. [56]

    Pretraining large brain language model for active bci: Silent speech

    Jinzhao Zhou, Zehong Cao, Yiqun Duan, Connor Barkley, Daniel Leong, Xiaowei Jiang, Quoc- Toan Nguyen, Ziyi Zhao, Thomas Do, Yu-Cheng Chang, et al. Pretraining large brain language model for active bci: Silent speech. arXiv preprint arXiv:2504.21214, 2025. 15

    2025 arXiv

  49. [57]

    The llama 3 herd of models

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024. 15

    2024 arXiv

  50. [58]

    You that read wrong again! a transposed- word effect in grammaticality judgments

    Jonathan Mirault, Joshua Snell, and Jonathan Grainger. You that read wrong again! a transposed- word effect in grammaticality judgments. Psychological Science, 29(12):1922–1929, 2018. 16

    1922

  51. [59]

    McGowan, Mahmoud M Elsherif, Michael G Cutter, Jingxin Wang, Zhiwei Liu, and Kevin B Paterson

    Sara V Milledge, Neya Bhatia, Loren Mensah-Mcleod, Pallvi Raghvani, Victoria A. McGowan, Mahmoud M Elsherif, Michael G Cutter, Jingxin Wang, Zhiwei Liu, and Kevin B Paterson. The transposed-word effect provides no unequivocal evidence for parallel processing. Attention, Percep...

    2023

  52. [60]

    When classifying grammatical role, bert doesn’t care about word order

    Isabel Papadimitriou, Richard Futrell, and Kyle Mahowald. When classifying grammatical role, bert doesn’t care about word order... except when it matters. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages...

    2022

  53. [61]

    When does word order matter and when doesn’t it? arXiv preprint arXiv:2402.18838, 2024

    Xuanda Chen, Timothy O’Donnell, and Siva Reddy. When does word order matter and when doesn’t it? arXiv preprint arXiv:2402.18838, 2024. 16

    2024 arXiv

  54. [62]

    To English:

    T Wolf. Huggingface’s transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771, 2019. 17 13 Appendix A Dataset Our modeling and evaluation protocols are tightly integrated with the experimental design of the ZuCo dataset, which features notab...

    1910 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.