REVIEW 5 cited by
BELT-2: Bootstrapping EEG-to-Language representation alignment for multi-task brain decoding
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
The remarkable success of large language models (LLMs) across various multi-modality applications is well established. However, integrating large language models with humans, or brain dynamics, remains relatively unexplored. In this paper, we introduce BELT-2, a pioneering multi-task model designed to enhance both encoding and decoding performance from EEG signals. To bolster the quality of the EEG encoder, BELT-2 is the first work to innovatively 1) adopt byte-pair encoding (BPE)-level EEG-language alignment and 2) integrate multi-task training and decoding in the EEG domain. Inspired by the idea of \textbf{\textit{Bridging the Brain with GPT}}, we further connect the multi-task EEG encoder with LLMs by utilizing prefix-tuning on intermediary output from the EEG encoder. These innovative efforts make BELT-2 a pioneering breakthrough, making it the first work in the field capable of decoding coherent and readable sentences from non-invasive brain signals. Our experiments highlight significant advancements over prior techniques in both quantitative and qualitative measures, achieving a decoding performance with a BLEU-1 score of 52.2\% on the ZuCo dataset. Furthermore, BELT-2 shows a remarkable improvement ranging from 31\% to 162\% on other translation benchmarks. Codes can be accessed via the provided anonymous link~\footnote{https://anonymous.4open.science/r/BELT-2-0048}.
Forward citations
Cited by 5 Pith papers
-
EEG2TEXT-CN: An Exploratory Study of Open-Vocabulary Chinese Text-EEG Alignment via Large Language Model and Contrastive Learning on ChineseEEG
A Chinese EEG-to-text system that aligns 128-channel EEG with per-character text embeddings achieves BLEU-1 6.38% on a held-out subject, claimed as the first open-vocabulary EEG-to-Chinese decoder.
-
WorldWeaver: Generating Long-Horizon Video Worlds via Rich Perception
WorldWeaver reduces temporal drift in long-horizon video generation by jointly modeling RGB and depth perceptual conditions with segmented noise scheduling.
-
Foundation Models for Cross-Domain EEG Analysis Application: A Survey
A survey that organizes EEG foundation-model research into five output-modality categories: native EEG, text, vision, audio, and multimodal fusion, with a claim to be the first such comprehensive taxonomy.
-
Large Language Models for EEG: A Comprehensive Survey and Taxonomy
A taxonomy and review of studies applying large language models to EEG signals, organized into four domains and three adaptation strategies.
-
Bridging Brain with Foundation Models through Self-Supervised Learning
A PRISMA-based survey maps self-supervised learning techniques, brain foundation models, datasets, and evaluation protocols for EEG and related neural signals, including a skeptical review of EEG-to-text decoding.
Discussion (0). Sign in to comment.