Pith. sign in

REVIEW 3 major objections 1 minor 7 cited by

WorldWeaver: Generating Long-Horizon Video Worlds via Rich Perception

T0 review · 3 major / 1 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Long AI videos stay coherent when depth is predicted alongside color in one pass.

desk verdict The submitted full text is an EEG survey, not WorldWeaver, so the technical claims are unreviewable; the abstract is intriguing but not enough. read the letter →

arxiv 2508.15720 v1 pith:D5DZ64FR submitted 2025-08-21 cs.CV

Zhiheng Liu , Xueqing Deng , Shoufa Chen , Angtian Wang , Qiushan Guo , Mingfei Han , Zeyue Xue , Mengzhao Chen
show 2 more authors
Ping Luo Linjie Yang
This is my paper
classification cs.CV
keywords long-horizonvideogenerationtemporalconsistencydepthmemorybankperceptualconditionsdiffusionmodelsrectifiedflowworlddriftreduction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Long-horizon video generation models that work only in RGB space accumulate errors: object structure and motion degrade as sequences stretch. WorldWeaver proposes training a video model to predict color frames and perceptual conditions, chiefly depth, jointly from a unified representation, so that the two outputs reinforce each other. The paper reports that depth is more resistant to drift than RGB, and builds a memory bank from predicted depth to carry context across long sequences, with a segmented noise schedule that cuts drift and compute. The claim matters because stable, long videos are the bottleneck for turning generative video into usable world models. If right, it offers a simple recipe: condition on geometry, not just pixels.

What carries the argument

The depth-derived memory bank: a store of predicted depth maps that carries scene structure forward across generation steps, exploiting the observation that depth estimates drift less than color estimates. It is paired with a unified representation that predicts perceptual conditions and RGB together, and with segmented noise scheduling for prediction groups that reduces drift and training cost.

What would settle it

Train WorldWeaver with an RGB memory bank in place of the depth memory bank, keeping everything else fixed, and compare temporal consistency and fidelity on long sequences (e.g., 100+ frames). If the RGB-memory variant matches or beats the depth-memory variant, the central claim is wrong. A second check: measure the drift of the model's own depth predictions versus its RGB predictions over the same rollout; the depth channel must be measurably more stable.

Watch

Extended reading notes

Core claim

WorldWeaver's central claim is that the usual RGB-only training objective is the wrong target for long-sequence video. Instead, the model should jointly predict perceptual conditions (depth) and color from one unified representation. The paper identifies depth as a drift-resistant signal and uses a memory bank of predicted depth maps to preserve clear context over long horizons, while segmented noise scheduling keeps training groups stable and cheaper. Experiments across diffusion- and rectified flow-based backbones show reduced temporal drift and improved fidelity relative to RGB-only baselines.

Load-bearing premise

The central assumption is that predicted depth stays more stable over long horizons than predicted RGB, so depth is a better channel for a context memory bank; if depth predictions drift as much as color, the main advantage collapses.

Editorial extensions

If this is right

  • Long-horizon video generation can be stabilized by adding a geometric side-channel rather than by enlarging the model or data alone.
  • Memory built on depth should preserve scene layout and object identity across hundreds of frames better than memory built on RGB.
  • The joint-prediction recipe transfers across generation families (diffusion and rectified flow), suggesting it is a training-objective property, not a backbone-specific fix.
  • Segmented noise scheduling makes long-sequence training practical at lower compute, lowering the barrier for longer outputs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If depth is genuinely the drift-resistant channel, the same memory-bank idea could extend to other geometric cues—surface normals, optical flow, or semantic maps—each with its own drift profile.
  • A testable consequence the paper leaves implicit: temporal consistency should degrade smoothly as depth-prediction error increases, making depth-predictor quality a measurable bottleneck.
  • The depth memory bank could double as a persistent 3D scaffold, enabling controllable or interactive generation where user edits to depth propagate to the video, though the paper does not explore this.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 1 minor

Summary. The submission is titled 'WorldWeaver: Generating Long-Horizon Video Worlds via Rich Perception' and consists of an abstract claiming a framework that jointly models RGB frames and perceptual conditions, uses a depth-derived memory bank, and employs segmented noise scheduling to reduce temporal drift and improve fidelity in long-horizon video generation. However, the full text supplied with the submission is an unrelated survey, 'Foundation Models for Cross-Domain EEG Analysis Application: A Survey,' carrying arXiv:2508.15716v2 [cs.HC]. No methods, equations, experimental setup, ablations, baselines, or quantitative results for WorldWeaver appear anywhere in the submitted material.

Significance. If the claims in the abstract were substantiated, WorldWeaver could represent a meaningful step toward long-horizon video generation by exploiting depth cues and perception-condition joint modeling. The idea that depth is more drift-resistant than RGB and can be used for a memory bank is interesting and potentially useful. However, no technical content is available for evaluation: there are no definitions, derivations, architectural details, training procedures, evaluation protocols, or comparisons to existing methods. The manuscript also does not provide reproducible artifacts, machine-checked proofs, or parameter-free derivations that could partially compensate for missing experimental detail. As submitted, the significance cannot be assessed beyond the abstract-level claim.

major comments (3)
  1. [Full text, page 1] The full text of the submission is not the paper described in the abstract. Page 1 begins with 'Foundation Models for Cross-Domain EEG Analysis Application: A Survey' and carries the identifier arXiv:2508.15716v2 [cs.HC]. None of the sections, equations, tables, or references pertain to WorldWeaver or to video generation. This is a document-integrity problem that makes the central claims unverifiable from the submitted material.
  2. [Abstract, claims 1–3] All three central claims—(1) joint prediction of perceptual conditions and color improves temporal consistency and motion dynamics; (2) a depth-derived memory bank preserves clearer contextual information because depth is 'more resistant to drift than RGB'; and (3) segmented noise scheduling mitigates drift and reduces computational cost—are stated without any supporting evidence in the submitted material. There are no equations defining the unified representation, the memory bank, or the noise scheduling, and no experiments demonstrating the claimed effects. The depth-vs-RGB drift-resistance premise is an empirical assertion that requires measurement; it is not self-evident and could fail if predicted depth is itself unstable or inconsistent with RGB.
  3. [Abstract, 'Extensive experiments'] The abstract states that extensive experiments on diffusion- and rectified flow-based models demonstrate effectiveness, but no experimental section, metrics, datasets, baselines, or tables are present. The reader cannot check whether the claimed improvements in temporal drift and fidelity are real, statistically meaningful, or obtained under fair comparisons. This is a load-bearing omission: the entire contribution is empirical, and the empirical record is absent.
minor comments (1)
  1. [General] The abstract provides no citations to prior work on long-horizon video generation, depth-conditioned diffusion, or memory-based temporal consistency, making it impossible to situate the contribution in context. If a corrected manuscript is submitted, the authors should add appropriate references.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity identifiable; submitted full text is a different paper, so no derivation chain is available to assess.

full rationale

The claimed paper (WorldWeaver) is represented only by its abstract; the supplied full text is actually 'Foundation Models for Cross-Domain EEG Analysis Application: A Survey' (arXiv:2508.15716v2). No methods, equations, ablations, or derivation chain from WorldWeaver are present, so there is no concrete reduction to exhibit. The abstract's statements—joint prediction of perceptual conditions and color, depth cues being more drift-resistant than RGB, and segmented noise scheduling—are empirical claims without supporting derivations; they do not define a quantity in terms of another quantity, do not fit a parameter and rename it a prediction, and do not rely on a self-citation to force a conclusion. The only potentially load-bearing premise (depth is more drift-resistant than RGB) is asserted as an observation, but an unsupported premise is a correctness/verifiability concern, not circularity. Per the hard rules, circularity must be shown by quoting the paper and exhibiting a specific reduction; that cannot be done here. The document mismatch is an integrity problem rather than a circularity signal. Therefore the honest finding is no significant circularity, score 0.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

No free parameters or invented entities are claimed in the abstract. The two axioms listed are empirical/design assumptions the framework rests on, both stated in the abstract but unverifiable without the full paper.

assumptions (2)
  • domain assumption Depth cues are more resistant to drift than RGB and can serve as a reliable memory bank for long-horizon generation.
    Stated as an observation in the abstract ('which we observe to be more resistant to drift than RGB'); it is load-bearing because the memory bank design depends on it, but no evidence is visible in the abstract.
  • domain assumption A unified representation can jointly predict perceptual conditions and color without task conflict that would worsen generation.
    The method's first advantage assumes joint prediction helps consistency; this is the core hypothesis, not yet supported by visible evidence.

how reviews work

0 comments
Cite this review

Pith. "Pith review of WorldWeaver: Generating Long-Horizon Video Worlds via Rich Perception." pith.science (2026). https://pith.science/paper/D5DZ64FR

@misc{pith2026250815720,
  author       = {Pith},
  title        = {Pith review of: WorldWeaver: Generating Long-Horizon Video Worlds via Rich Perception},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/D5DZ64FR}},
  note         = {Machine review of arXiv:2508.15720}
}
read the original abstract

Generative video modeling has made significant strides, yet ensuring structural and temporal consistency over long sequences remains a challenge. Current methods predominantly rely on RGB signals, leading to accumulated errors in object structure and motion over extended durations. To address these issues, we introduce WorldWeaver, a robust framework for long video generation that jointly models RGB frames and perceptual conditions within a unified long-horizon modeling scheme. Our training framework offers three key advantages. First, by jointly predicting perceptual conditions and color information from a unified representation, it significantly enhances temporal consistency and motion dynamics. Second, by leveraging depth cues, which we observe to be more resistant to drift than RGB, we construct a memory bank that preserves clearer contextual information, improving quality in long-horizon video generation. Third, we employ segmented noise scheduling for training prediction groups, which further mitigates drift and reduces computational cost. Extensive experiments on both diffusion- and rectified flow-based models demonstrate the effectiveness of WorldWeaver in reducing temporal drift and improving the fidelity of generated videos.

Discussion (0). Sign in to comment.

Forward citations

Cited by 7 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Divide and Conquer: Decoupled Representation Alignment for Multimodal World Models

    cs.CV 2026-05 unverdicted novelty 7.0 of 10

    M²-REPA decouples modality-specific features from diffusion intermediates and aligns them to complementary expert foundation models via a multi-modal alignment loss and modality-specific decoupling regularization for ...

  2. HYDRA-X: Native Unified Multimodal Models with Holistic Visual Tokenizers

    cs.CV 2026-06 unverdicted novelty 6.0 of 10

    HYDRA-X presents the first unified multimodal model using a single ViT for holistic image-video tokenization, with ablations on attention and compression plus a latent-level editing improvement.

  3. TunerDiT: Training-free Progressive Steering of Diffusion Transformer for Multi-Event Video Generation

    cs.CV 2026-05 unverdicted novelty 6.0 of 10

    TunerDiT adds event-partitioned masking and cross-event prompt fusion to diffusion transformers for training-free multi-event video generation, with gains scaling by event count on a new Meve benchmark.

  4. SWIFT: Prompt-Adaptive Memory for Efficient Interactive Long Video Generation

    cs.CV 2026-05 unverdicted novelty 6.0 of 10

    SWIFT introduces a semantic injection cache with head-wise updates and an adaptive dynamic window plus segment anchors to achieve efficient multi-prompt long video generation at 22.6 FPS while preserving quality in ca...

  5. Divide and Conquer: Decoupled Representation Alignment for Multimodal World Models

    cs.CV 2026-05 unverdicted novelty 6.0 of 10

    M²-REPA decouples modality-specific features inside a diffusion model and aligns each to its matching expert foundation model via an alignment loss plus a decoupling regularizer, yielding better visual quality and lon...

  6. Self-Forcing++: Towards Minute-Scale High-Quality Video Generation

    cs.CV 2025-10 conditional novelty 6.0 of 10

    Self-Forcing++ scales autoregressive video diffusion to over 4 minutes by using self-generated segments for guidance, reducing error accumulation and outperforming baselines in fidelity and consistency.

  7. Rolling Forcing: Autoregressive Long Video Diffusion in Real Time

    cs.CV 2025-09 unverdicted novelty 6.0 of 10

    Rolling Forcing generates multi-minute videos in real time by jointly denoising frames at increasing noise levels, anchoring attention to early frames, and using windowed distillation to limit error accumulation.

Reference graph

Works this paper leans on

125 extracted references · 50 canonical work pages · cited by 6 Pith papers

  1. [1]

    Dual-TSST: A dual-branch temporal- spectral-spatial Transformer model for EEG decoding,

    H. Li, H. Zhang and Y . Chen, “Dual-TSST: A dual-branch temporal- spectral-spatial Transformer model for EEG decoding,” J. Biomed. Heath Inform., doi: 10.1109/JBHI.2025.3577611

    arXiv 2025

  2. [2]

    Noninvasive EEG-based intelligent mobile robots: a systematic review,

    H. Li, X. Li and J. d. R. Mill ´an, “Noninvasive EEG-based intelligent mobile robots: a systematic review,” IEEE Trans. Autom. Sci. Eng., vol. 22, pp. 6291-6315, 2025

    2025

  3. [3]

    Neural decoding of EEG signals with machine learning: a systematic review,

    M. Saeidi et al. , “Neural decoding of EEG signals with machine learning: a systematic review,” Brain Sci. , vol. 11, no. 11, pp. 1524- 1569, 2021

    2021

  4. [4]

    Deep learning-based electroencephalography analysis: A systematic review,

    Yannick et al., “Deep learning-based electroencephalography analysis: A systematic review,” J. Neural Eng. , vol. 16, 2019, Art. no. 051001

    2019

  5. [5]

    Transformer-based EEG decoding: A survey,

    H. Zhang, and H. Li, “Transformer-based EEG decoding: A survey,” 2025, arXiv:2507.02320

    arXiv 2025

  6. [6]

    On the opportunities and risks of foundation models,

    R. Bommasani, et al. , “On the opportunities and risks of foundation models,” 2021, arXiv:2108.07258

    arXiv 2021

  7. [7]

    Improving language understanding by generative pre-training,

    A. Radford, and K. Narasimhan, “Improving language understanding by generative pre-training,” 2018

    2018

  8. [8]

    LLaMA: Open and efficient foundation language models

    H. Touvron et al. , “LLaMA: Open and efficient foundation language models.” 2023, arXiv:2302.13971

    arXiv 2023

Show all 125 references
  1. [9]

    An image is worth 16×16 words: Transformers for image recognition at scale,

    A. Dosovitskiy et al., “An image is worth 16×16 words: Transformers for image recognition at scale,” in Proc. Int. Conf. Learn. Represent. , 2021, pp. 1-21

    2021

  2. [10]

    Swin Transformer: Hierarchical vision transformer using shifted windows,

    Z. Liu et al., “Swin Transformer: Hierarchical vision transformer using shifted windows,” in Proc. IEEE/CVF Int. Conf. Comput. Vis. , 2021, pp. 10012-10022

    2021

  3. [11]

    Wav2Vec: Un- supervised pre training for speech recognition,

    S. Schneider, A. Baevski, R. Collobert, and M. Auli, “Wav2Vec: Un- supervised pre training for speech recognition,” in Proc. Interspeech, 2019, pp. 3465-3469

    2019

  4. [12]

    Whisper: Robust speech recognition via large scale weak supervision,

    A. Radford et al., “Whisper: Robust speech recognition via large scale weak supervision,” arXiv:2212.04356, 2022

    2022 arXiv

  5. [13]

    Learning transferable visual models from natural language supervision,

    A. Radford et al. , “Learning transferable visual models from natural language supervision,” in Proc. Int. Conf. Mach. Learn. , 2021, pp. 8748–8763

    2021

  6. [14]

    Flamingo: a visual language model for few-shot learning,

    J.B. Alayrac et al. , “Flamingo: a visual language model for few-shot learning,” Adv. Neural Inf. Process. Syst. , vol. 35, 2022, pp. 23716- 23736

    2022

  7. [15]

    Gemini: Efficient multimodal pretraining for vision and language,

    N. Goyal et al. , “Gemini: Efficient multimodal pretraining for vision and language,” 2024, arXiv:2401.01234

    2024 arXiv

  8. [16]

    PaLM-E: An embodied multimodal language model,

    D. Driess et al., “PaLM-E: An embodied multimodal language model,” 2023, arXiv:2303.13971

    2023 arXiv

  9. [17]

    A survey on bridging EEG signals and generative AI: From image and text to beyond,

    S. Shukla et al. , “A survey on bridging EEG signals and generative AI: From image and text to beyond,” 2025, arXiv:2502.12048

    2025

  10. [18]

    Decoding linguistic representations of human brain,

    Y . Wang, et al., “Decoding linguistic representations of human brain,” 2024, arXiv:2407.20622

    2024 arXiv

  11. [19]

    Unveiling thoughts: A review of ad- vancements in EEG brain signal decoding into text,

    S. A. Murad and N. Rahimi, “Unveiling thoughts: A review of ad- vancements in EEG brain signal decoding into text,”IEEE Trans. Cogn. Dev. Syst., vol. 17, no. 1, pp. 61-76, Feb. 2025

    2025

  12. [20]

    Decoding natural images from EEG for object recogni- tion,

    Y . Song et al., “Decoding natural images from EEG for object recogni- tion,” in Proc. Int. Conf. Learn. Represent. (ICLR), May 2024, pp. 1–6

    2024

  13. [21]

    Decoding speech perception from non-invasive brain recordings,

    A. D ´efossez et al. , “Decoding speech perception from non-invasive brain recordings,” Nat. Mach. Intell. , vol. 5, no. 10, pp. 1097-1107, Oct. 2023

    2023

  14. [22]

    Self-supervised Learning for Electroencephalog- raphy,

    M. H. Rafiei, et al., “Self-supervised Learning for Electroencephalog- raphy,” IEEE Trans. Neural Netw. Learn. Syst. , V ol. 35, no. 2, 2024, pp. 1457-1471

    2024

  15. [23]

    E2H: A two-stage non-invasive neural signal driven humanoid robotic whole-body control framework,

    Y . Duan et al. , “E2H: A two-stage non-invasive neural signal driven humanoid robotic whole-body control framework,” 2024, arXiv:2410.02141

    2024 arXiv

  16. [24]

    ChatBCI: A P300 speller BCI leveraging large language models for improved sentence composition in realistic scenarios,

    J. Hong et al., “ChatBCI: A P300 speller BCI leveraging large language models for improved sentence composition in realistic scenarios,” 2024, arXiv: 2411.15395

    2024

  17. [25]

    Sequential best-arm identification with application to P300 Speller,

    X. Zhou et al., “Sequential best-arm identification with application to P300 Speller,” Transact. Mach. Learn. Res. , Aug. 2024

    2024

  18. [26]

    Neural spelling: A spell-based BCI system for language neural decoding,

    X. Jiang et al., “Neural spelling: A spell-based BCI system for language neural decoding,” 2025, arXiv:2501.17489

    2025

  19. [27]

    Human-AI teaming using large language models: Boosting brain-computer interfacing (BCI) and brain research,

    M. Kapitnova and T. Ball, “Human-AI teaming using large language models: Boosting brain-computer interfacing (BCI) and brain research,” 2025, arXiv: 2501.01451

    2025 arXiv

  20. [28]

    Classification of non-invasive EEG signals during motor imagery tasks using a large language model,

    D. Lim, M. Cho and H. Kim, “Classification of non-invasive EEG signals during motor imagery tasks using a large language model,” in Proc. Int. Conf. Cyberworlds (CW) , Oct. 2024, pp. 378-379

    2024

  21. [29]

    BENDR: Using trans- formers and a contrastive self-supervised learning task to learn from massive amounts of EEG data,

    D. Kostas, S. Aroca-Ouellette and F. Rudzicz, “BENDR: Using trans- formers and a contrastive self-supervised learning task to learn from massive amounts of EEG data,” Front. Hum. Neurosci. , vol. 15, pp. 253, Jun. 2021

    2021

  22. [30]

    From word embedding to reading embedding using large language model, EEG and eye-tracking,

    Y . Zhang et al. , “From word embedding to reading embedding using large language model, EEG and eye-tracking,” inProc. IEEE Eng. Med. Biol. Soc. (EMBC) , Jul. 2024, pp. 1-4

    2024

  23. [31]

    Integrating large language model, EEG, and eye- tracking for word-level neural state classification in reading com- prehension,

    Y . Zhang et al. , “Integrating large language model, EEG, and eye- tracking for word-level neural state classification in reading com- prehension,” IEEE Trans. Neural Syst. Rehabil. Eng., vol. 32, pp. 3465- 3475, Aug. 2024

    2024

  24. [32]

    CBraMod: A criss-cross brain foundation model for EEG decoding,

    J. Wang et al. , “CBraMod: A criss-cross brain foundation model for EEG decoding,” in Proc. Int. Conf. Learn. Represent. (ICLR) , Apr. 2025, pp. 62056-62092

    2025

  25. [33]

    NeuroChat: A neuroadaptive AI chatbot for cus- tomizing learning experiences,

    D. Baradari et al. , “NeuroChat: A neuroadaptive AI chatbot for cus- tomizing learning experiences,” 2025, arXiv:2503.07599

    2025

  26. [34]

    EEG emotion copilot: Pruning LLMs for emotional EEG interpretation with assisted medical record generation,

    H. Chen et al. , “EEG emotion copilot: Pruning LLMs for emotional EEG interpretation with assisted medical record generation,” Nerual Netw., 107848, pp. 1-17, Jul. 2025

    2025

  27. [35]

    Exploring large-scale language models to evaluate EEG- based multimodal data for mental health,

    Y . Hu et al., “Exploring large-scale language models to evaluate EEG- based multimodal data for mental health,” in Proc. ACM Int. Jt. Conf. Pervasive Ubiquitous Comput. (UbiComp) , Oct. 2024, pp. 412-417

    2024

  28. [36]

    LLM-enhanced multi-teacher knowledge distillation for modality-incomplete emotion recognition in daily healthcare,

    Y . Zhang et al. , “LLM-enhanced multi-teacher knowledge distillation for modality-incomplete emotion recognition in daily healthcare,” IEEE J. Biomed. Heath Inform. , pp. 1–11, Sep. 2024

    2024

  29. [37]

    Advancing semi-supervised EEG emotion recognition through feature extraction with mixup and large language models,

    S. Yao et al. , “Advancing semi-supervised EEG emotion recognition through feature extraction with mixup and large language models,” in Proc. IEEE Int. Conf. Bioinform. Biomed. (BIBM) , Dec. 2024, pp. 2772-2779

    2024

  30. [38]

    Emotion analysis AI model for sensing architecture using EEG,

    S.-Y . Ji, M.-K. Kim and H.-J. Jun, “Emotion analysis AI model for sensing architecture using EEG,” Appl. Sci. , vol. 15, no.5, pp. 1-21, Mar. 2025

    2025

  31. [39]

    EEGM2: An efficient mamba-2-based self- supervised framework for long-sequence EEG modeling,

    J. Hong et al. , “EEGM2: An efficient mamba-2-based self- supervised framework for long-sequence EEG modeling,” 2025, arXiv:2502.17813

    2025 arXiv

  32. [40]

    BERT learns from electroencephalograms about Parkinson’s disease: Transformer-based models for aid diagnosis,

    A. Nogales et al. , “BERT learns from electroencephalograms about Parkinson’s disease: Transformer-based models for aid diagnosis,” IEEE Access, vol. 10, pp. 101672-101682, Aug. 2022

    2022

  33. [41]

    When neural implant meets multi- modal LLM: A dual-loop system for neuromodulation and naturalistic neuralbehav-ioral research,

    E. H. Wang and C. X. Wen, “When neural implant meets multi- modal LLM: A dual-loop system for neuromodulation and naturalistic neuralbehav-ioral research,” 2025, arXiv:2503.12334

    2025 arXiv

  34. [42]

    BrainWave: A brain signal foundation model for clinical applications,

    Z. Yuan et al. , “BrainWave: A brain signal foundation model for clinical applications,” 2024, arXiv:2402.10251

    2024 arXiv

  35. [43]

    Clinical grade prediction of therapeutic dosage for electroconvulsive therapy (ECT) based on patient’s pre-ictal EEG using fuzzy causal transformers,

    C. Kalantarpour et al., “Clinical grade prediction of therapeutic dosage for electroconvulsive therapy (ECT) based on patient’s pre-ictal EEG using fuzzy causal transformers,” in Proc. 3rd Int. Conf. Elect. Comput. Commun. Mechatronics Eng. (ICECCME) , Jul. 2023, pp. 1-6

    2023

  36. [44]

    Large transformers are better EEG learners,

    B. Wang et al. , “Large transformers are better EEG learners,” 2023, arXiv:2308.11654

    2023 arXiv

  37. [45]

    EEG-GPT: Exploring capabilities of large language models for EEG classification and interpretation,

    J. W. Kim, A. Alaa and D. Bernardo, “EEG-GPT: Exploring capabilities of large language models for EEG classification and interpretation,” 2024, arXiv:2401.18006

    2024 arXiv

  38. [46]

    NeuroLM: A universal multi-task foundation model for bridging the gap between language and EEG signals,

    W. B. Jiang et al., “NeuroLM: A universal multi-task foundation model for bridging the gap between language and EEG signals,” in Proc. Int. Conf. Learn. Represent. (ICLR) , Apr. 2025, pp. 46232-46253

    2025

  39. [47]

    A spatial-temporal transformer architecture using mul- ti-channel signals for sleep stage classification,

    H. Yao et al., “A spatial-temporal transformer architecture using mul- ti-channel signals for sleep stage classification,” IEEE Trans. Neural Syst. Rehabil. Eng. , vol. 31, pp. 3353-3362, Aug. 2023

    2023

  40. [48]

    Exploration of LLMs, EEG and behavioral data to measure and support attention and sleep,

    A. Sano et al. , “Exploration of LLMs, EEG and behavioral data to measure and support attention and sleep,” 2024, arXiv:2408.07822

    2024 arXiv

  41. [49]

    Are foundation models useful feature extractors for electroencephalography analysis?,

    ¨O. Turgut et al., “Are foundation models useful feature extractors for electroencephalography analysis?,” 2025, arXiv:2502.21086

    2025

  42. [50]

    LEAD: Large foundation model for EEG-based alz- heimer’s disease detection,

    Y . Wang et al. , “LEAD: Large foundation model for EEG-based alz- heimer’s disease detection,” 2025, arXiv:2502.01678

    2025

  43. [51]

    Can brain signals reveal inner alignment with human languages?,

    J. Qiu et al. , “Can brain signals reveal inner alignment with human languages?,” in Proc. Find. Assoc. Comput. Linguist (EMNLP) , Dec

  44. [52]

    Enhancing EEG-to-text decoding through transferable representations from pre-trained contrastive EEG-text masked auto- encoder,

    J. Wang et al., “Enhancing EEG-to-text decoding through transferable representations from pre-trained contrastive EEG-text masked auto- encoder,” in Proc. Annu. Meet. Assoc. Comput. Linguist. (ACL) , Aug. 2024, pp. 7278-7292

    2024

  45. [53]

    Aligning semantic in brain and language: A curricu- lum contrastive method for electroencephalography-to-text generation,

    X. Feng et al. , “Aligning semantic in brain and language: A curricu- lum contrastive method for electroencephalography-to-text generation,” IEEE Trans. Neural Syst. Rehabil. Eng. , vol.31, pp. 3874-3883, Sep. 2023

    2023

  46. [54]

    Towards linguistic neural representation learning and sentence retrieval from electroencephalogram recordings,

    J. Zhou et al. , “Towards linguistic neural representation learning and sentence retrieval from electroencephalogram recordings,” in Proc. Int. Workshop Brain-Comput. Interfaces (BCI) Multimed. Underst. (BCIMM), Oct. 2024, pp. 19-28

    2024

  47. [55]

    LLMs help alleviate the cross-subject variabil- ity in brain signal and language alignment,

    Y . Liu, H. Ye and S. Li, “LLMs help alleviate the cross-subject variabil- ity in brain signal and language alignment,” 2025, arXiv:2501.02621. JOURNAL OF LATEX CLASS FILES, VOL. 18, NO. 9, SEPTEMBER 2020 14

    2025 arXiv

  48. [56]

    BELT: Bootstrapped EEG-to-language training by natural language supervision,

    J. Zhou et al. , “BELT: Bootstrapped EEG-to-language training by natural language supervision,” IEEE Trans. Neural Syst. Rehabil. Eng., vol. 32, pp. 3278-3288, Aug. 2024

    2024

  49. [57]

    BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension,

    M. Lewi et al. , “BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension,” in Proc. Annu. Meet. Assoc. Comput Linguist. (ACL), Jul. 2020, pp. 7871- 7880

    2020

  50. [58]

    Open vocabulary electroencephalography-to-text decoding and zero-shot sentiment classification,

    Z. Wang, and H. Ji, “Open vocabulary electroencephalography-to-text decoding and zero-shot sentiment classification,” in Proc. AAAI Conf. Artif. Intell. (AAAI) , Jun. 2022, pp. 5350-5358

    2022

  51. [59]

    From thought to action: How a hierarchy of neural dynamics supports language production,

    J. L ´evy et al. , “From thought to action: How a hierarchy of neural dynamics supports language production,” 2025, arXiv:2502.07429

    2025 arXiv

  52. [60]

    Brain-to-text decoding: A non-invasive approach via typing,

    J. L ´evy, et al. , “Brain-to-text decoding: A non-invasive approach via typing,” 2025, arXiv:2502.17480

    2025 arXiv

  53. [61]

    Deep representation learning for open vocabulary electroencephalography-to-text decoding,

    H. Amrani, D. Micucci and P. Napoletano, “Deep representation learning for open vocabulary electroencephalography-to-text decoding,” IEEE J. Biomed. Health Inform. , pp. 1-12, Jun. 2024

    2024

  54. [62]

    SEE: Semantically aligned EEG-to-text translation,

    Y . Tao et al., “SEE: Semantically aligned EEG-to-text translation,” in Proc. IEEE Int. Conf. Acoust. Speech Signal Process. (ICASSP) , Apr. 2025, pp. 1-5

    2025

  55. [63]

    EEG2TEXT: Open vocabulary EEG-to-text de- coding with EEG pre-Training and multi-view transformer,

    H. Liu et al. , “EEG2TEXT: Open vocabulary EEG-to-text de- coding with EEG pre-Training and multi-view transformer,” 2024, arXiv:2405.02165

    2024 arXiv

  56. [64]

    Dewave: Discrete EEG waves encoding for brain dynamics to text translation,

    Y . Duan et al. , “Dewave: Discrete EEG waves encoding for brain dynamics to text translation,” in Proc. Int. Conf. Neural Inf. Process. Syst. (NeurIPS), Dec. 2023, pp. 1-22

    2023

  57. [65]

    BELT-2: Bootstrapping EEG-to-language representa- tion alignment for multi-task brain decoding,

    J. Zhou, et al. , “BELT-2: Bootstrapping EEG-to-language representa- tion alignment for multi-task brain decoding,” 2024, arXiv:2409.00121

    2024 arXiv

  58. [66]

    EEG-CLIP: Learning EEG repre-sentations from natural language descriptions,

    T. C. Ndir and R. T. Schirrmeister, “EEG-CLIP: Learning EEG repre-sentations from natural language descriptions,” 2025, arXiv:2503.16531

    2025 arXiv

  59. [67]

    Are EEG-to-text models working?,

    H. Jo et al. , “Are EEG-to-text models working?,” 2024, arXiv:2405.06459

    2024 arXiv

  60. [68]

    Learning robust deep visual representations from EEG brain recordings,

    P. Singh et al. , “Learning robust deep visual representations from EEG brain recordings,” in Proc. IEEE Winter Conf. Appl. Comput. Vis. (WACV), Jan. 2024, pp. 7553-7562

    2024

  61. [69]

    Human-aligned image models improve visual decod- ing from the brain,

    N. Rajabi et al., “Human-aligned image models improve visual decod- ing from the brain,” 2025, arXiv: 2502.03081

    2025 arXiv

  62. [70]

    MB2C: Multimodal bidirectional cycle consistency for learning robust visual neural representations,

    Y . Wei et al., “MB2C: Multimodal bidirectional cycle consistency for learning robust visual neural representations,” in Proc. ACM Int. Conf. Multimed. (MM), Oct. 2024, pp. 8992-9000

    2024

  63. [71]

    Neural-MCRL: Neural multimodal contrastive representa- tion learning for EEG-based visual decoding,

    Y . Li et al., “Neural-MCRL: Neural multimodal contrastive representa- tion learning for EEG-based visual decoding,” 2024,arXiv:2412.17337

    2024 arXiv

  64. [72]

    Visual neural decoding via improved visual-EEG se- mantic consistency,

    H. Chen et al., “Visual neural decoding via improved visual-EEG se- mantic consistency,” 2024, arXiv:2408.06788

    2024

  65. [73]

    RealMind: Advancing visual decoding and language in- teraction via EEG signals,

    D Li et al. , “RealMind: Advancing visual decoding and language in- teraction via EEG signals,” 2024, arXiv:2410.23754

    2024 arXiv

  66. [74]

    Visual decoding and reconstruction via EEG embeddings with guided diffusion,

    D. Li et.al., “Visual decoding and reconstruction via EEG embeddings with guided diffusion,” in Proc. 38th Int. Conf. Neural Inf. Process. Syst. (NeurIPS), Dec. 2024, pp. 1-35

    2024

  67. [75]

    DreamDiffusion: Generating high-quality images from brain EEG signals,

    Y . Bai et al. , “DreamDiffusion: Generating high-quality images from brain EEG signals,” 2023, arXiv:2306.16934

    2023 arXiv

  68. [76]

    Guess what I think: Streamlined EEG-to-image gen- eration with latent diffusion models,

    E. Lopez et al., “Guess what I think: Streamlined EEG-to-image gen- eration with latent diffusion models,” in Proc. IEEE Int. Conf. Acoust. Speech Signal Process. (ICASSP) , Apr. 2025, pp. 13-18

    2025

  69. [77]

    EEG decoding and visual reconstruction via 3D geometric with nonstationarity modelling,

    X. Xiao et al. , “EEG decoding and visual reconstruction via 3D geometric with nonstationarity modelling,” in Proc. IEEE Int. Conf. Acoust. Speech Signal Process. (ICASSP) , Apr. 2025, pp. 1-5

    2025

  70. [78]

    Seeing through the brain: Image reconstruction of visual perception from human brain signals,

    Y .-T. Lan et al. , “Seeing through the brain: Image reconstruction of visual perception from human brain signals,” 2023, arXiv:2308.02510

    2023 arXiv

  71. [79]

    BrainDecoder: Style-based visual decoding of EEG signals,

    M. Choi and H. Ishikawa, “BrainDecoder: Style-based visual decoding of EEG signals,” 2024, arXiv: 2409.05729

    2024 arXiv

  72. [80]

    BrainDreamer: Reasoning-coherent and controllable image generation from EEG brain signals via language guidance,

    L. Wang, C. Wu and L Wang, “BrainDreamer: Reasoning-coherent and controllable image generation from EEG brain signals via language guidance,” 2024, arXiv: 2409.14021

    2024 arXiv

  73. [81]

    BrainVis: Exploring the bridge between brain and visual signals via image reconstruction,

    H. Fu et al., “BrainVis: Exploring the bridge between brain and visual signals via image reconstruction,” in Proc. IEEE Int. Conf. Acoust. Speech Signal Process. (ICASSP) , Apr. 2025, pp. 4-5

    2025

  74. [82]

    Scaling laws for decoding images from brain activity,

    H. Banville et al. , “Scaling laws for decoding images from brain activity,” 2025, arXiv: 2501.15322

    2025 arXiv

  75. [83]

    Perceptogram: Reconstructing visual percepts from EEG,

    T. Fei et al., “Perceptogram: Reconstructing visual percepts from EEG,” 2024, arXiv: 2404.01250

    2024

  76. [84]

    EEG2Video: Towards decoding dynamic visual percep- tion from EEG signals,

    X. Liu et al., “EEG2Video: Towards decoding dynamic visual percep- tion from EEG signals,” in Proc. Int. Conf. Neural Inf. Process. Syst. (NeurIPS), Dec. 2024, pp. 72245-72273

    2024

  77. [85]

    EEG-driven 3D object reconstruction with style consistency and diffusion prior,

    X. Xiang, W. Zhou and G. Dai, “EEG-driven 3D object reconstruction with style consistency and diffusion prior,” 2024, arXiv:2410.20981

    2024 arXiv

  78. [86]

    Neuro-3D: Towards 3D visual decoding from EEG signals,

    Z. Guo et al. , “Neuro-3D: Towards 3D visual decoding from EEG signals,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2025, pp. 23820-23880

    2025

  79. [87]

    Mildenhall, et al

    B. Mildenhall, et al. , NeRF: Representing scenes as neural radiance fields for view synthesis,” Commun. ACM, vol. 65, no. 1, pp. 99–106, Jan. 2021

    2021

  80. [88]

    Contextual feature extraction hierarchies converge in large language models and the brain,

    G. Mischler et al., “Contextual feature extraction hierarchies converge in large language models and the brain,” Nat. Mach. Intell., vol. 6, no. 12, pp.1467-1477, Dec. 2024

    2024

  81. [89]

    A high-performance neuroprosthesis for speech decoding and avatar control,

    S. L. Metzger et al., “A high-performance neuroprosthesis for speech decoding and avatar control,” Nature, vol. 620, no. 7976, pp. 1037- 1046, Aug. 2023

    2023

  82. [90]

    Decoding of the speech envelope from EEG using the VLAAI deep neural network,

    B. Accou et al. , “Decoding of the speech envelope from EEG using the VLAAI deep neural network,” Sci. Rep., vol. 13, no. 1, pp. 1-12, Jan. 2023

    2023

  83. [91]

    Imagined speech reconstruction from neural signals—an overview of sources and methods,

    J. Tang et al., “Imagined speech reconstruction from neural signals—an overview of sources and methods,” IEEE Trans. Instrum. Meas. , vol. 73, pp. 1-21, Oct. 2024

    2024

  84. [92]

    Investigating self-supervised deep representations for EEG-based auditory attention decoding,

    K. Thakkar, J. Hia and M. Elhilali, “Investigating self-supervised deep representations for EEG-based auditory attention decoding,” in Proc. IEEE Int. Conf. Acoust. Speech Signal Process. (ICASSP) , Apr. 2024, pp. 1241-1245

    2024

  85. [93]

    Estimating musical surprisal in audio,

    M. R. Bjare et al. , “Estimating musical surprisal in audio,” in Proc. IEEE Int. Conf. Acoust. Speech Signal Process. (ICASSP) , Apr. 2025, pp. 1-5

    2025

  86. [94]

    Multimodal fusion for EEG emotion recognition in music with a multi-task learning framework,

    S. Huang et al. , “Multimodal fusion for EEG emotion recognition in music with a multi-task learning framework,” in Proc. IEEE Int. Conf. Acoust. Speech Signal Process. (ICASSP) , Apr. 2025, pp. 1-2

    2025

  87. [95]

    A review on EEG neural music system and application,

    Y . Ji et al., “A review on EEG neural music system and application,” in Proc. IEEE Int. Conf. Unmanned Syst. (ICUS), Oct. 2024, pp. 453-459

    2024

  88. [96]

    Generative AI and EEG-based music personali- zation for work stress reduction,

    V . Wijethuge et al. , “Generative AI and EEG-based music personali- zation for work stress reduction,” in Proc. Annu. Conf. IEEE Ind. Electron. Soc. (IECON) , Nov. 2024, pp.1-6

    2024

  89. [97]

    Simple and controllable music generation,

    J. Copet et al., “Simple and controllable music generation,” inProc. Int. Conf. Neural Inf. Process. Syst. (NIPS) , Dec. 2023, pp. 47704-47720

    2023

  90. [98]

    An investigation on the speech recovery from EEG signals using transformer,

    T. Mizuno et al., “An investigation on the speech recovery from EEG signals using transformer,” in Proc. Asia Pac. Signal Inf. Process. Assoc. Annu. Summit Conf. (APSIPA ASC) , Dec. 2024, pp. 1-6

    2024

  91. [99]

    Towards voice reconstruction from EEG imagined speech,

    Y .-E. Lee et al. , “Towards voice reconstruction from EEG imagined speech,” in Proc. AAAI Conf. Artif. Intell. (AAAI), Jun. 2023, pp. 6030- 6038

    2023

  92. [100]

    Towards EEG-based talking-face generation for brain signal-driven dynamic communication,

    J.-H. Park, S.-H. Lee and S.-W. Lee, “Towards EEG-based talking-face generation for brain signal-driven dynamic communication,” in Proc. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. (EMBC) , Jul. 2024, pp. 1-5

    2024

  93. [101]

    Towards speech synthesis of unconstrained sentences from speech-related biosignals,

    D.-S. Kim et al., “Towards speech synthesis of unconstrained sentences from speech-related biosignals,” in Proc. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. (EMBC) , Jul. 2024, pp. 52-57

    2024

  94. [102]

    Hubert: Self-supervised speech representation learn- ing by masked prediction of hidden units,

    W.-N. Hsu et al., “Hubert: Self-supervised speech representation learn- ing by masked prediction of hidden units,” IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 29, pp. 3451-3460, Oct. 2021

    2021

  95. [103]

    Music can be reconstructed from human auditory cortex activity using nonlinear decoding models,

    L. Bellier et al. , “Music can be reconstructed from human auditory cortex activity using nonlinear decoding models,” PLOS Biol., vol. 21, no. 8, pp. 1-27, Aug. 2023

    2023

  96. [104]

    Musical brainwaves: Decoding electroencephalog- raphy signals elicited by musical stimuli,

    C. Ankitha et al., “Musical brainwaves: Decoding electroencephalog- raphy signals elicited by musical stimuli,” in Proc. Int. Conf. Comput., Sci. Commun. (ICCSC) , Oct. 2024, pp. 1-7

    2024

  97. [105]

    Naturalistic music decoding from EEG data via latent diffusion models,

    E. Postolache et al., “Naturalistic music decoding from EEG data via latent diffusion models,” in Proc. IEEE Int. Conf. Acoust. Speech Signal Process. (ICASSP), Apr. 2025, pp. 1-5

    2025

  98. [106]

    Brain-conditional multimodal synthesis: A survey and taxonomy,

    W. Mai et al., “Brain-conditional multimodal synthesis: A survey and taxonomy,” IEEE Trans. Artif. Intell. , vol. 6, no. 5, pp. 1080-1099, May. 2025

    2025

  99. [107]

    VIT-LENS: Towards omni-modal representations,

    W. Lei et al. , “VIT-LENS: Towards omni-modal representations,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) , Jun. 2024, pp. 26637-26647

    2024

  100. [108]

    Imagebind: One embedding space to bind them all,

    R. Girdhar et al., “Imagebind: One embedding space to bind them all,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2023, pp. 15180-15190

    2023

  101. [109]

    CognitionCapturer: Decoding visual stimuli from human EEG signal with multimodal information,

    K. Zhang et al. , “CognitionCapturer: Decoding visual stimuli from human EEG signal with multimodal information,” in Proc. AAAI Conf. Artif. Intell. (AAAI) , Apr. 2025, pp. 14486-14493

    2025

  102. [110]

    Thought2Text: Text generation from EEG signal using large language models (LLMs),

    Mishra et al., “Thought2Text: Text generation from EEG signal using large language models (LLMs),” 2024, arXiv:2410. 07507

    2024

  103. [111]

    Milmer: A framework for multiple instance learning based multimodal emotion recognition,

    Z. Wang et al., “Milmer: A framework for multiple instance learning based multimodal emotion recognition,” 2025, arXiv:2502.00547. JOURNAL OF LATEX CLASS FILES, VOL. 18, NO. 9, SEPTEMBER 2020 15

    2025 arXiv

  104. [112]

    Revisiting your memory: Reconstruction of af-fect- contextualized memory via EEG guided audiovisual generation,

    J. Kwon et al. , “Revisiting your memory: Reconstruction of af-fect- contextualized memory via EEG guided audiovisual generation,” 2024, arXiv:2412.05296

    2024 arXiv

  105. [113]

    Towards dynamic neural communication and speech neuroprosthesis based on viseme decoding,

    J.-H. Park et al., “Towards dynamic neural communication and speech neuroprosthesis based on viseme decoding,” in Proc. IEEE Int. Conf. Acoust. Speech Signal Process. (ICASSP) , Apr. 2025, pp. 1-5

    2025

  106. [114]

    Neurobind: Towards unified multimodal representations for neural signals,

    F. Yang et al., “Neurobind: Towards unified multimodal representations for neural signals,” 2024, arXiv:2407.14020

    2024 arXiv

  107. [115]

    Towards neural foundation models for vision: Aligning EEG, MEG, and fMRI representations for decoding, encod- ing, and modality conversion,

    M. Ferrante et al. , “Towards neural foundation models for vision: Aligning EEG, MEG, and fMRI representations for decoding, encod- ing, and modality conversion,” 2024, arXiv: 2411.09723

    2024 arXiv

  108. [117]

    CineBrain: A large-scale multi-modal brain dataset during naturalistic audiovisual narrative processing,

    J. Gao et al. , “CineBrain: A large-scale multi-modal brain dataset during naturalistic audiovisual narrative processing,” 2025, arXiv:2503:06940

    2025

  109. [118]

    AnyGPT: Unified multimodal LLM with discrete sequence modeling,

    J. Zhan et al., “AnyGPT: Unified multimodal LLM with discrete sequence modeling,” in Proc. Annu. Meet. Assoc. Comput. Linguist. (ACL), Aug. 2024, pp. 9637-9662

    2024

  110. [119]

    NeuGPT: Unified multi-modal neural GPT,

    Y . Yang et al., “NeuGPT: Unified multi-modal neural GPT,” 2024, arXiv:2410.20916

    2024 arXiv

  111. [120]

    CATD: Unified representation learning for EEG-to- fMRI cross-modal generation,

    W. Yao et al., “CATD: Unified representation learning for EEG-to- fMRI cross-modal generation,” IEEE Trans. Med. Imag. , vol. 44, no. 7, pp. 2757-2767, Mar. 2025

    2025

  112. [121]

    Multi-modal cross-domain self-supervised pre-training for fMRI and EEG fusion,

    X. Wei et al., “Multi-modal cross-domain self-supervised pre-training for fMRI and EEG fusion,” Neural Networks, vol. 184, pp. 1-15, Apr. 2025

    2025

  113. [122]

    Adaptive modality balanced online knowledge distillation for brain-eye-computer based dim object detection,

    Z. Li et al., “Adaptive modality balanced online knowledge distillation for brain-eye-computer based dim object detection,” 2024, arXiv: 2407.01894

    2024

  114. [123]

    AdaBrain-bench: Benchmarking brain foundation models for brain-computer interface applications,

    J. Wu, et al., “AdaBrain-bench: Benchmarking brain foundation models for brain-computer interface applications,” 2025, arXiv: 2507.09882

    2025 arXiv

  115. [124]

    A novel multi-dynamic coupled neural mass model of SSVEP,

    H. Li, Y . Wang, and P. Fu, “A novel multi-dynamic coupled neural mass model of SSVEP,” Biomimetics, vol. 10, no. 3, pp. 171, 2025

    2025

  116. [125]

    EEG-based brain- computer interface enables real-time robotic hand control at indi-vidual finger level,

    Y . Ding, C. Udompanyawit, Y . Zhang, and B. He, “EEG-based brain- computer interface enables real-time robotic hand control at indi-vidual finger level,” Nat. Commun., 16, 5401, 2025

    2025

  117. [126]

    Building a digital twin framework for EEG neurofeedback in cricket batting motor imagery using simulated data,

    D. Pathak, R. Herrema, N. Heffer, and H. Yang, “Building a digital twin framework for EEG neurofeedback in cricket batting motor imagery using simulated data,” in Proc. 2024 IEEE Smart World Congr.,(SWC), 2024, pp. 2010-2017

    2024

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.