{"id":"7e49c9c4-958c-4b05-9ed3-3e54a901071b","arxiv_id":"2412.17337","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A multimodal contrastive learning framework with a spectral-temporal EEG encoder improves zero-shot visual decoding accuracy on THINGS-EEG.","lead":"Neural-MCRL pairs a new EEG encoder, NESTA, with text-anchored cross-attention to decode which image a person is viewing from their EEG. On THINGS-EEG it reports higher zero-shot accuracy than prior methods, but without error bars or released code.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"EITRA inference protocol is unspecified: if Eq. 15 mixes candidate test image embeddings into the EEG embedding at test time, the zero-shot comparison is not EEG-only and the SOTA claim may be inflated.","rationale":"The reader's weakest assumption was that the baselines were reproduced under identical conditions and that the seen/unseen class split was enforced with no test-class EEG leakage. I agree that this is a real concern, but the more load-bearing issue is the unspecified inference-time behavior of EITRA. The architecture as written (Eqs. 13-15) makes the final EEG embedding a function of both EEG and image features. If the image branch is populated with candidate test images at inference, the comparison against fixed-encoder baselines is not apples-to-apples; the model could be using the test gallery to condition the query representation, which would inflate accuracy and undermine the central SOTA claim. This is not an ad hominem or a disagreement with consensus; it is a concrete, checkable protocol ambiguity. The paper's ablation study (Table II) shows each component contributes, but it does not reveal the inference-time input to EITRA, and the text only says performance was evaluated on the zero-shot test set (Sec. IV-A). Because the issue can be resolved by inspecting code or running a simple controlled re-evaluation, a conditional verdict is appropriate: acceptance should require the authors to state and verify the inference protocol, specifically that A_I does not contain the candidate test image during EEG embedding computation, or to re-benchmark against baselines under the same image-conditioned protocol. The Table I arithmetic anomalies strengthen the need for code release and careful rechecking but are secondary to the architectural circularity concern.","tokens_in":9732,"tokens_out":10546,"duration_ms":108600,"concrete_test":"Inspect the released code (or ask the authors for the exact inference procedure) and determine whether A_I in Eqs. 13-15 is fed the candidate test image embeddings during zero-shot evaluation. Then rerun the 50-way and 200-way subject-dependent evaluations with A_I replaced by a fixed learned token or zero vector while keeping all trained weights fixed; if top-1 accuracy drops by more than approximately 2-3 points, the reported numbers depend on test-time image conditioning and the 'EEG decoding' claim must be revised. Separately, recompute all Table I averages and correct the impossible NICE-GA subject-5 row before drawing any SOTA conclusion.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's central claim is that Neural-MCRL outperforms fixed-EEG-encoder baselines (BraVL, NICE, ATMS, MB2C) in zero-shot visual decoding. The EITRA module (Sec. III-E) computes the final EEG embedding as ZE = ResNet(softmax(Q_C K_A^T / sqrt(d)) V_A), where K_A and V_A are built from the concatenation of semantically enriched EEG features A_E and image features A_I. The training loss (Eq. 16) then aligns this image-mixed ZE directly with ZI. The paper never specifies what A_I contains at inference time. If A_I contains the candidate test image embedding, then each test EEG is scored against an embedding that was itself conditioned on that candidate image. This is a fundamentally different and much easier task than the fixed-encoder zero-shot protocol used for the baselines, so the Table I comparison would not isolate EEG decoding ability. This is a load-bearing ambiguity independent of the baseline-reproduction issue: it concerns whether the method as described even implements the same retrieval protocol as the baselines. Supporting signs of reporting inconsistency exist in Table I: the NICE-GA row has an impossible subject-5 200-way entry (top-1 48.90 higher than top-5 29.70), and some Neural-MCRL 200-way averages do not match the per-subject sums (reported 32.25 vs computed 32.35 for top-1; reported 64.15 vs computed 64.05 for top-5). These do not by themselves disprove the method, but they reduce confidence that Table I is a clean, protocol-controlled comparison.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes Neural-MCRL, a multimodal contrastive learning framework for zero-shot visual decoding from EEG on the THINGS-EEG dataset. It combines a subject-specific spectral-temporal EEG encoder (NESTA) with semantic enhancement via class-wise text prototypes and noise-based augmentation, and an EEG-Image-Text Representation Alignment (EITRA) module that uses textual anchors and cross-modal attention. The authors report state-of-the-art accuracy on 50-way and 200-way zero-shot classification for both subject-dependent and subject-independent settings, and component ablations in Table II support the contribution of each module.","tokens_in":10073,"tokens_out":4905,"duration_ms":45638,"significance":"If the reported results are reproducible, the framework advances EEG-based visual decoding by integrating spectral attention, subject-specific adaptation, and semantic bridging in a principled way. The per-subject breakdowns, multiple baselines, and ablations are valuable. However, the absence of statistical validation and a protocol ambiguity in EITRA currently prevent full confidence in the headline claims.","major_comments":[{"comment":"The paper never states what A_I contains at inference time. In Eq. (15), the final EEG embedding ZE is computed from K_A and V_A, which concatenate semantically enriched image features A_I with EEG features A_E. If at test time A_I is the candidate test image embedding, then the EEG embedding is conditioned on the very image it is later scored against; the retrieval task is then not EEG-only, and the comparison with fixed-encoder baselines in Table I is not protocol-equivalent. If, instead, A_I is restricted to training-class image features or prototypes, that restriction must be stated and enforced in the implementation. Please specify the exact inference-time construction of A_I and, if needed, rerun the evaluation under the standard zero-shot protocol.","section":"III-E, Eqs. (14)-(15)"},{"comment":"Table I contains a physically impossible entry: in the 200-way subject-dependent block, NICE-GA subject 5 has top-1 = 48.90% and top-5 = 29.70%, violating the necessary inequality top-5 >= top-1. In the same table, the reported Neural-MCRL 200-way averages do not match the per-subject rows (computed top-1 = 32.35% vs reported 32.25%; computed top-5 = 64.05% vs reported 64.15%). The central comparison table must be corrected and re-audited.","section":"Table I"},{"comment":"All reported accuracies are single runs without error bars, confidence intervals, or significance tests. The abstract and conclusion call the improvements \"significant,\" but Section IV provides no statistical support. Given the high variance typical of EEG decoding, multi-seed runs or at least paired permutation tests are needed to establish that the observed gaps over the baselines are not noise.","section":"IV-B"},{"comment":"The statement that \"all models were configured with identical hyperparameters\" is not backed by any implementation details for the baselines. It is unclear whether BraVL, NICE, ATMS, and MB2C numbers were reproduced in this codebase or taken from their original papers. To make the state-of-the-art claim credible, the authors should release the evaluation code and training configuration, or at least report the exact hyperparameters and preprocessing used for each baseline.","section":"IV-A"}],"minor_comments":[{"comment":"The variable X_b in Eq. (8) is not defined; it presumably denotes the band-specific frequency components after masking (Gamma_b), but this should be stated explicitly.","section":"III-B, Eq. (8)"},{"comment":"The semantic generator is described as \"OPT-2.7b\" but the cited reference [18] is BLIP-2, not OPT; please correct the citation or the model name.","section":"III-D"},{"comment":"The notation P_b^{(f)} and P_b^{(c)} is confusing; define clearly that the superscript indicates aggregation over frequency or channel axes, and specify the dimensions of P_b before using it in the attention formulas.","section":"III-B, Eqs. (6)-(7)"},{"comment":"The GitHub URL in the abstract contains a space (\"NZW ANG\"); correct the typo.","section":"Abstract"},{"comment":"The symbol \"ResNet\" in Eq. (15) is not introduced; if it refers to a residual network or a residual connection, please define it and provide its architecture or dimensions.","section":"III-E, Eq. (15)"}],"recommendation":"major_revision","confidential_remarks":"The manuscript header states it was accepted by ICME 2025. If this is the same work being submitted to a journal, the prior conference publication and any dual-submission issues should be checked with the venue policy."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, the punchline: Neural-MCRL is a well-built combination of known pieces — subject-specific linear layers, FFT band attention, prototype text aggregation, noise augmentation, cross-attention — and it reports consistent gains over BraVL, NICE, ATMS, and MB2C on THINGS-EEG. The ablations in Table II show each module earns its keep. If the reported protocol is what the authors intend, this is a solid incremental advance.\n\nThe soft spot is real and load-bearing. Equation (15) computes the final EEG embedding using K_A and V_A built from [A_E; A_I]. The paper never states what A_I contains at inference time. If A_I is the CLIP embedding of the candidate test image, then the test EEG is scored against an embedding that was constructed using that same image. That is not the same zero-shot retrieval task the baselines face, and the Table I comparison would not measure EEG decoding ability. The stress-test note is exactly right: this is the first thing a referee should ask. It is not an obscure detail; it defines the task.\n\nTable I also has a typo that undermines confidence: the NICE-GA subject 5 row reports top-1 48.90 and top-5 29.70, which is impossible. And the Neural-MCRL 200-way averages don't match the per-subject sums. These could be transcription errors, but they signal sloppy table construction. There are no error bars or multi-seed runs, so the 3-4 point gains over MB2C could be noise. The paper claims code will be released, but the repo is not available at the time of reading, so the experiments are not independently checkable right now.\n\nOn the positive side, the method itself is coherent. The NESTA encoder is a reasonable EEG feature extractor, and the semantic enhancement via text prototypes and Gaussian noise is well-motivated. The cross-subject results are credible. If the EITRA inference protocol turns out to be EEG-only (e.g., A_I comes from the same image's text or is computed once per class from prototypes), then the method likely stands as a useful contribution.\n\nWho is this for? Researchers working on EEG-based zero-shot visual decoding and multimodal contrastive learning for brain signals. It deserves a serious referee because the core ambiguity is resolvable and the experiments are extensive. My recommendation: send it to review, but ask the authors to specify the exact inference-time inputs to EITRA, fix Table I, and release code or provide variance estimates. If the protocol is not EEG-only, the SOTA claim should be withdrawn.","headline":"A competent combination of known ingredients with consistent gains on THINGS-EEG, but the EITRA inference protocol is ambiguous and could invalidate the zero-shot comparison.","tokens_in":10632,"tokens_out":2475,"would_cite":false,"duration_ms":23700,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Neural-MCRL claims state-of-the-art zero-shot visual decoding from EEG by aligning brain signals with image and text embeddings through semantic-guided attention and a spectral-temporal encoder.","keywords":["EEG-based visual decoding","zero-shot learning","multimodal contrastive representation learning","semantic alignment","cross-attention","spectral-temporal EEG encoder","THINGS-EEG dataset","brain-computer interface"],"falsifier":"Reproduce the 50-way and 200-way subject-dependent zero-shot splits on THINGS-EEG, retraining BraVL, NICE, ATMS, and MB2C with exactly the hyperparameters stated in Section IV-A; if any baseline's average top-1 accuracy matches or exceeds Neural-MCRL's reported 52.55% or 32.25%, the central claim is falsified. Reporting variance across several seen/unseen class splits would also test whether the single reported partition drives the headline numbers.","tokens_in":9533,"feed_emoji":"🧠","tokens_out":12417,"duration_ms":94768,"temperature":0.7,"pith_summary":"The paper tries to establish that EEG-based visual decoding—identifying which image a person is viewing from their brain activity—can be made substantially more accurate by treating it as a multimodal alignment problem rather than simple pairwise matching. It proposes Neural-MCRL, which maps EEG, image, and text into a shared semantic space using a purpose-built EEG encoder and semantic-guided cross-attention. The framework reports state-of-the-art zero-shot accuracy on THINGS-EEG, including 52.55% top-1 in 50-way subject-dependent classification, and shows gains in cross-subject generalization. If the results hold, the approach would strengthen non-invasive brain-computer interfaces and neural rehabilitation applications that require recognizing novel visual content.","feed_headline":"Reach 52.55% top-1 in zero-shot EEG visual decoding","feed_subtitle":"Aligns EEG with image and text embeddings, beating prior zero-shot visual decoders on THINGS-EEG.","key_machinery":"The framework's load-bearing components are: (1) NESTA, an EEG encoder with a subject-specific linear layer (learnable per-subject channel mixing), an iTransformer block for temporal-spatial dynamics, and a Neural-Spectral Adaptation Block that applies a fast Fourier transform to each channel, decomposes the power spectrum into $\\delta$, $\\theta$, $\\alpha$, $\\beta$, and $\\gamma$ bands, applies channel and spectral attention, and recombines via inverse FFT with a residual connection; (2) semantic enhancement, which aggregates K CLIP text descriptions per class into prototype anchors and adds Gaussian noise to $\\ell^2$-normalized EEG embeddings to enforce intra-modal semantic consistency; and (3) EITRA, which uses learnable interaction matrices to query textual prototypes and produces semantically enriched EEG and image features, then aligns them with a cross-modal attention layer under a contrastive loss balancing EEG-image and EEG-text alignment. The text stream acts as a semantic bridge that anchors both modalities in the same conceptual space.","core_discovery":"The paper's central claim is that EEG responses to visual stimuli can be decoded into the correct object category without ever training on that category, provided the EEG encoder and the alignment objective preserve semantic structure. On THINGS-EEG, the proposed Neural-MCRL framework reports 52.55% top-1 and 84.70% top-5 accuracy in 50-way subject-dependent zero-shot classification, and 32.25% top-1 and 64.15% top-5 in 200-way, surpassing the reported BraVL, NICE, ATMS, and MB2C baselines. The authors attribute these gains to three components: NESTA, an EEG encoder that learns per-subject channel transforms and adaptively reweights five canonical EEG frequency bands; a semantic enhancement step that builds class-level text prototypes and adds noise-based augmentation to EEG embeddings; and EITRA, a cross-attention module that uses textual semantics as an anchor to align EEG and image features. The same components also improve subject-independent generalization, where the model is tested on a subject excluded from training.","pith_inferences":["The paper's ablations remove one component at a time, but they do not test combinations (e.g., NESTA alone without EITRA, or EITRA with a simpler encoder); running those combinations would show whether the components are complementary or partly redundant.","Because the method depends on CLIP text embeddings and generated descriptions, its accuracy is likely bounded by the semantic coverage of the language model; applying the framework to other EEG datasets or stimulus domains may require regenerating descriptions to preserve the semantic anchors.","The subject-specific layer (a learnable per-subject channel mixing matrix) suggests a natural few-shot calibration extension: adapt a pretrained model to a new user by fine-tuning only that matrix on a handful of trials; this is testable and would matter for practical brain-computer interfaces.","The noise-based semantic completion on the unit hypersphere is a generic low-SNR remedy that could transfer to other neural decoding modalities such as fMRI or MEG, where embeddings face similar information degradation."],"forward_implications":["Zero-shot EEG decoding of novel visual categories becomes practical at the reported accuracy levels (52.55% top-1 / 84.70% top-5 in 50-way subject-dependent settings).","Subject-independent decoding, where a held-out subject's EEG is classified using models trained on others, improves over the compared baselines (27.90%/64.80% top-1/top-5 in 50-way), indicating better cross-subject generalization.","The semantic similarity structure of the learned EEG embeddings reflects the category structure of the stimuli, as shown by the cosine-similarity matrix clustering into six semantic groups.","The NESTA encoder outperforms alternative EEG encoders (ATMS, TSConv, EEGNetV4, MLP) when plugged into the same framework across 5-way, 10-way, and 200-way tasks, suggesting it is a reusable component."],"supporting_citations":[{"why":"provides the THINGS-EEG visual EEG dataset and the train/test category split used in all experiments.","marker":"[11]"},{"why":"supplies the pre-trained CLIP image and text embeddings that define the shared semantic space.","marker":"[7]"},{"why":"BraVL, the trimodal baseline whose zero-shot decoding numbers Neural-MCRL must beat.","marker":"[1]"},{"why":"NICE, the baseline providing the TSConv EEG encoder and comparison accuracies.","marker":"[6]"},{"why":"ATMS, a comparison method whose EEG embeddings and scores appear in Table I.","marker":"[9]"},{"why":"MB2C, a baseline using cycle consistency for visual-neural representation, compared in Tables I and II.","marker":"[12]"},{"why":"iTransformer, the transformer block inside NESTA that captures temporal and spatial EEG dynamics.","marker":"[16]"},{"why":"referenced as the source of generated text descriptions used to build class-wise semantic anchors.","marker":"[18]"}],"fun_headline_variants":["Neural-MCRL hits 52.55% top-1 zero-shot EEG decoding","52.55% top-1: Neural-MCRL decodes unseen categories","Zero-shot EEG visual decoding achieves 52.55% top-1","Align EEG with text and image for 52.55% zero-shot","Neural-MCRL: 52.55% top-1 on THINGS-EEG zero-shot"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported gains assume that all baseline models were retrained under identical hyperparameters and that no test-class EEG trials entered training, so the comparison is apples-to-apples.","fun_headline_variants_meta":{"raw":{"variants":["Neural-MCRL hits 52.55% top-1 zero-shot EEG decoding","52.55% top-1: Neural-MCRL decodes unseen categories","Zero-shot EEG visual decoding achieves 52.55% top-1","Align EEG with text and image for 52.55% zero-shot","Neural-MCRL: 52.55% top-1 on THINGS-EEG zero-shot"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000841,"raw_usage":{"total_tokens":3665,"prompt_tokens":949,"completion_tokens":2716,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":565,"completion_tokens_details":{"reasoning_tokens":2606}},"tokens_in":565,"tokens_out":2716,"duration_ms":19297,"temperature":1.0,"reasoning_tokens":2606,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T05:33:45.729328+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Reproduce the 50-way and 200-way subject-dependent zero-shot splits on THINGS-EEG, retraining BraVL, NICE, ATMS, and MB2C with exactly the hyperparameters stated in Section IV-A; if any baseline's average top-1 accuracy matches or exceeds Neural-MCRL's reported 52.55% or 32.25%, the central claim is falsified. Reporting variance across several seen/unseen class splits would also test whether the single reported partition drives the headline numbers.","supporting_citations":[{"cited_title":"A large and rich EEG dataset for modeling human visual object recognition,","cited_arxiv_id":null,"evidence_quote":"provides the THINGS-EEG visual EEG dataset and the train/test category split used in all experiments."},{"cited_title":"Learning transferable visual models from natural language supervision,","cited_arxiv_id":null,"evidence_quote":"supplies the pre-trained CLIP image and text embeddings that define the shared semantic space."},{"cited_title":"Decoding visual neural representations by multimodal learning of brain-visual- linguistic features,","cited_arxiv_id":null,"evidence_quote":"BraVL, the trimodal baseline whose zero-shot decoding numbers Neural-MCRL must beat."},{"cited_title":"MB2C: Multimodal bidirectional cycle consistency for learning robust visual neural repre- sentations,","cited_arxiv_id":null,"evidence_quote":"MB2C, a baseline using cycle consistency for visual-neural representation, compared in Tables I and II."}],"review_version":1}