{"id":"9acdc6a3-db8a-4e24-a2e8-a2d06e2fdd8a","arxiv_id":"2411.17475","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A continual learning architecture with a frozen shared brain encoder and per-subject prompt modules improves fMRI-to-image reconstruction and avoids catastrophic forgetting.","lead":"COBRA is a brain-decoding system that lets an fMRI-to-image model learn from new people without forgetting earlier people, by keeping a shared brain-pattern module frozen and adding small per-person modules. It reports higher image-reconstruction and captioning scores than current baselines on the Natural Scenes Dataset.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reconstruction SOTA claim is not attributable to COBRA as written: Section 3.5 stops at CLIP-space alignment and no diffusion prior or image decoder is ever specified, yet Tables 2 and 14 and Figures 4–5 report pixel/image metrics.","rationale":"The continual-learning design is internally coherent: freezing prior modules and training new subject-specific PSS/MRIFormer modules does prevent forgetting by construction. The reader's weakest-assumption concern about the 2D fsaverage flattening is real and worth testing, since the model consumes only 224x224 2D patches and the paper gives no proof of bijectivity or geodesic preservation. However, that concern does not by itself undermine the empirical reconstruction numbers. The more decisive gap is that the reconstruction pipeline is never described: the method ends at CLIP-space alignment, yet the paper reports image-level metrics and qualitative reconstructions. Without knowing how images are generated, the SOTA reconstruction claim cannot be attributed to COBRA, and comparisons in Table 14 may conflate capabilities of an external generative model with COBRA itself. A conditional verdict requiring specification and controlled re-evaluation of the decoder is therefore appropriate, rather than accepting the reconstruction claim as stated.","tokens_in":28773,"tokens_out":7571,"duration_ms":78386,"concrete_test":"Require the authors to specify the exact image-generation pipeline: identify the diffusion prior, VAE/LDM decoder, or other generator that maps the output of Eq. 8 to pixels, and provide weights or code. Then re-run Table 14 end-to-end with a fixed decoder and check whether COBRA's SSIM = 0.375 and CLIP = 98.1 reproduce; also run the same decoder on MindEye2 and Psychometry features for a controlled comparison. If no decoder exists, the reconstruction claim is unsubstantiated.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's central claim is that COBRA achieves state-of-the-art vision-brain reconstruction. However, the method section only trains a contrastive alignment between fMRI-derived tokens (Eq. 8) and CLIP image embeddings (Eq. 9). There is no description of how the fmri feature is converted back into a reconstructed image: no diffusion prior, no latent decoder, no generator, and no image-space loss. Figures 4–5 and Tables 2/14 present SSIM, PixCorr, and CLIP scores, which require actual generated images. If those images are produced by an external pretrained decoder, that component must be identified and controlled for comparison; otherwise the reconstruction result is not COBRA's output. This is the most load-bearing concern because it directly governs the claimed SOTA in reconstruction. The reader's 2D-flattening concern is also legitimate: Section 3.2 asserts a 'bijective projection' with 'no information loss' and geodesic-distance preservation without proof, and flattening a curved cortical surface to a plane is not generally isometric. But even if the 2D representation is lossy, the absent image decoder still leaves the reconstruction claim untestable, which is why I focus on it.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces COBRA, a continual learning framework for vision-brain understanding (VBU) that processes fMRI signals projected onto a 2D fsaverage template. COBRA combines a frozen shared Subject Commonality (SC) module, a Prompt-based Subject Specific (PSS) module that selects top-k tokens per subject, and an MRIFormer transformer encoder-decoder that aligns fMRI features with CLIP image embeddings via a contrastive loss. In the continual setup, new subjects are accommodated by training new PSS and MRIFormer modules, leaving previous subjects' modules untouched. Experiments on the Natural Scenes Dataset are reported for two continual learning protocols, covering reconstruction metrics such as SSIM and CLIP score, captioning metrics across ten standard measures, ablations on prompt length and loss weights, and comparisons with prior VBU and continual learning methods.","tokens_in":29063,"tokens_out":5390,"duration_ms":48460,"significance":"If the empirical results hold, the modular architecture is a plausible and practical way to add new fMRI subjects without storing past data, and the explicit evaluation of subject-commonality generalization (F1=0.84 on unseen subjects) and subject-specific prompt selection (98.5% accuracy) are useful contributions. The paper includes extensive experiments with error bars, multiple continual learning scenarios, and a fairly complete set of ablations. The main limitation is that the reconstruction pipeline is not described, which makes the central reconstruction SOTA claim unverifiable as written.","major_comments":[{"comment":"The reconstruction results are not attributable to COBRA as written. The method section stops at producing fMRI-aligned CLIP features via the contrastive loss in Eq. (9), and no diffusion prior, image decoder, or generator is specified anywhere in the manuscript. Yet Tables 2 and 14 and Figures 4-5 report pixel-level and image-level metrics (SSIM, PixCorr, AlexNet/Inception/CLIP similarities) and show reconstructed images. The paper must specify the full image decoding pipeline, state whether any pretrained decoder is used, and control for that decoder in comparisons with prior methods. Without this, the reconstruction SOTA claim is untestable.","section":"Section 3.5, Tables 2 and 14, Figures 4-5"},{"comment":"The claim that the 3D-to-2D fsaverage flattening is a bijective projection with 'no information loss' and that it 'preserves geodesic distance and spatial relationships' is unsupported and, for a folded cortical surface, implausible as stated. A plane cannot be globally isometric to a curved surface; flattening necessarily introduces distortions or cuts unless the surface is developable. Appendix 7.2 only illustrates that 1D contiguous samples become scattered in 2D; it does not prove information preservation. Since all modules consume 224x224 2D patches, this assertion is load-bearing. Please revise to describe the specific projection used, cite authoritative references for its properties, and replace 'no information loss' with a qualified statement about the approximation.","section":"Section 3.2 and Appendix 7.2"},{"comment":"The 'no catastrophic forgetting' result is largely by construction under parameter isolation: previous PSS and MRIFormer modules are frozen, so old subjects' features cannot be overwritten. The continual learning comparison against LwF and PLOP, which update a shared network, is therefore not an apples-to-apples comparison. To support the claim of state-of-the-art continual learning performance, the authors should compare COBRA with architecture-based continual learning methods (e.g., progressive neural networks, PackNet, or prompt-based CL baselines) or clearly qualify that the comparison is limited to rehearsal-free regularization/distillation baselines.","section":"Section 3.6 and Tables 2-8"},{"comment":"It is unclear whether the SC module is frozen after the first training step or continues to be updated when new subjects are added. The text says 'we do not retrain or update the entire model' and only new PSS and MRIFormer modules are created, yet Algorithm 1 includes the SC loss Lc in the total loss at every step, and feeding new subjects' data through the SC module could involve gradient updates. Please specify the exact update schedule for the SC module across training steps and clarify how the commonality loss is used after the initial stage.","section":"Section 3.6 and Algorithm 1"}],"minor_comments":[{"comment":"The loss in Eq. (11) is written as λsc Lcon, but Algorithm 1 line 18 uses 'LsoftCLIP' for the same term; please unify the notation.","section":"Eq. (11) vs Algorithm 1"},{"comment":"The statement that the PSS module training is 'label-free' is misleading because the subject identity is used as a supervised label; this is a self-supervised pretext task, not label-free learning.","section":"Section 3.4"},{"comment":"The results in Table 14 are reported without error bars or the number of repeat evaluations, unlike the continual learning tables; please add error bars or state the evaluation protocol.","section":"Table 14"},{"comment":"The ablation labeled 'Effectiveness of the vision-brain data representation' varies both the input signal (partial ROI vs full signals) and the fMRI-to-CLIP transformation (linear vs MRIFormer) simultaneously, so the individual contributions of these two factors cannot be separated; please present a two-factor ablation.","section":"Section 4.7 and Table 17"},{"comment":"There is a typo: 'floc−face and floc−face' should likely be 'floc−face and floc−body'.","section":"Section 7.1"}],"recommendation":"major_revision","confidential_remarks":"The most serious issue is the missing reconstruction decoder, which makes the headline reconstruction SOTA claim untestable; this must be resolved before the paper can be evaluated on its merits. The flattening claim and the CL comparison fairness are also important but more easily fixable. The paper otherwise has a solid experimental structure and useful auxiliary evaluations."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: COBRA is a new and reasonably well-engineered combination of parameter-isolation continual learning and fMRI-to-CLIP alignment, with thorough experiments on NSD. But as written, the reconstruction claims are not reproducible because the method section stops at CLIP-space alignment and never says how the CLIP embedding becomes an image.\n\nThe core idea is sensible: freeze a commonality transformer trained across subjects to predict COCO object labels, add per-subject prompt selection, and use a transformer decoder to map into CLIP space. The paper deserves credit for real evidence: the SC module transfers to unseen subjects with F1=0.84, the ablations on prompt length and the regularization loss are informative, and the model-size comparison is useful. The continual-learning setup is honest in that old modules are frozen, so the absence of forgetting is by construction, but that is a legitimate design choice.\n\nThree soft spots, in increasing order of severity. First, the captioning SOTA is slightly overstated: Table 15 shows SPICE below UMBRAE (12.6 vs 12.79), and Table 14 has them second on SSIM. The abstract's \"state-of-the-art\" should be qualified. Second, the 2D flattening claim is unproven. Section 3.2 asserts a bijective, information-preserving projection that maintains geodesic distance, but the appendix only illustrates 1D vs 2D sparsity; it does not demonstrate isometry. Flattening a curved cortical surface to a 224x224 grid is not generally lossless, and the claim should be downgraded to an approximation. Third, and most load-bearing: no image decoder is ever described. Section 3.5 and Algorithm 1 output a CLIP-space feature, yet Tables 2 and 14 report SSIM and PixCorr. Unless the authors identify the pretrained diffusion prior or generator used to convert the CLIP embedding into pixels, and control for it across baselines, the reconstruction numbers are not attributable to COBRA. This is a reproducibility gap, not necessarily a fatal flaw in the architecture.\n\nWho this is for: researchers in cross-subject fMRI decoding and continual learning will find the modular design worth studying. I would send this to peer review, but with a required major revision: specify the full reconstruction pipeline, provide code or detailed hyperparameters, and soften the claims to match the tables. The framework is new and the experiments are extensive; it deserves a serious referee.\n\nRecommendation: engage with it, but insist on clarification of the image decoder before citing the reconstruction results.","headline":"A genuinely new modular continual-learning framework for fMRI-to-CLIP alignment with strong NSD experiments, but the reconstruction claim is untestable as written because no image decoder is specified.","tokens_in":29558,"tokens_out":2070,"would_cite":false,"duration_ms":21163,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"COBRA claims that continually adding new fMRI subjects to a vision-brain decoder does not have to erase knowledge of previous subjects, and that a modular design achieves state-of-the-art reconstruction and captioning without storing…","keywords":["continual learning","vision-brain understanding","fMRI decoding","image reconstruction from brain activity","brain captioning","prompt-based learning","catastrophic forgetting","Natural Scenes Dataset"],"falsifier":"Compute the round-trip error of the flattening: take a subject's 3D fMRI volume, project it to the 2D fsaverage grid, invert the projection, and compare voxel by voxel with the original. If the round trip is not identity, the paper's claim of a bijective, no-information-loss projection is false; a complementary check is to train COBRA directly on 3D patch inputs and see whether the reported SSIM and CLIP margins persist.","tokens_in":28603,"feed_emoji":"🧠","tokens_out":7459,"duration_ms":65317,"temperature":0.7,"pith_summary":"The paper introduces COBRA, a continual-learning framework for vision-brain understanding: reconstructing or captioning what a person saw from their fMRI activity when new participants are added over time. Existing per-subject decoders ignore patterns shared across people, while unified decoders suffer catastrophic forgetting when adapted to new subjects. COBRA separates the problem into a frozen shared module that captures brain-activation patterns common to all subjects and per-subject prompt and transformer modules that are trained only for the new person. The paper argues that this removes the need to retain prior subjects' data, prevents forgetting, and gives state-of-the-art image reconstruction and captioning results on the Natural Scenes Dataset.","feed_headline":"Adding fMRI subjects without erasing past ones","feed_subtitle":"A frozen shared brain module plus per-subject prompts keeps decoding quality high as new participants arrive.","key_machinery":"The key machinery is the three-module split with a frozen common encoder. The SC module maps 2D-flattened fMRI patches into a CLS token plus patch tokens and predicts object presence; the PSS module treats subject identity as a prompt by selecting the 30 tokens most similar to the CLS token; the MRIFormer concatenates common and specific tokens and uses a transformer decoder, rather than a linear layer, to translate them into CLIP feature length. The load-bearing design choice is that when a new subject arrives only the new PSS and MRIFormer modules are updated, so no gradient step can overwrite the knowledge stored in the shared module.","core_discovery":"COBRA's central claim is that continual vision-brain understanding can be solved by decomposing fMRI decoding into a commonality branch and a specificity branch. A Subject Commonality module, a vision transformer supervised by binary cross-entropy over 80 COCO object classes, learns shared visuo-brain patterns from the subjects seen so far and is then frozen. For each new subject, a Prompt-based Subject Specific module selects the 30 most relevant tokens using a key-query mechanism, and the MRIFormer encoder-decoder translates the combined common and specific features into CLIP space, where contrastive loss aligns them with the visual stimulus. Because only the new subject's PSS and MRIFormer parameters are trained, earlier modules are untouched, so knowledge from prior subjects is retained without rehearsal. The paper reports that this design reaches SSIM around 0.32-0.33 and CLIP scores around 92-97% in two rehearsal-free continual setups, outperforming LwF and PLOP, and also reports state-of-the-art reconstruction and captioning results when trained on all subjects.","pith_inferences":["Beyond the paper: the reported advantage could partly come from the 2D cortical-flattening representation itself rather than from the continual-learning modules; training the same modules on 3D patch inputs would isolate that contribution.","Beyond the paper: the SC module's object-prediction supervision is tied to 80 COCO classes, so on datasets with different stimulus categories the shared representation may need re-supervision, a limitation the paper only partially acknowledges.","Beyond the paper: the prompt-selection mechanism resembles retrieval-based continual learning, and a natural test is whether it generalizes to subjects with atypical cortical anatomy, where the fsaverage template fits poorly.","Beyond the paper: since the paper shows COBRA can be combined with PLOP for further gains, a follow-up could probe whether rehearsal-free performance saturates at some number of subjects and whether synthetic prompt replay would push it further."],"forward_implications":["New fMRI subjects could be added to a decoder without access to previous participants' recordings, which matters when brain data are too privacy-sensitive to store or share.","Per-subject training cost drops to training a small prompt module and transformer for roughly two hours per subject, instead of retraining a full model on all subjects.","If the shared module genuinely captures common brain patterns, performance should improve as more subjects contribute to it, rather than degrade with each new arrival.","Because MRIFormer maps into a shared CLIP space, the same continual scheme could extend to other decoding targets such as captions, speech, or visual-semantic queries.","The reported gains over prior continual baselines suggest that catastrophic forgetting in fMRI decoding is not inevitable and can be engineered away by modularity."],"supporting_citations":[{"why":"Supplies the Natural Scenes Dataset of fMRI-image pairs and the 80 COCO classes used for all experiments.","marker":"[74]"},{"why":"Provides the FreeSurfer fsaverage surface and flattening that the 2D input representation is built on.","marker":"[73]"},{"why":"MindBridge is the main unified-model baseline whose catastrophic forgetting and model growth COBRA targets.","marker":"[1]"},{"why":"MindEye2 is the shared-subject baseline used in reconstruction comparisons and model-size analysis.","marker":"[8]"},{"why":"Learning to Prompt for Continual Learning is the prompt-based continual learning method that motivates the PSS module.","marker":"[77]"},{"why":"Co2L is the rehearsal-based continual learning baseline that COBRA is compared against with buffers of 500, 1500, and 4000 samples.","marker":"[41]"},{"why":"PLOP is a continual learning baseline that COBRA outperforms and also combines with for higher scores.","marker":"[81]"},{"why":"LwF is the learning-without-forgetting baseline used in both rehearsal-free comparisons.","marker":"[82]"},{"why":"UMBRAE is the prior state-of-the-art brain captioning model whose protocol and adapter approach COBRA follows.","marker":"[26]"},{"why":"Shikra is the multimodal LLM that generates captions from the adapted fMRI features in the captioning task.","marker":"[83]"}],"fun_headline_variants":["COBRA splits common and specific brain signals to dodge forgetting","Vision-brain model adds subjects without erasing prior knowledge","Continual learning for fMRI decoding: no catastrophic forgetting","Frozen common module plus per-subject prompts beats forgetting","New subjects, old memories: COBRA's modular continual learning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole pipeline rests on the assumption that flattening the brain's curved cortical surface into a 2D $224\\times224$ image loses no information and preserves spatial relationships; if that projection distorts or drops data, every module downstream is working on an altered signal.","fun_headline_variants_meta":{"raw":{"variants":["COBRA splits common and specific brain signals to dodge forgetting","Vision-brain model adds subjects without erasing prior knowledge","Continual learning for fMRI decoding: no catastrophic forgetting","Frozen common module plus per-subject prompts beats forgetting","New subjects, old memories: COBRA's modular continual learning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000292,"raw_usage":{"total_tokens":1731,"prompt_tokens":998,"completion_tokens":733,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":614,"completion_tokens_details":{"reasoning_tokens":652}},"tokens_in":614,"tokens_out":733,"duration_ms":7128,"temperature":1.0,"reasoning_tokens":652,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T12:49:13.266086+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the round-trip error of the flattening: take a subject's 3D fMRI volume, project it to the 2D fsaverage grid, invert the projection, and compare voxel by voxel with the original. If the round trip is not identity, the paper's claim of a bijective, no-information-loss projection is false; a complementary check is to train COBRA directly on 3D patch inputs and see whether the reported SSIM and CLIP margins persist.","supporting_citations":[{"cited_title":"Human brain mapping 8(4), 272–284 (1999)","cited_arxiv_id":null,"evidence_quote":"Provides the FreeSurfer fsaverage surface and flattening that the 2D input representation is built on."},{"cited_title":"In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp","cited_arxiv_id":null,"evidence_quote":"Learning to Prompt for Continual Learning is the prompt-based continual learning method that motivates the PSS module."}],"review_version":1}