{"id":"a98f9250-3762-408d-9248-b99a3ddba518","arxiv_id":"2509.10620","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":9,"one_line_summary":"A SimCLR encoder pre-trained on 44,958 diverse 3D brain MRIs beats from-scratch supervised baselines on four downstream tasks, including Alzheimer's classification with 20% of labels.","lead":"This paper builds a self-supervised AI model for 3D brain MRI by pre-training on 44,958 scans from 11 public datasets spanning Alzheimer's, Parkinson's, and stroke. The fine-tuned model outperforms simple supervised baselines on four downstream tasks, and the code and model are released.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Universal superiority claim lacks statistical support: stroke and sex task differences are within one standard error.","rationale":"The reader's verdict correctly notes scope limitations (limited baselines, in-distribution split, lack of variance in data-efficiency plots), but the weakest assumption identified—cross-site confounds dominating the pre-training distribution—is not the most load-bearing concern for the central claim. The paper's strongest evidence for transfer is the out-of-distribution performance on AIBL and IXI, which are completely held out during pre-training, so cross-site confounding is largely mitigated by the evaluation design. The more serious correctness risk is that the headline claim of universal superiority is based on point estimates from only five runs, with no significance testing. On two of four tasks (stroke regression and sex classification), the differences between SimCLR-FT and the best baseline are within one standard error, meaning the observed improvement could easily arise from random seed variation. The data-efficiency claim about 20% data is even less robust because it is based on a single run per data fraction. The central argument—that a pre-trained SimCLR encoder beats training from scratch with full labels—needs statistical support to be credible. The paper does offer real strengths: it releases code and weights, uses public data, and applies a consistent preprocessing pipeline. The conditional acceptance recommendation remains appropriate, but the condition should explicitly require significance testing and error bars on the data-efficiency curves. This is a testable, concrete requirement that would settle whether the 'across all tasks' claim holds.","tokens_in":14244,"tokens_out":7935,"duration_ms":83967,"concrete_test":"Recompute Table 2 using paired permutation tests (or paired t-tests) across the five runs for each task, comparing SimCLR-FT to the best baseline for that task (e.g., ResNet-18 for stroke, sex, and age; SimCLR-LP for AD). Report p-values and 95% confidence intervals for each task. Also run the 20% data fine-tuning experiment with at least five seeds and report mean ± std. If any task yields p ≥ 0.05 or the 20% result shows high variance, the abstract's 'outperforms across all tasks' and 'superior with 20%' claims should be qualified to reflect statistical uncertainty.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The abstract's central claim is that the fine-tuned SimCLR model 'outperforms all other models across all tasks' and remains 'superior' when fine-tuned on only 20% of labeled data. In Table 2, results are reported as mean ± std over five independent runs, but no significance tests or confidence intervals are provided. For stroke-scale regression (MAE), SimCLR-FT (5.37 ± 0.24) vs ResNet-18 (5.47 ± 0.09) differs by 0.10, with a standard error of the difference approximately 0.11. For sex classification, SimCLR-FT (0.991 ± 0.004) vs ResNet-18 (0.988 ± 0.005) differs by 0.003, with SE ≈ 0.003. In both cases the difference is within one standard error, so the observed improvement is not statistically distinguishable from noise. Since the headline claim is explicitly 'across all tasks', the lack of significance on two of four tasks undermines the strength of the central assertion. Additionally, the data-efficiency experiments (Figures 4 and 5) show only a single run per data fraction, so the 'superior with 20% data' claim is a single-seed observation with no variance estimate. The comparison to the supervised ResNet-18 is fair (same backbone), so the architecture is not the issue; the evidence is simply not sufficient to support universal superiority without statistical testing.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper presents a 3D SimCLR-based self-supervised foundation model for T1-weighted brain MRI. The encoder is a 3D ResNet-18 pretrained on 44,958 scans from 18,759 patients across 11 public datasets, with the SOOP stroke dataset split before pretraining. The authors compare fine-tuned and linear-probed SimCLR against a 3D MAE (ViT-Tiny) and supervised ResNet-18/ViT-T baselines on four downstream tasks: NIH stroke-scale regression (in-distribution), AD vs healthy classification (AIBL, out-of-distribution), sex classification and age regression (IXI, out-of-distribution). The central claim is that the fine-tuned SimCLR model outperforms all other models across all tasks, and retains this advantage when fine-tuned on only 20% of labeled data for AD classification. The model and code are publicly released.","tokens_in":14582,"tokens_out":6723,"duration_ms":73372,"significance":"The practical contribution is substantial if the empirical claim holds: a high-resolution, publicly released 3D MRI encoder pretrained on a diverse multi-disease corpus, combined with a reproducible TurboPrep preprocessing pipeline and held-out evaluation on AIBL and IXI. The five-run reporting in Table 2 and the use of held-out datasets are good practices. However, the headline 'outperforms all models across all tasks' is only partially supported statistically; for two of the four tasks the margin over the strongest baseline is within sampling noise. The scientific significance of the paper therefore depends on tightening the statistical analysis and calibrating the claims.","major_comments":[{"comment":"The claim 'outperforms all other models across all tasks' (Abstract and Section 5.1) is not backed by any significance test or confidence interval. Using the reported mean±std over five runs, the stroke-scale difference between SimCLR-FT (5.37±0.24) and ResNet-18 (5.47±0.09) is 0.10, with approximate standard error of the difference sqrt(0.24^2/5 + 0.09^2/5) ≈ 0.11, so the observed difference is within one standard error. For sex classification, SimCLR-FT (0.991±0.004) vs ResNet-18 (0.988±0.005) differs by 0.003 with SE ≈ 0.003, again marginal. Thus two of four tasks do not provide statistically distinguishable improvements. Please add paired tests or bootstrap confidence intervals on the five runs, with multiple-comparison awareness, and temper the universal superiority claim if these comparisons are not significant.","section":"Section 5.2, Figures 4 and 5"},{"comment":"The data-efficiency experiments are reported from a single run at each data fraction ('each model is trained once using the same seed'). The claim that at 20% data the model 'still outperforms all supervised models' for Alzheimer's disease is therefore a single-seed observation with no variance estimate. Since this is one of the paper's headline contributions, please run at least three (preferably five) seeds per fraction, report error bars, and compare against supervised baselines trained on the same reduced data fractions as well as on full data. If the result is not stable across seeds, the claim should be softened accordingly.","section":"Section 5.2, Figures 4 and 5"}],"minor_comments":[{"comment":"Units are inconsistently formatted: '1.4 mm 3', '1×1×1 mm 3', and '10 4' should use superscripts (mm^3, 10^4).","section":"Section 2.1, 2.2, 4.1"},{"comment":"The 'NA' entries for AIBL and IXI scan counts are ambiguous; clarify that these datasets were only used for testing and their scans were not part of pretraining.","section":"Table 1"},{"comment":"In the in-distribution paragraph, 'superior performance over all other models' is stronger than the numbers support; please add a qualifier such as 'numerically' or 'not statistically significant' until formal tests are added.","section":"Section 5.1"},{"comment":"The MAE model-size comparison ('the performance of all three models are similar') lacks the criterion and the numbers. Please state whether this refers to pretraining loss, downstream performance, or something else, and report the actual values for ViT-Tiny/Base/Large.","section":"Section 4.2"},{"comment":"The manuscript has no limitations section. A brief discussion of statistical power, the near-ceiling sex-classification task, and the T1-only scope would help calibrate the claims.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"To the editor: the stress-test concern about statistical support is legitimate and lands. The headline universal-superiority claim is not statistically supported on two of the four tasks, and the data-efficiency result rests on a single seed. I do not see a circularity problem with the SOOP split: the held-out validation/test sets were not used in pretraining, and fine-tuning on the pretraining split is the intended setting. The main revision needed is additional statistical reporting and more conservative claims; with that, the paper would be a useful contribution to the community. No concerns about citation practice or scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe useful thing here is the resource: a SimCLR encoder pre-trained on 44,958 T1 scans across 11 datasets at 1 mm isotropic, with code and weights released. That is genuinely valuable for the medical imaging community, and the authors did the preprocessing with public tools (TurboPrep) so it is reproducible. The comparison against a same-backbone supervised ResNet-18 is fair, and the Alzheimer's result — 6% AUC gain on a held-out dataset with only 106 training patients — looks like a real effect.\n\nThe soft spots are in the headline claims. The abstract says the fine-tuned SimCLR model 'outperforms all other models across all tasks.' On the stroke scale regression, SimCLR-FT (5.37±0.24) is not statistically distinguishable from ResNet-18 (5.47±0.09); the difference is about 0.9 standard errors. Same for sex classification (0.991±0.004 vs 0.988±0.005). So two of the four tasks are within noise. Only the Alzheimer's AUC and age regression show clear wins. That is still a positive result, but the 'across all tasks' line should be softened.\n\nThe data-efficiency plots in Figures 4 and 5 are single runs with no variance. The claim that the model beats supervised training with 20% of labels rests on one seed. That is weak evidence, though the point estimate is plausible.\n\nThe baseline set is also narrow. There is no comparison to existing 3D brain MRI foundation models (BrainSegFounder, LaMIM, or Dufumier's SimCLR). The MAE baseline is a single ViT-T variant, and the authors admit it is not optimized. This makes it hard to know where the model sits relative to the field. The paper would be much stronger with a benchmark against at least one released foundation model.\n\nOne minor point: the in-distribution SOOP task uses the same 60% of SOOP for pre-training and fine-tuning, with test held out. That is not a true out-of-distribution evaluation, but they do have two fully held-out datasets, so it is not a fatal flaw.\n\nWho is this for? Anyone working on 3D brain MRI classification or regression who wants a strong, public, high-resolution pre-trained encoder. It deserves a serious referee, but the authors should be asked to add significance tests or confidence intervals, benchmark against at least one existing foundation model, and tone down the 'universal superiority' claim. I would recommend conditional acceptance after revision.","headline":"Useful released 3D brain MRI foundation model, but the 'outperforms all tasks' claim doesn't survive a look at the error bars on two of the four tasks.","tokens_in":15099,"tokens_out":2749,"would_cite":true,"duration_ms":24335,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A SimCLR foundation model pre-trained on 45,000 3D brain MRIs outperforms supervised baselines on every tested diagnostic task, including out-of-distribution tasks and with only 20% of labels for Alzheimer's prediction.","keywords":["self-supervised learning","SimCLR","3D brain MRI","foundation model","Alzheimer's disease","stroke outcome","contrastive learning","transfer learning"],"falsifier":"A concrete test: fine-tune the released SimCLR encoder on a held-out dataset from a different scanner vendor (e.g., a site not in the 11 used), with the same TurboPrep pipeline, and compare against a ResNet-18 trained from scratch on the same labeled data. If the SimCLR advantage vanishes or reverses on that unseen site, then the 'out-of-distribution' claim does not generalize beyond the tested sources. Also, an ablation that removes the intensity-shift and contrast augmentations could show whether anatomical invariance is the true driver.","tokens_in":14114,"feed_emoji":"🧠","tokens_out":3673,"duration_ms":35394,"temperature":0.7,"pith_summary":"The paper claims that a single self-supervised SimCLR encoder, pre-trained at native 1 mm isotropic resolution on 44,958 T1-weighted brain MRI scans from 11 public datasets spanning healthy aging, Alzheimer's, Parkinson's, frontotemporal dementia, and stroke, learns representations general enough to beat fully supervised models on four downstream tasks: stroke-severity regression, Alzheimer's classification, sex classification, and age regression. The fine-tuned model wins on both in-distribution (SOOP stroke) and held-out out-of-distribution (AIBL, IXI) data. Crucially, for Alzheimer's classification, fine-tuning on only 20% of the training labels still beats supervised baselines trained on 100% of the labels. If true, this shows that diverse unlabeled brain MRI can substitute for task-specific labeled data, making high-accuracy diagnostic models feasible in low-label clinical settings.","feed_headline":"SimCLR MRI model beats supervised baselines on all 4 brain tasks","feed_subtitle":"Pre-trained on 45,000 scans from 11 datasets, the encoder also wins Alzheimer's prediction with only 20% of labels.","key_machinery":"The load-bearing mechanism is the SimCLR contrastive objective (NT-Xent loss) applied to two random augmentations of each 3D volume, with a 3D ResNet-18 encoder and a 64-dimensional projection head. The augmentations—random spatial crop to 150×192×192, axial flip, 45° rotation, intensity shift, and contrast adjustment—force the encoder to become invariant to imaging nuisances while preserving anatomy. Pre-training uses one scan per patient per epoch across 18,759 patients, so frequent-scanner patients do not dominate. The same encoder is then either linearly probed or fine-tuned for each downstream task.","core_discovery":"On the paper's own terms, the central discovery is that contrastive self-supervision (SimCLR) applied to a large, heterogeneous corpus of 3D brain MRIs yields a transferable encoder whose fine-tuned performance exceeds that of ResNet-18 and ViT-Tiny trained from scratch with full supervision across all four evaluated tasks (Table 2). The margin is largest for Alzheimer's classification (AUC 0.929 vs 0.869 for the next-best supervised model), and the model holds its advantage even when fine-tuned with only 20% of the labeled training data (Figure 4). The authors attribute the result to SimCLR's augmentation-invariant learning of anatomical features at high resolution (1 mm iso), which the ViT","pith_inferences":["Because the pre-training corpus includes datasets with diverse acquisition protocols, the contrastive invariances may generalize to scanner/site shifts beyond the two held-out datasets; a systematic study across many sites could quantify this.","The Alzheimer's result with 10 positive training samples hints that even smaller label sets (e.g., a handful per class) could suffice for some classification tasks, but the upper bound of this regime remains untested.","The failure of MAE relative to SimCLR may be specific to the dataset scale and patch size; a larger corpus or different masking ratio could reverse the comparison.","Since sex classification reached near-perfect AUC (0.991), the model seems to encode fine anatomical sex dimorphisms; this could be a confound for disease-disparity studies and should be checked when fine-tuning on unrelated tasks."],"forward_implications":["A single pre-trained encoder can replace task-specific training for at least four distinct 3D brain MRI tasks, reducing the need for separate specialized models.","In settings with scarce labeled data, fine-tuning the foundation model beats from-scratch supervision; for Alzheimer's classification, 20% of labels suffice to outperform supervised baselines.","The CNN-based SimCLR approach is more effective than the ViT-based MAE approach at this dataset scale, suggesting inductive bias matters for 3D medical imaging with modest data.","The model's success on held-out datasets (AIBL, IXI) suggests the learned features are not overfit to the pre-training acquisition sources.","Release of code and model allows the community to fine-tune for other tasks without repeating the costly pre-training."],"fun_headline_variants":["SimCLR beats supervised models on all 4 brain MRI tasks","Pre-trained on 45,000 scans, SimCLR wins with just 20% labels","Contrastive SSL outperforms supervised baselines on neuro tasks","Self-supervised encoder tops full-supervision models on brain scans","SimCLR on 3D MRI: wins Alzheimer's with 20% of training data"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The paper assumes that after TurboPrep preprocessing (N4 bias correction, SynthStrip skull stripping, linear registration, intensity normalization), the 11 heterogeneous public datasets form a single learnable distribution, so the contrastively learned invariances reflect brain anatomy rather than scanner/site confounds, and that these features transfer to fully held-out datasets.","fun_headline_variants_meta":{"raw":{"variants":["SimCLR beats supervised models on all 4 brain MRI tasks","Pre-trained on 45,000 scans, SimCLR wins with just 20% labels","Contrastive SSL outperforms supervised baselines on neuro tasks","Self-supervised encoder tops full-supervision models on brain scans","SimCLR on 3D MRI: wins Alzheimer's with 20% of training data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000217,"raw_usage":{"total_tokens":1334,"prompt_tokens":869,"completion_tokens":465,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":613,"completion_tokens_details":{"reasoning_tokens":363}},"tokens_in":613,"tokens_out":465,"duration_ms":5188,"temperature":1.0,"reasoning_tokens":363,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T17:44:28.513448+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete test: fine-tune the released SimCLR encoder on a held-out dataset from a different scanner vendor (e.g., a site not in the 11 used), with the same TurboPrep pipeline, and compare against a ResNet-18 trained from scratch on the same labeled data. If the SimCLR advantage vanishes or reverses on that unseen site, then the 'out-of-distribution' claim does not generalize beyond the tested sources. Also, an ablation that removes the intensity-shift and contrast augmentations could show whether anatomical invariance is the true driver.","supporting_citations":[],"review_version":1}