{"id":"00500f87-8ad5-4ba6-821c-32192a59a56a","arxiv_id":"2506.23916","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A simple 3D convolutional network (SFCN) predicted age and sex from brain MRI more accurately and consistently across external cohorts than DenseNet or Swin Transformer.","lead":"This study compared three 3D deep learning models, a simple convolutional network, DenseNet, and a Swin Transformer, for predicting age and sex from brain MRI scans across four independent datasets. It found the simple network matched or beat the more complex models on nearly every cohort, suggesting that for brain imaging, architecture complexity alone does not guarantee better generalization.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Age-prediction MAEs are reported after a linear bias correction whose fitting set is unspecified; if the correction is fit on the same test labels used to compute MAE, the age-based half of the central claim is not a held-out comparison.","rationale":"The paper is best read as a controlled comparison of three 3D architectures on demographic prediction with external validation. The strongest support is sex classification: SFCN has AUC 1.00 internally and 0.85-0.91 externally, with CIs and DeLong comparisons showing it at or above DenseNet and above Swin; that part does not depend on the contested correction. The age prediction is the other load-bearing pillar for the abstract's 'simpler is better' conclusion, and its validity hinges on the unspecified bias-correction protocol. Standard practice in brain-age work is to fit bias correction on the training set (the cited Smith et al. reference does this), so the omission is probably an under-specification rather than a deliberate choice; but the text as written permits the leaky reading. Because the same correction is applied to all three models, the direction of the SFCN advantage might survive a clean re-analysis, which is why I would not reject the paper; the conditional verdict already captures this. The shared learning rate/batch size is a real limitation but is explicitly acknowledged in the Discussion, and it weakens the strength of the architectural conclusion rather than invalidating the measured sex results. The counting/MAE inconsistencies are concerning for quality control but are not by themselves decisive. Overall, the reader's CONDITIONAL verdict is the right one, and this stress-test does not move it.","tokens_in":25096,"tokens_out":4600,"duration_ms":53619,"concrete_test":"Recompute every age MAE, Pearson r, and Wilcoxon pairwise comparison with bias-correction coefficients estimated only from training/validation predictions (or from training folds via nested CV), never from test labels, and also report uncorrected MAE. If the corrected SFCN MAE remains below DenseNet/Swin by a similar margin with p<0.017 on all external cohorts, the age claim survives; if the gap narrows, flips, or loses significance, the stated generalizability conclusion must be restricted to sex classification and the age comparison treated as unverified.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Under the Methods section 'Bias Correction', the paper states that for each model, predicted age values were regressed against their corresponding true age labels and the resulting coefficients were applied to adjust predictions, after which MAE and Pearson r were recomputed. It does not state whether the regression is fit on training/validation predictions or on the same test set whose labels are used for evaluation. The reported age results (SFCN MAE 2.66 in UKB, 4.98-5.81 in external cohorts, and the pairwise Wilcoxon comparisons against DenseNet/Swin) are computed on these corrected predictions. On the natural but unstated reading that correction is fit on each test set, every age MAE absorbs information from the test labels and is partially in-sample; correction can reduce MAE and change the SFCN-vs-others gap. Because age prediction is one of the two pillars of the abstract's claim that simpler convolutional networks generalize better, this ambiguity is the most load-bearing weakness. It is not rescued by sex classification, which is measured before any such correction and does robustly show SFCN at or above DenseNet and above Swin on external cohorts. The paper's own Discussion acknowledges the fixed learning-rate/batch-size limitation, so that issue is disclosed; the bias-correction ambiguity is not, and the numerical inconsistencies (e.g., UKB Swin MAE 4.22 in Table 2 vs 4.86 in text; DLBS n=132 in Table 1 vs 108 in Results) further reduce confidence in the age numbers.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper compares three 3D deep learning architectures—SFCN, DenseNet121, and Swin Transformer—for sex classification and age regression from T1-weighted brain MRI. Models are trained on 34,918 UK Biobank participants from three centers and tested on the held-out Newcastle center (12,472 participants) and on three external cohorts (DLBS, PPMI, IXI). The paper reports SFCN achieving essentially perfect internal sex AUC and 0.85-0.91 external AUC, and lower age MAE than DenseNet and Swin Transformer after a linear bias-correction step, with DeLong and Wilcoxon tests, subgroup analyses, and explainability analyses. The manuscript concludes that simpler convolutional networks generalize better across datasets than denser or attention-based architectures.","tokens_in":25344,"tokens_out":8522,"duration_ms":90917,"significance":"If the results hold, the paper is a useful contribution to the architectural-choice debate in medical image analysis: it provides large-scale UK Biobank training, three external validation cohorts, confidence intervals, pairwise statistical tests, open code (https://github.com/jrad9921/RadBrainDL), and reproducible preprocessing using standard tools. The sex-classification results are strong and consistently reported. The main caveat is that the age-prediction half of the central claim depends on a bias-correction procedure whose fitting set is not specified; until that is clarified, the age-based superiority and generalizability conclusion is not fully established.","major_comments":[{"comment":"The bias-correction step is not sufficiently specified, and this is load-bearing for the age-prediction half of the central claim. The Methods state that for each model, predicted age values were regressed against their corresponding true age labels and that the resulting coefficients were applied to adjust predictions, after which MAE and Pearson r were recomputed; they do not state whether the regression is fitted on training/validation predictions or on the same test set whose labels are later used to compute MAE and r. If the latter, then the age MAEs in Table 2 and in the Results (e.g., SFCN 2.66 years in UKB, 4.97-5.80 years external) and the pairwise Wilcoxon comparisons are not held-out estimates, because each evaluation would absorb test-label information through the fitted slope and intercept. Please specify the fitting set explicitly; if the correction was fitted on test labels, the age results and the model comparisons must be recomputed with a correction fitted only on training/validation predictions, or using a properly nested procedure. If the authors followed Smith et al. (ref 62), where the correction is normally fitted on training data, a sentence confirming this and stating the resulting coefficients would resolve the concern.","section":"Methods, 'Bias Correction'; Results, age prediction; Table 2"},{"comment":"The architecture comparison uses a single learning rate and batch size for all models, and this weakens the reported superiority of SFCN. The Methods state that a batch size of 4 and a learning rate of 10e-05 were fixed across all models, while also stating that hyperparameters were tuned on the validation set; these statements are not reconciled, and no per-model tuning results are reported. The Discussion acknowledges the fixed-parameter limitation, but the abstract and conclusion present the finding unconditionally ('simpler convolutional networks outperform...'). Because DenseNet and Swin Transformer may require different learning rates or batch sizes, the head-to-head comparison cannot rule out that the gap is partly an artifact of the common training protocol. Please provide per-model tuning results or explicitly qualify the central claim as holding under a fixed common protocol.","section":"Methods, 'Training Procedures'; Discussion; Abstract"},{"comment":"The reported age results contain internal inconsistencies that must be reconciled. The Results text gives UKB SwinTransformer MAE=4.86 and r=0.60, while Table 2 reports MAE=4.22 (4.16-4.28) and r=0.71 (0.70-0.72); similarly, DLBS is listed as n=132 in Table 1 but as n=108 in the Results section. Because the age comparisons are central to the paper's conclusion, the text, tables, and figures should be checked and the final numbers used consistently.","section":"Results, age prediction; Table 2; Table 1"}],"minor_comments":[{"comment":"The expression '10e-05' should be written as 1e-5 (or 10^-5) if that is the intended learning rate; as written, 10e-05 equals 1e-4, which is a different value.","section":"Methods, 'Training Procedures'"},{"comment":"The Abstract states that 'No significant demographic subgroup biases were detected', but the Results do not report statistical comparisons for the subgroup differences; the age/sex subgroup MAEs in Fig. 3 are presented descriptively. Please either add appropriate tests or soften the claim.","section":"Abstract; Results, 'Minimal bias...'"},{"comment":"The text refers to 'attention heatmaps' for SFCN, while the Fig. 4 caption correctly identifies the method as Grad-CAM; these are gradient-based saliency maps, not attention weights, so the terminology should be made consistent.","section":"Results, 'DL reveals consistent task-specific attention patterns'; Fig. 4 caption"},{"comment":"The sample sizes of the training and validation subsets after the 2:1 split are not reported; please state them for reproducibility.","section":"Methods, 'Training Procedures'"},{"comment":"The phrase 'Bonferroni corrections confirmed SFCN's superiority over Swin Transformer across most cohorts (p<0.017, for three comparisons)' is imprecise: p<0.017 is the Bonferroni-corrected threshold for three pairwise comparisons, and the Results report different significance patterns for sex and age; the Abstract should reflect the Results accurately.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":"The bias-correction ambiguity is the main risk to the paper's age-based claim; if the correction was fitted on test labels, the age half of the paper would require substantial re-analysis. The DLBS sample-size discrepancy should also be checked because external validation is a core contribution. Please ensure the fitting set for the bias correction is stated and that the age metrics are computed under a genuinely held-out protocol before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a look. This is the first multi-cohort comparison I know of that pits SFCN, DenseNet, and Swin Transformer on the same 3D T1 MRI preprocessing and tests them on three external cohorts. That in itself is a useful contribution for people choosing a default architecture for demographic prediction.\n\nThe sex-classification half is solid. SFCN is at or above DenseNet and clearly above Swin on all external cohorts, with confidence intervals and DeLong tests. That part is reproducible and internally consistent. The paper also posts code and data links, which is more than many papers in this area.\n\nThe age-prediction half is where I'd push back. The Methods say predicted ages were regressed against their corresponding true age labels and the coefficients applied to adjust predictions, then MAE and r recomputed. It never says whether that regression is fit on the training/validation set or on the same test set used for evaluation. If it's the latter, every age MAE in the paper is partly in-sample, and the SFCN-vs-others gap could be an artifact of the correction. That is the load-bearing part of the abstract's 'simpler networks generalize better' claim, so it needs to be fixed. The authors also used a single learning rate and batch size for all three architectures; they disclose this in the limitations, which is honest, but it means the age comparison is partly about tuning effort rather than pure architecture.\n\nThere are also numeric inconsistencies that should be reconciled before publication: DLBS is n=132 in Table 1 but n=108 in the Results, and Swin's UKB MAE is 4.22 in Table 2 but 4.86 in the text. None of these are damning on their own, but they erode confidence in the age numbers.\n\nThe central setup—training on UKB and validating on three independent cohorts—is the right way to study generalizability, and the sex result is probably robust. With a clarification of the bias-correction fitting set and a per-model tuning sensitivity check, this becomes a genuinely useful reference for neuroimaging pipeline decisions. I'd send it to peer review, not desk reject it. The referee should ask for those two things plus a reconciliation of the n and MAE discrepancies.","headline":"A useful multi-cohort benchmark for 3D T1 MRI age/sex prediction; the sex result is solid, but the age-superiority claim hinges on an unspecified bias-correction fit that needs clarification.","tokens_in":25997,"tokens_out":2863,"would_cite":false,"duration_ms":28892,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A simple 3D convolutional network outperforms DenseNet and Swin Transformer at predicting age and sex from brain MRI, and generalizes better across cohorts.","keywords":["deep learning","brain MRI","age prediction","sex classification","external validation","generalizability","convolutional neural network","Swin Transformer"],"falsifier":"Re-run the age prediction with bias-correction coefficients estimated only on the training or validation split and re-measure external MAE; if SFCN's advantage over DenseNet and Swin shrinks or disappears, the headline generalizability gap is an artifact of post-hoc label fitting. A separate check would retrain DenseNet and Swin with their own tuned learning rates and batch sizes and see whether they match SFCN on external cohorts.","tokens_in":24836,"feed_emoji":"🧠","tokens_out":6210,"duration_ms":60768,"temperature":0.7,"pith_summary":"The paper tests whether heavier, more expressive 3D deep-learning architectures are better than a lightweight convolutional network at reading demographic traits from T1-weighted brain MRI. It claims they are not: the Simple Fully Connected Network (SFCN) beats DenseNet and Swin Transformer on internal and external cohorts for both age regression and sex classification. The paper wants to establish that simpler convolutional models generalize better across imaging sites and populations, and that external validation should be standard before trusting such models. This matters because brain-age and sex predictions underpin many proposed clinical biomarkers, and choosing the right default architecture affects how those biomarkers behave in new data.","feed_headline":"Simple 3D CNN beats attention models on brain MRI age and sex","feed_subtitle":"Across four independent cohorts, the lightweight network keeps errors low where heavier models degrade.","key_machinery":"The load-bearing object is SFCN (Simple Fully Connected Network), a lightweight 3D convolutional architecture: a stack of 3D convolution, batch-normalization, and ReLU layers, followed by global average pooling and a fully connected head for a scalar age or binary sex output. It is compared against Monai's DenseNet121 and a 3D Swin Transformer under identical fixed training hyperparameters (batch size 4, learning rate $10^{{-5}}$, early stopping), on brain images preprocessed via registration to standard space, brain extraction, intensity normalization, and center-cropping to $180^{3}$ voxels. The comparison protocol, including a held-out center for internal testing and three external cohorts, is what carries the generalizability claim; the bias-correction step in age evaluation is what carries the reported MAE values.","core_discovery":"On the paper's own terms, the central discovery is a consistent ordering of architectures: across four cohorts, SFCN, a shallow 3D convolutional network with batch normalization and ReLU, achieved an AUC of 1.00 [1.00-1.00] for sex classification on the internal UK test set and 0.85-0.91 on three external sets, while DenseNet lagged at 0.82-0.87 and Swin Transformer dropped to 0.67-0.80 outside the training distribution. For age prediction, SFCN reached a mean absolute error of 2.66 years (r=0.89) internally and 4.98-5.81 years (r=0.55-0.70) externally; both DenseNet and Swin produced MAEs near 6-7 years with near-zero correlation on external data. Statistical tests with Bonferroni correction support SFCN over Swin Transformer, and the paper argues the attention-based model is overparameterized for the available training data. The authors additionally find no substantial age or sex bias in SFCN's performance, and they observe that its attention maps and volume correlations align with known neuroanatomy.","pith_inferences":["An implication the authors leave implicit: because the same fixed batch size and learning rate were used for all three architectures, the comparison likely penalizes the larger models more than their capacity deserves; a tuned Swin could close part of the age-prediction gap.","A testable extension is to run the same three architectures on downstream clinical targets, such as brain-age delta in multiple sclerosis, to see whether SFCN's simplicity advantage persists beyond healthy demographic prediction.","Because the code and external cohorts are public, these data could serve as a reusable out-of-distribution benchmark for future 3D brain MRI models, letting the field track whether new architectures actually generalize rather than merely overfit the UK cohort.","The volume-correlation check (predictions matching ground-truth correlations with regional volumes) offers a cheap sanity test for any new brain-MRI model: if a new architecture's prediction gradients do not correlate with the same regions, it is likely exploiting site-specific artifacts."],"forward_implications":["For sex classification, architecture choice barely matters within the training distribution (all models hit AUC 1.00), but it determines robustness cross-cohort: the simple CNN stays at 0.85-0.91 where Swin drops to 0.67.","Age prediction from T1 MRI degrades by roughly 2-3 years of MAE across imaging sites even for the best model, so external-site error ranges should be reported alongside internal results in future brain-age studies.","Attention-based architectures like Swin may need substantially more training data or per-model hyperparameter tuning before they can beat a well-designed lightweight CNN on 3D volumetric medical images.","The consistency between model-attention heatmaps and gray-matter volume correlations suggests SFCN is using neuroanatomical features, not image artifacts, to make its predictions."],"supporting_citations":[{"why":"Defines the SFCN architecture and its use for brain age prediction; supplies the model that the paper finds best.","marker":"[37]"},{"why":"Introduces the Swin Transformer architecture, the attention-based model this paper compares against.","marker":"[48]"},{"why":"Introduces DenseNet, the densely connected CNN used as the second comparison architecture.","marker":"[61]"},{"why":"Describes the UK Biobank imaging resource that provides the training and internal test cohorts.","marker":"[54]"},{"why":"Provides the linear bias-correction method used before computing age MAE; the reported age errors rely on it.","marker":"[62]"},{"why":"Shows in a prior comparison that simpler models can match or beat larger ones in medical imaging, which the paper cites as support for its finding.","marker":"[65]"}],"fun_headline_variants":["Simple 3D CNN outshines complex models on brain MRI tasks","Simplicity wins: shallow CNN beats transformers for brain age and sex","Lightweight CNN generalizes better than attention models on MRI","SFCN beats Swin and DenseNet across four brain MRI cohorts","Efficient 3D CNN tops heavier networks for neuroimaging prediction"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported age errors are trustworthy only if the linear correction that shifts predicted ages toward true ages is fitted on data that is separate from the test set where the errors are measured, and the paper does not state which set is used for that fit.","fun_headline_variants_meta":{"raw":{"variants":["Simple 3D CNN outshines complex models on brain MRI tasks","Simplicity wins: shallow CNN beats transformers for brain age and sex","Lightweight CNN generalizes better than attention models on MRI","SFCN beats Swin and DenseNet across four brain MRI cohorts","Efficient 3D CNN tops heavier networks for neuroimaging prediction"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000628,"raw_usage":{"total_tokens":2989,"prompt_tokens":1119,"completion_tokens":1870,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":735,"completion_tokens_details":{"reasoning_tokens":1778}},"tokens_in":735,"tokens_out":1870,"duration_ms":13884,"temperature":1.0,"reasoning_tokens":1778,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T21:28:16.289077+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the age prediction with bias-correction coefficients estimated only on the training or validation split and re-measure external MAE; if SFCN's advantage over DenseNet and Swin shrinks or disappears, the headline generalizability gap is an artifact of post-hoc label fitting. A separate check would retrain DenseNet and Swin with their own tuned learning rates and batch sizes and see whether they match SFCN on external cohorts.","supporting_citations":[{"cited_title":"F., Vedaldi, A","cited_arxiv_id":null,"evidence_quote":"Defines the SFCN architecture and its use for brain age prediction; supplies the model that the paper finds best."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Describes the UK Biobank imaging resource that provides the training and internal test cohorts."},{"cited_title":"M., Vidaurre, D., Alfaro-Almagro, F., Nichols, T","cited_arxiv_id":null,"evidence_quote":"Provides the linear bias-correction method used before computing age MAE; the reported age errors rely on it."},{"cited_title":"& Bengio, S","cited_arxiv_id":null,"evidence_quote":"Shows in a prior comparison that simpler models can match or beat larger ones in medical imaging, which the paper cites as support for its finding."}],"review_version":1}