REVIEW 4 major objections 6 minor 21 references
3DSiameseNet to Analyze Brain MRI
T0 review · 4 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read A 3D siamese convolutional network trained on pairs of whole-brain MRI scans taken a year apart is claimed to classify cognitively declining versus stable patients with 90% accuracy.
desk verdict A reasonable 3D siamese adaptation with a 90% validation accuracy on 247 ADNI subjects, but the lack of an independent test set and undocumented subject selection keep the headline claim unsupported. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the 3D siamese architecture with difference fusion. A siamese network is a pair of identical neural-network branches that share weights, so any difference in their outputs comes from the inputs, not from different learned filters; here the inputs are the baseline and follow-up MRI volumes and the outputs are subtracted elementwise. That subtraction is the mechanism that isolates brain change over the year, and it is what lets the network avoid segmentation, ROI atlases, and cortical-thickness computation. The choice of average pooling rather than max pooling is meant to prevent the network from biasing toward edge detection, and the small 3x3x3 kernels preserve local volumetric details.
What would settle it
Re-run the same network and validation protocol on a cohort where decline is confirmed independently (for example, by expert diagnosis or documented conversion to mild cognitive impairment) using labels not derived from the trajectory script; if accuracy falls to around 75% or to chance, the central claim is not supported.
Extended reading notes
Core claim
The central claim is that a 3D siamese convolutional network, called 3D-SiameseNet, classifies cognitively declining versus stable patients with 90% mean validation accuracy using only pairs of raw MRI volumes. The model has two branches with shared weights, one processing the baseline scan and the other the 12-month follow-up scan; each branch stacks three blocks of 3D convolution, batch normalization, leaky ReLU, and average pooling. Instead of concatenating the branch outputs as standard siamese networks do, the authors subtract them, so the network is forced to represent morphological change between visits. The labels come from a published trajectory-modeling script, and the comparison is to four imaging-only baselines whose reported accuracies range from 75% to 77%; the paper also reports a mean squared logarithmic error of 0.04 on validation, meaning predicted probabilities sit close to the true labels.
Load-bearing premise
The division of patients into 'stable' and 'declining' comes from a published script that labels trajectories from mental-status test scores; if those labels are noisy or wrong, the 90% accuracy may measure prediction of the script's labels rather than true cognitive decline.
Editorial extensions
If this is right
- Imaging-only longitudinal prediction may not require segmentation, ROI selection, or cortical-thickness measures; raw whole-brain volumes are enough for the reported 90% accuracy.
- Difference fusion in a siamese network is a viable way to represent disease-related change, and the t-SNE plots show the representation becomes class-separated only after the subtraction and dense layers.
- The model's 90% validation accuracy exceeds the 75–77% accuracies of the logistic regression, SVM, random forest, and ANN baselines that also use only imaging, while using far fewer subjects.
- Adding clinical data to the imaging branch, as the authors propose for future work, could plausibly move accuracy beyond the 90% imaging-only level toward the 94% reported for the earlier model that used both imaging and clinical scores.
Reading between the lines
- If the label script is reliable, a natural next test is external validation on an independent cohort with clinician-confirmed progression; a drop to near the 75% baseline level would indicate the 90% is tied to the specific label assignment.
- The three clusters visible in the t-SNE projection before the subtraction layer may encode anatomy such as head size; a regressor trained on those pre-subtraction features could test whether the network has learned non-diagnostic confounds.
- A permutation test that swaps the two time-point images within each subject would show whether the model is truly reading temporal change; if swapped-pair accuracy stays high, the model may be exploiting visit-specific artifacts rather than progression.
- Because the authors note memory limits forced downscaling, running the same architecture at native resolution with more filters is a direct test of whether the reported accuracy underestimates what the approach can do.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes 3D-SiameseNet, a siamese convolutional network that takes a pair of whole-brain 3D MRI scans (baseline and 12-month follow-up) and classifies subjects as cognitively declining or stable. The model replaces the fully-connected branches of Bhagwat et al.'s Longitudinal Siamese Network with 3D convolution blocks and fuses the two branches by a subtraction layer. Using ADNI data, the authors select 247 subjects, follow Bhagwat et al.'s script to assign Stable/Decline labels from MMSE-based trajectory clustering, and train with data augmentation. Over 10 random 40-subject validation splits, they report a mean validation accuracy of 90%, a validation MSLE of 0.04, and a validation cross-entropy loss of 0.992. The paper also includes t-SNE visualizations of intermediate feature maps to argue that the network learns discriminative representations.
Significance. If the 90% accuracy held on an independent test set with clinically validated labels, the contribution would be meaningful: it would show that a relatively small 3D siamese CNN can predict cognitive decline directly from raw whole-brain longitudinal MRI, without segmentation, ROI selection, or cognitive scores as inputs. The paper provides source code, uses a public dataset, and the architecture is simple and reproducible. However, the current evaluation protocol does not support the headline claim. The accuracy is measured only on repeated validation splits rather than a held-out test set, the comparison with other models uses published numbers from a different and larger cohort, and the reported validation cross-entropy is inconsistent with the claimed accuracy. These issues need to be addressed before the central claim can be taken as established.
major comments (4)
- [Section 4, Table 1] The reported 90% mean validation accuracy is computed over 10 random sub-sampling runs with only 40 validation subjects each, and no independent test set is ever held out. Because the same validation subjects are reused across runs and the model is trained on the remaining 207 subjects each time, the reported accuracy is a repeated random-split estimate, not an independent evaluation of a final model. This protocol does not support the abstract's claim that the network 'achieves an accuracy of 90%' on the general patient population. An independent test cohort, or at least a single fixed test split never used in any model-selection or early-stopping decision, is needed.
- [Section 4, Table 2] The comparison to LR, SVM, RF, and ANN uses accuracy values taken from Bhagwat et al. (S4 File), which were obtained on a different cohort of 1116 subjects (Section 4 states 'Bhagwat et al. did performance evaluation experiments using 1116 subjects'). This is not a head-to-head comparison: the baseline models were evaluated on a different dataset, so the statement that 'our model outperforms LR, SVM, RF, and ANN models' is not supported by the evidence presented. A fair comparison would require running the same baselines on the same 247-subject subset with the same validation protocol.
- [Section 3.3 and Section 4, Figure 4] The assignment of Stable/Decline labels follows an external script based on MMSE trajectory clustering, and the authors then 'selected a subset of these subjects' (134 stable, 113 decline). No inclusion or exclusion criteria for this subset are reported, and no evidence is provided that the subset is representative of the ADNI1 cohort or that the script's clusters correspond to clinically meaningful cognitive decline. If the selection preferentially kept subjects with extreme or easy-to-separate trajectories, the 90% accuracy could be high while the model performs poorly on the broader patient population. The label-generation script and the exact selection procedure must be described, and the resulting labels should be validated against clinical outcomes.
- [Section 4, Figure 4] The reported validation cross-entropy loss of 0.992 is greater than ln(2) ≈ 0.693, which is the expected cross-entropy of a random classifier on balanced classes, yet the validation accuracy is reported as 90%. This inconsistency is unexplained. If the model is 90% accurate, a mean cross-entropy above 0.7 suggests either severely miscalibrated probabilities or a mismatch in how accuracy and loss are computed. The authors should clarify how the loss and accuracy are aggregated over the 10 runs and explain the relationship between the reported loss and the reported accuracy, or correct the metric if there is an error.
minor comments (6)
- [Section 2] The phrase 'Logintudinal Siamese Network' appears to be a typo for 'Longitudinal Siamese Network'; please correct it.
- [Section 5, Figure 5] In the paragraph describing the penultimate Dense layer, the text says 'see Fig.5.3', but the penultimate layer corresponds to the fourth subplot (Fig.5.4); the reference should be updated.
- [Abstract and Section 3.3] The abstract states that the result is obtained 'without the help of a cognitive score', but the Stable/Decline labels are derived from MMSE-based trajectory clustering (Section 3.3). Please clarify that cognitive scores are not used as model inputs, but are used to define the labels.
- [Section 4, Table 1] The metric MSLE (Mean Squared Logarithmic Error) is not standard for classification probabilities; please define how it is computed and why it is appropriate for this task.
- [Section 3.2] The word 'tridimentional' should be 'three-dimensional'.
- [Section 3.1] The claim that average pooling is used 'to avoid a bias of the network towards edge detection' is not self-evident, because the convolution filters are learned; a brief explanation or reference would help.
Circularity Check
No significant circularity: the reported 90% accuracy is an empirical classification result trained on labels from an external script, with no prediction that reduces to its own inputs by construction.
full rationale
The paper's central claim is an empirical result: a 3D siamese CNN classifies ADNI subjects into 'Stable' and 'Decline' based on pairs of whole-brain MRIs. The labels are assigned by following the script of Bhagwat et al. (Section 3.3), which uses MMSE clinical scores to model trajectories. This is an external label source, not a quantity derived from the network's inputs or outputs. Cognitive scores are not used as model inputs, and no fitted parameter is renamed as a prediction. The network's input consists of raw MRI volumes, and its output is compared to externally produced labels. The comparison to Bhagwat et al.'s previously published accuracy values is also an external benchmark, not a self-citation chain that forces the result. The paper does not define its target variable in terms of its own predictions, and the reported validation accuracy is a standard train/validation evaluation. The only notable weakness is that the paper does not report the exact inclusion/exclusion criteria for selecting 247 subjects from the ADNI1 cohort, and the external script's label quality is not independently validated here; however, that is a correctness or generalizability concern, not circularity. Under the specified rules, no self-definitional step, fitted-input-as-prediction, load-bearing self-citation, imported uniqueness theorem, ansatz-smuggling citation, or renaming of a known result is present. Therefore the circularity score is 0.
Assumptions & free parameters
free parameters (10)
- Downscaling factor =
2
- Convolution filter counts =
Not stated in text (shown in Fig.2)
- Dropout rate =
0.5
- L2 regularization strength =
Not stated
- Learning rate =
0.001
- Batch size =
20
- Training epochs =
800
- Data augmentation rotation range =
0 to 5 degrees
- Number of convolution blocks per branch =
3
- Kernel size =
3x3x3
assumptions (3)
- domain assumption The label-generation script from Bhagwat et al. assigns correct Stable/Decline labels based on MMSE trajectories.
- domain assumption Downscaling by 2 using order 3 spline interpolation preserves the morphological changes relevant to cognitive decline.
- domain assumption The 247-subject ADNI subset is representative, and the 40-subject validation split is sufficient to estimate generalization.
Cite this review
Pith. "Pith review of 3DSiameseNet to Analyze Brain MRI." pith.science (2026). https://pith.science/paper/3KNLRTB5
@misc{pith2026190901098,
author = {Pith},
title = {Pith review of: 3DSiameseNet to Analyze Brain MRI},
year = {2026},
howpublished = {\url{https://pith.science/paper/3KNLRTB5}},
note = {Machine review of arXiv:1909.01098}
}
read the original abstract
Prediction of the cognitive evolution of a person susceptible to develop a neurodegenerative disorder is crucial to provide an appropriate treatment as soon as possible. In this paper we propose a 3D siamese network designed to extract features from whole-brain 3D MRI images. We show that it is possible to extract meaningful features using convolution layers, reducing the need of classical image processing operations such as segmentation or pre-computing features such as cortical thickness. To lead this study we used the Alzheimer's Disease Neuroimaging Initiative (ADNI), a public data base of 3D MRI brain images. A set of 247 subjects has been extracted, all of the subjects having 2 images in a range of 12 months. In order to measure the evolution of the patients states we have compared these 2 images. Our work has been inspired at the beginning by an article of Bhagwat et al. in 2018, who have proposed a siamese network to predict the status of patients but without any convolutional layers and reducing the MRI images to a vector of features extracted from predefined ROIs. We show that our network achieves an accuracy of 90\% in the classification of cognitively declining VS stable patients. This result has been obtained without the help of a cognitive score and with a small number of patients comparing to the current datasets size claimed in deep learning domain.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[1]
The clinical use of structural mri in alzheimer disease,
G. B. Frisoni, N. C. Fox, C. R. Jack Jr, P. Scheltens, and P. M. Thompson, “The clinical use of structural mri in alzheimer disease,” Nature Reviews Neurology, vol. 6, no. 2, p. 67, 2010
work page 2010
-
[2]
N. Bhagwat, J. D. Viviano, A. N. V oineskos, M. M. Chakravarty, et al., “Modeling and prediction of clinical symptom trajectories in alzheimers disease using longitu- dinal data,” PLoS computational biology , vol. 14, no. 9, p. e1006376, 2018
work page 2018
-
[3]
Gradient-based learning applied to document recogni- tion,
Y . LeCun, L. Bottou, Y . Bengio, and P. Haffner, “Gradient-based learning applied to document recogni- tion,” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278– 2324, 1998
1998
-
[4]
Siamese neu- ral networks for one-shot image recognition,
G. Koch, R. Zemel, and R. Salakhutdinov, “Siamese neu- ral networks for one-shot image recognition,” in ICML Deep Learning Workshop, vol. 2, 2015
work page 2015
-
[5]
Learning to compare image patches via convolutional neural networks,
S. Zagoruyko and N. Komodakis, “Learning to compare image patches via convolutional neural networks,” inPro- ceedings of the IEEE conference on computer vision and pattern recognition, pp. 4353–4361, 2015
work page 2015
-
[6]
Homemade ts-net for auto- matic face recognition,
S. Lin, Z. Zhao, and F. Su, “Homemade ts-net for auto- matic face recognition,” in Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval , pp. 135–142, ACM, 2016
work page 2016
-
[7]
S. Sarraf and G. Tofighi, “Classification of alzheimer’s disease using fmri data and deep learning convolutional neural networks,” arXiv preprint arXiv:1603.08631 , 2016
arXiv 2016
-
[8]
C. D. Billones, O. J. L. D. Demetria, D. E. D. Hostallero, and P. C. Naval, “Demnet: A convolutional neural net- work for the detection of alzheimer’s disease and mild cognitive impairment,” in Region 10 Conference (TEN- CON), 2016 IEEE, pp. 3724–3727, IEEE, 2016
work page 2016
Show all 21 references
-
[9]
Alzheimer’s disease diagnostics by a deeply supervised adaptable 3d convolutional network,
E. Hosseini-Asl, G. Gimel’farb, and A. El-Baz, “Alzheimer’s disease diagnostics by a deeply supervised adaptable 3d convolutional network,” arXiv preprint arXiv:1607.00556, 2016
2016 arXiv
-
[10]
Predicting alzheimer’s dis- ease: a neuroimaging study with 3d convolutional neural networks,
A. Payan and G. Montana, “Predicting alzheimer’s dis- ease: a neuroimaging study with 3d convolutional neural networks,” arXiv preprint arXiv:1502.02506, 2015
2015 arXiv
-
[11]
Residual and plain convolutional neural networks for 3d brain mri classification,
S. Korolev, A. Safiullin, M. Belyaev, and Y . Dodonova, “Residual and plain convolutional neural networks for 3d brain mri classification,” in Biomedical Imaging (ISBI 2017), 2017 IEEE 14th International Symposium on , pp. 835–838, IEEE, 2017
2017
-
[12]
3d cnn-based classification using smri and md-dti images for alzheimer disease studies,
A. Khvostikov, K. Aderghal, J. Benois-Pineau, A. Krylov, and G. Catheline, “3d cnn-based classification using smri and md-dti images for alzheimer disease studies,” arXiv preprint arXiv:1801.05968, 2018
2018 arXiv
-
[13]
Classifica- tion of smri for alzheimer’s disease diagnosis with cnn: Single siamese networks with 2d+? approach and fusion on adni,
K. Aderghal, J. Benois-Pineau, and K. Afdel, “Classifica- tion of smri for alzheimer’s disease diagnosis with cnn: Single siamese networks with 2d+? approach and fusion on adni,” in Proceedings of the 2017 ACM on Interna- tional Conference on Multimedia Retrieval, pp. 494–498, ...
2017
-
[14]
Batch normalization: Acceler- ating deep network training by reducing internal covariate shift,
S. Ioffe and C. Szegedy, “Batch normalization: Acceler- ating deep network training by reducing internal covariate shift,” arXiv preprint arXiv:1502.03167, 2015
2015 arXiv
-
[15]
Rectifier non- linearities improve neural network acoustic models,
A. L. Maas, A. Y . Hannun, and A. Y . Ng, “Rectifier non- linearities improve neural network acoustic models,” in Proc. icml, vol. 30, p. 3, 2013
2013
-
[16]
ADNI | Alzheimer’s Disease Neuroimaging Initiative
“ADNI | Alzheimer’s Disease Neuroimaging Initiative.”
-
[17]
3d u-net for skull stripping in brain mri,
H. Hwang, H. Rehman, and S. Lee, “3d u-net for skull stripping in brain mri,” Applied Sciences, vol. 9, p. 569, 02 2019
2019
-
[18]
Adam: A method for stochastic optimization,
D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014
2014 arXiv
-
[19]
Chollet et al., “Keras,” 2015
F. Chollet et al., “Keras,” 2015
2015
-
[20]
TensorFlow: Large-scale machine learn- ing on heterogeneous systems,
M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. Goodfellow, A. Harp, G. Irving, M. Is- ard, Y . Jia, R. Jozefowicz, L. Kaiser, M. Kudlur, J. Leven- berg, D. Man´e, R. Monga, S. Moore, D. Murray, C. Olah...
2015
-
[21]
Visualizing data us- ing t-sne,
L. v. d. Maaten and G. Hinton, “Visualizing data us- ing t-sne,” Journal of machine learning research , vol. 9, no. Nov, pp. 2579–2605, 2008
2008
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.