REVIEW 3 major objections 5 minor 46 references
Tutorial: VAE as an inference paradigm for neuroimaging
T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read This tutorial argues that variational autoencoders, trained through the ELBO objective and the reparameterization trick, make high-dimensional neuroimaging data interpretable enough to expose patterns tied to neurodegeneration.
desk verdict Competent, unoriginal VAE tutorial: standard derivations correct, but Appendix A's concentration-of-measure argument doesn't support the neuroimaging manifold premise. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the Evidence Lower Bound (ELBO) in its practical form, $\log p(x) \geq \mathbb{E}_{q_\phi(z|x)}[\log p(x|z)] - \mathrm{KL}(q_\phi(z|x)\,\|\,p(z))$, together with the reparameterization trick. The ELBO converts Bayesian inference over an intractable posterior into a tractable sum of a reconstruction term and a regularization term, and the reparameterization trick replaces stochastic sampling from $q_\phi(z|x)$ by a differentiable transformation $z = \mu_\phi(x) + \sigma_\phi(x)\epsilon$ with $\epsilon \sim \mathcal{N}(0,I)$, which lets gradients propagate to the encoder. The tutorial also relies on the manifold hypothesis, supported by the volume-concentration calculation in Appendix A, as the geometric reason why low-dimensional latent variables can summarize brain scans.
What would settle it
Train a VAE on a large set of structural MRI scans and inspect the latent space: if the latent variables show no significant correlation with known biological variables (age, ventricle size, diagnosis) while reconstruction quality remains high, the tutorial's claim that VAEs produce clinically meaningful representations is falsified for that setting.
Extended reading notes
Core claim
The paper claims that the variational autoencoder's value for neuroimaging rests on the manifold hypothesis: high-dimensional brain scans concentrate near a lower-dimensional latent manifold, so a probabilistic encoder-decoder can capture the biologically meaningful axes of variation. The mathematical core is the ELBO inequality, which turns the intractable marginal likelihood into an optimizable objective: the reconstruction term $\mathbb{E}_{q_\phi(z|x)}[\log p(x|z)]$ plus a regularizer $-\mathrm{KL}(q_\phi(z|x)\,\|\,p(z))$. Because $q_\phi$ is a Gaussian whose mean and variance are outputs of a convolutional encoder, the reparameterization trick $z = \mu_\phi(x) + \sigma_\phi(x)\epsilon$ makes gradient flow possible. The paper then argues that the resulting latent space is interpretable and can be analyzed with linear or generalized linear models to associate latent coordinates with clinical scores, as illustrated by studies that fuse structural and functional data, recover Parkinson's UPDRS scores, map longitudinal fMRI dynamics, and predict brain-age trajectories.
Load-bearing premise
Everything in the tutorial assumes that real brain scans actually concentrate near a low-dimensional manifold, so the latent variables learned by a VAE correspond to meaningful biological variability; the paper states this assumption but gives no direct empirical check on neuroimaging data.
Editorial extensions
If this is right
- VAE-derived latent coordinates can be treated as statistical variables: after training, one can regress them against diagnosis or symptom scores with interpretable models like GLMs, rather than treating the encoder as a black box.
- If the decoder is too flexible, the ELBO objective alone will not guarantee that the latent space carries information; the tutorial shows that choosing a divergence that maximizes mutual information between x and z, such as the InfoVAE-style MMD-based ELBO with a large weight, is a remedy.
- The mean-attractor failure mode means practitioners should watch for reconstructions that are identical across subjects; that symptom identifies a model stuck at a local minimum where the approximate posterior collapses to the prior.
- For neuroimaging, replacing mean squared error with structural similarity (DSSIM) as the reconstruction loss should yield reconstructions that preserve anatomical structure, matching human perception of image quality better than pixel-wise loss.
Reading between the lines
- The paper stops short of claiming the latent space is identifiable; a natural extension is that VAE latent coordinates are only a useful representation up to the symmetries of the trained network, so cross-subject or cross-study comparisons should either fix the encoder or test invariance under re-training.
- The volume-concentration argument in Appendix A implies a concrete design heuristic the authors do not state: choosing the latent dimensionality roughly equal to the shell thickness of the data in the ambient space should balance keeping information and avoiding over-regularization; this could be tested by sweeping latent dimension against reconstruction and KL loss.
- A testable extension of the InfoVAE discussion: on a neuroimaging cohort, comparing latent variables trained with KL versus MMD objectives on their correlation with clinical scores and their anatomical interpretability would tell whether the information-preference fix actually improves biological insight.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript is a tutorial on variational autoencoders (VAEs) aimed at neuroimaging researchers. It derives the evidence lower bound (ELBO) from the marginal log-likelihood, explains the reparameterization trick, discusses practical issues such as posterior collapse and the mean local minimum, and reviews several applications of VAEs to neuroimaging data. The paper does not present new experimental results; its contribution is pedagogical, with the central derivation being Eq. (6), log p(x) ≥ E_q[log p(x|z)] − KL(q_φ(z|x)||p(z)).
Significance. If the presentational and technical issues are fixed, the tutorial offers a compact and mostly correct introduction to VAEs for the neuroimaging community. The ELBO derivation (Eqs. 3–6) and the reparameterization trick (Section 2.2.1) are standard and correctly explained. The paper usefully connects concepts such as posterior collapse and the mean local minimum to concrete neuroimaging failure modes, and Figure 2 provides a clear illustration of the mean local minimum. The applications section surveys four relevant recent works, giving readers pointers to the literature. The main weakness is the unsupported mathematical justification of the manifold hypothesis in Appendix A, which is used to motivate the entire approach; this needs substantial revision, along with a misstatement of the InfoVAE objective in Section 2.2.2.
major comments (3)
- [Appendix A, Eqs. (12)–(15)] The concentration-of-measure argument does not establish the manifold hypothesis. The shell fraction nε/R shows that volume concentrates near the boundary sphere, which is a codimension-1 surface of dimension n−1, not a lower-dimensional manifold. The concluding sentence of Appendix A, 'points tend to cluster close to a lower-dimensional manifold,' is therefore a non-sequitur. Since Section 1.1 explicitly cites Appendix A as mathematical support for the manifold hypothesis, this is a load-bearing gap in the tutorial's motivation. The authors should either remove the claim that Appendix A supports the manifold hypothesis, or replace it with a correct statement (e.g., concentration of measure implies most volume lies near the boundary, not near a low-dimensional set) and present the manifold hypothesis as an empirical assumption backed by references on intrinsic dimensionality estimation for neuroimaging data.
- [Section 2.2.2, Eq. (11)] The description of the InfoVAE objective is inaccurate. The text says that under some assumptions the ELBO can be rewritten as E_q[log p(x|z)] − λD(q_φ(z|x)||p(z)) 'with the constraint that λ → ∞' and that it is sufficient for λ to be of the same order as the log-likelihood. In the cited InfoVAE work [35], λ is a finite hyperparameter, typically chosen to balance reconstruction and regularization; there is no λ → ∞ constraint, and the 'same order' statement is not standard. This misstates a practical recommendation that the tutorial is explicitly providing. Please correct the description and cite the original formulation accurately.
- [Appendix A, Eq. (16)] Equation (16) is dimensionally inconsistent as written: it states r^n = x_1^2 + ... + x_{n+1}^2, which is not an equation for the radius of a hypersphere. It should be r^2 = x_1^2 + ... + x_{n+1}^2. Additionally, in Eq. (13) the notation 'R ∈ R^{n+1}' is incorrect; R is a positive scalar radius. These are mathematical typos in the only derivational appendix, which further weakens the already problematic argument in Appendix A.
minor comments (5)
- [Section 2.2.1] The word 'Pythoch' is a typo for 'PyTorch'.
- [Section 2.2.2] The text writes 'SSMI' where the intended acronym is 'SSIM' (Structural Similarity Index), and also writes '1 − SSMI' instead of '1 − SSIM'.
- [Section 2.1] There are several typos: 'both sites' should be 'both sides', 'paramererized' should be 'parameterized', and 'It is not hard to image' should be 'It is not hard to imagine'.
- [Section 2.3, bulleted list] The first bullet is garbled: 'F using F unctional and Structural Neuroimaging Data' should be 'Fusing Functional and Structural Neuroimaging Data'. Also, the numbering in the list is inconsistent: the second item begins with '2.' but contains a stray '3.' before the next item.
- [Title and Abstract] The title and abstract consistently write 'V AE' with a space, which should be 'VAE'.
Circularity Check
No circularity: the VAE derivation is standard, self-contained mathematics, and the sole self-citation is an application example that is not load-bearing.
full rationale
This paper is a tutorial whose central mathematical contribution is the derivation of the evidence lower bound (ELBO), Eqs. (4)-(6). That derivation is self-contained: it starts from the definition of the joint distribution p(x,z), introduces the variational posterior q_phi(z|x), takes expectations, and uses the non-negativity of the KL divergence to obtain the bound. Eq. (6) follows algebraically from Eq. (5) by writing p(x,z)=p(z)p(x|z), so no fitted parameter is renamed as a prediction and no result is assumed to prove itself. The only self-citation, Ref. [42], appears in the applications review section describing the authors' own cross-modality VAE for Parkinson's disease; it is presented as one example among several external works ([41], [43], [44]) and is not used to justify the theoretical derivation. Appendix A's concentration-of-measure argument may be a weak or incomplete justification of the manifold hypothesis for neuroimaging -- the hypersphere shell calculation establishes surface concentration for uniform distributions in high-dimensional balls, not the intrinsic dimensionality of real brain scans -- but that is a gap in empirical support, not a circular reduction. The tutorial even concedes that real-world latent manifolds are more complex. Accordingly, no step in the paper's derivation chain is equivalent to its own input by construction, and the appropriate circularity score is 0.
Assumptions & free parameters
assumptions (3)
- standard math Bayes rule and KL divergence properties (non-negativity)
- domain assumption Manifold hypothesis: high-dimensional data concentrate near lower-dimensional manifolds
- standard math Reparameterization trick differentiability
Cite this review
Pith. "Pith review of Tutorial: VAE as an inference paradigm for neuroimaging." pith.science (2026). https://pith.science/paper/6WBQXNYF
@misc{pith2026250108009,
author = {Pith},
title = {Pith review of: Tutorial: VAE as an inference paradigm for neuroimaging},
year = {2026},
howpublished = {\url{https://pith.science/paper/6WBQXNYF}},
note = {Machine review of arXiv:2501.08009}
}
read the original abstract
In this tutorial, we explore Variational Autoencoders (VAEs), an essential framework for unsupervised learning, particularly suited for high-dimensional datasets such as neuroimaging. By integrating deep learning with Bayesian inference, VAEs enable the generation of interpretable latent representations. This tutorial outlines the theoretical foundations of VAEs, addresses practical challenges such as convergence issues and over-fitting, and discusses strategies like the reparameterization trick and hyperparameter optimization. We also highlight key applications of VAEs in neuroimaging, demonstrating their potential to uncover meaningful patterns, including those associated with neurodegenerative processes, and their broader implications for analyzing complex brain data.
Figures
Reference graph
Works this paper leans on
-
[42]
Crist´ obal V´ azquez-Garc ´ ıa, Francisco Jes´ us Mart ´ ınez-Murcia, Juan E Arco, Ignacio A Ill´ an, Carmen Jim´ enez-Mesa, Javier Ram ´ ırez, and Juan M G´ orriz. A cross-modality latent representation for the prediction of clinical symptomatology in parkinson’s disease. In International Work-Conference on the Interplay Between Natural and Artificial C...
work page 2024
-
[35]
Infovae: Information maximizing variational autoencoders
Shengjia Zhao, Jiaming Song, and Stefano Ermon. Infovae: Information maximizing variational autoencoders. arXiv preprint arXiv:1706.02262 , 2017
arXiv 2017
-
[1]
Auto-encoding variational bayes
Diederik P Kingma. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013
arXiv 2013
-
[2]
Stochastic backpropagation and approximate inference in deep generative models
Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic backpropagation and approximate inference in deep generative models. In International conference on machine learning , pages 1278–1286. PMLR, 2014
work page 2014
-
[3]
Weight uncertainty in neural network
Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wier- stra. Weight uncertainty in neural network. In International conference on machine learning, pages 1613–1622. PMLR, 2015. 14
work page 2015
-
[4]
Long-term neural and physiological phenotyping of a single human
Russell A Poldrack, Timothy O Laumann, Oluwasanmi Koyejo, Brenda Gregory, Ashleigh Hover, Mei-Yen Chen, Krzysztof J Gorgolewski, Jeffrey Luci, Sung Jun Joo, Ryan L Boyd, et al. Long-term neural and physiological phenotyping of a single human. Nature communications, 6(1):8885, 2015
work page 2015
-
[5]
Adam: A method for stochastic optimization
Diederik P Kingma. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014
arXiv 2014
-
[6]
Automatic differentiation variational inference
Alp Kucukelbir, Dustin Tran, Rajesh Ranganath, Andrew Gelman, and David M Blei. Automatic differentiation variational inference. Journal of machine learning research, 18(14):1–45, 2017
work page 2017
Show all 46 references
-
[7]
Machine learning for neuroimaging with scikit-learn
Alexandre Abraham, Fabian Pedregosa, Michael Eickenberg, Philippe Gervais, Andreas Mueller, Jean Kossaifi, Alexandre Gramfort, Bertrand Thirion, and Ga¨ el Varoquaux. Machine learning for neuroimaging with scikit-learn. Frontiers in neuroinformatics, 8:71792, 2014
2014
-
[8]
The cognitive neuroscience of working memory
Mark D’Esposito and Bradley R Postle. The cognitive neuroscience of working memory. Annual review of psychology , 66(1):115–142, 2015
2015
-
[9]
The kdd process for extracting useful knowledge from volumes of data
Usama Fayyad, Gregory Piatetsky-Shapiro, and Padhraic Smyth. The kdd process for extracting useful knowledge from volumes of data. Communi- cations of the ACM , 39(11):27–34, 1996
1996
-
[10]
A global geometric framework for nonlinear dimensionality reduction
Joshua B Tenenbaum, Vin de Silva, and John C Langford. A global geometric framework for nonlinear dimensionality reduction. science, 290(5500):2319–2323, 2000
2000
-
[11]
Reducing the dimension- ality of data with neural networks
Geoffrey E Hinton and Ruslan R Salakhutdinov. Reducing the dimension- ality of data with neural networks. science, 313(5786):504–507, 2006
2006
-
[12]
Generative adversarial nets
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde- Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems , 27, 2014
2014
-
[13]
High-resolution image synthesis with latent diffusion mod- els
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨ orn Ommer. High-resolution image synthesis with latent diffusion mod- els. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022
2022
-
[14]
Modern multidimensional scaling: Theory and applications
Ingwer Borg and Patrick JF Groenen. Modern multidimensional scaling: Theory and applications . Springer Science & Business Media, 2007
2007
-
[15]
Nonlinear dimensionality reduction by locally linear embedding
Sam T Roweis and Lawrence K Saul. Nonlinear dimensionality reduction by locally linear embedding. science, 290(5500):2323–2326, 2000
2000
-
[16]
Deep learning
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature, 521(7553):436–444, 2015. 15
2015
-
[17]
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 770–778, 2016
2016
-
[18]
Cross-orientation suppression in visual area v2
Ryan J Rowekamp and Tatyana O Sharpee. Cross-orientation suppression in visual area v2. Nature communications, 8(1):15739, 2017
2017
-
[19]
On information and sufficiency
Solomon Kullback and Richard A Leibler. On information and sufficiency. The annals of mathematical statistics , 22(1):79–86, 1951
1951
-
[20]
What’s wrong with mean-squared error? In Digital images and human vision , pages 207–220
Bernd Girod. What’s wrong with mean-squared error? In Digital images and human vision , pages 207–220. 1993
1993
-
[21]
Image quality measures and their performance
Ahmet M Eskicioglu and Paul S Fisher. Image quality measures and their performance. IEEE Transactions on communications , 43(12):2959–2965, 1995
1995
-
[22]
A universal image quality index
Zhou Wang and Alan C Bovik. A universal image quality index. IEEE signal processing letters, 9(3):81–84, 2002
2002
-
[23]
Image quality assessment: from error visibility to structural similarity
Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing , 13(4):600–612, 2004
2004
-
[24]
A comprehensive guide to bayesian convolutional neural network with variational inference
Kumar Shridhar, Felix Laumann, and Marcus Liwicki. A comprehensive guide to bayesian convolutional neural network with variational inference. arXiv preprint arXiv:1901.02731 , 2019
1901 arXiv
-
[25]
Facenet: A unified embedding for face recognition and clustering
Florian Schroff, Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 815–823, 2015
2015
-
[26]
You only look once: Unified, real-time object detection
J Redmon. You only look once: Unified, real-time object detection. In Proceedings of the IEEE conference on computer vision and pattern recog- nition, 2016
2016
-
[27]
U-net: Convolu- tional networks for biomedical image segmentation
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolu- tional networks for biomedical image segmentation. In Medical image com- puting and computer-assisted intervention–MICCAI 2015: 18th interna- tional conference, Munich, Germany, October 5-9, 2015, proceedings,...
2015
-
[28]
A survey on deep learning in medical image analysis
Geert Litjens, Thijs Kooi, Babak Ehteshami Bejnordi, Arnaud Arindra Adiyoso Setio, Francesco Ciompi, Mohsen Ghafoorian, Jeroen Awm Van Der Laak, Bram Van Ginneken, and Clara I S´ anchez. A survey on deep learning in medical image analysis. Medical image analysis, 42:60–88, 2017
2017
-
[29]
Monte carlo statistical methods, 1999
CP Robert. Monte carlo statistical methods, 1999. 16
1999
-
[30]
An introduction to variational autoencoders
Diederik P Kingma, Max Welling, et al. An introduction to variational autoencoders. Foundations and Trends® in Machine Learning, 12(4):307– 392, 2019
2019
-
[31]
Pytorch: An imperative style, high-performance deep learning library
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Brad- bury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems...
2019
-
[32]
Multilayer feedfor- ward networks are universal approximators.Neural networks, 2(5):359–366, 1989
Kurt Hornik, Maxwell Stinchcombe, and Halbert White. Multilayer feedfor- ward networks are universal approximators.Neural networks, 2(5):359–366, 1989
1989
-
[33]
Neural networks and the bias/variance dilemma
Stuart Geman, Elie Bienenstock, and Ren´ e Doursat. Neural networks and the bias/variance dilemma. Neural computation, 4(1):1–58, 1992
1992
-
[34]
Variational lossy autoencoder
Xi Chen, Diederik P Kingma, Tim Salimans, Yan Duan, Prafulla Dhari- wal, John Schulman, Ilya Sutskever, and Pieter Abbeel. Variational lossy autoencoder. arXiv preprint arXiv:1611.02731 , 2016
2016 arXiv
-
[36]
Understanding deep learning (still) requires rethinking generaliza- tion
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning (still) requires rethinking generaliza- tion. Communications of the ACM , 64(3):107–115, 2021
2021
-
[37]
On the importance of initialization and momentum in deep learning
Ilya Sutskever, James Martens, George Dahl, and Geoffrey Hinton. On the importance of initialization and momentum in deep learning. In Interna- tional conference on machine learning , pages 1139–1147. PMLR, 2013
2013
-
[38]
Deep learning and the information bottleneck principle
Naftali Tishby and Noga Zaslavsky. Deep learning and the information bottleneck principle. In 2015 ieee information theory workshop (itw) , pages 1–5. IEEE, 2015
2015
-
[39]
Please help! i’m stuck in a local minimum and how i climbed out, 2024
Linh V Nguyen. Please help! i’m stuck in a local minimum and how i climbed out, 2024. Accedido: 2024-10-08
2024
-
[40]
beta-vae: Learning basic visual concepts with a constrained variational framework
Irina Higgins, Loic Matthey, Arka Pal, Christopher P Burgess, Xavier Glo- rot, Matthew M Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-vae: Learning basic visual concepts with a constrained variational framework. ICLR (Poster) , 3, 2017
2017
-
[41]
Fusing multimodal neuroimaging data with a varia- tional autoencoder
Eloy Geenjaar, Noah Lewis, Zening Fu, Rohan Venkatdas, Sergey Plis, and Vince Calhoun. Fusing multimodal neuroimaging data with a varia- tional autoencoder. In 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) , pages 3630–
2021
-
[43]
Deep variational autoencoder for mapping functional brain networks
Ning Qiang, Qinglin Dong, Fangfei Ge, Hongtao Liang, Bao Ge, Shu Zhang, Yifei Sun, Jie Gao, and Tianming Liu. Deep variational autoencoder for mapping functional brain networks. IEEE Transactions on Cognitive and Developmental Systems , 13(4):841–852, 2020
2020
-
[44]
An image feature mapping model for continuous longitudinal data completion and generation of syn- thetic patient trajectories
Cl´ ement Chadebec, Evi MC Huijben, Josien PW Pluim, St´ ephanie Allas- sonni` ere, and Maureen AJM van Eijnatten. An image feature mapping model for continuous longitudinal data completion and generation of syn- thetic patient trajectories. In MICCAI Workshop on Deep Generati...
2022
-
[45]
The concentration of measure phenomenon
Michel Ledoux. The concentration of measure phenomenon . Number 89. American Mathematical Soc., 2001
2001
-
[46]
High-dimensional data analysis: The curses and blessings of dimensionality
David L Donoho et al. High-dimensional data analysis: The curses and blessings of dimensionality. AMS math challenges lecture, 1(2000):32, 2000. 18
2000
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.