REVIEW 5 major objections 4 minor 1 cited by
Self-Clustering Graph Transformer Approach to Model Resting-State Functional Brain Activity
T0 review · 5 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read This paper claims that a graph transformer whose attention parameters are functions of Laplacian positional encodings learns cluster-specific update rules, improves fMRI-based cognitive-score and sex prediction, and yields interpretable…
desk verdict SCGT's attention parameterization is a legitimate new idea, but the interpretability claim uses the wrong matrix and the gains are not yet validated. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the self-clustering graph attention block. It starts from node features $\hat h_i = h_i+\lambda_i$, where $\lambda_i$ is the Laplacian positional encoding (eigenvectors of the normalized graph Laplacian). The key, query, and value transformations are not shared globally: they are parameterized as functions of $\lambda_i$ through a shared matrix $\theta_1^{(l)}\in\mathbb{R}^{N\times k_r}$, followed by ReLU and separate projections, so that nodes in different positions can adopt different attention weights. The number $k_r=7$ is chosen to match seven canonical functional networks, and the same $\theta_1^{(l)}$ matrix is reused for interpretation: assigning each node to $\arg\max_c\theta_1^{(l)}[i,c]$ yields the learned subclusters.
What would settle it
Train the same architecture with $\theta_1$ fixed to a random matrix (or replaced by a single global parameter vector shared by all nodes) and measure test MSE and correlation on the same split; if performance does not drop, the self-clustering mechanism is not load-bearing. For the interpretability claim, compute the overlap between the argmax community assignments from $\theta_1$ and an independent seven-network atlas; chance-level overlap would falsify the claim that the learned subclusters align with known functional networks.
Extended reading notes
Core claim
The central claim is that conditioning the attention parameter matrices on Laplacian positional encodings through a shared matrix $\theta_1^{(l)}$ produces cluster-specific update rules, and that this self-clustering is what lets the model capture brain subnetworks. Concretely, in layer $l$ each node's key, query, and value vectors are computed with parameter vectors $w_i^{(k,l)} = \theta_{k2}^{(l)}\mathrm{ReLU}(\theta_1^{(l)}\lambda_i)+b_k^{(l)}$ (and analogously for query and value), so nodes with different positional encodings get different attention projections. The authors report that on the resting-state functional connectivity data of 7,957 participants this lowers cognitive-score prediction MSE from 283.43 (vanilla graph transformer) to 271.02, raises correlation from 0.25 to 0.283, and improves sex classification from 82.3% to 84.5% accuracy, while also beating BrainGNN, BrainNetCNN, and SVR baselines. They further claim that the rows of $\theta_1^{(l)}$, when each node is assigned to its maximum community, produce clusters that align with known functional networks such as the default mode and frontoparietal networks.
Load-bearing premise
The method's advantage rests on the assumption that making attention parameters depend on positional encodings through one shared matrix really gives nodes in different clusters different update rules, and that this difference, not merely the extra parameters, explains the reported improvements.
Editorial extensions
If this is right
- If the reported numbers hold, SCGT is the best of the compared models on this protocol for cognitive-score prediction, with MSE 271.02 versus 283.43 for the vanilla graph transformer.
- Gender classification from the same 100-node functional connectivity graph rises to 84.5% accuracy and F1 0.86, up from 82.3% and 0.84.
- The model produces a seven-community partition of the 100 brain regions from the shared matrix, so prediction and parcellation come from the same learned object.
- Because the self-clustering block is defined for any graph, the same idea extends to other graphs with subnetwork structure, as the authors state they intend to test.
Reading between the lines
- One check the paper leaves open is ablating $\theta_1$: replacing it with a fixed random matrix, or with one global parameter vector, would show whether the cluster-specific mechanism itself, rather than extra parameters or the positional encodings, drives the reported gains.
- The interpretability claim is visual; a quantitative overlap score between the $\theta_1$-derived assignments and an independent seven-network atlas would give a measurable test.
- An extension the authors do not explore is learning the number of communities from data instead of fixing it at seven, which would show whether the seven-network prior is necessary for the improvement.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces the Self-Clustering Graph Transformer (SCGT), an attention mechanism for graphs with subnetworks. The key/query/value parameters are made functions of Laplacian positional encodings through a shared matrix θ1, with the intent of performing cluster-specific node updates. The model is evaluated on the ABCD dataset (7,957 participants) for predicting total cognitive score and classifying gender, reporting improved MSE and correlation over a vanilla graph transformer and several baselines. The paper also claims that the learned θ1 matrix yields interpretable brain subclusters aligning with known functional networks such as the default mode network. The manuscript is preliminary and identifies future work on broader applicability.
Significance. The problem addressed — uniform node updates in graph transformers applied to brain functional connectivity — is relevant, and the idea of making attention parameters position-dependent is a plausible direction. The study uses a large dataset (ABCD, n=7,957) and compares against several baselines, which are strengths. However, the central claims currently rest on an interpretability step that is inconsistent with the forward-pass equations, a lack of ablations isolating the clustering mechanism, missing statistical significance testing, and an incomplete baseline comparison. If corrected and validated, the approach could be a useful contribution to interpretable graph transformers for neuroimaging, but in its present form the evidence is not sufficient to support the stated conclusions.
major comments (5)
- [Section IV] The interpretability assignment is inconsistent with the forward-pass use of θ1. In Eqs. (3)-(5), θ1 ∈ R^{N×kr} is applied to the positional encoding vector λ_i ∈ R^N, so the cluster-score vector for node i is (θ1 λ_i) ∈ R^{kr}, and the rows of θ1 index Laplacian eigenvector coordinates, not brain regions. The assignment ci = argmax_c θ1[i,c] is therefore not the quantity used in the model, and the claimed alignment of the learned clusters with Yeo's seven networks is unsupported. Please use ci = argmax_c (θ1 λ_i)_c and quantitatively validate the resulting assignment against a reference parcellation.
- [Section III.B.1 and Table I] BrainRGIN is listed as a baseline in Section III.A, but no results for BrainRGIN appear in Table I or anywhere in the results. Without these results, the claim of outperforming 'other recent models' is incomplete. Please add the missing results or explicitly explain their omission.
- [Section III.B.1, Table I] No statistical significance tests are reported. The SCGT advantage over BrainGNN (271.02 vs 272.02/273.02 MSE) is within one standard deviation of the reported fold-level variation (SCGT 271.02±3.21, BrainGNN 273.02±3.01), so the text's characterization of 'significant improvement' is not supported. Report per-fold results, confidence intervals, and appropriate paired significance tests. The table also has an internal inconsistency: BrainGNN's Avg. MSE is 272.02 while its MSE±Std Dev is 273.02±3.01; please correct.
- [Section II.C.2.b and Section III.B] The design does not isolate the self-clustering mechanism. The gain of SCGT over the vanilla graph transformer (MSE 271.02 vs 283.43) could be due to the extra parameters introduced in Eqs. (3)-(5) or to the Laplacian positional encodings, rather than to any learned cluster-specific update. Please include ablations, for example: (i) a variant with θ1 randomized or frozen, (ii) a variant where the per-node parameter modulation is applied without any clustering interpretation, and (iii) a variant with an explicit cluster-wise aggregation step.
- [Section II.C.2.b, Eqs. (3)-(8)] The equations are dimensionally underspecified. λ_i ∈ R^N and θ1 ∈ R^{N×kr} imply θ1 λ_i ∈ R^{kr}; it is unclear how this kr-dimensional vector becomes the key/query/value parameter matrices used in Eqs. (6)-(8), which act on N-dimensional node features (or the hidden dimension D). Please specify the shapes of θ_k2, θ_q2, θ_v2, and explain how w_i^(k) is reshaped or broadcast to form W^(K)(l), W^(Q)(l), and W^(V)(l).
minor comments (4)
- [Abstract vs Section I] The abstract states that SCGT 'outperforms the vanilla graph transformer method and other recent models,' while Section I states that SCGT 'outperforms the vanilla graph transformer method and is competitive with other baseline models.' Please align the wording with the actual results.
- [Equation (1)] The adjacency matrix Ak is written with a superscript k that is not used in the rest of the paper; likely a typo for A. Also, the definition of the thresholded element e_i,j should be made precise (e.g., threshold value and whether negative correlations are included).
- [Section III.B.2, Table II] The gender classification comparison reports only point estimates (84.5% vs 82.3% accuracy) without variance, confidence intervals, or significance tests. The phrase 'marginal yet significant' is unsupported; please provide per-fold results and a significance test.
- [Section III.A] The hyperparameters are stated as D=64, kr=7, and two layers, but Eq. (9) also uses K attention heads or clusters. Please clarify the relationship between K, kr, and the number of attention heads.
Circularity Check
Predictive comparisons are independent, but the interpretability claim reduces to a preset Yeo-derived kr and the paper's own labeling of theta_1 rows as communities.
-
self definitional
[Section II.C.2.b (Eq. 3) and Section IV; hyperparameter choice in Section III.A]
"The number of clustered communities (kr) was set to 7, inspired by the seven functional networks defined by Yeo et al. [17]. ... This interpretability is facilitated by the shared parameter matrix theta^(l)_1, which captures the association between each of the 100 brain regions (nodes) and the 7 predefined functional communities. ... ci = argmax_{c in {1,...,7}} theta^(l)_1[i,c]. By assigning each node to the community with the highest score, we obtain distinct functional clusters that align with known brain networks, such as the default mode network or frontoparietal network [17]."
The cluster count kr is imported from Yeo's seven-network parcellation, and theta_1 is declared to be the node-to-community association matrix. Section IV then reads the clusters directly off theta_1 rows and asserts alignment with the same Yeo networks, without any external overlap metric or ablation. Moreover, in the forward pass (Eq. 3) theta_1 is applied to positional encodings (theta_1 * lambda_i), so the argmax over theta_1 rows is not the quantity that determines per-node attention weights. The 'learned subclusters' reported as aligning with known networks are therefore a relabeling of a fitted matrix plus the preset Yeo-inspired kr, rather than a prediction independently derived from the clustering mechanism.
full rationale
The central predictive claim (SCGT vs vanilla GT and other baselines on the ABCD test split) is an external empirical comparison and does not reduce to any fitted parameter or self-citation. Author self-citations appear only as background or baseline references and are not load-bearing. The partial circularity is confined to the interpretability section: kr is set to 7 because of Yeo's seven-network parcellation, theta_1 is described post-hoc as a node-community association matrix, and the resulting seven clusters are said to align with Yeo's known networks without a quantitative overlap test. This makes the interpretability claim self-referential rather than a derived result. A separate, non-circular validity concern is the dimensional mismatch between Eq. (3), where theta_1 is applied to lambda_i, and Section IV, where argmax over theta_1 rows defines clusters; this weakens the interpretability claim but is not itself a circular reduction. Overall, the predictive results have independent content, so the circularity score is moderate and localized.
Assumptions & free parameters
free parameters (4)
- kr (number of communities) =
7
- D (output feature dimension) =
64
- Number of Laplacian eigenvectors used as positional encodings =
100
- K (number of attention heads or clusters in Eq. 9) =
not reported
assumptions (5)
- domain assumption Neuromark fMRI 1.0 template's 100 spatially constrained ICA components are valid functional nodes for all subjects.
- domain assumption Static FC correlation features plus Laplacian positional encodings are sufficient to learn cognitive and demographic signals from rs-fMRI.
- domain assumption Yeo's seven-network parcellation is the correct cluster count for the brain's functional organization.
- domain assumption Regression of age and site from cognitive scores removes confounds, and no other confounds (head motion, family structure) affect the FC-outcome association.
- domain assumption Laplacian eigenvectors provide positional encodings that allow meaningful per-node parameter specialization.
invented entities (1)
-
Learned cluster-assignment matrix theta_1
Cite this review
Pith. "Pith review of Self-Clustering Graph Transformer Approach to Model Resting-State Functional Brain Activity." pith.science (2026). https://pith.science/paper/HQAPORCE
@misc{pith2026250116345,
author = {Pith},
title = {Pith review of: Self-Clustering Graph Transformer Approach to Model Resting-State Functional Brain Activity},
year = {2026},
howpublished = {\url{https://pith.science/paper/HQAPORCE}},
note = {Machine review of arXiv:2501.16345}
}
read the original abstract
Resting-state functional magnetic resonance imaging (rs-fMRI) offers valuable insights into the human brain's functional organization and is a powerful tool for investigating the relationship between brain function and cognitive processes, as it allows for the functional organization of the brain to be captured without relying on a specific task or stimuli. In this study, we introduce a novel attention mechanism for graphs with subnetworks, named Self-Clustering Graph Transformer (SCGT), designed to handle the issue of uniform node updates in graph transformers. By using static functional connectivity (FC) correlation features as input to the transformer model, SCGT effectively captures the sub-network structure of the brain by performing cluster-specific updates to the nodes, unlike uniform node updates in vanilla graph transformers, further allowing us to learn and interpret the subclusters. We validate our approach on the Adolescent Brain Cognitive Development (ABCD) dataset, comprising 7,957 participants, for the prediction of total cognitive score and gender classification. Our results demonstrate that SCGT outperforms the vanilla graph transformer method and other recent models, offering a promising tool for modeling brain functional connectivity and interpreting the underlying subnetwork structures.
Figures
Forward citations
Cited by 1 Pith paper
-
Fast Clustering of Categorical Big Data
Bisecting K-Modes initializes K-Modes clustering via hierarchical splitting and shows promising speed and quality on large categorical datasets, but the experimental validation is limited.
Reference graph
Works this paper leans on
-
[1]
Resting-state fMRI: A review of methods and clinical applications,
M.H. Lee, C.D. Smyser, and J.S. Shimony, “Resting-state fMRI: A review of methods and clinical applications,” American Journal of Neuroradiology, vol. 34, no. 10, pp. 1866–1872, Aug. 2012
work page 2012
-
[2]
Ram Sapkota, Bishal Thapaliya, Pranav Suresh, Bhaskar Ray, Vince D. Calhoun, and Jingyu Liu, “Multimodal imaging feature extraction with reference canonical correlation analysis underlying intelligence,” in ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . Apr. 2024, p. 2071–2075, IEEE
work page 2024
-
[3]
Effective training strategy for NN models of working memory classification with limited samples,
Pranav Suresh, Bhaskar Ray, Bishal Thapaliya, Britny Farahdel, Behnam Kazemivash, Jiayu Chen, Kuaikuai Duan, Vince D. Calhoun, and Jingyu Liu, “Effective training strategy for NN models of working memory classification with limited samples,” in 2023 IEEE 20th International Symposium on Biomedical Imaging (ISBI) . Apr. 2023, IEEE
work page 2023
-
[4]
Replication and refinement of brain age model for adolescent development,
Bhaskar Ray, Jiayu Chen, Zening Fu, Pranav Suresh, Bishal Thapaliya, Britny Farahdel, Vince D. Calhoun, and Jingyu Liu, “Replication and refinement of brain age model for adolescent development,” Aug. 2023
work page 2023
-
[5]
Dsam: A deep learning framework for analyzing temporal and spatial dynamics in brain networks,
Bishal Thapaliya, Robyn Miller, Jiayu Chen, Yu Ping Wang, Esra Akbas, Ram Sapkota, Bhaskar Ray, Pranav Suresh, Santosh Ghimire, Vince D. Calhoun, and Jingyu Liu, “Dsam: A deep learning framework for analyzing temporal and spatial dynamics in brain networks,” Medical Image Analysis , vol. 101, pp. 103462, Apr. 2025
work page 2025
-
[6]
Michael A. Ferguson, Jeffrey S. Anderson, and R. Nathan Spreng, “Fluid and flexible minds: Intelligence reflects synchrony in the brain’s intrinsic network architecture,” Network Neuroscience, vol. 1, no. 2, pp. 192–207, June 2017
work page 2017
-
[7]
Tong He, Ru Kong, Avram J. Holmes, Minh Nguyen, Mert R. Sabuncu, Simon B. Eickhoff, Danilo Bzdok, Jiashi Feng, and B.T. Thomas Yeo, “Deep neural networks and kernel regression achieve comparable accuracies for functional connectivity prediction of behavior and demographics,” NeuroImage, vol. 206, pp. 116276, Feb. 2020
work page 2020
-
[8]
Julien Dubois, Paola Galdi, Lynn K. Paul, and Ralph Adolphs, “A distributed brain network predicts general intelligence from resting- state human neuroimaging data,” Philosophical Transactions of the Royal Society B: Biological Sciences , vol. 373, no. 1756, pp. 20170284, Aug. 2018
work page 2018
Show all 19 references
-
[9]
Functional connectomics from resting-state fMRI,
Stephen M. Smith, Diego Vidaurre, Christian F. Beckmann, Matthew F. Glasser, Mark Jenkinson, Karla L. Miller, Thomas E. Nichols, Emma C. Robinson, Gholamreza Salimi-Khorshidi, Mark W. Wool- rich, Deanna M. Barch, Kamil U ˘gurbil, and David C. Van Essen, “Functional connectomic...
2013
-
[10]
Graph-based deep learning models in the prediction of early-stage alzheimers,
Bishal Thapaliya, Zundong Wu, Ram Sapkota, Bhaskar Ray, Pranav Suresh, Santosh Ghimire, Vince Calhoun, and Jingyu Liu, “Graph-based deep learning models in the prediction of early-stage alzheimers,” in 2024 46th Annual International Conference of the IEEE Engineering in Medici...
2024
-
[11]
BrainGNN: Interpretable brain graph neural network for fMRI analysis,
Xiaoxiao Li, Yuan Zhou, Nicha Dvornek, Muhan Zhang, Siyuan Gao, Juntang Zhuang, Dustin Scheinost, Lawrence H. Staib, Pamela Ventola, and James S. Duncan, “BrainGNN: Interpretable brain graph neural network for fMRI analysis,” Medical Image Analysis , vol. 74, pp. 102233, Dec. 2021
2021
-
[12]
Brain networks and intelligence: A graph neural network based approach to resting state fmri data,
Bishal Thapaliya, Esra Akbas, Jiayu Chen, Ram Sapkota, Bhaskar Ray, Pranav Suresh, Vince D. Calhoun, and Jingyu Liu, “Brain networks and intelligence: A graph neural network based approach to resting state fmri data,” Medical Image Analysis , vol. 101, pp. 103433, Apr. 2025
2025
-
[13]
NeuroMark: An automated and adaptive ICA based pipeline to identify reproducible fMRI markers of brain disorders,
Yuhui Du, Zening Fu, Jing Sui, Shuang Gao, Ying Xing, Dongdong Lin, Mustafa Salman, Anees Abrol, Md Abdur Rahaman, Jiayu Chen, L. Elliot Hong, Peter Kochunov, Elizabeth A. Osuch, and Vince D. Calhoun, “NeuroMark: An automated and adaptive ICA based pipeline to identify reprodu...
2020
-
[14]
A generalization of transformer networks to graphs,
Vijay Prakash Dwivedi and Xavier Bresson, “A generalization of transformer networks to graphs,” 2020
2020
-
[15]
Pytorch: An imperative style, high-performance deep learning library,
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas K ¨opf, Edward Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chil- amkurthy, Benoit Steiner, Lu...
2019
-
[16]
Fast graph representation learning with pytorch geometric,
Matthias Fey and Jan Eric Lenssen, “Fast graph representation learning with pytorch geometric,” 2019
2019
-
[17]
The organization of the human cerebral cortex estimated by intrinsic functional connectivity,
B. T. Thomas Yeo, Fenna M. Krienen, Jorge Sepulcre, Mert R. Sabuncu, Danial Lashkari, Marisa Hollinshead, Joshua L. Roffman, Jordan W. Smoller, Lilla Z ¨ollei, Jonathan R. Polimeni, Bruce Fischl, Hesheng Liu, and Randy L. Buckner, “The organization of the human cerebral cortex...
2011
-
[18]
Adam: A method for stochastic optimization,
Diederik P. Kingma and Jimmy Ba, “Adam: A method for stochastic optimization,” 2014
2014
-
[19]
Brainnetcnn: Convolutional neural networks for brain networks; towards predicting neurodevelopment,
Jeremy Kawahara, Colin J. Brown, Steven P. Miller, Brian G. Booth, Vann Chau, Ruth E. Grunau, Jill G. Zwicker, and Ghassan Hamarneh, “Brainnetcnn: Convolutional neural networks for brain networks; towards predicting neurodevelopment,” NeuroImage, vol. 146, pp. 1038–1049, Feb. 2017
2017
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.