{"id":"f4e38409-0cd2-43a7-848d-f88e655f9d4d","arxiv_id":"2501.16345","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A self-clustering graph transformer that tailors attention per brain subnetwork slightly improves cognitive score and gender prediction from resting-state fMRI and yields interpretable network assignments.","lead":"A brain-imaging study introduces a graph transformer whose attention weights adapt to each brain region's network, rather than treating all regions identically. Trained on 7,957 adolescents, it modestly improves prediction of cognitive scores and gender over a standard graph transformer, and it can visualize region-to-network assignments.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The self-clustering mechanism is not actually validated: the θ1 matrix used for interpretability (Section IV) is not the quantity that Eq. 3 applies to node PEs, and no ablation isolates cluster-specific updates.","rationale":"I agree with the reader that the weakest assumption concerns the self-clustering mechanism, but my stress-test sharpens it into an internal inconsistency. With θ1 ∈ R^{N×kr} and λ_i ∈ R^N, Eq. 3 computes community scores as a linear combination of PE dimensions, so the row index of θ1 is not the node index. Section IV's row-argmax is therefore not the model's cluster assignment. This is not merely a missing ablation: the interpretability claim in the abstract and Section IV is tied to a quantity that the forward pass does not use in the way stated. The reported performance gap is modest (MSE 271.02 vs 283.43), with no significance testing and no code or data released; the baseline table also omits BrainRGIN despite citing it. Independent support is limited to a small number of descriptive results and a visual alignment assertion. I recommend keeping the conditional verdict, with conditions that include fixing the θ1 interpretation, recomputing cluster alignment against an external parcellation, and providing an ablation that isolates the self-clustering component from added parameters and positional encodings.","tokens_in":7043,"tokens_out":8846,"duration_ms":95865,"concrete_test":"Fix N=100, kr=7, and let λ_i be row i of the 100 Laplacian eigenvectors. Recompute the Section IV assignment two ways: (a) c_i = argmax_c θ1[i,c] as printed; (b) c_i = argmax_c (θ1 λ_i)_c as implied by Eq. (3). Compare both to the Yeo 7-network labels with adjusted Rand index. If (b) does not align with Yeo while (a) appears to, the paper's interpretation has been obtained from the wrong quantity. In the same script, run an ablation: freeze θ1 to random values versus Yeo one-hot versus learned; if test MSE and correlation do not change across these conditions, the self-clustering component is not the driver of the reported improvement.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Load-bearing concern: the defining mechanism of SCGT, as written, does not support the paper's two central claims (cluster-specific updates and interpretable subnetworks). In Eqs. (3)-(5), θ1 ∈ R^{N×kr} is applied to the positional encoding vector λ_i ∈ R^N, so the resulting kr-dimensional vector (θ1 λ_i) is a linear combination of Laplacian eigenvector coordinates. The rows of θ1 therefore index PE coordinates, not brain regions. Yet Section IV assigns node i to argmax_c θ1[i,c], treating θ1 rows as node-to-community affinities. That is not the quantity used in the forward pass; the correct per-node community score would be argmax_c (θ1 λ_i)_c. The paper provides no argument that these two assignments coincide, and for genuine Laplacian eigenvectors they generally will not. In addition, the forward pass never forms an explicit cluster assignment or cluster-wise aggregation; it only modulates per-head K/Q/V projections by a function of λ_i. Thus the reported gain over vanilla GT (MSE 271.02 vs 283.43) could be due to extra parameters or PE conditioning rather than to learned self-clustering. The absence of an ablation with θ1 fixed or randomized leaves this unresolved.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces the Self-Clustering Graph Transformer (SCGT), an attention mechanism for graphs with subnetworks. The key/query/value parameters are made functions of Laplacian positional encodings through a shared matrix θ1, with the intent of performing cluster-specific node updates. The model is evaluated on the ABCD dataset (7,957 participants) for predicting total cognitive score and classifying gender, reporting improved MSE and correlation over a vanilla graph transformer and several baselines. The paper also claims that the learned θ1 matrix yields interpretable brain subclusters aligning with known functional networks such as the default mode network. The manuscript is preliminary and identifies future work on broader applicability.","tokens_in":7256,"tokens_out":4924,"duration_ms":46468,"significance":"The problem addressed — uniform node updates in graph transformers applied to brain functional connectivity — is relevant, and the idea of making attention parameters position-dependent is a plausible direction. The study uses a large dataset (ABCD, n=7,957) and compares against several baselines, which are strengths. However, the central claims currently rest on an interpretability step that is inconsistent with the forward-pass equations, a lack of ablations isolating the clustering mechanism, missing statistical significance testing, and an incomplete baseline comparison. If corrected and validated, the approach could be a useful contribution to interpretable graph transformers for neuroimaging, but in its present form the evidence is not sufficient to support the stated conclusions.","major_comments":[{"comment":"The interpretability assignment is inconsistent with the forward-pass use of θ1. In Eqs. (3)-(5), θ1 ∈ R^{N×kr} is applied to the positional encoding vector λ_i ∈ R^N, so the cluster-score vector for node i is (θ1 λ_i) ∈ R^{kr}, and the rows of θ1 index Laplacian eigenvector coordinates, not brain regions. The assignment ci = argmax_c θ1[i,c] is therefore not the quantity used in the model, and the claimed alignment of the learned clusters with Yeo's seven networks is unsupported. Please use ci = argmax_c (θ1 λ_i)_c and quantitatively validate the resulting assignment against a reference parcellation.","section":"Section IV"},{"comment":"BrainRGIN is listed as a baseline in Section III.A, but no results for BrainRGIN appear in Table I or anywhere in the results. Without these results, the claim of outperforming 'other recent models' is incomplete. Please add the missing results or explicitly explain their omission.","section":"Section III.B.1 and Table I"},{"comment":"No statistical significance tests are reported. The SCGT advantage over BrainGNN (271.02 vs 272.02/273.02 MSE) is within one standard deviation of the reported fold-level variation (SCGT 271.02±3.21, BrainGNN 273.02±3.01), so the text's characterization of 'significant improvement' is not supported. Report per-fold results, confidence intervals, and appropriate paired significance tests. The table also has an internal inconsistency: BrainGNN's Avg. MSE is 272.02 while its MSE±Std Dev is 273.02±3.01; please correct.","section":"Section III.B.1, Table I"},{"comment":"The design does not isolate the self-clustering mechanism. The gain of SCGT over the vanilla graph transformer (MSE 271.02 vs 283.43) could be due to the extra parameters introduced in Eqs. (3)-(5) or to the Laplacian positional encodings, rather than to any learned cluster-specific update. Please include ablations, for example: (i) a variant with θ1 randomized or frozen, (ii) a variant where the per-node parameter modulation is applied without any clustering interpretation, and (iii) a variant with an explicit cluster-wise aggregation step.","section":"Section II.C.2.b and Section III.B"},{"comment":"The equations are dimensionally underspecified. λ_i ∈ R^N and θ1 ∈ R^{N×kr} imply θ1 λ_i ∈ R^{kr}; it is unclear how this kr-dimensional vector becomes the key/query/value parameter matrices used in Eqs. (6)-(8), which act on N-dimensional node features (or the hidden dimension D). Please specify the shapes of θ_k2, θ_q2, θ_v2, and explain how w_i^(k) is reshaped or broadcast to form W^(K)(l), W^(Q)(l), and W^(V)(l).","section":"Section II.C.2.b, Eqs. (3)-(8)"}],"minor_comments":[{"comment":"The abstract states that SCGT 'outperforms the vanilla graph transformer method and other recent models,' while Section I states that SCGT 'outperforms the vanilla graph transformer method and is competitive with other baseline models.' Please align the wording with the actual results.","section":"Abstract vs Section I"},{"comment":"The adjacency matrix Ak is written with a superscript k that is not used in the rest of the paper; likely a typo for A. Also, the definition of the thresholded element e_i,j should be made precise (e.g., threshold value and whether negative correlations are included).","section":"Equation (1)"},{"comment":"The gender classification comparison reports only point estimates (84.5% vs 82.3% accuracy) without variance, confidence intervals, or significance tests. The phrase 'marginal yet significant' is unsupported; please provide per-fold results and a significance test.","section":"Section III.B.2, Table II"},{"comment":"The hyperparameters are stated as D=64, kr=7, and two layers, but Eq. (9) also uses K attention heads or clusters. Please clarify the relationship between K, kr, and the number of attention heads.","section":"Section III.A"}],"recommendation":"major_revision","confidential_remarks":"The paper is preliminary, and the core methodological claims are not yet adequately supported. The interpretability error in Section IV in particular undermines one of the two advertised contributions. The authors should be asked to correct the interpretability mapping, add ablations, report significance tests, and complete the baseline table. Fit with this venue is borderline until these issues are resolved."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"SCGT is a graph transformer whose attention parameters are functions of Laplacian positional encodings through a shared matrix θ1, with the stated goal of cluster-specific updates instead of the uniform updates of a vanilla transformer. That parameterization (Eqs. 3–5) is the genuinely new bit; it is a reasonable extension of the authors' own BrainRGIN and worth testing on brain graphs. The empirical setup is serious: roughly 8,000 ABCD subjects, 5-fold CV, comparisons against BrainGNN, BrainNetCNN, SVR, and a vanilla GT. The gains are modest (MSE 271 vs 283, r 0.283 vs 0.25) but consistently in one direction.\n\nThe soft spots are real and, in one case, load-bearing. First, the interpretability section assigns node i to argmax_c θ1[i,c], but θ1 in Eqs. 3–5 is multiplied by λi, a PE vector, so rows of θ1 index PE coordinates, not nodes. The quantity actually used in the forward pass is (θ1 λi)_c. The paper gives no reason these two assignments coincide, and for Laplacian eigenvectors they generally won't. That undercuts the claim that SCGT learns interpretable brain subnetworks. Second, there is no ablation that isolates the self-clustering mechanism from the extra parameters or from PE conditioning; the improvement over vanilla GT could come from either. Third, no significance tests are reported; the gender table lacks variance; and BrainRGIN, listed as a baseline, is missing from Table I. Minor: K in Eq. 9 is called \"number of attention heads or clusters\" while kr is the number of communities; the relationship is unexplained.\n\nNone of this sank the core idea for me, but the paper as written overclaims. The authors should fix the interpretability computation, add ablations with θ1 randomized or fixed, run significance tests, include BrainRGIN, and release code. This is preliminary work and they say so, which helps.\n\nWho's this for? People working on graph transformers for neuroimaging, or on conditioning attention on graph structure. It deserves a serious referee—the mechanism is novel enough and the dataset large enough—but it needs a major revision before publication. I wouldn't cite it in its current form.","headline":"SCGT's attention parameterization is a legitimate new idea, but the interpretability claim uses the wrong matrix and the gains are not yet validated.","tokens_in":7862,"tokens_out":2820,"would_cite":false,"duration_ms":29493,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a graph transformer whose attention parameters are functions of Laplacian positional encodings learns cluster-specific update rules, improves fMRI-based cognitive-score and sex prediction, and yields interpretable…","keywords":["graph transformer","self-clustering attention","functional connectivity","resting-state fMRI","cognitive score prediction","gender classification","Laplacian positional encoding","brain subnetwork interpretation"],"falsifier":"Train the same architecture with $\\theta_1$ fixed to a random matrix (or replaced by a single global parameter vector shared by all nodes) and measure test MSE and correlation on the same split; if performance does not drop, the self-clustering mechanism is not load-bearing. For the interpretability claim, compute the overlap between the argmax community assignments from $\\theta_1$ and an independent seven-network atlas; chance-level overlap would falsify the claim that the learned subclusters align with known functional networks.","tokens_in":6790,"feed_emoji":"🧠","tokens_out":12474,"duration_ms":107550,"temperature":0.7,"pith_summary":"Resting-state fMRI captures how brain regions co-activate at rest, and graph transformers are a natural way to read such data, but standard transformers apply the same attention update to every node, missing the brain's community structure. The paper introduces the Self-Clustering Graph Transformer (SCGT), which lets attention rules vary by cluster by making the key, query, and value parameters functions of Laplacian positional encodings through a shared matrix. On 7,957 children aged 9-10, the authors report that SCGT predicts total cognitive score with MSE 271.02 and correlation 0.283, outperforming the vanilla graph transformer (MSE 283.43, correlation 0.25) and several baselines, and raises sex-classification accuracy from 82.3% to 84.5%. The same shared matrix also assigns each brain region to one of seven communities, giving an interpretable subnetwork parcellation. The paper treats the result as preliminary but argues that the mechanism is not brain-specific.","feed_headline":"Self-clustering attention beats vanilla graph transformer","feed_subtitle":"On 7,957 brain scans it improves cognitive-score and sex prediction; learned clusters match known networks.","key_machinery":"The load-bearing object is the self-clustering graph attention block. It starts from node features $\\hat h_i = h_i+\\lambda_i$, where $\\lambda_i$ is the Laplacian positional encoding (eigenvectors of the normalized graph Laplacian). The key, query, and value transformations are not shared globally: they are parameterized as functions of $\\lambda_i$ through a shared matrix $\\theta_1^{(l)}\\in\\mathbb{R}^{N\\times k_r}$, followed by ReLU and separate projections, so that nodes in different positions can adopt different attention weights. The number $k_r=7$ is chosen to match seven canonical functional networks, and the same $\\theta_1^{(l)}$ matrix is reused for interpretation: assigning each node to $\\arg\\max_c\\theta_1^{(l)}[i,c]$ yields the learned subclusters.","core_discovery":"The central claim is that conditioning the attention parameter matrices on Laplacian positional encodings through a shared matrix $\\theta_1^{(l)}$ produces cluster-specific update rules, and that this self-clustering is what lets the model capture brain subnetworks. Concretely, in layer $l$ each node's key, query, and value vectors are computed with parameter vectors $w_i^{(k,l)} = \\theta_{k2}^{(l)}\\mathrm{ReLU}(\\theta_1^{(l)}\\lambda_i)+b_k^{(l)}$ (and analogously for query and value), so nodes with different positional encodings get different attention projections. The authors report that on the resting-state functional connectivity data of 7,957 participants this lowers cognitive-score prediction MSE from 283.43 (vanilla graph transformer) to 271.02, raises correlation from 0.25 to 0.283, and improves sex classification from 82.3% to 84.5% accuracy, while also beating BrainGNN, BrainNetCNN, and SVR baselines. They further claim that the rows of $\\theta_1^{(l)}$, when each node is assigned to its maximum community, produce clusters that align with known functional networks such as the default mode and frontoparietal networks.","pith_inferences":["One check the paper leaves open is ablating $\\theta_1$: replacing it with a fixed random matrix, or with one global parameter vector, would show whether the cluster-specific mechanism itself, rather than extra parameters or the positional encodings, drives the reported gains.","The interpretability claim is visual; a quantitative overlap score between the $\\theta_1$-derived assignments and an independent seven-network atlas would give a measurable test.","An extension the authors do not explore is learning the number of communities from data instead of fixing it at seven, which would show whether the seven-network prior is necessary for the improvement."],"forward_implications":["If the reported numbers hold, SCGT is the best of the compared models on this protocol for cognitive-score prediction, with MSE 271.02 versus 283.43 for the vanilla graph transformer.","Gender classification from the same 100-node functional connectivity graph rises to 84.5% accuracy and F1 0.86, up from 82.3% and 0.84.","The model produces a seven-community partition of the 100 brain regions from the shared matrix, so prediction and parcellation come from the same learned object.","Because the self-clustering block is defined for any graph, the same idea extends to other graphs with subnetwork structure, as the authors state they intend to test."],"supporting_citations":[{"why":"Supplies the vanilla graph transformer baseline and the Laplacian positional encoding scheme that SCGT builds on.","marker":"[14]"},{"why":"Provides an interpretable brain graph neural network baseline with clustering-based embedding that SCGT must beat.","marker":"[11]"},{"why":"Provides a recent brain graph network baseline against which SCGT reports competitive or better results.","marker":"[12]"},{"why":"Supplies the ICA template that defines the 100 brain regions used as graph nodes.","marker":"[13]"},{"why":"Supplies the seven functional networks that set the number of clustered communities and ground the interpretability discussion.","marker":"[17]"},{"why":"Provides the BrainNetCNN baseline for brain network prediction.","marker":"[19]"}],"fun_headline_variants":["Self-clustering transformer maps brain subnetworks from fMRI","Graph transformer with self-clustering cuts cognitive score errors","SCGT: self-clustering attention predicts brain scores better","Self-clustering attention improves brain activity modeling","Cluster-aware transformer beats vanilla on 7,957 brain scans"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method's advantage rests on the assumption that making attention parameters depend on positional encodings through one shared matrix really gives nodes in different clusters different update rules, and that this difference, not merely the extra parameters, explains the reported improvements.","fun_headline_variants_meta":{"raw":{"variants":["Self-clustering transformer maps brain subnetworks from fMRI","Graph transformer with self-clustering cuts cognitive score errors","SCGT: self-clustering attention predicts brain scores better","Self-clustering attention improves brain activity modeling","Cluster-aware transformer beats vanilla on 7,957 brain scans"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000265,"raw_usage":{"total_tokens":1635,"prompt_tokens":999,"completion_tokens":636,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":615,"completion_tokens_details":{"reasoning_tokens":559}},"tokens_in":615,"tokens_out":636,"duration_ms":6301,"temperature":1.0,"reasoning_tokens":559,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T19:08:31.782085+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same architecture with $\\theta_1$ fixed to a random matrix (or replaced by a single global parameter vector shared by all nodes) and measure test MSE and correlation on the same split; if performance does not drop, the self-clustering mechanism is not load-bearing. For the interpretability claim, compute the overlap between the argmax community assignments from $\\theta_1$ and an independent seven-network atlas; chance-level overlap would falsify the claim that the learned subclusters align with known functional networks.","supporting_citations":[{"cited_title":"A generalization of transformer networks to graphs,","cited_arxiv_id":null,"evidence_quote":"Supplies the vanilla graph transformer baseline and the Laplacian positional encoding scheme that SCGT builds on."},{"cited_title":"BrainGNN: Interpretable brain graph neural network for fMRI analysis,","cited_arxiv_id":null,"evidence_quote":"Provides an interpretable brain graph neural network baseline with clustering-based embedding that SCGT must beat."},{"cited_title":"Brain networks and intelligence: A graph neural network based approach to resting state fmri data,","cited_arxiv_id":null,"evidence_quote":"Provides a recent brain graph network baseline against which SCGT reports competitive or better results."},{"cited_title":"NeuroMark: An automated and adaptive ICA based pipeline to identify reproducible fMRI markers of brain disorders,","cited_arxiv_id":null,"evidence_quote":"Supplies the ICA template that defines the 100 brain regions used as graph nodes."},{"cited_title":"The organization of the human cerebral cortex estimated by intrinsic functional connectivity,","cited_arxiv_id":null,"evidence_quote":"Supplies the seven functional networks that set the number of clustered communities and ground the interpretability discussion."},{"cited_title":"Brainnetcnn: Convolutional neural networks for brain networks; towards predicting neurodevelopment,","cited_arxiv_id":null,"evidence_quote":"Provides the BrainNetCNN baseline for brain network prediction."}],"review_version":1}