{"id":"0c40bd9b-ea2b-4dc5-85aa-81d28a4f1213","arxiv_id":"2412.16197","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"MeTSK, a meta-learning plus self-supervised strategy on fMRI graphs, improves linear-probing classification of post-traumatic epilepsy, Alzheimer's disease, and Parkinson's disease compared with direct deep learning and fMRI foundation models in low-data settings.","lead":"The authors combine self-supervised learning and meta-learning to train an fMRI model on healthy controls and a small clinical dataset, then test the learned features on four held-out neurological disorder classification tasks. The method, MeTSK, reports higher area-under-the-curve than direct deep learning and two fMRI foundation models, but the improvements are noisy on the small datasets.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The Neurocon 5-fold split may leak the same subject's replicate scans into both train and test folds, inflating the reported MeTSK AUC and undermining the 'best on all four datasets' claim.","rationale":"The reader's CONDITIONAL verdict is appropriate, and the condition should explicitly include verifying the Neurocon split. The reader's rationale already flags the underspecified Neurocon cross-validation split, so we partially agree. However, the reader's stated weakest_assumption is about healthy-control feature transferability, whereas the most load-bearing, concretely testable threat to the central claim is the potential subject-level leakage in the Neurocon evaluation. The paper does provide public code, so the split protocol can be checked directly. If the code shows a scan-level split, the Neurocon result is unreliable and the central claim loses one of its four datasets. If the code shows a subject-level split, the concern is resolved and the conditional accept stands on that point. The additional RF-row inconsistency in Table 2 is secondary but should be addressed by defining the exact criterion for 'best performance.' No ad hominem is intended; this is a standard experimental-design check. We therefore see no reason to change the reader's verdict, but the stated acceptance condition should include the Neurocon split verification.","tokens_in":23075,"tokens_out":9948,"duration_ms":82290,"concrete_test":"Inspect the public repository's cross-validation code for the Neurocon dataset. If it uses plain KFold or StratifiedKFold on scans rather than GroupKFold on subject IDs, rerun the Neurocon linear-probing experiments with GroupKFold or LeaveOneSubjectOut, keeping both replicate scans of each subject in the same fold. Then compare the SVM AUC to the reported 0.7529 and to BrainLM's 0.6476; if the MeTSK AUC falls to near the BrainLM level, the 'best on all four' claim is unsupported for Neurocon. Also report explicitly which probe classifier defines 'best performance' and recompute the mean AUC across SVM/RF/LR for each dataset.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim in Sec 5.2.2 is that 'MeTSK achieved the best performance on all four datasets, outperforming the two latest fMRI foundation models.' One of the four datasets, Neurocon, is described in Sec 4.6 as containing 27 PD patients and 16 controls with 2 replicate scans per subject. The paper never states that the 5-fold cross-validation split is subject-level. If the split is by scan rather than by subject, replicate scans from the same subject can appear in both the training and test folds of the linear probe. Because the probe classifier can then memorize subject-specific signal, the reported SVM AUC of 0.7529 (Table 2) could be inflated relative to the fair subject-level estimate. This margin over BrainLM (0.6476) is large enough that losing it would remove Neurocon from the 'best on all four' claim. In addition, even within Table 2 the unqualified 'best on all four' statement is not uniformly supported: BrainLM's RF AUC exceeds MeTSK's RF AUC on both PTE (0.5589 vs 0.5392) and Neurocon (0.6774 vs 0.6230). Thus the headline claim depends on either a specific probe classifier or an average across probes, and the split-leakage issue is the most direct threat to the claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MeTSK, a representation learning strategy that combines meta-learning with contrastive self-supervised learning for fMRI-based neurological disorder identification. MeTSK is trained on HCP healthy-control data (source) and a small clinical target dataset (ADHD-Peking or ABIDE-UM), then evaluated by linear probing on four held-out clinical datasets (PTE, OASIS-3, TaoWu, Neurocon) for binary classification. The central claim is that MeTSK achieves the best linear-probing AUC on all four held-out datasets, outperforming two fMRI foundation models (Thomas et al. 2022 and BrainLM). The paper also includes comparisons with traditional classifiers, ablations of meta-learning and self-supervised components, fine-tuning experiments for foundation models, and a domain-similarity analysis.","tokens_in":23382,"tokens_out":3946,"duration_ms":36462,"significance":"The proposed method addresses a real and important problem: learning transferable fMRI representations from abundant healthy-control data to scarce and heterogeneous clinical datasets. If the central claim is valid, the approach would provide a practical feature extractor for low-data neurological disorder classification. The paper has clear strengths: it evaluates on four independent clinical datasets, includes comparisons with two recent fMRI foundation models, provides an ablation study isolating the contributions of self-supervised learning and meta-learning, and makes code publicly available. The domain-similarity analysis is a useful addition. However, the evaluation protocol has several unresolved threats—potential subject-level leakage in the Neurocon dataset, unclear handling of PCA in the feature pipeline, inconsistent support for the headline 'best on all four' claim across classifiers, and absent significance testing against foundation models—so the current evidence does not fully establish the central claim.","major_comments":[{"comment":"The Neurocon dataset is described in §4.6 as containing 27 PD patients and 16 controls with two replicate scans per subject, but the 5-fold cross-validation in §5.2.2 is not stated to be at the subject level. If the folds are split by scan rather than by subject, replicate scans from the same subject can appear in both the training and test sets of the linear probe, inflating the reported SVM AUC of 0.7529. The authors must clarify the splitting scheme and, if it is scan-level, rerun the evaluation with subject-level splits. Because Neurocon is one of four datasets supporting the 'best on all four' claim, this issue is load-bearing.","section":"§4.6, §5.2.2"},{"comment":"The feature evaluation pipeline applies PCA to reduce dimensionality 'before feeding the features into classifiers,' but the manuscript does not state that PCA is fit on the training folds only. If PCA is fit on the entire dataset before cross-validation, test information leaks into the probe training, which would bias the reported AUCs upward. The authors should specify the PCA fitting procedure and, if it was global, repeat the evaluation with PCA fit inside each training fold.","section":"§5.3, Evaluation"},{"comment":"The statement in §5.2.2 that 'MeTSK achieved the best performance on all four datasets, outperforming the two latest fMRI foundation models' is not uniformly supported by Table 2. On PTE, BrainLM's RF AUC is 0.5589 versus MeTSK's 0.5392; on Neurocon, BrainLM's RF AUC is 0.6774 versus MeTSK's 0.6230, and Thomas et al.'s RF AUC is 0.5813 versus MeTSK's 0.6230 as well. The claim appears to depend on choosing the SVM probe or on averaging across probes, but no aggregation rule is stated. The authors should either refine the claim to the specific probe and metric used or provide a principled aggregation across the three classifiers and confirm that the conclusion holds.","section":"§5.2.2, Table 2"},{"comment":"No significance tests are reported for the differences between MeTSK and the foundation models in Table 2. Many of the reported standard deviations overlap substantially (e.g., PTE SVM: MeTSK 0.6415±0.0312 vs BrainLM 0.6011±0.0465; TaoWu SVM: 0.6831±0.1431 vs BrainLM 0.5725±0.1502), so the observed differences may be within noise. The authors should provide paired significance tests across folds (e.g., Wilcoxon signed-rank or permutation tests) for the MeTSK versus foundation-model comparisons on each dataset and classifier, and interpret the results accordingly.","section":"§5.2.2, Table 2"},{"comment":"The key hyperparameters λ=30 and τ=30 are selected 'empirically by observing the convergence of the training loss,' but no sensitivity analysis is provided. Because λ scales the meta-learning target loss relative to the self-supervised loss and τ controls the sharpness of the contrastive loss, the reported performance could be sensitive to these choices. The authors should include a sensitivity analysis over reasonable ranges of λ and τ (e.g., on the ADHD-Peking or ABIDE-UM validation tasks) to demonstrate that the benefit of MeTSK is not confined to a single tuned setting.","section":"§5.3, §6"}],"minor_comments":[{"comment":"The word 'Pinciple' in 'applied Pinciple Component Analysis' should be 'Principal.'","section":"§5.3, Evaluation"},{"comment":"The heading 'P-values Computed from Wilconxon Signed-Rank Tests' misspells 'Wilcoxon.'","section":"Appendix A.2"},{"comment":"The text refers to 'the same steps as described in Section 4.1.2,' but there is no Section 4.1.2; the intended reference appears to be Section 4.2 (ADHD-Peking) or the general preprocessing description in Section 4.","section":"§6.1"},{"comment":"The feature importance map in Figure 3 would benefit from a colorbar or an explicit statement of the color scale, since the reader cannot otherwise gauge the magnitude of the coefficients.","section":"Figure 3"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is already published in TMLR (05/2025), but as a submitted paper it requires major revision before the central claim can be accepted. The most immediate concerns are the Neurocon subject-level split and the PCA fitting protocol; both are fixable with additional experiments and would directly determine whether the 'best on all four datasets' claim survives. The inconsistent RF results in Table 2 and the absence of significance tests are also important to address. I would encourage the editor to request a revised version that clarifies these evaluation details rather than reject, because the underlying idea is useful and the current evidence is suggestive but not conclusive."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: this is a competent engineering paper, not a conceptual breakthrough. The new piece is the combination: meta-learning with a contrastive self-supervised source task on a ST-GCN, transferring healthy-control features to scarce clinical datasets. The authors evaluate on four held-out datasets, include a useful ablation (self-supervision beats sex classification as a source task; meta-learning beats fine-tuning and MTL), and release code. That is enough to take seriously.\n\nWhere it gets soft: the statistics do not support the strength of the claims. Tables 1 and 2 report mean AUC with standard deviations, and the error bars overlap with several baselines. No significance tests are run against the two foundation models in the main table. And the 'best on all four datasets' sentence in Sec 5.2.2 is not uniformly true: for PTE and Neurocon, BrainLM's RF AUC is higher than MeTSK's RF AUC (0.5589 vs 0.5392; 0.6774 vs 0.6230). The statement only holds if you aggregate across the three linear probes in a particular way, and the paper should say how.\n\nThe sharpest concern is Neurocon. That dataset is described as 27 PD patients and 16 controls with two replicate scans per subject. The paper never states that the 5-fold CV split is at the subject level. If folds are split by scan, replicate scans from the same subject can appear in both training and test folds, which would inflate MeTSK's 0.7529 SVM AUC. This is a specific, checkable issue. It does not kill the paper's overall point -- the held-out evaluation design is sound and the other three datasets do not have this leakage problem -- but the Neurocon result should be redone or explicitly justified with a subject-level split.\n\nMinor: lambda=30 and tau=30 were chosen by watching loss convergence, with no sensitivity analysis. For a method that is mainly an engineering recipe, that is a small gap, but worth noting. The paper is otherwise transparent: it reports the overfitting of deep baselines, includes a convergence analysis, and fine-tunes foundation models as an extra fairness check. The citation pattern is appropriate -- the meta-learning and contrastive SSL ideas are attributed to their sources.\n\nVerdict: worth a serious referee. If I were editing, I would send it out and ask for subject-level Neurocon splits, a definition of what 'best' means across classifiers, and either significance tests or paired bootstrap intervals. The code and ablations make this a useful baseline for anyone working on low-data clinical fMRI.","headline":"Solid engineering paper combining meta-learning and contrastive SSL for fMRI; the four-dataset claim is plausible but the Neurocon split must be subject-level, and the stats are weaker than the headline.","tokens_in":23896,"tokens_out":3003,"would_cite":true,"duration_ms":27055,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MeTSK, a meta-learned self-supervised fMRI representation, transfers to four scarce neurological-disorder tasks and beats both fMRI foundation models and connectivity-feature classifiers.","keywords":["fMRI","representation learning","meta-learning","self-supervised learning","neurological disorder classification","functional connectivity","ST-GCN","domain generalization"],"falsifier":"Train the same MeTSK pipeline without the source domain—removing the HCP self-supervised loss and keeping only the bi-level target-head adaptation, as in the paper's MeL ablation—and run the four held-out linear-probing tasks; any dataset on which MeL matches or exceeds MeTSK would falsify the claim that healthy-control self-supervision drives the generalization.","tokens_in":22890,"feed_emoji":"🧠","tokens_out":8647,"duration_ms":74352,"temperature":0.7,"pith_summary":"This paper sets out to show that deep learning can be made useful for neurological-disorder classification from fMRI even when clinical training data are scarce, by first learning general features from abundant healthy-control scans. The proposed strategy, MeTSK, combines contrastive self-supervised learning on the HCP healthy-control dataset with a bi-level meta-learning loop that adapts a target head to a small clinical dataset while updating a shared feature extractor for both domains. Evaluated by linear probing on four held-out clinical datasets—post-traumatic epilepsy, Alzheimer's disease, and two Parkinson's disease cohorts—the MeTSK features give classification AUCs above those of functional-connectivity features and above two recent fMRI foundation models. If the claim holds, MeTSK is a reusable feature extractor for low-data clinical tasks where end-to-end deep learning typically overfits.","feed_headline":"MeTSK beats fMRI foundation models on four scarce clinical datasets","feed_subtitle":"This healthy-control pretraining recipe transfers to epilepsy, Alzheimer's, and Parkinson's tasks.","key_machinery":"The load-bearing machinery is MeTSK (Meta Transfer of Self-supervised Knowledge), built on a spatio-temporal graph convolutional network (ST-GCN) whose graph nodes are 116 AAL brain regions with Pearson-correlation edges. Training alternates two loops: an inner loop fast-adapts only the target head on a meta-training split of the small clinical dataset, and an outer loop updates the source head and the shared feature extractor by minimizing a source-domain graph-contrastive loss plus the target head's loss on a held-out meta-validation split. The graph-contrastive loss treats two randomly sampled fMRI sub-sequences from the same subject as positive pairs and views different subjects as negatives, so the feature extractor must encode subject-specific but temporally invariant functional patterns. This nested optimization is what carries the transfer argument: the target head's fast adaptation defines what the extractor must preserve, and the source loss keeps those features from collapsing to the source task.","core_discovery":"The central discovery claimed is that the features learned by MeTSK—pre-trained with a self-supervised contrastive loss on healthy controls and shaped by meta-learning on one small clinical target—transfer to unseen clinical populations. On all four held-out datasets, linear classifiers trained on MeTSK features achieved the best mean AUC among the compared methods, exceeding both the frozen and fine-tuned versions of the two fMRI foundation models and beating classifiers trained directly on functional-connectivity features. The authors attribute this to the combination: self-supervision keeps the representation tied to intrinsic functional patterns rather than to a supervised source task, while the inner-loop adaptation of the target head forces the feature extractor to preserve information useful for the clinical domain. The direct consequence is that a model pretrained once on HCP plus a single small clinical site can serve as an off-the-shelf feature extractor for other disorder-classification tasks.","pith_inferences":["Beyond the paper, a testable extension is to replace the ST-GCN backbone with a foundation-model encoder while keeping MeTSK's bi-level adaptation; the paper's fine-tuning results suggest foundation models are not necessarily improved by the same meta-adaptation unless their scale is handled carefully.","Beyond the paper, the domain-similarity metric used in the appendix could be turned into a screening tool: compute the earth mover's distance between source and candidate target features to decide in advance whether MeTSK will transfer before collecting a clinical dataset.","Beyond the paper, the method's success across epilepsy, Alzheimer's, and Parkinson's suggests the recipe could extend to other fMRI classification targets such as psychiatric disorders or stroke recovery, although the paper does not test these.","Beyond the paper, because the inner loop re-initializes the target head and re-samples meta-train and meta-validation splits each iteration, the reported stability may depend on the small meta-validation set; a sensitivity analysis over target-split size is an implicit next test."],"forward_implications":["MeTSK trained once on HCP plus one small clinical site can be frozen and reused as a feature extractor for new disorder-classification tasks without retraining the encoder.","Linear probing on MeTSK features outperforms direct classifiers on functional-connectivity features, so the learned representation captures disorder-relevant signal that raw connectivity misses.","Training MeTSK with the full multi-site ADHD dataset yields no significant gain over a single site, implying that target-data scarcity is not the bottleneck once the representation is learned.","Frozen MeTSK features beat frozen features from two large fMRI foundation models, indicating that massive-scale pretraining is not necessary for low-data clinical transfer.","The SVM feature-importance map for PTE highlights temporal, parietal, and occipital regions, consistent with known epilepsy localization and with an independent PTE study."],"supporting_citations":[{"why":"It supplies the bi-level model-agnostic meta-learning formalism that MeTSK adapts to the fMRI setting.","marker":"Finn et al., 2017"},{"why":"It provides the single-target-domain meta-representation learning formulation that MeTSK builds on.","marker":"Liu et al., 2020a"},{"why":"It contributes the ST-GCN backbone, the sub-sequence sampling strategy, and the voting procedure used throughout the paper.","marker":"Gadgil et al., 2020"},{"why":"It provides the graph contrastive learning with augmentations that underlies the source-domain self-supervised loss.","marker":"You et al., 2020a"},{"why":"It supplies the contrastive loss formulation used in the self-supervised objective.","marker":"Chen et al., 2020b"},{"why":"It defines the HCP dataset that serves as the large healthy-control source domain.","marker":"Van Essen et al., 2013"},{"why":"It supplies the ADHD-200 data used as a small clinical target domain and as the training target for the main model.","marker":"Bellec et al., 2017"},{"why":"It supplies the OASIS-3 dataset used as one of the four held-out generalization evaluation tasks.","marker":"LaMontagne et al., 2019"},{"why":"It provides the pre-trained fMRI GPT foundation model against which MeTSK is compared in the generalization experiments.","marker":"Thomas et al., 2022"},{"why":"It provides the BrainLM foundation model against which MeTSK is compared in the generalization experiments.","marker":"Ortega Caro et al., 2023"}],"fun_headline_variants":["MeTSK beats fMRI foundation models on four scarce datasets","Self-supervised meta-learning makes fMRI features generalize","Healthy-control pretraining transfers to rare disorder tasks","One fMRI model, four disorders: MeTSK wins on all"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that self-supervised features learned from resting-state scans of healthy young adults capture intrinsic functional brain patterns that remain invariant enough to transfer to patient populations with different disorders, scanners, and age ranges.","fun_headline_variants_meta":{"raw":{"variants":["MeTSK beats fMRI foundation models on four scarce datasets","Self-supervised meta-learning makes fMRI features generalize","Healthy-control pretraining transfers to rare disorder tasks","One fMRI model, four disorders: MeTSK wins on all"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000187,"raw_usage":{"total_tokens":1314,"prompt_tokens":918,"completion_tokens":396,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":534,"completion_tokens_details":{"reasoning_tokens":331}},"tokens_in":534,"tokens_out":396,"duration_ms":4150,"temperature":1.0,"reasoning_tokens":331,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T14:08:42.281815+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same MeTSK pipeline without the source domain—removing the HCP self-supervised loss and keeping only the bi-level target-head adaptation, as in the paper's MeL ablation—and run the four held-out linear-probing tasks; any dataset on which MeL matches or exceeds MeTSK would falsify the claim that healthy-control self-supervision drives the generalization.","supporting_citations":[{"cited_title":"Oasis-3: longitudinal neuroimaging, clinical, and cognitive dataset for normal aging and alzheimer disease.MedRxiv, pp","cited_arxiv_id":null,"evidence_quote":"It supplies the OASIS-3 dataset used as one of the four held-out generalization evaluation tasks."},{"cited_title":"Brainlm: A foundation model for brain activity recordings.bioRxiv, pp","cited_arxiv_id":null,"evidence_quote":"It provides the BrainLM foundation model against which MeTSK is compared in the generalization experiments."}],"review_version":1}