REVIEW 3 major objections 5 minor 45 references
A Self-supervised Multimodal Deep Learning Approach to Differentiate Post-radiotherapy Progression from Pseudoprogression in Glioblastoma
T0 review · 3 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash
Pith's one-line read A multimodal transformer that combines self-supervised MRI encoding with clinical and radiotherapy dose features can distinguish pseudoprogression from true progression in glioblastoma, achieving an external-test AUC of 0.753 and…
desk verdict Sensible engineering and honest limitations, but the SOTA claim is undermined by a label-correlated time-to-event feature and an n=20 external test. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing component is a frozen Vision Transformer encoder pretrained with contrastive learning and context restoration on 2,317 unlabeled glioma MRI studies, which turns 160x160x160 two-channel MRI volumes into 512 tokens. A guided cross-attention layer then lets four selected clinical and dose tokens (age, IDH and MGMT status, time to progression, and dose statistics) query those imaging tokens, yielding a compact fused representation that proceeds through self-attention and classification heads. The mechanism's dual role is to fuse modalities and to shrink the token sequence from 512 to 4, avoiding the quadratic cost of self-attention on long sequences.
What would settle it
Retest the same model on an external cohort where every PsP and TP label is histologically confirmed; if the AUC falls below the reported 0.753 or below the baselines, the central claim is weakened. A second check is to recompute the dose features using deformable registration and see whether the contribution of dose features such as D98 and minimum dose changes materially.
Extended reading notes
Core claim
The paper's central claim is that combining a self-supervised Vision Transformer encoder for whole-brain FLAIR and T1 post-contrast volumes with clinical and radiotherapy dose information through guided cross-attention discriminates pseudoprogression from true progression better than previous data-driven methods. On a 20-patient external test set from a different institution, the model reaches an AUC of 0.753, accuracy 0.750, sensitivity 0.727, and specificity 0.800, compared with AUCs of 0.686, 0.677, and 0.530 for the CNN-LSTM, CNN-SVM, and random-forest baselines. The paper attributes the improvement to the transformer's ability to model long-range spatial dependencies in whole-volume MRI and to the cross-attention mechanism that lets a small set of clinical and dose tokens guide the imaging representation.
Load-bearing premise
The external-test labels are treated as ground truth even though 12 of the 20 patients were classified by longitudinal imaging follow-up rather than histology; if any of those labels are wrong, the reported AUC is directly biased.
Editorial extensions
If this is right
- A clinically usable, non-invasive tool for distinguishing PsP from TP, potentially reducing unnecessary biopsies and treatment changes.
- Evidence that self-supervised pretraining on unlabeled MRI can compensate for scarce labeled data in medical imaging classification.
- A general pattern for fusing a few structured clinical or dose variables with thousands of imaging tokens via cross-attention.
- A model that relies only on routinely acquired FLAIR and T1 post-contrast MRI plus standard clinical and RT-planning data, making deployment in routine care feasible.
Reading between the lines
- The reported gain of self-supervised pretraining over transfer learning (0.753 vs 0.727 AUC) comes from a single 20-patient test set; the confidence intervals likely overlap, so the superiority of SSL over transfer learning should be confirmed on larger cohorts.
- Because dose features are extracted by registering follow-up MRI to the planning CT, registration errors could corrupt those inputs; the paper does not quantify this sensitivity, and deformable registration may change both performance and which features matter.
- Adding diffusion or perfusion MRI, which the authors list as a limitation, is a natural next test and may push the AUC above 0.75.
- The most influential feature, time to progression, encodes the clinical prior that PsP usually appears earlier; an ablation using only clinical and time features would clarify how much the imaging and dose fusion actually adds.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a multimodal transformer-based deep learning model to distinguish pseudoprogression (PsP) from true progression (TP) in glioblastoma after radiotherapy. The model combines a self-supervised Vision Transformer (ViT) encoding two MRI sequences (T1CE, FLAIR), structured clinical features, and radiotherapy dose information via guided cross-modal attention. It is trained on 59 patients from the Burdenko Glioblastoma Progression Dataset with 5-fold cross-validation and tested on 20 patients from the external GlioCMV UKER dataset. The authors report an external AUC of 0.753, which they state outperforms current state-of-the-art data-driven methods (Jang et al., Akbari et al., Sun et al.). The paper also analyzes unimodal vs. multimodal performance, the impact of self-supervised pretraining versus transfer learning or end-to-end training, and SHAP-based feature importance showing TIME_UNTIL_PROGRESSION as the most influential feature.
Significance. If the central claim were secure, the work would be a valuable contribution to a clinically difficult problem: a non-invasive, externally validated tool for PsP/TP differentiation, enhanced by self-supervised pretraining on large public MRI datasets. The use of an external test set and the comparison of multiple training strategies are strengths, as is the integration of dose features from radiotherapy planning. However, the central claim of outperforming state-of-the-art as stated in the abstract and results is not yet supported because the comparison is confounded by a label-correlated clinical feature and because the added value of imaging over clinical data alone is small and statistically unquantified. The paper is relevant to the community, but the evidence in its current form requires substantial revision.
major comments (3)
- [Results (Table 2), Figure 3, Discussion] The external state-of-the-art comparison is confounded by TIME_UNTIL_PROGRESSION. Figure 3 and the Discussion state that time from end of radiotherapy to the index progression has the greatest influence on the model's prediction. The baselines (Jang et al., Akbari et al., Sun et al.) were not given this time-to-event feature, and PsP is by definition an early transient post-radiotherapy phenomenon. Moreover, 12 of the 20 external test patients were labeled by longitudinal imaging follow-up rather than histology (Methods: Datasets), so the time feature is directly correlated with the label-generation process. To support the claim that the proposed architecture outperforms existing methods, the authors must either rerun all baselines with the same clinical features (including the time feature) or retrain the proposed model without TIME_UNTIL_PROGRESSION. Without this, the observed AUC gap (0.753 vs. 0.686 for the best baseline) cannot be attributed to the self-supervised ViT, cross-modal attention, or multimodal fusion.
- [Results (Table 3), Unimodal vs. Multimodal] The claim that multimodal integration improves prediction is not statistically supported on the external test. Clinical data alone achieves AUC 0.727, and adding imaging raises it to 0.753, an improvement of 0.026 that is smaller than the reported standard deviation of 0.038 for the multimodal model. The external test has only 20 patients, and no confidence intervals, DeLong tests, or other significance measures are reported for any AUC comparison. The authors should report confidence intervals for all external-test metrics and perform paired significance tests between the multimodal model and both unimodal models and each baseline. Without these, the assertion that the fusion mechanism provides a meaningful advantage over clinical data alone is not justified.
- [Methods: Model Architecture and Training, Feature Importance Analysis] The feature-selection procedure (choosing the first M features with highest AUC on the cross-validation sets) constitutes model selection performed on the validation folds, and the reported validation performance is therefore optimistically biased. While the external test set remains untouched, the stability of the chosen M=4 features across the five folds is not reported, and it is not clear whether the same feature subset was selected in every fold or whether TIME_UNTIL_PROGRESSION was always selected. The authors should either embed feature selection inside the cross-validation loop and report the selected features per fold, or fix the feature set a priori based on clinical knowledge, to avoid selection-induced optimism and to clarify the role of the time-to-event feature.
minor comments (5)
- [Abstract and Introduction] The abstract repeats the phrase 'outperforming the current state-of-the-art data-driven approaches' twice in slightly different forms; the second occurrence should be replaced with a quantitative statement that is supported by the results after the additional analyses are performed.
- [Methods: Datasets] The term 'colinear' should be 'collinear', and 'miss-classification' later in the Discussion should be 'misclassification'. The affiliations list also contains typos such as 'Germnay' for 'Germany' and an inconsistent university name; these should be corrected before publication.
- [Figure 1 caption and Figure 2 caption] The caption text in Figure 1 contains garbled mathematical symbols and duplicated loss terms; the caption should be rewritten so that the self-supervised loss is clearly expressed. In Figure 2, the individual fold ROC curves are informative, but the figure would benefit from labeling the ensembled AUC values on the plots for direct comparison with Table 2.
- [Results: Feature Importance Analysis] The order of feature names in Figure 3 is ambiguous because the vertical axis labels are not shown; the figure should explicitly label the axis and indicate whether TIME_UNTIL_PROGRESSION is the top or bottom bar, since the text claims it is the most influential feature.
- [Discussion: Limitations] The limitations paragraph correctly notes the small sample sizes, but it does not mention the reliance on longitudinal imaging follow-up for 12 of 20 external test labels as a limitation. This should be added, as it directly affects the reference standard for the reported AUC.
Circularity Check
External evaluation is a genuine held-out prediction; the only self-citation is non-load-bearing, and no fitted parameter is relabeled as a prediction, so circularity is minimal.
full rationale
The central claim, an external AUC of 0.753, is based on evaluation on the independent GlioCMV UKER test set, which was not used for model selection or feature selection; the predictions are therefore genuine held-out predictions rather than refits of the test labels. The self-supervised ViT encoder was pre-trained on unlabeled public glioma MRI datasets (BraTS2021, UPenn-GBM, UCSF-PDGM) and does not use progression labels, so the upstream representation learning is not circular. The guided cross-attention formula in Eq. (1) is a standard cross-attention expression; although ref. [38] is a self-citation, the same mechanism is also supported by independent refs. [37] and [39], and no uniqueness theorem or tailored ansatz is imported from the authors' prior work. Feature selection via SHAP was performed on cross-validation folds, and the external test set was not used to choose the number of features. A non-circular but important validity concern is that TIME_UNTIL_PROGRESSION, identified as the most influential feature, is label-correlated in the 12/20 external-test patients whose labels were assigned by longitudinal imaging follow-up; this is a potential confounding factor in the state-of-the-art comparison and could inflate the clinical-data contribution, but it is not a circular step in the derivation because the model does not define pseudoprogression in terms of this feature and the multimodal model still contains independent imaging and dosimetric information. Overall, no load-bearing circularity, self-definitional reduction, or fitted-input-as-prediction was found.
Assumptions & free parameters
free parameters (3)
- Number of clinical features selected M =
4
- Self-supervised loss weights alpha and beta =
not reported
- Input crop size =
160x160x160
assumptions (4)
- domain assumption Reference standard labels for PsP and TP in both datasets are correct.
- domain assumption nnU-Net tumor segmentation and ANTs registration accurately map the follow-up enlarging lesion to the planning CT and dose distribution.
- domain assumption Self-supervised pretraining on BraTS2021, UPenn-GBM, and UCSF-PDGM transfers to the target progression-classification domain.
- domain assumption Standard deep learning training assumptions (i.i.d. samples within cohorts, no hidden data leakage) hold.
Cite this review
Pith. "Pith review of A Self-supervised Multimodal Deep Learning Approach to Differentiate Post-radiotherapy Progression from Pseudoprogression in Glioblastoma." pith.science (2026). https://pith.science/paper/MUTSW5WT
@misc{pith2026250203999,
author = {Pith},
title = {Pith review of: A Self-supervised Multimodal Deep Learning Approach to Differentiate Post-radiotherapy Progression from Pseudoprogression in Glioblastoma},
year = {2026},
howpublished = {\url{https://pith.science/paper/MUTSW5WT}},
note = {Machine review of arXiv:2502.03999}
}
read the original abstract
Accurate differentiation of pseudoprogression (PsP) from True Progression (TP) following radiotherapy (RT) in glioblastoma (GBM) patients is crucial for optimal treatment planning. However, this task remains challenging due to the overlapping imaging characteristics of PsP and TP. This study therefore proposes a multimodal deep-learning approach utilizing complementary information from routine anatomical MR images, clinical parameters, and RT treatment planning information for improved predictive accuracy. The approach utilizes a self-supervised Vision Transformer (ViT) to encode multi-sequence MR brain volumes to effectively capture both global and local context from the high dimensional input. The encoder is trained in a self-supervised upstream task on unlabeled glioma MRI datasets from the open BraTS2021, UPenn-GBM, and UCSF-PDGM datasets to generate compact, clinically relevant representations from FLAIR and T1 post-contrast sequences. These encoded MR inputs are then integrated with clinical data and RT treatment planning information through guided cross-modal attention, improving progression classification accuracy. This work was developed using two datasets from different centers: the Burdenko Glioblastoma Progression Dataset (n = 59) for training and validation, and the GlioCMV progression dataset from the University Hospital Erlangen (UKER) (n = 20) for testing. The proposed method achieved an AUC of 75.3%, outperforming the current state-of-the-art data-driven approaches. Importantly, the proposed approach relies on readily available anatomical MRI sequences, clinical data, and RT treatment planning information, enhancing its clinical feasibility. The proposed approach addresses the challenge of limited data availability for PsP and TP differentiation and could allow for improved clinical decision-making and optimized treatment plans for GBM patients.
Figures
Reference graph
Works this paper leans on
-
[1]
Price, M. et al. CBTRUS statistical report: American brain tumor association & NCI neuro-oncology branch adolescent and young adult primary brain and other central nervous system tumors diagnosed in the united states in 2016–2020. Neuro-oncology 26, iii1–iii53 (2024)
work page 2024
-
[2]
Ostrom, Q. T. et al. CBTRUS statistical report: primary brain and other central nervous system tumors diagnosed in the united states in 2013–2017. Neuro-oncology 22, iv1–iv96 (2020)
work page 2020
-
[3]
Hagag, A. et al. Deep learning for cancer prognosis prediction using portrait photos by stylegan embedding. InInternational Conference on Medical Image Computing and Computer-Assisted Intervention , 198–208 (Springer, 2024)
work page 2024
-
[4]
Zhou, Q. et al. Integrated MRI radiomics, tumor microenvironment, and clinical risk factors for improving survival prediction in patients with glioblastomas. Strahlentherapie und Onkologie 1–13 (2024)
work page 2024
-
[5]
Lau, D., Magill, S. & Aghi, M. Molecularly targeted therapies for recurrent glioblastoma: current and future targets. Neurosurg. focus 37 6, E15, DOI: 10.3171/2014.9.FOCUS14519 (2014)
-
[6]
Stupp, R. et al. Radiotherapy plus concomitant and adjuvant temozolomide for glioblastoma. New Engl. journal medicine 352, 987–996 (2005)
work page 2005
-
[7]
Koshy, M. et al. Improved survival time trends for glioblastoma using the seer 17 population-based registries. J. neuro-oncology 107, 207–212 (2012)
work page 2012
-
[8]
Brandsma, D. & van den Bent, M. J. Pseudoprogression and pseudoresponse in the treatment of gliomas. Curr. opinion neurology 22, 633–638 (2009)
work page 2009
Show all 45 references
-
[9]
H., Rodriguez, I., Domingues, R., Gasparetto, E
Da Cruz, L. H., Rodriguez, I., Domingues, R., Gasparetto, E. & Sorensen, A. Pseudoprogression and pseudoresponse: imaging challenges in the assessment of posttreatment glioma. Am. J. Neuroradiol. 32, 1978–1985 (2011)
2011
-
[10]
Adding DSC PWI and DWI to BT-RADS can help identify postoperative recurrence in patients with high-grade gliomas
Yang, Y .et al. Adding DSC PWI and DWI to BT-RADS can help identify postoperative recurrence in patients with high-grade gliomas. J. Neuro-Oncology 146, 363–371 (2020)
2020
-
[11]
Principles of artificial intelligence in radiooncology
Huang, Y .et al. Principles of artificial intelligence in radiooncology. Strahlentherapie und Onkologie 1–26 (2024)
2024
-
[12]
Erdur, A. C. et al. Deep learning for autosegmentation for radiotherapy treatment planning: State-of-the-art and novel perspectives. Strahlentherapie und Onkologie 1–19 (2024)
2024
-
[13]
Akbari, H. et al. Histopathology-validated machine learning radiographic biomarker for noninvasive discrimination between true progression and pseudo-progression in glioblastoma. Cancer 126, 2625–2636 (2020)
2020
-
[14]
Lee, J. et al. Discriminating pseudoprogression and true progression in diffuse infiltrating glioma using multi-parametric MRI data through deep learning. Sci. reports 10, 20331 (2020)
2020
-
[15]
Moassefi, M. et al. A deep learning model for discriminating true progression from pseudoprogression in glioblastoma patients. J. neuro-oncology 159, 447–455 (2022)
2022
-
[16]
Sun, Y .-Z.et al. Differentiation of pseudoprogression from true progression in glioblastoma patients after standard treatment: a machine learning strategy combined radiomics features from T1-weighted contrast-enhanced imaging. BMC medical imaging 21, 1–12 (2021)
2021
-
[17]
H., Kim, I
Jang, B.-S., Jeon, S. H., Kim, I. H. & Kim, I. A. Prediction of pseudoprogression versus progression using machine learning algorithm in glioblastoma. Sci. reports 8, 12516 (2018)
2018
-
[18]
Dosovitskiy, A. et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020). 10/12
2020 arXiv
-
[19]
& Feiszli, M
Wang, W., Tran, D. & Feiszli, M. What makes training multi-modal classification networks hard? In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 12695–12705 (2020)
2020
-
[20]
He, K. et al. Transformers in medical image analysis. Intell. Medicine 3, 59–78 (2023)
2023
-
[21]
& Dosovitskiy, A
Raghu, M., Unterthiner, T., Kornblith, S., Zhang, C. & Dosovitskiy, A. Do vision transformers see like convolutional neural networks? Adv. Neural Inf. Process. Syst. 34, 12116–12128 (2021)
2021
-
[22]
Chen, L. et al. Self-supervised learning for medical image analysis using image context restoration. Med. image analysis 58, 101539 (2019)
2019
-
[23]
& Duwairi, R
Shurrab, S. & Duwairi, R. Self-supervised learning methods and applications in medical imaging analysis: A survey. PeerJ Comput. Sci. 8, e1045 (2022)
2022
-
[24]
Calabrese, E. et al. The university of california san francisco preoperative diffuse glioma mri dataset. Radiol. Artif. Intell. 4, e220058 (2022)
2022
-
[25]
Clark, K. et al. The cancer imaging archive (TCIA): maintaining and operating a public information repository. J. digital imaging 26, 1045–1057 (2013)
2013
-
[26]
Baid, U. et al. The RSNA-ASNR-MICCAI BraTS 2021 benchmark on brain tumor segmentation and radiogenomic classification (2021). 2107.02314
2021 arXiv
-
[27]
Menze, B. H. et al. The multimodal brain tumor image segmentation benchmark (BRATS). IEEE transactions on medical imaging 34, 1993–2024 (2014)
2014
-
[28]
Bakas, S. et al. Advancing the cancer genome atlas glioma MRI collections with expert segmentation labels and radiomic features. Sci. data 4, 1–13 (2017)
2017
-
[29]
Bakas, S. et al. The university of pennsylvania glioblastoma (UPenn-GBM) cohort: Advanced MRI, clinical, genomics, & radiomics. Sci. data 9, 453 (2022)
2022
-
[30]
V .et al
Zolotova, S. V .et al. Burdenko’s glioblastoma progression dataset (Burdenko-GBM-Progression) (version 1) (2023). Data accessed: 2023-04-10
2023
-
[31]
F., Kohl, S
Isensee, F., Jaeger, P. F., Kohl, S. A., Petersen, J. & Maier-Hein, K. H. nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nat. methods 18, 203–211 (2021)
2021
-
[32]
Avants, B. B. et al. A reproducible evaluation of ANTs similarity metric performance in brain image registration. Neuroimage 54, 2033–2044 (2011)
2011
-
[33]
Goerig, N. L. et al. Early mortality of brain cancer patients and its connection to cytomegalovirus reactivation during radiochemotherapy. Clin. Cancer Res. 26, 3259–3270 (2020)
2020
-
[34]
Goerig, N. L. et al. Frequent occurrence of therapeutically reversible CMV-associated encephalopathy during radiotherapy of the brain. Neuro-oncology 18, 1664–1672 (2016)
2016
-
[35]
Lundberg, S. M. & Lee, S.-I. A unified approach to interpreting model predictions. Adv. neural information processing systems 30 (2017)
2017
-
[36]
Azad, R. et al. Advances in medical image analysis with vision transformers: a comprehensive review. Med. Image Analysis 103000 (2023)
2023
-
[37]
& Lee, S
Lu, J., Batra, D., Parikh, D. & Lee, S. Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and- language tasks. Adv. neural information processing systems 32 (2019)
2019
-
[38]
Gomaa, A. et al. Comprehensive multimodal deep learning survival prediction enabled by a transformer architecture: A multicenter study in glioblastoma. Neuro-Oncology Adv. 6, vdae122 (2024)
2024
-
[39]
Tsai, Y .-H. H. et al. Multimodal transformer for unaligned multimodal language sequences. In Proceedings of the conference. Association for computational linguistics. Meeting , vol. 2019, 6558 (NIH Public Access, 2019)
2019
-
[40]
& van den Bent, M
Brandsma, D., Stalpers, L., Taal, W., Sminia, P. & van den Bent, M. J. Clinical features, mechanisms, and management of pseudoprogression in malignant gliomas. The lancet oncology 9, 453–461 (2008)
2008
-
[41]
Brandes, A. A. et al. MGMT promoter methylation status can predict the incidence and outcome of pseudoprogression after concomitant radiochemotherapy in newly diagnosed glioblastoma patients. J. Clin. Oncol. 26, 2192–2197 (2008)
2008
-
[42]
Zhou, J. et al. Review and consensus recommendations on clinical apt-weighted imaging approaches at 3t: application to brain tumors. Magn. resonance medicine 88, 546–574 (2022). 11/12
2022
-
[43]
Y .et al
Wen, P. Y .et al. Updated response assessment criteria for high-grade gliomas: response assessment in neuro-oncology working group. J. clinical oncology 28, 1963–1972 (2010)
2010
-
[44]
W., Ulmer, S
Henson, J. W., Ulmer, S. & Harris, G. Brain tumor imaging in clinical trials. Am. J. Neuroradiol. 29, 419–424 (2008)
2008
-
[45]
Smith, J. S. et al. Serial diffusion-weighted magnetic resonance imaging in cases of glioma: distinguishing tumor recurrence from postresection injury. J. neurosurgery 103, 428–438 (2005). Author contributions statement The study was designed and conceived by FP, YH, and AG. A...
2005
Reviewed August 8, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.