{"id":"ea7879c1-5127-45ad-8229-dac6060ef7a7","arxiv_id":"2502.03999","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"A self-supervised transformer using MRI, clinical, and dosimetric features distinguishes glioblastoma pseudoprogression from true progression with an external-test AUC of 0.753.","lead":"This paper tests a machine learning model that combines MRI scans, clinical records, and radiotherapy dose information to tell apart true tumor growth from treatment-related pseudo-progression in glioblastoma patients. On a 20-patient external test set, the model reached an AUC of 75.3%, but the small sample makes the result uncertain.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The external SOTA comparison is confounded by TIME_UNTIL_PROGRESSION, a label-correlated clinical feature missing from the baselines; imaging adds only +0.026 AUC over clinical data alone.","rationale":"The reader correctly identified the external test as the fragile part of the argument, and the label-reliability issue is real. However, the single most load-bearing weakness is more specific: the proposed model includes TIME_UNTIL_PROGRESSION, a feature that is strongly correlated with the outcome definition and that the comparison baselines do not use. This directly threatens the headline claim that the multimodal transformer architecture outperforms SOTA, because the gain could come from a confounded clinical variable rather than from the imaging model. The ablation evidence in Table 3 supports this worry: clinical data alone achieves 0.727, and the imaging modality contributes only a small increment to 0.753. The proposed concrete test — retraining with a matched feature set that omits time-to-event — would settle whether the architecture itself adds value. This is addressable in revision, so the conditional verdict remains appropriate rather than moving to rejection.","tokens_in":12301,"tokens_out":7125,"duration_ms":74824,"concrete_test":"Retrain the proposed model and all three baselines on the same Burdenko folds using an identical input feature set that excludes TIME_UNTIL_PROGRESSION and all date-derived covariates, and re-evaluate on the GlioCMV external set. If the proposed ensemble AUC falls from 0.753 to below the best baseline or to the ~0.72 imaging-only level, the SOTA claim is confounded by the temporal feature.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is the 0.753 external AUC and the assertion that this outperforms current SOTA data-driven models. The most load-bearing condition for that claim is that the comparison actually tests the proposed multimodal imaging architecture. That condition is not met. Figure 3 and the Discussion identify TIME_UNTIL_PROGRESSION (time from end of RT to the index progression) as the single most influential feature, and Table 3 shows clinical data alone reaches AUC 0.727 while adding imaging only raises it to 0.753, with imaging alone at 0.717. The external GlioCMV test set labels 12/20 patients by longitudinal imaging follow-up (Methods: Datasets), and PsP is by definition an early, transient post-RT phenomenon; thus time-to-event is correlated with the label-generation process itself. The SOTA baselines (Jang et al., Akbari et al., Sun et al.) were not given this time-to-event feature, so the comparison is not apples-to-apples. The observed advantage may reflect a known clinical heuristic encoded in the input features rather than the self-supervised ViT, cross-modal attention, or multimodal fusion that the paper claims as contributions. Without removing or matching this feature, the 'outperforming SOTA' claim is not secure.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a multimodal transformer-based deep learning model to distinguish pseudoprogression (PsP) from true progression (TP) in glioblastoma after radiotherapy. The model combines a self-supervised Vision Transformer (ViT) encoding two MRI sequences (T1CE, FLAIR), structured clinical features, and radiotherapy dose information via guided cross-modal attention. It is trained on 59 patients from the Burdenko Glioblastoma Progression Dataset with 5-fold cross-validation and tested on 20 patients from the external GlioCMV UKER dataset. The authors report an external AUC of 0.753, which they state outperforms current state-of-the-art data-driven methods (Jang et al., Akbari et al., Sun et al.). The paper also analyzes unimodal vs. multimodal performance, the impact of self-supervised pretraining versus transfer learning or end-to-end training, and SHAP-based feature importance showing TIME_UNTIL_PROGRESSION as the most influential feature.","tokens_in":12671,"tokens_out":3707,"duration_ms":38065,"significance":"If the central claim were secure, the work would be a valuable contribution to a clinically difficult problem: a non-invasive, externally validated tool for PsP/TP differentiation, enhanced by self-supervised pretraining on large public MRI datasets. The use of an external test set and the comparison of multiple training strategies are strengths, as is the integration of dose features from radiotherapy planning. However, the central claim of outperforming state-of-the-art as stated in the abstract and results is not yet supported because the comparison is confounded by a label-correlated clinical feature and because the added value of imaging over clinical data alone is small and statistically unquantified. The paper is relevant to the community, but the evidence in its current form requires substantial revision.","major_comments":[{"comment":"The external state-of-the-art comparison is confounded by TIME_UNTIL_PROGRESSION. Figure 3 and the Discussion state that time from end of radiotherapy to the index progression has the greatest influence on the model's prediction. The baselines (Jang et al., Akbari et al., Sun et al.) were not given this time-to-event feature, and PsP is by definition an early transient post-radiotherapy phenomenon. Moreover, 12 of the 20 external test patients were labeled by longitudinal imaging follow-up rather than histology (Methods: Datasets), so the time feature is directly correlated with the label-generation process. To support the claim that the proposed architecture outperforms existing methods, the authors must either rerun all baselines with the same clinical features (including the time feature) or retrain the proposed model without TIME_UNTIL_PROGRESSION. Without this, the observed AUC gap (0.753 vs. 0.686 for the best baseline) cannot be attributed to the self-supervised ViT, cross-modal attention, or multimodal fusion.","section":"Results (Table 2), Figure 3, Discussion"},{"comment":"The claim that multimodal integration improves prediction is not statistically supported on the external test. Clinical data alone achieves AUC 0.727, and adding imaging raises it to 0.753, an improvement of 0.026 that is smaller than the reported standard deviation of 0.038 for the multimodal model. The external test has only 20 patients, and no confidence intervals, DeLong tests, or other significance measures are reported for any AUC comparison. The authors should report confidence intervals for all external-test metrics and perform paired significance tests between the multimodal model and both unimodal models and each baseline. Without these, the assertion that the fusion mechanism provides a meaningful advantage over clinical data alone is not justified.","section":"Results (Table 3), Unimodal vs. Multimodal"},{"comment":"The feature-selection procedure (choosing the first M features with highest AUC on the cross-validation sets) constitutes model selection performed on the validation folds, and the reported validation performance is therefore optimistically biased. While the external test set remains untouched, the stability of the chosen M=4 features across the five folds is not reported, and it is not clear whether the same feature subset was selected in every fold or whether TIME_UNTIL_PROGRESSION was always selected. The authors should either embed feature selection inside the cross-validation loop and report the selected features per fold, or fix the feature set a priori based on clinical knowledge, to avoid selection-induced optimism and to clarify the role of the time-to-event feature.","section":"Methods: Model Architecture and Training, Feature Importance Analysis"}],"minor_comments":[{"comment":"The abstract repeats the phrase 'outperforming the current state-of-the-art data-driven approaches' twice in slightly different forms; the second occurrence should be replaced with a quantitative statement that is supported by the results after the additional analyses are performed.","section":"Abstract and Introduction"},{"comment":"The term 'colinear' should be 'collinear', and 'miss-classification' later in the Discussion should be 'misclassification'. The affiliations list also contains typos such as 'Germnay' for 'Germany' and an inconsistent university name; these should be corrected before publication.","section":"Methods: Datasets"},{"comment":"The caption text in Figure 1 contains garbled mathematical symbols and duplicated loss terms; the caption should be rewritten so that the self-supervised loss is clearly expressed. In Figure 2, the individual fold ROC curves are informative, but the figure would benefit from labeling the ensembled AUC values on the plots for direct comparison with Table 2.","section":"Figure 1 caption and Figure 2 caption"},{"comment":"The order of feature names in Figure 3 is ambiguous because the vertical axis labels are not shown; the figure should explicitly label the axis and indicate whether TIME_UNTIL_PROGRESSION is the top or bottom bar, since the text claims it is the most influential feature.","section":"Results: Feature Importance Analysis"},{"comment":"The limitations paragraph correctly notes the small sample sizes, but it does not mention the reliance on longitudinal imaging follow-up for 12 of 20 external test labels as a limitation. This should be added, as it directly affects the reference standard for the reported AUC.","section":"Discussion: Limitations"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a clinically relevant problem and uses an external test set, which is commendable. However, the central claim of outperforming the state of the art is not secure because of the time-to-progression confound and the lack of statistical testing on the small external cohort. The authors should be encouraged to reframe the claim to match the evidence, for example by presenting the model as a promising approach pending larger validation, rather than as a definitive improvement over prior methods. The feature-selection procedure also needs to be clarified or embedded in cross-validation. If the authors perform the suggested reanalyses and the results remain largely unchanged, the paper could be acceptable; without them, the current form is not suitable for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: the paper is a solid engineering effort on a real clinical problem, but the headline claim—outperforming state-of-the-art—does not hold up under scrutiny. The main reason is that the model's most influential feature is TIME_UNTIL_PROGRESSION, which the baseline models were not given. That feature is essentially a shortcut: pseudoprogression is an early post-RT phenomenon, and 12 of the 20 external test labels were assigned by longitudinal imaging follow-up, so time-to-event is correlated with the label generation itself. The comparison against Jang, Akbari, and Sun is therefore not apples-to-apples.\n\nWhat the paper does well: it uses an external test cohort, compares three published baselines, includes unimodal ablations, and demonstrates that a self-supervised ViT pretrained on large public glioma MRI datasets can be adapted to this task with limited labeled data. The dose-feature extraction (nnU-Net segmentation, ANTs registration to the planning CT) is nontrivial and clearly described. The SHAP analysis is informative, even if it exposes the shortcut.\n\nThe soft spots are real. With n=20, the external AUC of 0.753 has wide confidence intervals, and no CIs or significance tests are reported. The gain from adding imaging to clinical data is only 0.026 AUC (0.727 to 0.753), which is well within the noise. The feature count M=4 was selected on validation folds, which can bias the external estimate. And the label uncertainty in the 12 imaging-follow-up cases is not addressed with a sensitivity analysis.\n\nNone of this kills the underlying idea. The architecture is reasonable, the self-supervised pretraining makes sense in this data-scarce setting, and the paper is honest about its limitations. But the central claim needs to be scoped down until the model is tested without the time-to-event feature, or at least with baselines that get the same inputs.\n\nI would send this to peer review—a good referee could push the authors to fix the comparison and report uncertainty. I wouldn't cite the AUC as evidence yet.","headline":"Sensible engineering and honest limitations, but the SOTA claim is undermined by a label-correlated time-to-event feature and an n=20 external test.","tokens_in":13164,"tokens_out":3835,"would_cite":false,"duration_ms":33913,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A multimodal transformer that combines self-supervised MRI encoding with clinical and radiotherapy dose features can distinguish pseudoprogression from true progression in glioblastoma, achieving an external-test AUC of 0.753 and…","keywords":["pseudoprogression","true progression","glioblastoma","self-supervised learning","vision transformer","multimodal deep learning","radiotherapy dose","cross-modal attention"],"falsifier":"Retest the same model on an external cohort where every PsP and TP label is histologically confirmed; if the AUC falls below the reported 0.753 or below the baselines, the central claim is weakened. A second check is to recompute the dose features using deformable registration and see whether the contribution of dose features such as D98 and minimum dose changes materially.","tokens_in":12113,"feed_emoji":"🧠","tokens_out":9992,"duration_ms":80999,"temperature":0.7,"pith_summary":"The paper sets out to show that a multimodal transformer can non-invasively distinguish pseudoprogression (PsP) from true progression (TP) in glioblastoma after radiotherapy, using routine MRI, clinical variables, and radiotherapy dose features. Getting this distinction right matters because treating PsP as progression can lead to unnecessary toxic therapy, while mistaking TP for PsP delays effective intervention. The authors demonstrate that a Vision Transformer pretrained in a self-supervised way on 2,317 unlabeled glioma MRI scans, then fused with clinical and dose tokens via guided cross-attention, achieves an external-test AUC of 0.753 and beats three published baselines. They also report that the self-supervised encoder outperforms both end-to-end training and transfer learning, suggesting the approach can work even when labeled examples are scarce.","feed_headline":"Multimodal AI distinguishes true progression from pseudoprogression","feed_subtitle":"A self-supervised vision transformer plus clinical and dose features hits 0.753 AUC on an external test.","key_machinery":"The load-bearing component is a frozen Vision Transformer encoder pretrained with contrastive learning and context restoration on 2,317 unlabeled glioma MRI studies, which turns 160x160x160 two-channel MRI volumes into 512 tokens. A guided cross-attention layer then lets four selected clinical and dose tokens (age, IDH and MGMT status, time to progression, and dose statistics) query those imaging tokens, yielding a compact fused representation that proceeds through self-attention and classification heads. The mechanism's dual role is to fuse modalities and to shrink the token sequence from 512 to 4, avoiding the quadratic cost of self-attention on long sequences.","core_discovery":"The paper's central claim is that combining a self-supervised Vision Transformer encoder for whole-brain FLAIR and T1 post-contrast volumes with clinical and radiotherapy dose information through guided cross-attention discriminates pseudoprogression from true progression better than previous data-driven methods. On a 20-patient external test set from a different institution, the model reaches an AUC of 0.753, accuracy 0.750, sensitivity 0.727, and specificity 0.800, compared with AUCs of 0.686, 0.677, and 0.530 for the CNN-LSTM, CNN-SVM, and random-forest baselines. The paper attributes the improvement to the transformer's ability to model long-range spatial dependencies in whole-volume MRI and to the cross-attention mechanism that lets a small set of clinical and dose tokens guide the imaging representation.","pith_inferences":["The reported gain of self-supervised pretraining over transfer learning (0.753 vs 0.727 AUC) comes from a single 20-patient test set; the confidence intervals likely overlap, so the superiority of SSL over transfer learning should be confirmed on larger cohorts.","Because dose features are extracted by registering follow-up MRI to the planning CT, registration errors could corrupt those inputs; the paper does not quantify this sensitivity, and deformable registration may change both performance and which features matter.","Adding diffusion or perfusion MRI, which the authors list as a limitation, is a natural next test and may push the AUC above 0.75.","The most influential feature, time to progression, encodes the clinical prior that PsP usually appears earlier; an ablation using only clinical and time features would clarify how much the imaging and dose fusion actually adds."],"forward_implications":["A clinically usable, non-invasive tool for distinguishing PsP from TP, potentially reducing unnecessary biopsies and treatment changes.","Evidence that self-supervised pretraining on unlabeled MRI can compensate for scarce labeled data in medical imaging classification.","A general pattern for fusing a few structured clinical or dose variables with thousands of imaging tokens via cross-attention.","A model that relies only on routinely acquired FLAIR and T1 post-contrast MRI plus standard clinical and RT-planning data, making deployment in routine care feasible."],"supporting_citations":[{"why":"CNN-LSTM baseline that the proposed model is compared against on the external test.","marker":"17"},{"why":"CNN-SVM baseline using pretrained CNN features and SVM classification.","marker":"13"},{"why":"Random-forest baseline combining radiomics and clinical features.","marker":"16"},{"why":"Training dataset of glioblastoma progression cases used for cross-validation.","marker":"30"},{"why":"External test dataset from a different center, providing independent evaluation.","marker":"33"},{"why":"Companion report from the same prospective study providing the external test cohort.","marker":"34"},{"why":"Segmentation network used to delineate the enlarging lesion in follow-up MRI.","marker":"31"},{"why":"Registration method used to map follow-up MRI to the planning CT for dose extraction.","marker":"32"},{"why":"Vision Transformer architecture that underpins the self-supervised imaging encoder.","marker":"18"},{"why":"Context-restoration self-supervised method used in the pretraining objective.","marker":"22"}],"fun_headline_variants":["AI separates GBM true progression from pseudoprogression after RT","Self-supervised transformer with dose data beats baselines for GBM progression","External test: multimodal AI for GBM progression hits 0.753 AUC","Guided cross-attention AI improves GBM progression classification","0.753 AUC: AI for post-RT GBM progression on external test"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The external-test labels are treated as ground truth even though 12 of the 20 patients were classified by longitudinal imaging follow-up rather than histology; if any of those labels are wrong, the reported AUC is directly biased.","fun_headline_variants_meta":{"raw":{"variants":["AI separates GBM true progression from pseudoprogression after RT","Self-supervised transformer with dose data beats baselines for GBM progression","External test: multimodal AI for GBM progression hits 0.753 AUC","Guided cross-attention AI improves GBM progression classification","0.753 AUC: AI for post-RT GBM progression on external test"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000498,"raw_usage":{"total_tokens":2491,"prompt_tokens":1049,"completion_tokens":1442,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":665,"completion_tokens_details":{"reasoning_tokens":1347}},"tokens_in":665,"tokens_out":1442,"duration_ms":13252,"temperature":1.0,"reasoning_tokens":1347,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T23:54:02.061650+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retest the same model on an external cohort where every PsP and TP label is histologically confirmed; if the AUC falls below the reported 0.753 or below the baselines, the central claim is weakened. A second check is to recompute the dose features using deformable registration and see whether the contribution of dose features such as D98 and minimum dose changes materially.","supporting_citations":[{"cited_title":"H., Kim, I","cited_arxiv_id":null,"evidence_quote":"CNN-LSTM baseline that the proposed model is compared against on the external test."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"CNN-SVM baseline using pretrained CNN features and SVM classification."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Random-forest baseline combining radiomics and clinical features."},{"cited_title":"V .et al","cited_arxiv_id":null,"evidence_quote":"Training dataset of glioblastoma progression cases used for cross-validation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"External test dataset from a different center, providing independent evaluation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Companion report from the same prospective study providing the external test cohort."},{"cited_title":"F., Kohl, S","cited_arxiv_id":null,"evidence_quote":"Segmentation network used to delineate the enlarging lesion in follow-up MRI."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Context-restoration self-supervised method used in the pretraining objective."}],"review_version":1}