Pith. sign in

REVIEW 4 major objections 5 minor 21 references

AI-based response assessment and prediction in longitudinal imaging for brain metastases treated with stereotactic radiosurgery

T0 review · 4 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash

Pith's one-line read The paper claims that routine longitudinal MRI, processed by a fully automated pipeline, can accurately assess and predict one-year response of brain metastases to stereotactic radiosurgery.

desk verdict Useful longitudinal BM dataset and pipeline, but the 0.90 AUC overstates early prediction because the CV split is not patient-level and CR persistence dominates. read the letter →

arxiv 2509.06396 v1 pith:QZNZNQZM submitted 2025-09-08 cs.CV

classification cs.CV
keywords longitudinalMRIbrainmetastasesstereotacticradiosurgeryresponsepredictiongrowthtrajectoriesgraphmachinelearningradiomicsUNetresegmentation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a fully automated imaging pipeline can turn routine follow-up scans of brain metastases into reliable lesion-level response labels, and that one-year response can be predicted as early as the first follow-up scan. On a curated cohort of 896 lesions in 177 patients, gradient boosting on pre-treatment and 60-day post-treatment MRI reaches 0.90 AUC for complete response versus not, and 0.82 AUC for the broader responding/non-responding split; a graph model reaches 0.88 AUC with one flexible model across input time-point configurations. The same data-driven analysis finds five characteristic volume trajectories, including an early-shrinking group, a pseudoprogression-like group, moderate responders, and progressive groups. If the evaluation is unbiased, these results imply that routine clinical images alone can support early, personalized treatment decisions without manual annotation.

What carries the argument

The load-bearing mechanism is the lesion-level longitudinal volume trajectory: pre-treatment delineations from radiosurgery planning are propagated through registered follow-up MRIs by a UNet-based resegmentation model, individual lesions are matched by overlap and centroid distance, and volumes are resampled to seven uniform 60-day intervals. Relative volume trajectories are clustered with a Gaussian-mixture-style model into five patterns, and per-time-point radiomic and volume features feed either gradient-boosted trees or a graph attention network whose nodes are time points. The graph formulation lets one model take any subset of follow-up time points, while the classical model needs a s

What would settle it

Recompute one-year response labels on a random subset of lesions from manual expert segmentations of the same MRIs, then retrain and evaluate the same features and models. If AUC for CR vs non-CR drops substantially below 0.90, the automated-label noise is the source of the performance. An external multi-centre cohort would also settle generalizability.

Watch

Extended reading notes

Core claim

The central claim is that the one-year response of a brain metastasis to stereotactic radiosurgery is largely legible in the first post-treatment MRI, and that a data-curation pipeline can extract this signal at scale. Lesions are followed across uniformly resampled 60-day time points; clustering identifies five dominant growth trajectories. For CR vs non-CR, classification reaches 0.90 AUC (CI 0.88–0.92) using only t0 and t1 features with gradient boosting; a graph attention network reaches 0.88 AUC (CI 0.86–0.90) and handles variable input configurations in one model. Performance rises with each additional time point but the largest jump comes from adding t1.

Load-bearing premise

The automated volume measurements, lesion matching, and feature extraction are accurate enough that the response labels and the predictors are not corrupted by shared segmentation or registration errors; the paper's own quality check reports about 16% partial segmentation failures and 1% misregistrations.

Editorial extensions

If this is right

  • One-year complete response can be predicted with high AUC at 60 days post-treatment from routine MRI, so treatment decisions could be adapted a year earlier.
  • Five data-driven trajectory clusters give clinicians a compact vocabulary for interpreting lesion growth after radiosurgery.
  • Automated curation makes large longitudinal cohorts feasible without manual annotation, enabling larger studies of growth patterns and biomarkers.
  • Including more follow-up time points improves prediction monotonically, with the largest gain from the first follow-up scan.
  • The graph model trades a small amount of accuracy for deployment flexibility, using a single model across different available time-point configurations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the 0.90 AUC holds under external validation, response prediction could be recast as a screening tool to select lesions for early salvage, rather than only retrospective assessment.
  • The pseudoprogression-like cluster suggests a trajectory-based classifier could eventually separate radiation necrosis from true progression using only volumes, a clinically urgent distinction the paper does not directly test.
  • The reported 16% partial segmentation failure rate implies that a human-in-the-loop correction step or uncertainty filter may be needed before clinical deployment; the reported AUC could be optimistic if label noise from automated volumes is correlated with features.
  • Because most one-year CR lesions are already CR at t1, the high AUC may partly reflect an early binary state rather than true predictive signal; testing on lesions not already CR at t1 would isolate genuine prediction.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents an automated pipeline for curating longitudinal MRI/CT/RTStruct data of brain metastases treated with stereotactic radiosurgery, yielding 896 lesions in 177 patients with follow-up to one year. The authors use Gaussian-mixture clustering of normalized volume trajectories to identify five response patterns, and then predict lesion-level one-year response (CR vs. non-CR and responding vs. non-responding) using gradient boosting (LGBM) and graph attention networks on time-point graphs. The headline result is an AUC of 0.90 (CI95%=0.88–0.92) for CR vs. non-CR using only t0 and t1 features with LGBM, with GML reaching 0.88. The paper emphasizes the scalability of the curation pipeline and the potential for early-response decision support.

Significance. If the evaluation is unbiased, this is a useful contribution: it demonstrates that a fully automated longitudinal curation pipeline can assemble a large real-world lesion-level dataset, and that early follow-up imaging carries substantial predictive signal for one-year response. The trajectory clustering is clinically interpretable, and the code is made available, which supports reproducibility. The reported AUCs are high but rest on a small number of methodological choices; the central claim therefore hinges on whether the evaluation protocol excludes leakage and whether the automated volumetry is reliable enough to support both the labels and the features.

major comments (4)
  1. [Section 2.3 (Cross-validation)] The 5-fold cross-validation is described without stating that folds were split by patient. With 896 lesions from only 177 patients, random lesion-level splits will place lesions from the same patient in both training and test folds. Since the feature set includes patient-level variables (primary tumor site, histology, total lesion burden, lesion count) and lesions from one patient share systemic treatment and imaging conditions, the model can exploit patient identity or patient-level confounding to inflate the test AUC. The manuscript reports pooled test predictions across folds without any patient-level grouping. This directly affects the headline 0.90 AUC. Please re-run the evaluation with patient-stratified folds and report the patient-level cross-validated AUC; alternatively, justify why lesion-level independence is acceptable here and quantify the amount of same-patient leakage.
  2. [Section 3.3 / Table 2] The CR vs. non-CR prediction at t0:t1 likely benefits from the trivial persistence of complete response. The authors note in Section 3.2 and Figure 2 that more than 50% of lesions achieving one-year CR are already in CR at the first follow-up. Since the t1 volume (specifically zero volume) is a feature and the one-year label is derived from the same volumetric resegmentation pipeline, a lesion already at CR at t1 is almost deterministically labeled CR at t6. The paper acknowledges this pattern only as a possible contributor to separability, but does not quantify it. To support the early-prediction claim, report AUC on the subset of lesions not already in CR at t1, or at least report the fraction of test predictions that are explained by this persistence effect. Without this, the 0.90 AUC cannot be interpreted as novel predictive information beyond what is already visible at the first fol
  3. [Section 3.1 / Section 2.3] The quality control subsection reports that roughly 16% of lesion time series had partial segmentation or other failures on a random subset of 66 series. The response labels and the volume/radiomic features are both derived from these resegmentations. If segmentation errors are systematic (e.g., more frequent for small lesions or for lesions near the skull), they can create a shared error component in labels and features that inflates the apparent predictive performance. The paper also applies a 'semi-automatic sample rejection based on suspicious trajectory detection and marginal manual observation' without reporting how many lesions were excluded or on what exact criteria. Please provide a sensitivity analysis excluding QC-flagged time points or lesions, and report the rejection rate and its possible effect on the reported AUC.
  4. [Section 2.3 / Section 3.3 (statistical inference)] The permutation test is used only to compare models, and the reported confidence intervals are obtained by bootstrapping pooled fold predictions. This procedure treats predictions as independent, which is not valid when lesions are nested in patients and when folds may contain the same patient. The significance of the improvement from adding t1 is also tested with multiple comparisons across time-point configurations, but no correction is applied. At minimum, the statistical analysis should account for patient-level clustering and should state whether any multiple-comparison correction was used.
minor comments (5)
  1. [Abstract and Section 1] Duplicate phrase 'Graph Machine Learning Graph Machine Learning (GML)' in the abstract. Also 'machine learning Machine Learning (ML)' in Section 1. Please proofread the notation for expanded acronyms.
  2. [Section 2.2] The choice of target cluster count is fixed to 5, but no sensitivity analysis is provided. Since the clustering is exploratory, a small paragraph reporting stability across k values would strengthen the interpretation of the five trajectories.
  3. [Section 2.3] The definition of 'responding' vs. 'non-responding' groups is clinically meaningful, but the class imbalance and the exact RANO-BM thresholds are stated only in Section 1. It would help to restate the volumetric criteria near the prediction task definition, since the thresholds drive both the labels and the feature engineering.
  4. [Section 3.1] The sentence 'For many patients (76.5%), RTStruct files were stored on separate databases that cannot be queried' is important for the generalizability of the curation pipeline, but it is not clear whether this refers to all 1135 initial patients or to the 177-patient cohort. Please clarify.
  5. [Section 3.2 / Figure 3] The clusters are labeled only by number in the figure and text. Naming them descriptively (e.g., 'fast responders', 'pseudoprogression-like', 'moderate responders', 'rapid progressors', 'late progressors') would improve readability, especially for clinical readers.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the t0/t1-to-t6 prediction uses distinct time points and externally defined response labels; acknowledged CR persistence is a clinical effect, not a definitional tautology.

full rationale

The paper's central predictive claim is that features from t0 and t1 predict the one-year (t6) response. The response labels are assigned according to published volumetric RANO-BM criteria using the re-segmented volumes at t6, while the t1 volume feature is an earlier measurement of the same lesion; this is a longitudinal prediction, not a self-definitional identity. The paper explicitly notes in Section 3.3 that most one-year CR lesions are already CR at the first follow-up, which explains the high AUC for CR vs. non-CR; this is an acknowledged persistence effect, not a case where the label is defined from the feature. The UNet re-segmentation model from the authors' prior work [2] is used as an external tool with a reported Dice of 0.78 and is not itself the object of the prediction. No fitted parameter is renamed as a prediction, no uniqueness theorem is imported, and no ansatz is smuggled in via citation. The main threats to the reported AUC are validation-related (e.g., possible absence of patient-level fold splitting in the 5-fold CV), but these are correctness/leakage concerns rather than circularity under the definitions used here. Accordingly, no circular step is identified.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim relies on assumptions about segmentation accuracy, registration quality, tracking correctness, and the clinical validity of volumetric response thresholds. No new physical or conceptual entities are introduced; the 'five trajectories' are clusters derived from the data rather than invented entities.

free parameters (4)
  • number of clusters = 5
    Target cluster count fixed to 5 for interpretability, not selected by model selection (Section 2.2).
  • resampling time grid = 7 points at 60-day intervals
    All trajectories resampled to t0..t6 to handle irregular scan timing; interval matches the reported mean follow-up of 60±20 days (Section 2.2).
  • graph connectivity = fully connected, directed to past
    Selected as optimal based on volume-only baselines in the same dataset (Section 2.3).
  • GAT training hyperparameters = learning rate 1e-4, warm restarts every 50 epochs, early stopping patience 20, up to 1000 epochs
    Chosen by the authors without a reported hyperparameter search (Section 2.3).
assumptions (5)
  • domain assumption Volumetric RANO-BM thresholds (34.3% and 172.8% volume ratios) define clinically meaningful response categories at lesion level.
    Section 1 adapts RANO-BM to lesion level; these thresholds determine every label and the prediction target.
  • domain assumption The UNet resegmentation model from [2] (Dice 0.78 on a separate test set) generalizes to the CHUV cohort and produces volumes accurate enough for labels and features.
    Used for all follow-up segmentations; the paper's own QC reports approximately 16% partial segmentation failures (Section 3.1).
  • domain assumption Affine registration is sufficient to align CT-defined ROIs to all follow-up MRIs and to the t0 MRI.
    Section 2.1 states all registrations are affine with ANTs; large deformations could misalign small lesions.
  • domain assumption Lesion correspondence over time is correctly established by maximum overlap and minimum centroid distance matching.
    Used to build trajectories; no quantitative evaluation of tracking errors is provided.
  • standard math Gaussian mixture clustering with StepMix and a fixed target of 5 clusters reveals meaningful biological trajectory groups.
    Section 2.2 fixes the cluster count for interpretability; the clusters are descriptive, not externally validated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AI-based response assessment and prediction in longitudinal imaging for brain metastases treated with stereotactic radiosurgery." pith.science (2026). https://pith.science/paper/QZNZNQZM

@misc{pith2026250906396,
  author       = {Pith},
  title        = {Pith review of: AI-based response assessment and prediction in longitudinal imaging for brain metastases treated with stereotactic radiosurgery},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QZNZNQZM}},
  note         = {Machine review of arXiv:2509.06396}
}
read the original abstract

Brain Metastases (BM) are a large contributor to mortality of patients with cancer. They are treated with Stereotactic Radiosurgery (SRS) and monitored with Magnetic Resonance Imaging (MRI) at regular follow-up intervals according to treatment guidelines. Analyzing and quantifying this longitudinal imaging represents an intractable workload for clinicians. As a result, follow-up images are not annotated and merely assessed by observation. Response to treatment in longitudinal imaging is being studied, to better understand growth trajectories and ultimately predict treatment success or toxicity as early as possible. In this study, we implement an automated pipeline to curate a large longitudinal dataset of SRS treatment data, resulting in a cohort of 896 BMs in 177 patients who were monitored for >360 days at approximately two-month intervals at Lausanne University Hospital (CHUV). We use a data-driven clustering to identify characteristic trajectories. In addition, we predict 12 months lesion-level response using classical as well as graph machine learning Graph Machine Learning (GML). Clustering revealed 5 dominant growth trajectories with distinct final response categories. Response prediction reaches up to 0.90 AUC (CI95%=0.88-0.92) using only pre-treatment and first follow-up MRI with gradient boosting. Similarly, robust predictive performance of up to 0.88 AUC (CI95%=0.86-0.90) was obtained using GML, offering more flexibility with a single model for multiple input time-points configurations. Our results suggest potential automation and increased precision for the comprehensive assessment and prediction of BM response to SRS in longitudinal MRI. The proposed pipeline facilitates scalable data curation for the investigation of BM growth patterns, and lays the foundation for clinical decision support systems aiming at optimizing personalized care.

Figures

Figures reproduced from arXiv: 2509.06396 by the authors.

Figure 1
Figure 1. Data curation pipeline. (1) Data is extracted and depersonalized from the [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Sankey flow diagram of lesion response categories over time. Imaging time [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Dominant lesion growth trajectories revealed by StepMix with a target [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

21 extracted references · 19 canonical work pages

  1. [1]

    Nature Reviews Disease Primers5(1), 5 (Jan 2019)

    Achrol, A.S., Rennert, R.C., Anders, C., Soffietti, R., Ahluwalia, M.S., Nayak, L., Peters, S., Arvold, N.D., Harsh, G.R., Steeg, P.S., Chang, S.D.: Brain metastases. Nature Reviews Disease Primers5(1), 5 (Jan 2019). https://doi.org/10.1038/s41572-018-0055-y

  2. [2]

    Scientific reports14(1), 31603 (2024)

    Andrearczyk, V., Schiappacasse, L., Abler, D., Wodzinski, M., Hottinger, A., Raccaud, M., Bourhis, J., Prior, J.O., Dunet, V., Depeurnge, A.: Auto- matic detection and multi-component segmentation of brain metastases in longitudinal mri. Scientific reports14(1), 31603 (2024)

  3. [3]

    Neuro- Oncology Advances7(1), vdae216 (01 2025)

    Andrearczyk, V., Schiappacasse, L., Raccaud, M., Bourhis, J., Prior, J.O., Cuendet, M.A., Hottinger, A.F., Dunet, V., Depeursinge, A.: The value of ai for assessing longitudinal brain metastases treatment response. Neuro- Oncology Advances7(1), vdae216 (01 2025). https://doi.org/10.1093/ noajnl/vdae216

  4. [4]

    Insight j2(365), 1–35 (2009)

    Avants, B.B., Tustison, N., Song, G., et al.: Advanced normalization tools (ants). Insight j2(365), 1–35 (2009)

  5. [5]

    Cancers15(16) (2023)

    Cao, Y., Parekh, V.S., Lee, E., Chen, X., Redmond, K.J., Pillai, J.J., Peng, L., Jacobs, M.A., Kleinberg, L.R.: A multidimensional connectomics- and radiomics-based advanced machine-learning framework to distinguish radi- ation necrosis from true progression in brain metastases. Cancers15(16) (2023)

  6. [6]

    Scientific Reports14(1), 11085 (May 2024)

    Cho, S.J.e.a.: Prediction of treatment response after stereotactic radio- surgery of brain metastasis using deep learning and radiomics on lon- gitudinal mri data. Scientific Reports14(1), 11085 (May 2024). https: //doi.org/10.1038/s41598-024-60781-5

  7. [7]

    Current Oncology30(11), 9382–9391 (2023)

    Douri, K., Iorio-Morin, C., Mercure-Cyr, R., Figueiredo, G., Touchette, C.J., Masson-Côté, L., Mathieu, D.: Response assessment in brain metas- tases managed by stereotactic radiosurgery: A reappraisal of the rano-bm criteria. Current Oncology30(11), 9382–9391 (2023). https://doi.org/10. 3390/curroncol30110679, https://www.mdpi.com/1718-7729/30/11/679

  8. [8]

    Scientific Data3(1), 160044 (2016)

    Gorgolewski, K.J., et al.: The brain imaging data structure, a format for organizing and describing outputs of neuroimaging experiments. Scientific Data3(1), 160044 (2016). https://doi.org/10.1038/sdata.2016.44, https:// doi.org/10.1038/sdata.2016.44

Show all 21 references
  1. [9]

    Cancer Research77(21), e104–e107 (10 2017)

    van Griethuysen, J.J., et al.: Computational radiomics system to de- code the radiographic phenotype. Cancer Research77(21), e104–e107 (10 2017). https://doi.org/10.1158/0008-5472.CAN-17-0339, https://doi.org/ 10.1158/0008-5472.CAN-17-0339

  2. [10]

    Physics and Imaging in Radiation Oncology27, 100452 (Jul 2023)

    Hsu, D.G., Åse Ballangrud, Prezelski, K., Swinburne, N.C., Young, R., Beal, K., Deasy, J.O., Cerviño, L., Aristophanous, M.: Automatically tracking brain metastases after stereotactic radiosurgery. Physics and Imaging in Radiation Oncology27, 100452 (Jul 2023). https://doi.org...

  3. [11]

    Radiation Oncology19(1), 182 (2024)

    Kanakarajan, H., De Baene, W., Hanssens, P., Sitskoorn, M.: Predicting local control of brain metastases after stereotactic radiotherapy with clini- cal, radiomics and deep learning features. Radiation Oncology19(1), 182 (2024). https://doi.org/10.1186/s13014-024-02573-9

  4. [12]

    Surgical Neurology Interna- tional11, 254 (2020)

    Kobets, A., Backus, R., Fluss, R., Lee, A., Lasala, P.: Evaluating the natural growth rate of metastatic cancer to the brain. Surgical Neurology Interna- tional11, 254 (2020). https://doi.org/10.25259/SNI_291_2020

  5. [13]

    The Lancet Oncology16(6), e270–e278 (2015)

    Lin, N.U., et al.: Response assessment criteria for brain metastases: proposal from the rano group. The Lancet Oncology16(6), e270–e278 (2015). https: //doi.org/10.1016/S1470-2045(15)70057-4

  6. [14]

    ArXiv [Preprint] (Dec 2024)

    Moawad, A.W., et al.: The brain tumor segmentation - metastases (brats- mets) challenge 2023: Brain metastasis segmentation on pre-treatment mri. ArXiv [Preprint] (Dec 2024). https://doi.org/arXiv:2306.00838v3, preprint published on arXiv. PMID: 37396600. PMCID: PMC10312806

  7. [15]

    arXiv preprint arXiv:2304.03853 (2023)

    Morin, S., Legault, R., Laliberté, F., Bakk, Z., Giguère, C.É., de la Sablon- nière, R., Lacourse, É.: Stepmix: A python package for pseudo-likelihood estimation of generalized mixture models with external variables. arXiv preprint arXiv:2304.03853 (2023)

  8. [16]

    Neuro-Oncology Advances5(1), vdac179 (12 2022)

    Ocaña-Tienda, B., Pérez-Beteta, J., Molina-García, D., Asenjo, B., Or- tiz de Mendivil, A., Albillo, D., Pérez-Romasanta, L., González del Por- tillo, E., Llorente, M., Carballo, N., Arana, E., Pérez-García, V.: Growth dynamics of brain metastases differentiate radiation necro...

  9. [17]

    https://doi.org/10.1093/noajnl/vdad161

    Ocaña-Tienda, B., Pérez-Beteta, J., Romero-Rosales, J.A., Asenjo, B., Or- tizdeMendivil,A.,PérezRomasanta,L.A.,AlbilloLabarra,J.D.,Nagib,F., VidalDenis,M.,Luque,B.,Arana,E.,Pérez-García,V.M.:Volumetricanal- ysis:Rethinkingbrainmetastasesresponseassessment.Neuro-OncologyAd- van...

  10. [18]

    Frontiers in Oncology10(2021)

    Oft, D., Schmidt, M.A., Weissmann, T., Roesch, J., Mengling, V., Masitho, S., Bert, C., Lettmaier, S., Frey, B., Distel, L.V., Fietkau, R., Putz, F.: Volumetric regression in brain metastases after stereotactic radiotherapy: Time course, predictors, and significance. Frontiers...

  11. [19]

    American Journal of Neuroradiology32(10), 1885–1892 (2011)

    Patel, T., McHugh, B., Bi, W., Minja, F., Knisely, J., Chiang, V.: A compre- hensive review of mr imaging changes following radiosurgery to 500 brain metastases. American Journal of Neuroradiology32(10), 1885–1892 (2011). https://doi.org/10.3174/ajnr.A2668

  12. [20]

    Veličković, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., Bengio, Y.: Graph attention networks (2018)

  13. [21]

    International Jour- nalofMolecularSciences14(1),1383–1411(2013).https://doi.org/10.3390/ ijms14011383, https://www.mdpi.com/1422-0067/14/1/1383

    Wilhelm, I., Molnár, J., Fazakas, C., Haskó, J., Krizbai, I.A.: Role of the blood-brain barrier in the formation of brain metastases. International Jour- nalofMolecularSciences14(1),1383–1411(2013).https://doi.org/10.3390/ ijms14011383, https://www.mdpi.com/1422-0067/14/1/1383

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.