pith. sign in

arxiv: 2607.00500 · v1 · pith:AQ5JG26Wnew · submitted 2026-07-01 · ⚛️ physics.med-ph · cs.CV

Closed-loop coupling of personalised and foundation models for real-time treatment guidance with MRI

James Grover , Emily A. Hewson , Andrew Phair , Michael Ferraro , Hilary L. Byrne , Paul Keall , Michael G. Jameson , David E.J. Waddington This is my paper

Pith reviewed 2026-07-02 02:12 UTC · model grok-4.3

classification ⚛️ physics.med-ph cs.CV
keywords closed-loop couplingpersonalised modelsfoundation modelsMRI-guided radiotherapymotion predictionreal-time imagingdosimetric erroranatomical segmentation
0
0 comments X

The pith

Coupling a personalised predictor with a streaming foundation model in a closed loop improves real-time anatomical forecasts during MRI-guided treatment.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces a framework that pairs a patient-specific model, which forecasts future anatomy to offset imaging delays, with a foundation model that supplies ongoing anatomical segmentations to keep the predictor grounded and updated during a session. Personalised models alone lose anatomical accuracy over time, while foundation models alone cannot adjust to individual motion patterns, so the closed loop uses each to correct the other in real time. Tests on a digital phantom and on intrafraction MRI scans from radiotherapy patients show gains in prediction quality and lower dosimetric error at a 400 ms horizon, all while meeting clinical latency limits. The authors present this coupling as a reusable pattern for any motion-affected image-guided procedure.

Core claim

A closed-loop coupling framework that combines patient-specific temporal prediction with continuous segmentation-based anatomical interpretation from a foundation model improves anatomical prediction and reduces dosimetric error for a 400 ms prediction horizon compared with existing approaches, while remaining within clinically relevant latency constraints, as shown on digital phantoms and patient intrafraction MRI data.

What carries the argument

Closed-loop coupling framework in which the personalised model predicts future anatomy to compensate for latency and the streaming foundation model supplies real-time anatomical supervision that continuously updates the personalised predictor.

If this is right

  • The method compensates for system latency while staying inside clinical time limits.
  • It yields better anatomical forecasts and lower dose errors than either personalised models or foundation models used alone.
  • The same coupling pattern applies across image-guided therapies that suffer from motion, including biopsy and deep brain stimulation.
  • Validation used both synthetic phantoms and real intrafraction MRI from radiotherapy patients.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The approach may extend to other real-time imaging modalities if the foundation model can be adapted to produce comparable segmentations.
  • Continuous supervision from the foundation model could lower the frequency of full retraining for the personalised component.
  • If the coupling proves robust, it could support adaptive treatment plans that respond to anatomical changes detected mid-session.

Load-bearing premise

The streaming foundation model must supply stable, accurate anatomical segmentations that update the personalised predictor without introducing new instabilities or errors during live treatment.

What would settle it

A side-by-side test on the same patient MRI sequences showing that the closed-loop method produces equal or higher anatomical prediction error or dosimetric error than the best existing personalised or foundation-only baseline at 400 ms horizon.

read the original abstract

Image-guided therapies, including radiotherapy, biopsy and deep brain stimulation, rely on real-time targeting of anatomical structures. However, in the presence of motion, imaging latencies create a temporal misalignment between observed and true anatomy, compromising treatment accuracy. Artificial intelligence-based frameworks have increasingly been presented to close this latency gap, but leading personalised models can fail due to a lack of stable anatomical grounding. Foundation models can provide grounded behaviour, but they do not adapt to real-time, individual patient dynamics. Here we introduce a closed-loop coupling framework that synergises patient-specific temporal prediction with continuous segmentation-based anatomical interpretation from a foundation model. A personalised model predicts future anatomy to compensate for system latency, while a streaming foundation model provides anatomical supervision used to continuously update the temporal predictor in real time during treatment. We validate the framework using a digital phantom and intrafraction magnetic resonance imaging (MRI) from patients undergoing MRI-guided radiotherapy. For a prediction horizon of 400 ms, the proposed method improves anatomical prediction and reduces dosimetric error compared with existing approaches, within clinically relevant latency constraints. These results establish closed-loop coupling as a general strategy for real-time image-guided intervention.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 0 minor

Summary. The manuscript introduces a closed-loop coupling framework that combines a patient-specific temporal prediction model with continuous anatomical supervision from a streaming foundation model to compensate for imaging latencies in real-time image-guided therapies (radiotherapy, biopsy, deep brain stimulation). A personalised model predicts future anatomy while the foundation model provides segmentation-based updates to the predictor during treatment. Validation is performed on a digital phantom and intrafraction MRI from patients undergoing MRI-guided radiotherapy, with the claim that for a 400 ms prediction horizon the method improves anatomical prediction accuracy and reduces dosimetric error relative to existing approaches while satisfying clinical latency constraints.

Significance. If the closed-loop stability holds, the framework would represent a meaningful advance by addressing the lack of stable anatomical grounding in personalised models through continuous foundation-model supervision, potentially enabling more accurate motion compensation in MRI-guided interventions. The use of both phantom and patient intrafraction data provides a reasonable starting point for validation, though the absence of explicit stability analysis limits the strength of the current evidence.

major comments (1)
  1. [Validation] Validation (as described): the experiments on the digital phantom and intrafraction patient MRI do not report analysis of closed-loop divergence, sensitivity to segmentation jitter from the foundation model, or long-horizon error propagation when foundation-model outputs deviate from ground truth. This is load-bearing for the central claim because the 400 ms dosimetric improvement requires that the streaming foundation model provides stable supervision that updates the personalised predictor without introducing instabilities or error accumulation over treatment duration.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the constructive feedback on our manuscript. We address the single major comment below.

read point-by-point responses

  1. Referee: [Validation] Validation (as described): the experiments on the digital phantom and intrafraction patient MRI do not report analysis of closed-loop divergence, sensitivity to segmentation jitter from the foundation model, or long-horizon error propagation when foundation-model outputs deviate from ground truth. This is load-bearing for the central claim because the 400 ms dosimetric improvement requires that the streaming foundation model provides stable supervision that updates the personalised predictor without introducing instabilities or error accumulation over treatment duration.

    Authors: We agree that explicit analysis of closed-loop divergence, sensitivity to foundation-model segmentation jitter, and long-horizon error propagation would strengthen the validation and directly support the stability claim. The reported experiments on the digital phantom and patient intrafraction MRI demonstrate consistent anatomical and dosimetric improvements at 400 ms without observed instabilities, but do not include the requested sensitivity or propagation studies. In the revised manuscript we will add these analyses, including controlled perturbations to foundation-model outputs and evaluation over longer sequences, to address this point. revision: yes

Circularity Check

0 steps flagged

No circularity; framework description contains no load-bearing derivations or self-referential reductions

full rationale

The provided abstract and description introduce a closed-loop coupling of personalised temporal predictors with streaming foundation-model segmentation but contain no equations, fitted parameters presented as predictions, self-citations invoked as uniqueness theorems, or ansatzes smuggled via prior work. Validation claims rest on phantom and patient MRI experiments without any reduction of the 400 ms improvement result to the inputs by construction. The central premise of stable anatomical supervision is stated as an empirical outcome rather than derived from self-referential definitions, satisfying the criteria for a self-contained proposal.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only input provides no identifiable free parameters, axioms, or invented entities; full text would be needed to audit these.

pith-pipeline@v0.9.1-grok · 5761 in / 922 out tokens · 35270 ms · 2026-07-02T02:12:58.508490+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

65 extracted references · 2 canonical work pages · 1 internal anchor

  1. [1]

    Chun, J. et al. Intentional deep overfit learning (IDOL): A novel deep learning strategy for adaptive radiation therapy. Med Phys 49, 488-496 (2022)

    2022

  2. [2]

    Lombardo, E. et al. Patient -Specific Deep Learning Tracking Framework for Real-Time 2D Target Localization in Magnetic Resonance Imaging- Guided Radiation Therapy. Int J Radiat Oncol Biol Phys 122, 827-837 (2025)

    2025

  3. [3]

    & Keall, P

    Hindley, N., Shieh, C.C. & Keall, P. A patient-specific deep learning framework for 3D motion estimation and volumetric imaging during lung cancer radiotherapy. Phys Med Biol 68, 14NT01 (2023)

    2023

  4. [4]

    Mylonas, A. et al. Patient -specific prostate segmentation in kilovoltage images for radiation therapy intrafraction monitoring via deep learning. Commun Med 5, 212 (2025)

    2025

  5. [5]

    Ma, J. et al. Segment anything in medical images. Nat Commun 15, 654 (2024)

    2024

  6. [6]

    & Wang, L

    Sun, Y., Wang, L., Li, G., Lin, W. & Wang, L. A foundation model for enhancing magnetic resonance images and downstream segmentation, registration and diagnostic tasks. Nat Biomed Eng 9, 521-538 (2025)

    2025

  7. [7]

    Lombardo, E. et al. Experimental comparison of linear regression and LSTM motion prediction models for MLC-tracking on an MRI -linac. Med Phys 50, 7083-7092 (2023)

    2023

  8. [8]

    & Steeden, J

    Nayak, K.S., Lim, Y., Campbell - Washburn, A.E. & Steeden, J. Real - Time Magnetic Resonance Imaging. J Magn Reson Imaging 55, 81-99 (2022)

    2022

  9. [9]

    Keall, P.J. et al. Integrated MRI-guided radiotherapy - opportunities and challenges. Nat Rev Clin Oncol 19, 458-470 (2022)

    2022

  10. [10]

    Zafar, A. et al. MRI -Guided High- Intensity Focused Ultrasound as an Emerging Therapy for Stroke: A Review. J Neuroimaging 29, 5 -13 (2019)

    2019

  11. [11]

    & Hansen, M.S

    Campbell-Washburn, A.E., Faranesh, A.Z., Lederman, R.J. & Hansen, M.S. Magnetic Resonance Sequences and Rapid Acquisition for MR -Guided Interventions. Magn Reson Imaging Clin N Am 23, 669-679 (2015)

    2015

  12. [12]

    He, Z. et al. A deep unrolled neural network for real-time MRI-guided brain intervention. Nat Commun 14, 8257 (2023)

    2023

  13. [13]

    & Richardson, R.M

    Lee, P.S. & Richardson, R.M. Interventional MRI-Guided Deep Brain Stimulation Lead Implantation. Neurosurg Clin N Am 28, 535- 544 (2017)

    2017

  14. [14]

    & Kumar, S

    Yoganathan, S.A., Maria Das, K.J., Agarwal, A. & Kumar, S. Magnitude, Impact, and Management of Respiration-induced Target Motion in Radiotherapy Treatment: A Comprehensive Review. J Med Phys 42, 101-115 (2017)

    2017

  15. [15]

    Keall, P.J. et al. The management of respiratory motion in radiation oncology report of AAPM Task Group 76. Med Phys 33, 3874-3900 (2006)

    2006

  16. [16]

    & Low, D.A

    Thorwarth, D. & Low, D.A. Technical Challenges of Real-Time Adaptive MR- Guided Radiotherapy. Front Oncol 11, 634507 (2021)

    2021

  17. [17]

    Waddington, D.E.J. et al. Real -time radial reconstruction with domain transform manifold learning for MRI - guided radiotherapy. Med Phys 50, 1962-1974 (2023)

    1962

  18. [18]

    & Raaymakers, B.W

    Glitzner, M., Woodhead, P.L., Borman, P.T.S., Lagendijk, J.J.W. & Raaymakers, B.W. Technical note: MLC-tracking performance on the Elekta unity MRI -linac. Phys Med Biol 64, 15NT02 (2019)

    2019

  19. [19]

    Uijtewaal, P. et al. Dosimetric evaluation of MRI -guided multi -leaf collimator tracking and trailing for lung stereotactic body radiation therapy. Med Phys 48, 1520-1532 (2021)

    2021

  20. [20]

    Uijtewaal, P. et al. First experimental demonstration of VMAT combined with MLC tracking for single and multi fraction lung SBRT on an MR -linac. Radiother Oncol 174, 149-157 (2022)

    2022

  21. [21]

    Liu, P.Z.Y. et al. First experimental investigation of simultaneously tracking two independently moving targets on an MRI-linac using real -time MRI and MLC tracking. Med Phys 47, 6440- 6449 (2020)

    2020

  22. [22]

    Hu, P. et al. Dosimetry impact of gating latency in cine magnetic resonance image guided breath- hold pancreatic cancer radiotherapy. Phys Med Biol 67, 055008 (2022)

    2022

  23. [23]

    Shan, S. et al. Distortion- corrected image reconstruction with deep learning on an MRI-Linac. Magn Reson Med 90, 963-977 (2023)

    2023

  24. [24]

    Keall, P.J. et al. AAPM Task Group 264: The safe clinical implementation of MLC tracking in radiotherapy. Med Phys 48, e44-e64 (2021)

    2021

  25. [25]

    & Hyer, D.E

    Smith, B.R., St- Aubin, J. & Hyer, D.E. Commissioning of a motion management system for a 1.5T Elekta Unity MR -Linac: A single institution experience. J Appl Clin Med Phys 26, e70005 (2025)

    2025

  26. [26]

    Paganelli, C. et al. Quantification of lung tumor rotation with automated landmark extraction using orthogonal cine MRI images. Phys Med Biol 60, 7165-7178 (2015)

    2015

  27. [27]

    Feng, M. et al. Characterization of pancreatic tumor motion using cine MRI: surrogates for tumor position should be used with caution. Int J Radiat Oncol Biol Phys 74, 884- 891 (2009)

    2009

  28. [28]

    Weng, J. et al. Convolutional LSTM model for cine image prediction of abdominal motion. Phys Med Biol 69, 085024 (2024)

    2024

  29. [29]

    Romaguera, L.V. et al. Prediction of in- plane organ deformation during free- breathing radiotherapy via discriminative spatial transformer networks. Med Image Anal 64, 101754 (2020)

    2020

  30. [30]

    Lombardo, E. et al. Evaluation of real - time tumor contour prediction using LSTM networks for MR -guided radiotherapy. Radiother Oncol 182, 109555 (2023)

    2023

  31. [31]

    Jin, G. et al. Prediction of real -time cine-MR images during MRI -guided radiotherapy of liver cancer using a GAN-ConvLSTM network. Med Phys 52, 3161-3172 (2025)

    2025

  32. [32]

    Ma, J. et al. Medsam2: Segment anything in 3d medical images and videos. arXiv preprint arXiv:2504.03600 (2025)

    work page arXiv 2025

  33. [33]

    Shi, X.J. et al. Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting. Advances in Neural Information Processing Systems 28 (Nips 2015) 28 (2015)

    2015

  34. [34]

    & Tang, R

    Tang, S., Li, C., Zhang, P. & Tang, R. in Proceedings of the IEEE/CVF international conference on computer vision 13470-13479 (2023)

    2023

  35. [35]

    & Li, S.Z

    Gao, Z., Tan, C., Wu, L. & Li, S.Z. in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition 3170-3180 (2022)

    2022

  36. [36]

    Lehtinen, J. et al. Noise2Noise: Learning Image Restoration without Clean Data. International Conference on Machine Learning, Vol 80 80 (2018)

    2018

  37. [37]

    & Tsui, B.M

    Segars, W.P., Sturgeon, G., Mendonca, S., Grimes, J. & Tsui, B.M. 4D XCAT phantom for multimodality imaging research. Med Phys 37, 4902- 4915 (2010)

    2010

  38. [38]

    Paganelli, C. et al. A tool for validating MRI-guided strategies: a digital breathing CT/MRI phantom of the abdominal site. Med Biol Eng Comput 55, 2001-2014 (2017)

    2001

  39. [39]

    Keall, P.J. et al. Real -Time Dose- Guided Radiation Therapy. Int J Radiat Oncol Biol Phys 122, 787-801 (2025)

    2025

  40. [40]

    Grimm, J. et al. High Dose per Fraction, Hypofractionated Treatment Effects in the Clinic (HyTEC): An Overview. Int J Radiat Oncol Biol Phys 110, 1 -10 (2021)

    2021

  41. [41]

    & Raaymakers, B.W

    Waddington, D.E.J., Keall, P.J. & Raaymakers, B.W. MRI -guided radiotherapy: is the best still to come? Phys Med Biol 71, 092001 (2026)

    2026

  42. [42]

    Blocker, T.J. et al. MRIgRT real -time target tracking: TrackRAD2025 challenge report. Med Image Anal 112, 104134 (2026)

    2026

  43. [43]

    Yang, J. et al. Is diaphragm motion a good surrogate for liver tumor motion? Int J Radiat Oncol Biol Phys 90, 952- 958 (2014)

    2014

  44. [44]

    Shimizu, S. et al. Non -stationary transformers-based model for predicting liver motion for interleaved two-dimensional cine magnetic resonance imaging. Med Phys 53, e70241 (2026)

    2026

  45. [45]

    & Glitzner, M

    Borman, P.T.S., Raaymakers, B.W. & Glitzner, M. ReconSocket: a low - latency raw data streaming interface for real-time MRI -guided radiotherapy. Phys Med Biol 64, 185008 (2019)

    2019

  46. [46]

    & Sorensen, T.S

    Hansen, M.S. & Sorensen, T.S. Gadgetron: an open source framework for medical image reconstruction. Magn Reson Med 69, 1768- 1776 (2013)

    2013

  47. [47]

    Grover, J. et al. Super-resolution neural networks improve the spatiotemporal resolution of adaptive MRI -guided radiation therapy. Commun Med 4, 64 (2024)

    2024

  48. [48]

    & Fast, M.F

    Hewson, E.A., Borman, P.T.S., Raaymakers, B.W., Keall, P.J. & Fast, M.F. The first experimental implementation of real -time dose- guided MLC tracking using an MRI - linac. Radiother Oncol 219, 111536 (2026)

    2026

  49. [49]

    de Leon, J. et al. Analysis of data to Advance Personalised Therapy with MR-Linac (ADAPT -MRL). Clin Transl Radiat Oncol 31, 64-70 (2021)

    2021

  50. [50]

    Paszke, A. et al. PyTorch: An Imperative Style, High- Performance Deep Learning Library. Advances in Neural Information Processing Systems 32 (Nips 2019) 32 (2019)

    2019

  51. [51]

    Vaswani, A. et al. Attention Is All You Need. Advances in Neural Information Processing Systems 30 (Nips 2017) 30 (2017)

    2017

  52. [52]

    & Brox, T

    Ronneberger, O., Fischer, P. & Brox, T. in International Conference on Medical Image Computing and Computer - assisted Intervention 234- 241 (Springer, 2015)

    2015

  53. [53]

    & Yang, E

    Kim, J., Yoon, H., Park, G., Kim, K. & Yang, E. in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 11353- 11364 (2024)

    2024

  54. [54]

    Kingma, D.P. & Ba, J. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

    work page internal anchor Pith review Pith/arXiv arXiv 2014

  55. [55]

    & Topin, N

    Smith, L.N. & Topin, N. in Artificial Intelligence and Machine Learning for Multi-domain Operations Applications, Vol. 11006 369-386 (SPIE, 2019)

    2019

  56. [56]

    van der Walt, S. et al. scikit -image: image processing in Python. PeerJ 2, e453 (2014)

    2014

  57. [57]

    Virtanen, P. et al. SciPy 1.0: fundamental algorithms for scientific computing in Python. Nat Methods 17, 261-272 (2020)

    2020

  58. [58]

    & Klen, R

    Rainio, O., Teuho, J. & Klen, R. Evaluation metrics and statistical tests for machine learning. Sci Rep 14, 6086 (2024)

    2024

  59. [59]

    & Perktold, J

    Seabold, S. & Perktold, J. Statsmodels: econometric and statistical modeling with python. scipy 7, 92-96 (2010)

    2010

  60. [60]

    Lee, Y.Y.D. et al. Study protocol of the LARK (TROG 17.03) clinical trial: a phase II trial investigating the dosimetric impact of Liver Ablative Radiotherapy using Kilovoltage intrafraction monitoring. BMC Cancer 21, 494 (2021)

    2021

  61. [61]

    & Keall, P.J

    Hewson, E.A., Mejnertsen, L., Booth, J.T. & Keall, P.J. Adaptive radiation therapy for real -time deformations using dose- optimised multileaf collimator tracking: an in- silico proof of concept for multiple lung lesions. Phys Med Biol 70, 225006 (2025)

    2025

  62. [62]

    & Keall, P

    Mejnertsen, L., Hewson, E., Nguyen, D.T., Booth, J. & Keall, P. Dose -based optimisation for multi -leaf collimator tracking during radiation therapy. Phys Med Biol 66, 065027 (2021)

    2021

  63. [63]

    & Dempsey, J.F

    Low, D.A. & Dempsey, J.F. Evaluation of the gamma dose distribution comparison method. Med Phys 30, 2455-2464 (2003)

    2003

  64. [64]

    Miften, M. et al. Tolerance limits and methodologies for IMRT measurement- based verification QA: Recommendations of AAPM Task Group No. 218. Med Phys 45, e53-e83 (2018)

    2018

  65. [65]

    & Alber, M

    Jelen, U., Sohn, M. & Alber, M. A finite size pencil beam for IMRT dose optimization. Phys Med Biol 50, 1747- 1766 (2005). SUPPLEMENTARY INFORMATION Supplementary Figure 1: Comparison of baseline architectures with and without closed- loop coupling. a) ConvLSTM, b) SwinLSTM, and c) SimVP where each panel contains the data for all patients and each data po...

    2005