pith. sign in

arxiv: 2606.22101 · v1 · pith:3YHQW6H5new · submitted 2026-06-20 · 💻 cs.LG · cs.CV

OphthaDT: Generative Digital Twins for Forecasting Visual Acuity Trajectories in Ophthalmology

Pietro Belligoli , Nikita Makarov , Sayedali Shetab Boushehri , Fabian Schmich , Raul Rodriguez-Esteban , Michael Menden This is my paper

Pith reviewed 2026-06-26 12:23 UTC · model grok-4.3

classification 💻 cs.LG cs.CV
keywords digital twinophthalmologyvisual acuity forecastingLLMlongitudinal predictionnAMDclinical trials
0
0 comments X

The pith

OphthaDT forecasts visual acuity trajectories more accurately than baselines by turning patient histories into LLM-readable narratives.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces OphthaDT as an LLM-based digital twin that converts longitudinal records from over three thousand ophthalmology trial patients into structured text narratives. This setup is used to predict best-corrected visual acuity up to one hundred weeks ahead without imputing missing values. A sympathetic reader would care because many eye conditions show highly variable responses over time, and better forecasts could support more tailored monitoring or treatment adjustments. The reported results indicate the largest gains occur in cases with high trajectory variability.

Core claim

OphthaDT serializes histories from 3,220 patients across four Phase III trials into structured narratives, then applies an LLM to forecast BCVA; it records the lowest mean absolute error in neovascular age-related macular degeneration with a 6.0 percent average reduction versus all baselines, remains competitive in diabetic macular edema while beating Random Forest and XGBoost by 2.6 percent and 6.9 percent respectively, and manages irregular sampling intervals without imputation.

What carries the argument

Serialization of longitudinal patient histories into structured narratives that serve as direct input to the LLM-based digital twin.

If this is right

  • The predictive edge grows with the degree of longitudinal variability in the disease course.
  • Linear models suffice for stable trajectories while the narrative-LLM route is needed for fluctuating ones.
  • Irregular visit schedules can be used directly, removing the requirement for complete or evenly spaced data.
  • Such modeling is positioned as a route to lower patient monitoring burden and faster evaluation of new therapies.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same narrative serialization step could be tested on longitudinal records from other chronic conditions that produce irregular time series.
  • Performance on data collected outside controlled trials would show whether the reported advantage survives differences in documentation style and missingness patterns.
  • Adding imaging or biomarker text to the narratives might further tighten forecasts if the current mechanism already extracts useful structure from text alone.

Load-bearing premise

Converting raw patient records into fixed narrative text supplies the LLM with enough signal to capture high-variability trajectories without any additional feature engineering or data imputation.

What would settle it

Running the same model on a fresh set of ophthalmology trial records and observing that its mean absolute error is not lower than the strongest baseline by at least the reported margins.

Figures

Figures reproduced from arXiv: 2606.22101 by Fabian Schmich, Michael Menden, Nikita Makarov, Pietro Belligoli, Raul Rodriguez-Esteban, Sayedali Shetab Boushehri.

Figure 1
Figure 1. Figure 1: OphthaDT pipeline overview. (1) Patient history and baseline measurements are extracted from four Phase III clinical trials. (2) Longitudinal records are serialized into structured text prompts with forecasting instructions. (3) MedGemma 4B is fine-tuned on instruction–completion pairs. (4) The model predicts BCVA trajectories for unseen patients. 2. Related Work Digital Twins for Clinical Trajectory Forec… view at source ↗
Figure 2
Figure 2. Figure 2: Stratified forecasting performance across clinical subgroups. Mean absolute error (in letters) for nAMD and DME. Rows correspond to demographic (age, gender) and clinical (treat￾ment arm, TNFL) strata; columns to weeks 8, 24, 52, 100. nAMD and DME by serializing fragmented clinical records into structured narratives. This approach achieved the low￾est prediction error across all timepoints in nAMD and re￾m… view at source ↗
read the original abstract

Precision medicine in ophthalmology requires accurate longitudinal predictions, but the fragmented nature of multimodal clinical data remains a barrier to forecasting. We introduce OphthaDT, an LLM-based digital twin for ophthalmology that serializes longitudinal patient histories from 3,220 patients across four Phase III clinical trials into structured narratives to forecast best corrected visual acuity (BCVA). In benchmarks spanning up to 100 weeks, OphthaDT demonstrated the lowest prediction error in neovascular age-related macular degeneration (nAMD), achieving an average mean absolute error (MAE) reduction of 6.0% compared to all baselines. In diabetic macular edema (DME), OphthaDT demonstrated competitive performance against all baselines while outperforming Random Forest and XGBoost by an average MAE reduction of 2.6% and 6.9%, respectively. Results reveal that OphthaDT's predictive advantage scales with trajectory complexity: whereas linear models remain effective for the more stable treatment responses of DME, OphthaDT's capacity is better suited for capturing the high longitudinal variability of nAMD. Finally, OphthaDT handles irregular sampling without imputation, positioning LLM-based clinical trajectory modeling as a methodology that could reduce patient burden and accelerate drug development.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper introduces OphthaDT, an LLM-based digital twin for ophthalmology that serializes longitudinal patient histories from 3,220 patients across four Phase III trials into structured narratives to forecast best corrected visual acuity (BCVA). It claims the lowest prediction error in neovascular age-related macular degeneration (nAMD) with a 6.0% average MAE reduction versus all baselines, competitive performance in diabetic macular edema (DME) with 2.6% and 6.9% MAE reductions over Random Forest and XGBoost respectively, an advantage that scales with trajectory complexity, and the ability to handle irregular sampling without imputation.

Significance. If the empirical claims hold after full methodological disclosure and validation, the work would position LLM-based generative modeling of serialized clinical narratives as a viable alternative for longitudinal forecasting in ophthalmology, particularly for high-variability trajectories where traditional models underperform, with potential downstream benefits for precision medicine, reduced patient burden, and accelerated trial design.

major comments (2)
  1. Abstract: the central empirical claim of a 6.0% average MAE reduction in nAMD (and the DME comparisons) is presented without any description of the LLM architecture, training procedure, cross-validation scheme, statistical tests, or error bars, rendering it impossible to verify whether the stated reductions are supported by the data or arise from baseline differences.
  2. Abstract: the assertion that OphthaDT's predictive advantage 'scales with trajectory complexity' and is 'better suited for capturing the high longitudinal variability of nAMD' lacks supporting quantitative evidence (e.g., variability metrics, per-patient error distributions, or subgroup analyses) that would be required to substantiate the scaling claim over linear models.
minor comments (1)
  1. The abstract states benchmarks 'spanning up to 100 weeks' but provides no explicit prediction horizons, evaluation windows, or details on how irregular sampling intervals are encoded in the narrative serialization.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive comments on our manuscript. We address each major comment point by point below and commit to revisions that strengthen the presentation of our empirical claims.

read point-by-point responses

  1. Referee: Abstract: the central empirical claim of a 6.0% average MAE reduction in nAMD (and the DME comparisons) is presented without any description of the LLM architecture, training procedure, cross-validation scheme, statistical tests, or error bars, rendering it impossible to verify whether the stated reductions are supported by the data or arise from baseline differences.

    Authors: We agree that the abstract, as currently written, presents the key performance claims without sufficient methodological context. In the revised version we will expand the abstract by one to two sentences to briefly note the LLM-based narrative serialization approach, the use of patient-level cross-validation, and the reporting of error bars with statistical comparisons in the main text. Full details on architecture, training procedure, cross-validation scheme, and all statistical tests will remain in the Methods and Results sections, but the abstract revision will make the claims more verifiable at first reading. revision: yes

  2. Referee: Abstract: the assertion that OphthaDT's predictive advantage 'scales with trajectory complexity' and is 'better suited for capturing the high longitudinal variability of nAMD' lacks supporting quantitative evidence (e.g., variability metrics, per-patient error distributions, or subgroup analyses) that would be required to substantiate the scaling claim over linear models.

    Authors: We acknowledge that the scaling claim in the abstract currently lacks explicit quantitative backing within the abstract itself. In the revision we will add supporting evidence in both the abstract (via a short qualifier) and the main Results section, including: (i) a variability metric (standard deviation of BCVA across visits per patient), (ii) per-patient error distributions stratified by variability quartile, and (iii) subgroup MAE comparisons between OphthaDT and linear baselines. These additions will directly substantiate the statement that the advantage increases with trajectory complexity. revision: yes

Circularity Check

0 steps flagged

No significant circularity identified

full rationale

The paper reports an empirical ML application: serializing patient histories into narratives for an LLM-based forecaster, then benchmarking MAE on held-out nAMD and DME trajectories against Random Forest, XGBoost and other baselines. No derivation chain, equations, uniqueness theorems, or first-principles claims appear; performance numbers are direct experimental outcomes rather than quantities forced by construction from fitted parameters or self-citations. The central claim (lowest MAE in nAMD) rests on external test-set comparisons and does not reduce to any of the enumerated circular patterns.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract supplies no information on free parameters, background axioms, or new postulated entities.

pith-pipeline@v0.9.1-grok · 5768 in / 1039 out tokens · 24535 ms · 2026-06-26T12:23:53.638612+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

23 extracted references · 4 canonical work pages · 2 internal anchors

  1. [1]

    M., Chakravarthy, U., Gale, R., et al

    Amoaku, W. M., Chakravarthy, U., Gale, R., et al. Defining response to anti-VEGF therapies in neovascular AMD. Eye (Lond), 29(6), 2015

    2015

  2. [2]

    W., Maguire, M

    Beck, R. W., Maguire, M. G., Bressler, N. M., et al. Visual acuity as an outcome measure in clinical trials of retinal diseases. Ophthalmology, 114(10), 2007

    2007

  3. [3]

    J., Makarov, N., et al

    Bordukova, M., Arneth, A. J., Makarov, N., et al. Generative AI and digital twins: shaping a paradigm shift from precision to truly personalized medicine. Frontiers in Genetics, 15, 2024

    2024

  4. [4]

    A., Antaki, F., Zhou, Y., Turner, A

    Chia, M. A., Antaki, F., Zhou, Y., Turner, A. W., Lee, A. Y., & Keane, P. A. Foundation models in ophthalmology. British Journal of Ophthalmology, 108(10):1341--1348, 2024

    2024

  5. [5]

    G., Richman, E

    Csaky, K. G., Richman, E. A., & Ferris 3rd, F. L. Report from the NEI/FDA ophthalmic clinical trial design and endpoints symposium. Investigative Ophthalmology & Visual Science, 49(2):479--489, 2008

    2008

  6. [6]

    U., Hillenkamp, J., Sivaprasad, S., et al

    Dugel, P. U., Hillenkamp, J., Sivaprasad, S., et al. Baseline visual acuity strongly predicts visual acuity gain in patients with diabetic macular edema following anti-vascular endothelial growth factor treatment across trials. Clinical Ophthalmology, 10, 2016

    2016

  7. [7]

    R., Bourne, R

    Flaxman, S. R., Bourne, R. R. A., Resnikoff, S., et al. Global causes of blindness and distance vision impairment 1990--2020. The Lancet Global Health, 5(12), 2017

    1990

  8. [8]

    L., Steinberg, E., Fleming, S

    Guo, L. L., Steinberg, E., Fleming, S. L., et al. EHR foundation models improve robustness in the presence of temporal distribution shift. Scientific Reports, 13(1), 2023

    2023

  9. [9]

    Large Language Models are Powerful Electronic Health Record Encoders

    Hegselmann, S., von Arnim, G., Rheude, T., Kronenberg, N., Sontag, D., Hindricks, G., Eils, R., & Wild, B. Large language models are powerful electronic health record encoders. arXiv preprint arXiv:2502.17403, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  10. [10]

    J., Cho, E., Moon, S.-E., Kim, Y.-H., Atallah, L., & Choi, E

    Hur, K., Oh, J., Kim, J., Kim, J., Lee, M. J., Cho, E., Moon, S.-E., Kim, Y.-H., Atallah, L., & Choi, E. GenHPF : General healthcare predictive framework for multi-task multi-source learning. IEEE Journal of Biomedical and Health Informatics, 2023

    2023

  11. [11]

    Kamel Boulos, M. N. & Zhang, P. Digital twins: From personalised medicine to precision public health. Journal of Personalized Medicine, 11(8), 2021

    2021

  12. [12]

    G., Anegondi, N., et al

    Kikuchi, Y., Kawczynski, M. G., Anegondi, N., et al. Machine Learning to Predict Faricimab Treatment Outcome in Neovascular Age-Related Macular Degeneration. Ophthalmology Science, 4(2), 2024

    2024

  13. [13]

    TwinWeaver : An LLM -based foundation model framework for pan-cancer digital twins

    Makarov, N., Bordukova, M., Voith von Voithenberg, L., et al. TwinWeaver : An LLM -based foundation model framework for pan-cancer digital twins. arXiv preprint arXiv:2601.20906, 2026

    work page arXiv 2026

  14. [14]

    EHR2Path : Scalable modeling of longitudinal patient pathways from multimodal electronic health records

    Pellegrini, C., Özsoy, E., Bani-Harouni, D., Keicher, M., & Navab, N. EHR2Path : Scalable modeling of longitudinal patient pathways from multimodal electronic health records. arXiv preprint arXiv:2506.04831, 2026

    work page arXiv 2026

  15. [15]

    Development and validation of a multimodal multitask vision foundation model for generalist ophthalmic artificial intelligence

    Qiu, J., Wu, J., Wei, H., et al. Development and validation of a multimodal multitask vision foundation model for generalist ophthalmic artificial intelligence. NEJM AI, 1(12), 2024

    2024

  16. [16]

    B., Boyer, D

    Rofagha, S., Bhisitkul, R. B., Boyer, D. S., et al. Seven-year outcomes in ranibizumab-treated patients in ANCHOR, MARINA, and HORIZON: a multicenter cohort study (SEVEN-UP). Ophthalmology, 120(11), 2013

    2013

  17. [17]

    Predicting visual acuity by using machine learning in patients treated for neovascular age-related macular degeneration

    Rohm, M., Tresp, V., M \"u ller, M., et al. Predicting visual acuity by using machine learning in patients treated for neovascular age-related macular degeneration. Ophthalmology, 125(7), 2018

    2018

  18. [18]

    Machine learning to analyze the prognostic value of current imaging biomarkers in neovascular age-related macular degeneration

    Schmidt-Erfurth, U., Bogunovic, H., Sadeghipour, A., et al. Machine learning to analyze the prognostic value of current imaging biomarkers in neovascular age-related macular degeneration. Ophthalmology Retina, 2(1), 2018

    2018

  19. [19]

    MedGemma Technical Report

    Sellergren, A., Kazemzadeh, S., Jaroensri, T., et al. MedGemma technical report. arXiv preprint arXiv:2507.05201, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  20. [20]

    A multimodal visual--language foundation model for computational ophthalmology

    Shi, D., Zhang, W., Yang, J., et al. A multimodal visual--language foundation model for computational ophthalmology. npj Digital Medicine, 8:381, 2025

    2025

  21. [21]

    A., Xu, Y., & Shah, N

    Steinberg, E., Fries, J. A., Xu, Y., & Shah, N. H. MOTOR : A time-to-event foundation model for structured medical records. International Conference on Learning Representations (ICLR), 2024

    2024

  22. [22]

    The shaky foundations of large language models and foundation models for electronic health records

    Wornow, M., Xu, Y., Thapa, R., et al. The shaky foundations of large language models and foundation models for electronic health records. npj Digital Medicine, 6(1), 2023

    2023

  23. [23]

    A., Wagner, S

    Zhou, Y., Chia, M. A., Wagner, S. K., et al. A foundation model for generalizable disease detection from retinal images. Nature, 622, 2023

    2023