REVIEW 4 major objections 38 references
Cross-Modal Fusion of OCT and OCT angiography enface for Improved Diagnostics of Diabetic Retinopathy
T0 review · 4 major / 0 minor · reviewed 2026-07-11 · grok-4.5
Pith's one-line read Attention fusion of OCT with single-channel enface OCTA beats OCT-only DR detection, and OCT-translated OCTA can replace hardware OCTA.
desk verdict Solid multi-scenario evidence that OCT + single-channel enface OCTA attention fusion beats a strong OCT-only baseline; the hardware-replacement claim for diffusion TR OCTA is the soft spot and needs stress-testing, not rejection of the whole paper. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
Bidirectional cross-modal attention network (CRD-Net backbone): independent CNN streams extract features from OCT and enface OCTA, self-attention focuses each stream, then each modality queries the other via scaled dot-product attention before multi-branch classification; the same pipeline accepts either ground-truth or diffusion-translated OCTA.
What would settle it
Train the identical fusion pipeline on a third independent multi-scanner cohort that includes mild DR and measure whether TR-OCTA fusion still matches or exceeds GT-OCTA fusion in recall and cross-dataset AUC; a clear drop for TR relative to GT would falsify the substitutability claim.
Extended reading notes
Core claim
Cross-modal attention fusion of OCT B-scans with single-channel enface OCTA consistently and substantially outperforms a strong unimodal OCT baseline for binary DR-versus-normal classification across all five evaluation scenarios, and computationally translated OCTA can substitute for hardware-acquired OCTA while preserving or improving accuracy, sensitivity, and cross-dataset robustness.
Load-bearing premise
The assumption that OCT-to-OCTA translation produces genuine, generalizable vascular disease signal rather than dataset-specific texture or denoising patterns that merely inflate metrics.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes bidirectional cross-modal attention fusion (CRD-Net) of OCT B-scans with single-channel enface OCTA for binary DR vs normal classification. Using OCT500 and UIC (730 subjects total), it evaluates five scenarios: within-dataset, combined, and bidirectional cross-dataset tests. A ConvNeXt V2 OCT-only model is the unimodal baseline. In addition to ground-truth (GT) OCTA, the authors substitute translated (TR) OCTA generated from OCT by a conditional diffusion model trained on OCT500. Tables I–III report that fusion consistently outperforms the unimodal baseline; TR OCTA matches or exceeds GT OCTA in most settings and improves recall in the OCT500→UIC direction. The central clinical claim is that attention-based OCT–OCTA enface fusion improves DR detection and that computationally generated OCTA can replace hardware OCTA for screening in resource-limited settings.
Significance. If the directional fusion benefit and the TR-as-surrogate claim hold under scrutiny, the work is clinically relevant: OCTA hardware is expensive and scarce, while standard OCT is widely available. The multi-scenario design (patient-level splits, combined training, bidirectional cross-dataset tests) and bootstrap 95% CIs on fusion metrics are strengths relative to many single-cohort multimodal OCT papers. Using only single-channel enface OCTA also matches a realistic clinical data constraint. The TR pathway, if it truly injects generalizable vascular signal rather than domain-specific texture, would be a practical bridge for high-sensitivity DR screening without OCTA devices. These contributions are incremental but useful for the eess.IV / ophthalmic imaging community.
major comments (4)
- Tables II–III and Discussion (TR OCTA paragraphs): The hardware-replacement claim rests on TR OCTA supplying genuine, generalizable vascular disease signal. The translator was trained only on OCT500 (§III.A, ref. [12]). On that same cohort, TR fusion accuracy rises from 0.8788 (GT) to 0.9697 and recall from 0.7143 to 0.9286, under a severe 251:64 Normal:DR imbalance. Discussion attributes this to TR being a “denoised, average-representative” map, but that is equally consistent with re-emitted OCT500-specific texture or label-correlated artifacts that the fusion head can exploit. Cross-dataset results are mixed (TR helps OCT500→UIC recall; UIC→OCT500 accuracy falls to 0.7005). Without an independent check—e.g., vessel-density / FAZ correlation of TR vs GT, ablation of TR quality, or a translator trained off OCT500—the claim that TR can replace hardware OCTA does not follow from the tables
- Tables I–III: Specificity equals Recall in essentially every row (e.g., OCT500 unimodal 0.6872/0.6872; GT OCT500 0.7143/0.7143; TR OCT500 0.9286/0.9286; same pattern for UIC and Combined). For binary classification this is only expected under balanced classes or a particular threshold/definition; with the reported 251:64 imbalance it is anomalous and suggests either a reporting error (specificity copied from recall), macro-averaging over classes without stating it, or an unintended operating point. This undermines trust in the metric suite that supports the “consistently outperforms” claim. Please recompute and report standard binary specificity (TN/(TN+FP) for the DR-positive definition) and clarify the positive class and averaging convention.
- §III.B / Results: The unimodal baseline is described as “ConvNeXt Large” with ImageNet-1k weights, while the abstract, contributions, and Table I caption say “ConvNeXt V2.” These are different architectures. The baseline must be named and configured consistently; otherwise the fusion gains relative to a “strong unimodal ConvNeXt V2 baseline” cannot be verified. Please also state whether the same backbone family and capacity are used inside CRD-Net’s two streams.
- §III.A and Tables I–III: How multiple (~80) fovea-centered OCT B-scans per patient are aggregated into a single patient-level prediction is not specified (slice-wise majority vote, feature pooling, multi-instance head, etc.). Patient-level stratified splits are claimed, but without the aggregation rule the reported accuracies and cross-dataset numbers are not reproducible and the comparison to a single enface OCTA is ambiguous. This is load-bearing for all five scenarios.
Circularity Check
No circularity: empirical held-out fusion metrics are not forced by construction from self-cited TR OCTA generation or the adopted CRD-Net backbone.
full rationale
This paper is supervised multimodal classification with patient-level held-out, combined, and bidirectional cross-dataset evaluation on OCT500 and UIC. The central claims (fusion beats unimodal ConvNeXt V2; TR OCTA matches or exceeds GT OCTA in most settings) are measured accuracy/AUC/recall numbers on fixed splits, not algebraic identities of fitted parameters. The self-citation to the authors’ diffusion OCT→OCTA translator [12] only supplies the TR enface input modality; it does not define or force the fusion classifier’s test metrics. CRD-Net [38] and ImageNet-pretrained backbones are standard external components, not uniqueness theorems or self-definitional premises. Concerns that TR>GT gains on OCT500 may reflect translator domain leakage or denoising rather than true vascular signal are validity/correctness risks, not circular reductions of prediction to input. No self-definitional loop, fitted-input-as-prediction, load-bearing uniqueness import, or renaming of a known result was found.
Assumptions & free parameters
free parameters (5)
- Learning rate and decay schedule (1e-4, decay ×0.1 at epochs 50 and 100)
- Training epochs (100), batch size (32), SGD momentum (0.9), weight decay (1e-4)
- Number of OCT B-scans per patient (~80, fovea-centered)
- Train/val/test split ratios (80-10-10 within-dataset; 90-10 for cross-dataset train)
- Diffusion model parameters for TR OCTA (from prior work, trained on OCT500 only)
assumptions (5)
- domain assumption Structural OCT and enface OCTA provide complementary, non-redundant information about DR that intermediate bidirectional cross-attention can exploit better than OCT alone.
- domain assumption Patient-level stratified splits prevent leakage and make within- and cross-dataset metrics valid estimates of clinical generalization.
- ad hoc to paper A conditional diffusion model trained on OCT500 produces TR OCTA that is a valid vascular surrogate for fusion, including when tested on UIC.
- domain assumption Binary Normal vs DR labels on the selected OCT500 subset (251 Normal, 64 DR) and UIC (64 Normal, 351 DR) are clinically consistent across devices and demographics.
- standard math ImageNet-pretrained CNN backbones plus CRD-Net-style multi-head self- and cross-attention are appropriate feature extractors for OCT/OCTA fusion.
invented entities (1)
-
TR OCTA as hardware-free second modality for CRD-Net fusion in DR screening
Cite this review
Pith. "Pith review of Cross-Modal Fusion of OCT and OCT angiography enface for Improved Diagnostics of Diabetic Retinopathy." pith.science (2026). https://pith.science/paper/MXUOSNMQ
@misc{pith2026260703959,
author = {Pith},
title = {Pith review of: Cross-Modal Fusion of OCT and OCT angiography enface for Improved Diagnostics of Diabetic Retinopathy},
year = {2026},
howpublished = {\url{https://pith.science/paper/MXUOSNMQ}},
note = {Machine review of arXiv:2607.03959}
}
read the original abstract
Diabetic retinopathy (DR) is a leading cause of vision impairment worldwide, highlighting the need for accurate and accessible screening tools. Optical Coherence Tomography (OCT) provides high-resolution structural information of the retina, whereas OCT angiography (OCTA) offers complementary vascular information that is highly relevant for DR diagnosis. In this study, we propose a cross-modal fusion of OCT B-scans with single-channel en face OCTA using a bidirectional cross-modal attention network for automated DR classification. Two independent datasets, OCT500 and UIC, comprising 730 subjects in total, were utilized to evaluate performance under within-dataset, combined-dataset, and cross-dataset generalization settings. A ConvNeXt V2 model trained solely on OCT images served as the unimodal baseline. In addition to ground-truth (GT) OCTA, we explored the use of translated (TR) OCTA generated from OCT scans, eliminating the requirement for dedicated OCTA hardware. Experimental results demonstrate that cross-modal fusion consistently outperforms unimodal OCT classification across all evaluation scenarios. Fusion with GT OCTA improved classification accuracy and discriminative performance, while TR OCTA achieved comparable or superior results in most settings. Furthermore, TR OCTA improved sensitivity and cross-dataset generalization, indicating enhanced robustness to domain shifts. These findings demonstrate that attention-based OCT-OCTA en face fusion provides clinically meaningful improvements for DR detection and suggest that computationally generated OCTA can serve as a practical, low-cost alternative to hardware-acquired OCTA, enabling broader deployment of high-performance retinal screening systems in resource-limited clinical environments.
Figures
Reference graph
Works this paper leans on
-
[12]
Diffusion model based OCT to OCTA translation,
R. H. Badhonet al., “Diffusion model based OCT to OCTA translation,”Frontiers in Medicine, vol. 12, 2025, doi: 10.3389/fmed.2025.1655453
-
[1]
Optical coherence tomography,
D. Huanget al., “Optical coherence tomography,”Science, vol. 254, no. 5035, pp. 1178–1181, 1991
1991
-
[2]
Optical coherence tomography—principles and applications,
A. F. Fercheret al., “Optical coherence tomography—principles and applications,”Rep. Prog. Phys., vol. 66, no. 2, p. 239, 2003
2003
-
[3]
Optical coherence tomography: technology and applications,
W. Drexler and J. G. Fujimoto, “Optical coherence tomography: technology and applications,”Springer, 2008
2008
-
[4]
Artificial intelligence in retina,
E. Schmidt-Erfurthet al., “Artificial intelligence in retina,”Prog. Retin. Eye Res., vol. 67, pp. 1–29, 2018
2018
-
[5]
Optical coherence tomography angiography-based capillary velocimetry,
R. K. Wanget al., “Optical coherence tomography angiography-based capillary velocimetry,”J. Biomed. Opt., vol. 22, no. 6, p. 066008, 2017
2017
-
[6]
Quantitative optical coherence tomography angiography,
S. S. Gaoet al., “Quantitative optical coherence tomography angiography,”Biomed. Opt. Express, vol. 7, no. 2, pp. 323–346, 2016
2016
-
[7]
Optical coherence tomography angiography,
R. F. Spaideet al., “Optical coherence tomography angiography,”Prog. Retin. Eye Res., vol. 64, pp. 1–55, 2018
2018
Show all 38 references
-
[8]
Quantitative comparisons between optical coherence tomography angiography and matched histology in the human eye,
L. Anet al., “Quantitative comparisons between optical coherence tomography angiography and matched histology in the human eye,” Exp. Eye Res., vol. 146, pp. 10–17, 2016
2016
-
[9]
Development of a deep learning algorithm for optical coherence tomography angiography in diabetic retinopathy,
A. R. Ranet al., “Development of a deep learning algorithm for optical coherence tomography angiography in diabetic retinopathy,” Graefes Arch. Clin. Exp. Ophthalmol., vol. 257, pp. 1353–1361, 2019
2019
-
[10]
Deep learning for detecting retinal detachment and discerning macular status using ultra-widefield fundus images,
Z. Liet al., “Deep learning for detecting retinal detachment and discerning macular status using ultra-widefield fundus images,”Commun. Biol., vol. 3, no. 1, p. 15, 2020
2020
-
[11]
Fully automated detection of diabetic macular edema and dry age-related macular degeneration from optical coherence tomography images,
P. P. Srinivasanet al., “Fully automated detection of diabetic macular edema and dry age-related macular degeneration from optical coherence tomography images,”Biomed. Opt. Express, vol. 5, no. 10, pp. 3568–3577, 2014
2014
-
[13]
Multimodal deep learning for macular disease detection,
Q. Zhanget al., “Multimodal deep learning for macular disease detection,”IEEE Trans. Med. Imaging, vol. 39, no. 12, pp. 4093–4104, 2020
2020
-
[14]
Joint classification and segmentation of diabetic retinopathy using deep learning,
Y . Zhouet al., “Joint classification and segmentation of diabetic retinopathy using deep learning,”IEEE J. Biomed. Health Inform., vol. 25, no. 7, pp. 2671–2679, 2021
2021
-
[15]
Cross-modal learning for multi-modal medical image analysis,
J. Wuet al., “Cross-modal learning for multi-modal medical image analysis,”Med. Image Anal., vol. 78, p. 102394, 2022
2022
-
[16]
A review of deep learning-based information fusion techniques for multimodal medical image classification,
Y . Liet al., “A review of deep learning-based information fusion techniques for multimodal medical image classification,”Comput. Biol. Med., vol. 177, p. 108635, 2024
2024
-
[17]
A review on multimodal medical image fusion: Compendious analysis of medical modalities, multimodal databases, fusion techniques and quality metrics,
M. A. Azamet al., “A review on multimodal medical image fusion: Compendious analysis of medical modalities, multimodal databases, fusion techniques and quality metrics,”Comput. Biol. Med., vol. 144, p. 105253, 2022
2022
-
[18]
Recent advancements in multimodal medical image fusion techniques for better diagnosis: an overview,
M. Haribabu, V . Guruviah, and P. Yogarajah, “Recent advancements in multimodal medical image fusion techniques for better diagnosis: an overview,”Curr. Med. Imaging, vol. 19, pp. 673–694, 2023
2023
-
[19]
A review of the application of multi-modal deep learning in medicine: bibliometrics and future directions,
X. Peiet al., “A review of the application of multi-modal deep learning in medicine: bibliometrics and future directions,”Int. J. Comput. Intell. Syst., vol. 16, p. 44, 2023
2023
-
[20]
Optimizing the OCTA layer fusion option for deep learning classification of diabetic retinopathy,
G. Ryuet al., “Optimizing the OCTA layer fusion option for deep learning classification of diabetic retinopathy,”Biomed. Opt. Express, vol. 14, no. 9, pp. 4706–4722, 2023
2023
-
[21]
Multi-modal retinal image classification with modality-specific attention network,
X. Heet al., “Multi-modal retinal image classification with modality-specific attention network,”IEEE Trans. Med. Imaging, vol. 40, no. 6, pp. 1591–1602, 2021
2021
-
[22]
Ensemble deep learning for diabetic retinopathy detection using optical coherence tomography angiography,
M. Heisleret al., “Ensemble deep learning for diabetic retinopathy detection using optical coherence tomography angiography,”Transl. Vis. Sci. Technol., vol. 9, no. 2, p. 20, 2020
2020
-
[23]
A comprehensive review of multimodal deep learning for enhanced medical diagnostics,
Y . Huanget al., “A comprehensive review of multimodal deep learning for enhanced medical diagnostics,”Clin. eHealth, 2025, in press
2025
-
[24]
CMAF-Net: a cross-modal attention fusion-based deep neural network for incomplete multi-modal brain tumor segmentation,
K. Sunet al., “CMAF-Net: a cross-modal attention fusion-based deep neural network for incomplete multi-modal brain tumor segmentation,”Quant. Imaging Med. Surg., vol. 14, no. 7, pp. 4579–4604, 2024. 15
2024
-
[25]
CrossFuse: A novel cross attention mechanism based infrared and visible image fusion approach,
H. Li and X.-J. Wu, “CrossFuse: A novel cross attention mechanism based infrared and visible image fusion approach,”Inf. Fusion, vol. 103, p. 102147, 2024
2024
-
[26]
TransMEF: A transformer-based multi-exposure image fusion framework using self-supervised multi-task learning,
L. Quet al., “TransMEF: A transformer-based multi-exposure image fusion framework using self-supervised multi-task learning,” in Proc. AAAI Conf. Artif. Intell., vol. 36, 2022, pp. 2126–2134
2022
-
[27]
MATR: Multimodal medical image fusion via multiscale adaptive transformer,
W. Tanget al., “MATR: Multimodal medical image fusion via multiscale adaptive transformer,”IEEE Trans. Image Process., vol. 31, pp. 5134–5149, 2022
2022
-
[28]
MACTFusion: Lightweight cross transformer for adaptive multimodal medical image fusion,
Z. Wanget al., “MACTFusion: Lightweight cross transformer for adaptive multimodal medical image fusion,”J. King Saud Univ. Comput. Inf. Sci., vol. 36, no. 4, p. 102039, 2024
2024
-
[29]
A novel multimodel medical image fusion framework with edge enhancement and cross-scale transformer,
X. Yanget al., “A novel multimodel medical image fusion framework with edge enhancement and cross-scale transformer,”Sci. Rep., vol. 15, p. 6245, 2025
2025
-
[30]
A multimodal vision transformer for interpretable fusion of functional and structural neuroimaging data,
Y . Liuet al., “A multimodal vision transformer for interpretable fusion of functional and structural neuroimaging data,”Hum. Brain Mapp., vol. 45, p. e26850, 2024
2024
-
[31]
A V-Net: deep learning for fully automated artery-vein classification in optical coherence tomography angiography,
M. Alamet al., “A V-Net: deep learning for fully automated artery-vein classification in optical coherence tomography angiography,” Biomed. Opt. Express, vol. 11, no. 9, pp. 5249–5257, 2020
2020
-
[32]
MF-A V-Net: an open-source deep learning network with multimodal fusion options for artery-vein segmentation in OCT angiography,
M. Abtahiet al., “MF-A V-Net: an open-source deep learning network with multimodal fusion options for artery-vein segmentation in OCT angiography,”Biomed. Opt. Express, vol. 13, no. 9, pp. 4870–4888, 2022
2022
-
[33]
Hybrid fusion of high-resolution and ultra-widefield OCTA acquisitions for the automatic diagnosis of diabetic retinopathy,
Y . Liuet al., “Hybrid fusion of high-resolution and ultra-widefield OCTA acquisitions for the automatic diagnosis of diabetic retinopathy,” Diagnostics, vol. 13, p. 2881, 2023
2023
-
[34]
Multi-modal classification of retinal disease based on convolutional neural network,
H. Panet al., “Multi-modal classification of retinal disease based on convolutional neural network,”Biomedical Physics & Engineering Express, vol. 11, no. 4, 2025, doi: 10.1088/2057-1976/adeb92
2025 doi
-
[35]
MultiEYE: Dataset and benchmark for OCT-enhanced retinal disease recognition from fundus images,
Z. Liuet al., “MultiEYE: Dataset and benchmark for OCT-enhanced retinal disease recognition from fundus images,” arXiv preprint arXiv:2412.09402, 2024
2024 arXiv
-
[36]
OCTA-500: A retinal dataset for optical coherence tomography angiography study,
M. Liet al., “OCTA-500: A retinal dataset for optical coherence tomography angiography study,”Medical Image Analysis, vol. 93, p. 103092, 2024. :contentReference[oaicite:0]index=0
2024
-
[37]
ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders,
S. Wooet al., “ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders,” arXiv preprint arXiv:2301.00808, 2023
2023 arXiv
-
[38]
Cross-modal attention network for retinal disease classification based on multi-modal images,
Z. Liuet al., “Cross-modal attention network for retinal disease classification based on multi-modal images,”Biomedical Optics Express, vol. 15, no. 6, pp. 3699–3714, 2024
2024
Reviewed July 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.