{"id":"9723d0e6-2f75-46f8-aa17-b7e748537901","arxiv_id":"2504.18046","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A siamese ResNet-152 with multi-scale pooling and cross-eye attention modules reports 82.9% accuracy on ODIR-5K binocular fundus classification.","lead":"This paper presents DMS-Net, a deep learning model that classifies eye diseases by looking at both eyes' fundus images together instead of one at a time. It reports top results on the ODIR-5K benchmark, but its code and data are not yet available and it does not compare with prior state-of-the-art systems.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"SOTA claim depends on an unreported evaluation protocol; without a defined split, a leakage-safe augmentation order, and external method comparisons, the reported 0.5% accuracy edge over RFB could be noise or artifact.","rationale":"The reader's weakest assumption identifies the missing data split and training protocol; that is indeed the primary blocker. I sharpen it in two ways. First, the ordering of CutMix relative to the split creates a specific leakage mechanism that could explain the headline numbers even if the architecture is sound. Second, the 'state-of-the-art' wording in the abstract is not backed by any comparison to complete prior ODIR-5K models; Tables I-III only compare internal variants. Both points reinforce the same conditional disposition rather than changing it. The architecture is plausible, the ablations are internally consistent, and I do not see an internal mathematical contradiction, so I would not move to reject. The correct disposition remains conditional on releasing the evaluation protocol, code, and external baselines.","tokens_in":11842,"tokens_out":7092,"duration_ms":74142,"concrete_test":"Release the exact patient-level data split, class-to-label mapping, and training script, then recompute the Table III ALL and RFB rows under that split with five random seeds, both with CutMix applied only after splitting and with the paper's current ordering. If the 0.5% accuracy and 0.004 kappa gap disappears or reverses, the claimed advantage over the best baseline is not supported. In addition, run at least one published binocular ODIR-5K method, such as the Siamese CNN from Zeng et al. [18] or the Siamese DR-grading network from Nirthika et al. [19], on the same split to test the 'state-of-the-art' label.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is a new state-of-the-art result on ODIR-5K (Acc 82.9%, Recall 84.5%, Kappa 83.2%). For that claim to hold, the numbers must come from a sound, comparable evaluation. Section IV describes preprocessing (non-uniform illumination correction and CutMix) and explicitly preserves class imbalance, but never states how ODIR-5K was split into train/validation/test, whether the split was patient-level, whether CutMix was applied before or after the split, how many seeds were averaged, or whether the task is multi-class or multi-label. Section V then compares only DMS-Net variants: backbones in Table I, module ablations in Table II, and OSIM replaced by SPP/ASPP/RFB variants in Table III. No complete published ODIR-5K method is included, so the abstract's 'state-of-the-art' label has no external reference point. In Table III, the RFB variant already reaches 82.4% accuracy and 0.828 kappa; without confidence intervals, the reported 0.5% accuracy and 0.004 kappa advantages are within plausible run-to-run variation. More concretely, if CutMix is applied to the pooled dataset before partitioning, augmented samples built from test images can leak into the training set and inflate every reported number. The paper's own statement that 'Code and the processed dataset will be released subsequently' confirms the results are currently unreproducible. These issues do not prove the architecture is wrong, but they make the headline performance claim unverifiable as written.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript introduces DMS-Net, a Siamese-network architecture for classifying paired left/right fundus photographs from ODIR-5K. It combines a weight-shared ResNet-152 backbone with four proposed modules (OSIM, CASFM, CCAM, CIAM) and reports accuracy 82.9%, recall 84.5%, and Cohen's kappa 83.2%, claiming state-of-the-art performance on the ODIR-5K benchmark. Three ablation tables compare backbone choices, module removal, and alternative multi-scale pooling modules.","tokens_in":12167,"tokens_out":4048,"duration_ms":38913,"significance":"If the reported results are reproducible and the comparisons are fair, the paper offers a modular, well-motivated architecture for binocular fundus classification. The strength is the internal consistency of the ablations: removing CASFM, CCAM, or CIAM degrades most metrics, and OSIM generally outperforms ASPP/SPP-style variants. However, the headline state-of-the-art claim cannot be assessed without an external comparison and a specified evaluation protocol, so the current significance is moderate.","major_comments":[{"comment":"The evaluation protocol is not specified. Section IV describes preprocessing and augmentation but never states how ODIR-5K is split into training, validation, and test sets, whether the split is at patient level (to prevent left/right images of the same patient from appearing in both training and test), how many random seeds or runs are averaged, or the exact task formulation (single-label eight-class versus multi-label). CutMix is described as applied to illumination-corrected images (Section IV, Figures 4-5) with no statement about its order relative to the split; if augmentation is performed before partitioning, synthetic samples can bridge train and test. Because the abstract's numbers are the central claim, this missing protocol is load-bearing and must be supplied before the results can be evaluated.","section":"Section IV / V"},{"comment":"The label 'state-of-the-art' is unsupported: all comparisons are against DMS-Net variants (backbones, ablated modules, or replacement pooling modules). No previously published binocular fundus classifier or ODIR-5K result is included as an external baseline. To substantiate state-of-the-art, the authors should add published methods on ODIR-5K (and, if available, other binocular datasets) evaluated under the same protocol.","section":"Section V, Tables I-III"},{"comment":"The claimed superiority of OSIM over the strongest alternative is within plausible noise. OSIM achieves accuracy 0.829 and kappa 0.832 versus RFB with 0.824 and 0.828, differences of 0.5 and 0.4 percentage points, respectively. Without multiple seeds, confidence intervals, or significance tests, these differences do not establish that OSIM outperforms RFB. The conclusion's statement that OSIM improves kappa by 0.8% over the best baseline is also inconsistent with these numbers.","section":"Table III"},{"comment":"No implementation details are reported: optimizer, learning-rate schedule, batch size, input resolution, number of training epochs, and class-imbalance handling beyond 'class distribution preservation' are absent. This further impedes reproducibility, especially given the statement that code and processed data will be released subsequently.","section":"Section V"}],"minor_comments":[{"comment":"T_left and T_right are used in the residual connection but never defined; specify their relationship to F'_left/right, and also define W^O.","section":"Section III-C, Eq. (12)"},{"comment":"'Improving Kappa coefficient by 0.8% over the best baseline' conflicts with Table III, where the best baseline RFB has kappa 0.828 and OSIM has 0.832; please correct the percentage.","section":"Conclusion, Section VI"},{"comment":"References [5] and [6] are identical; one of them appears to be a duplicate citation that should be replaced or removed.","section":"References [5] and [6]"},{"comment":"'THEPROPSOEDMETHOD' should be 'THE PROPOSED METHOD'.","section":"Section III heading"},{"comment":"The relation between Figures 3-5 and the final training set is unclear; specify the augmentation probability and mixing coefficient for CutMix, and clarify which classes correspond to the eight abbreviations.","section":"Section IV"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is not ready for acceptance as is; the missing evaluation protocol and lack of external comparisons are the main blockers. I see no evidence of intentional misreporting, but the headline numbers are unverifiable in their current form. The authors should be asked to provide split details, multiple-run statistics, and comparisons against published ODIR-5K methods."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe short version: DMS-Net is a sensible engineering combo of Siamese ResNet-152 with multi-scale pooling and attention modules, and its ablations are internally consistent. But the paper claims state-of-the-art on ODIR-5K without reporting the split, seeds, or any comparison to published binocular classifiers. Those numbers are currently unverifiable, and the edge over RFB (0.5% accuracy) is well within plausible run-to-run noise.\n\nWhat is new: the particular stack (OSIM + CASFM + CCAM + CIAM) is not in the cited prior work, and the ablation tables show each module helps when removed. That is decent evidence the architecture is doing something. The writing is clear, and the figures are fine. The paper is honest in that it says code will be released, but that is a promise, not a result.\n\nWhere it is soft: the elephant is the evaluation. Section IV describes preprocessing and CutMix, but never says whether the split is patient-level, whether augmentation is done before or after splitting, how many seeds, or if the task is multi-class or multi-label. If CutMix is applied on the pooled data before partition, augmented samples built from test images can leak into training. That alone could explain the gains. The paper also compares only to its own backbone and module variants; the abstract's 'state-of-the-art' has no external reference. The reported differences between OSIM and RFB are tiny, and with no confidence intervals the selectivity of the module is not demonstrated.\n\nI would not say the architecture is wrong. The ablation pattern is sensible and the idea of explicitly modeling asymmetric versus symmetric interocular pathology has merit. But as written, the empirical claim is a set of numbers in need of verification. I would send it to a serious referee with the explicit demand that the authors release the exact split, seeds, and code, and add comparisons to at least the Zeng et al. and Nirthika et al. methods they cite. If the numbers hold under a defined protocol, this is a reasonable incremental contribution to the binocular fundus literature.\n\nWho it is for: researchers working on paired-eye diagnosis, especially with Siamese networks, will find the module design worth a look. The framework is plausible but the headline result should not be trusted until the method section is complete.\n\nRecommendation: merit peer review, but conditional on the authors fixing the evaluation transparency. Not a desk reject.","headline":"A plausible Siamese architecture with internally consistent ablations, but the SOTA claim is unverifiable because the evaluation protocol and external comparisons are missing.","tokens_in":12717,"tokens_out":2106,"would_cite":false,"duration_ms":19508,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A weight-sharing Siamese network that fuses left- and right-eye features tops the ODIR-5K binocular fundus benchmark.","keywords":["binocular fundus image classification","Siamese network","cross-modal attention","multi-scale pooling","ODIR-5K","ophthalmic diagnostic robot","fundus disease classification","deep learning"],"falsifier":"Take the released code and a fixed public split of ODIR-5K, rerun the full model and ablations across several random seeds, and check whether the accuracy gap over the RFB baseline and the CASFM ablation gap persist; if either collapses to within run-to-run variation, the main performance claim is not supported.","tokens_in":11641,"feed_emoji":"👁️","tokens_out":5404,"duration_ms":49439,"temperature":0.7,"pith_summary":"This paper tries to establish that binocular fundus image classification is better done by reading the two eyes together than by classifying each eye separately. It proposes DMS-Net, a weight-sharing Siamese ResNet-152 that extracts features from the left and right fundus images in parallel, then applies four attention-and-alignment modules to combine them. On the ODIR-5K benchmark the model reaches an accuracy of 82.9%, recall of 84.5%, and a kappa coefficient of 83.2%, which the authors report as state-of-the-art. The clinical motivation is that pathologies often appear asymmetrically or symmetrically across the two eyes, so explicit modeling of both contrast and correlation could support ophthalmic diagnostic robots.","feed_headline":"Two-eye model tops fundus benchmark with 82.9% accuracy","feed_subtitle":"Weight-sharing Siamese ResNet-152 plus cross-eye attention reaches 82.9% accuracy on ODIR-5K.","key_machinery":"The load-bearing mechanism is a set of paired-image feature interactions stacked on a Siamese ResNet-152. OSIM, a multi-scale adaptive pooling plus spatial attention module, aggregates lesion features at several resolutions; CASFM, a bidirectional cross-attention module with learned residual weights, builds a modality-agnostic shared representation of both fundus structures; CCAM, a cross-attention-driven contrastive alignment module, highlights asymmetric lesion differences between eyes; and CIAM, a cross-attention-driven integrative alignment module, reinforces symmetric interocular patterns. Together these modules turn two independent feature maps into one fused representation that the classifier uses to predict a single disease label per patient.","core_discovery":"The paper's central claim is that explicitly modeling inter-eye relations improves binocular fundus classification beyond what monocular models or simple feature concatenation achieve. The authors argue that asymmetric lesion patterns, such as hemorrhage in one eye, and symmetric co-evolving patterns, such as cup-to-disc changes in both eyes, are complementary diagnostic signals, and they design modules to capture each. DMS-Net combines a weight-sharing Siamese ResNet-152 backbone with OSIM for multi-scale pooling, CASFM for cross-eye semantic fusion, and the parallel CCAM and CIAM modules for contrastive and integrative alignment. Reported results on ODIR-5K show the complete model at 82.9% accuracy, 84.5% recall, 83.2% kappa, and 0.973 AUC, with ablations attributing the largest accuracy drop to removing CASFM.","pith_inferences":["The same contrast-and-integration design could be applied to longitudinal same-eye image pairs, where the two inputs are the same retina at two time points and the contrast signal becomes disease progression.","Adapting the fused representation to output per-eye risk scores, instead of one patient-level label, would make the model more directly useful for asymmetric-disease staging.","Because the reported gains over the RFB baseline are small, an independent replication with a fixed public split and multiple seeds would test whether the module improvements generalize beyond the paper's experimental setup."],"forward_implications":["On the ODIR-5K benchmark, DMS-Net reports higher accuracy, recall, kappa, F1, and AUC than every backbone and module variant the paper compares against.","Removing CASFM costs the most performance, indicating that cross-eye semantic fusion is the main contributor among the proposed modules.","Removing either CCAM or CIAM lowers accuracy, supporting the claim that asymmetric and symmetric lesion information are complementary signals.","The OSIM module outperforms ASPP, SPPF, SPP, simSPPF, and RFB in the same backbone setup, suggesting multi-scale pooling with spatial attention is well suited to diffuse fundus lesions.","Deeper ResNet backbones improve results monotonically, so backbone capacity remains a relevant factor even with the Siamese pairing."],"supporting_citations":[{"why":"Supplies the ODIR-5K binocular fundus dataset with paired left-right images and physician diagnostic labels used in all evaluations.","marker":"[34]"},{"why":"Introduces the Siamese weight-sharing architecture that DMS-Net adapts for paired eye images.","marker":"[15]"},{"why":"Shows an earlier binocular Siamese-like CNN for diabetic retinopathy, establishing the baseline paradigm the paper extends.","marker":"[18]"},{"why":"Demonstrates Siamese networks with fine-grained classification for diabetic retinopathy grading, another baseline for interocular modeling.","marker":"[19]"},{"why":"Provides the clinical motivation that unilateral ocular pathology raises contralateral risk, justifying binocular classification.","marker":"[7]"},{"why":"ASPP is one of the alternative multi-scale feature extraction modules that OSIM is compared against in the module replacement experiments.","marker":"[35]"},{"why":"RFB is the strongest alternative feature-extraction baseline that OSIM must outperform.","marker":"[38]"},{"why":"SPP is another compared multi-scale pooling baseline in the module replacement experiments.","marker":"[37]"}],"fun_headline_variants":["Two-eye Siamese network hits 82.9% on ODIR-5K fundus set","Binocular fundus net pairs eyes with attention, 82.9% accurate","DMS-Net: cross-eye fusion for better fundus diagnosis","Fundus AI that looks at both eyes beats single-eye models","82.9% accuracy: deep net leverages inter-eye pathology"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported performance rests on an unreported division of the ODIR-5K data into training and test sets and a single training run, so the small accuracy gaps over comparison models could reflect which patients were used for testing rather than the architecture alone.","fun_headline_variants_meta":{"raw":{"variants":["Two-eye Siamese network hits 82.9% on ODIR-5K fundus set","Binocular fundus net pairs eyes with attention, 82.9% accurate","DMS-Net: cross-eye fusion for better fundus diagnosis","Fundus AI that looks at both eyes beats single-eye models","82.9% accuracy: deep net leverages inter-eye pathology"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00026,"raw_usage":{"total_tokens":1630,"prompt_tokens":1028,"completion_tokens":602,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":644,"completion_tokens_details":{"reasoning_tokens":503}},"tokens_in":644,"tokens_out":602,"duration_ms":6450,"temperature":1.0,"reasoning_tokens":503,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T10:25:17.737921+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the released code and a fixed public split of ODIR-5K, rerun the full model and ablations across several random seeds, and check whether the accuracy gap over the RFB baseline and the CASFM ablation gap persist; if either collapses to within run-to-run variation, the main performance claim is not supported.","supporting_citations":[{"cited_title":"Odir-5k,","cited_arxiv_id":null,"evidence_quote":"Supplies the ODIR-5K binocular fundus dataset with paired left-right images and physician diagnostic labels used in all evaluations."},{"cited_title":"Signature verification using a siamese time delay neural network,","cited_arxiv_id":null,"evidence_quote":"Introduces the Siamese weight-sharing architecture that DMS-Net adapts for paired eye images."},{"cited_title":"Automated diabetic retinopathy detection based on binocular siamese-like convolutional neural network,","cited_arxiv_id":null,"evidence_quote":"Shows an earlier binocular Siamese-like CNN for diabetic retinopathy, establishing the baseline paradigm the paper extends."},{"cited_title":"Siamese network based fine grained classification for diabetic retinopathy grading,","cited_arxiv_id":null,"evidence_quote":"Demonstrates Siamese networks with fine-grained classification for diabetic retinopathy grading, another baseline for interocular modeling."},{"cited_title":"Global causes of blindness and distance vision impairment 1990–2020: a systematic review and meta-analysis,","cited_arxiv_id":null,"evidence_quote":"Provides the clinical motivation that unilateral ocular pathology raises contralateral risk, justifying binocular classification."},{"cited_title":"Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs,","cited_arxiv_id":null,"evidence_quote":"ASPP is one of the alternative multi-scale feature extraction modules that OSIM is compared against in the module replacement experiments."},{"cited_title":"Receptive field block net for accurate and fast object detection,","cited_arxiv_id":null,"evidence_quote":"RFB is the strongest alternative feature-extraction baseline that OSIM must outperform."},{"cited_title":"Spatial pyramid pooling in deep convolutional networks for visual recognition,","cited_arxiv_id":null,"evidence_quote":"SPP is another compared multi-scale pooling baseline in the module replacement experiments."}],"review_version":1}