{"id":"bc275918-c2cd-43b4-979d-0ab8e68a9de2","arxiv_id":"2505.24792","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":7,"one_line_summary":"A combination of task interpolation, relational embedding, and bi-level routing attention reaches 90.1% accuracy on the Kvasir GI image classification benchmark.","lead":"The authors assembled a few-shot learning model that combines task interpolation, relational embedding, and bi-level routing attention to classify gastrointestinal endoscopic images, reporting 90.1% accuracy on the Kvasir dataset. The paper is a test of whether combining published few-shot and attention components improves medical image classification when labeled data are scarce.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Fine-tuning on Hyper-Kvasir likely trains on the exact Kvasir-v2 test images, so the reported 90.1% does not measure few-shot generalization.","rationale":"The single most load-bearing issue is the training/test split. The method section (Sec II-B) explicitly requires Ctrain ∩ Ctest = ∅, while the experiment (Sec III-A) fine-tunes on all 10,662 Hyper-Kvasir images, 23 classes, then evaluates on Kvasir-v2, 8 classes, 8,000 images. The cited Hyper-Kvasir dataset is publicly documented to include the entire Kvasir dataset; thus the test images are contained in the fine-tuning set. Even without that external fact, the disease classes in Kvasir-v2 (ulcerative colitis, polyps, esophagitis, etc.) certainly overlap with the 23 Hyper-Kvasir classes, violating the disjoint-class requirement. The reported metrics therefore cannot be interpreted as few-shot generalization and the state-of-the-art comparison is invalid. I concur with the reader's REJECT verdict, and the concern is more severe than the reader's weakest assumption: not merely class overlap but exact image overlap. A secondary concern is that the proposed model uses Conv4 with 640 channels while the ProtoNet baseline presumably uses the standard 64-channel Conv4, making the '11% better' comparison confounded by capacity; this is not the central issue because the split leakage alone invalidates the headline. The correct next step is to verify the dataset overlap and, if confirmed, re-run with a truly disjoint split; the paper provides no code or split for such a check.","tokens_in":8054,"tokens_out":5843,"duration_ms":52114,"concrete_test":"Verify dataset composition: download the official Hyper-Kvasir image list and intersect it with the Kvasir-v2 8,000-image list. If the intersection is non-empty (expected to be 8,000), re-run the experiment using only the 2,662 Hyper-Kvasir images disjoint from Kvasir-v2 for fine-tuning, or construct a class-disjoint split (e.g., 5 Kvasir-v2 classes for training, 3 for testing). If accuracy on the disjoint split drops materially below 90.1%, the state-of-the-art claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that the model surpasses state-of-the-art on Kvasir-v2 under a few-shot learning paradigm, but Sec II-B requires Ctrain ∩ Ctest = ∅. The experimental protocol (Sec III-A) fine-tunes on all 10,662 Hyper-Kvasir images (23 classes) and evaluates on Kvasir-v2 (8,000 images, 8 classes). Public documentation of the Hyper-Kvasir dataset (Borgli et al., Scientific Data 2020, ref [20]) states that Hyper-Kvasir includes the Kvasir dataset: the 10,662-image set contains the 8,000 Kvasir-v2 images plus 2,662 new images. Consequently, the exact images used for testing were present in the fine-tuning set, and every Kvasir-v2 class was seen during training. Even if one ignored the direct image overlap, the 23 Hyper-Kvasir classes include the same disease labels (e.g., ulcerative colitis, polyps, esophagitis) as Kvasir-v2, violating the disjoint-class condition. The reported 0.901 accuracy / 0.891 F1 therefore reflect memorization or test-set-aware adaptation, not generalization to unseen classes, invalidating any comparison to MAML, ProtoNet, Transformer, and ResNet50. No code or data split is provided to rule this out, and the ablations inherit the same leakage.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a few-shot learning pipeline for gastrointestinal endoscopy image classification, combining a wide Conv4 feature extractor (640 channels), task interpolation, self- and cross-correlational relational embeddings, and a bi-level routing attention mechanism. The authors fine-tune on Hyper-Kvasir and report 90.1% accuracy, 0.845 precision, 0.942 recall, and 0.891 F1 on Kvasir-v2, claiming state-of-the-art performance. Ablation studies remove or replace individual components to attribute contributions. The central claim is that this architecture surpasses MAML, ProtoNet, Transformer, and ResNet50 under the few-shot paradigm described in Section II-B.","tokens_in":8317,"tokens_out":4119,"duration_ms":45554,"significance":"If the reported results were obtained under a valid few-shot protocol, the paper would offer a useful composition of existing components (Conv4, task interpolation from Yao et al., relational embedding from Kang et al., and BiFormer-style attention) applied to a clinically relevant domain. The task-interpolation idea is a reasonable response to limited medical data, and the ablation study is internally consistent. However, the manuscript provides no code, no variance estimates, and most importantly the experimental protocol does not satisfy the few-shot disjoint-class requirement. As it stands, the evaluation does not support the state-of-the-art claim, and the main contribution cannot be assessed without redoing the experiments under a proper split.","major_comments":[{"comment":"The FSL formulation in Section II-B explicitly requires Ctrain ∩ Ctest = ∅, but Section III-A fine-tunes on Hyper-Kvasir (10,662 images, 23 classes) and tests on Kvasir-v2 (8,000 images, 8 classes). The Hyper-Kvasir dataset, cited as reference [20], is publicly documented to include the Kvasir-v2 images, so the exact test images and all test classes are contained in the fine-tuning set. This violates both the image-level and class-level disjointness requirements. The reported 90.1% accuracy therefore does not measure few-shot generalization, and the comparisons in Table III are invalid. Please provide an evaluation with a truly disjoint class split, or report results under a conventional transfer-learning protocol without claiming few-shot generalization.","section":"II-B and III-A"},{"comment":"Tables III and IV report point estimates with no standard deviations, confidence intervals, or number of seeds. Few-shot evaluation is stochastic because tasks, support sets, and augmentations are randomly sampled; a single run cannot support a claim of superiority, especially when the margin over the Transformer baseline is only 0.031 in accuracy (0.901 vs. 0.870). Report mean ± std over at least five independent runs and, if possible, a paired significance test across the same task draws.","section":"III-B and Table III"},{"comment":"The authors state that they 'eliminated the validation process,' and the final configuration appears to be selected using the Kvasir-v2 test set: Table IV compares architectural choices such as 640 versus 64 channels, ResNet12 versus Conv4, and attention variants, and the best accuracy is then reported as the final result. This is a form of test-set model selection that inflates the reported accuracy. A held-out validation split or nested cross-validation is needed to make the reported 90.1% an unbiased estimate.","section":"III-B"},{"comment":"The baseline comparison is underspecified. No implementation details are given for MAML, ProtoNet, Transformer, or ResNet50: the backbone, optimizer, learning rate, episode count, data preprocessing, or whether the baselines were subjected to the same Hyper-Kvasir fine-tuning. Without matched protocols, the numbers in Table III are uninterpretable. In particular, 'Transformer' needs a concrete architecture and training schedule, and each baseline should be evaluated under the same split and with the same number of seeds.","section":"III-C"}],"minor_comments":[{"comment":"The phrase 'Nature Language Processing' should be 'Natural Language Processing'.","section":"II-E"},{"comment":"There are typos and grammatical errors: 'Keeping Con4 64 layer channels and change relational layer to 64' should be 'Keeping Conv4 with 64 channels and changing the relational layer to 64 channels,' and 'Con4' should be 'Conv4'.","section":"III-B and Table IV"},{"comment":"The sentence about 'the static size of the structuring element in the morphological operator' complicating segmentation, with vessels evolving 'along the periphery of the liver,' appears tangential and confusing; it should be removed or substantially clarified.","section":"I"},{"comment":"The phrase 'following the criteria in [1]' is vague because reference [1] is a classification paper, not a dataset curation standard; specify what criteria are meant.","section":"III-A"},{"comment":"The abstract and several places say 'Kvasir dataset' while the experimental section uses 'Kvasir-v2'; use the versioned name consistently throughout.","section":"Tables and Abstract"}],"recommendation":"major_revision","confidential_remarks":"The Hyper-Kvasir/Kvasir-v2 overlap is a well-documented fact, and the authors' own reference [20] contains the relevant statement. Before sending the manuscript back, the editor may wish to verify that the authors have access to a proper disjoint split, since the current evaluation cannot be salvaged by minor edits. No code or data split is provided, which will further hinder verification of any revised claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The headline result is not trustworthy. The protocol fine-tunes on Hyper-Kvasir (10,662 images, 23 classes) and tests on Kvasir-v2 (8,000 images, 8 classes), but Hyper-Kvasir contains the Kvasir-v2 images. Even if the exact image overlap were somehow avoided, the disease classes are the same, so Ctrain ∩ Ctest is not empty as the paper's own Section II-B requires. The 90.1% accuracy therefore reflects memorization or test-set-aware adaptation, not generalization to unseen classes, and the comparison to MAML, ProtoNet, Transformer, and ResNet50 is invalid.\n\nWhat the paper does well: the architecture is a clean assembly of published components—task interpolation, self-correlational representation, and bi-level routing attention—and the ablation study is internally consistent, showing each added piece contributes a modest gain. The motivation for few-shot learning in GI endoscopy is reasonable, and the attention heatmaps suggest the mechanism focuses on plausible regions.\n\nThe soft spots are load-bearing beyond the leakage. All metrics come from a single run with no error bars, so even the 2-3 point gaps over ablations are not quantified. Baseline configurations (training episodes, data splits, hyperparameters) are unspecified, and no code or data splits are provided. The paper also does not report per-class results, which matters for an 8-class medical task.\n\nThe citation pattern is honest: the method is explicitly built on [25], [16], and [18], and the authors do not claim to invent those components. But the empirical claim that this surpasses state-of-the-art is unsupported because of the contamination.\n\nThis is a clear reject for the stated claim. A resubmission needs a proper disjoint split (train only on Hyper-Kvasir images that are not in Kvasir-v2, or use a different target dataset), multiple runs with error bars, and public code and splits. Without those, the result is not salvageable. I would desk reject the current version; the topic is relevant and a corrected study could be useful, but this version does not warrant referee time.","headline":"Fine-tuning on Hyper-Kvasir contaminates the test set: the reported 90.1% accuracy is not few-shot generalization.","tokens_in":8853,"tokens_out":2302,"would_cite":false,"duration_ms":24105,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper proposes a few-shot network that reaches 90.1% accuracy on Kvasir gastrointestinal images by combining task interpolation, self- and cross-correlational embedding, and bi-level routing attention.","keywords":["AI","Deep Learning","Few-Shot Learning","Gastrointestinal disease classification","Kvasir","Task interpolation","Relational embedding","Bi-level routing attention"],"falsifier":"Compare the eight Kvasir-v2 class names (ulcerative colitis, polyps, normal cecum, normal pylorus, normal z-line, esophagitis, dyed resection margins, dyed lifted polyps) against the 23 Hyper-Kvasir classes; if any of these eight appears in the fine-tuning set, the reported 90.1% accuracy is not a measure of few-shot generalization. Then retrain on Hyper-Kvasir classes strictly excluding those eight and test on held-out images; if accuracy falls materially below 90.1%, the few-shot generalization claim is refuted.","tokens_in":7822,"feed_emoji":"🩺","tokens_out":14250,"duration_ms":143840,"temperature":0.7,"pith_summary":"Endoscopic images of the gastrointestinal tract are scarce, visually similar, and captured from many viewpoints, which makes classification hard for standard CNNs. The paper builds a few-shot learning network that combines a Conv4 encoder widened to 640 channels, task interpolation to synthesize intermediate endoscopic viewpoints, self- and cross-correlational embedding to capture intra-image details and inter-image transitions, and a bi-level routing attention mechanism that focuses on pertinent regions while cutting attention cost. Evaluated on the 8-class Kvasir-v2 dataset, the model reports 90.1% accuracy, 0.845 precision, 0.942 recall, and an F1 score of 0.891, higher in the paper's comparison than ResNet50, MAML, ProtoNet, and a vanilla Transformer. The intended significance is that a lightweight model, able to adapt from a handful of examples, can improve early colorectal-cancer screening by making endoscopy image analysis more accurate and less dependent on large labeled datasets.","feed_headline":"A few-shot model hits 90.1% on gastrointestinal images","feed_subtitle":"A model trained on a few examples per class beats larger baselines, pointing toward data-efficient endoscopic screening","key_machinery":"The load-bearing object is the task-interpolated relational embedding pipeline. Task interpolation densifies the task distribution by mixing hidden representations and labels of two tasks at a random layer; the self-correlational representation (SCR) module highlights intra-image regions of interest via a channel-wise Hadamard product inside a neighbor window, and the cross-correlational attention (CCA) module captures inter-image transitions by refining a 4-dimensional cross-correlation tensor between query and support. The bi-level routing attention mechanism prunes a coarse region-region affinity graph to top-k key-value pairs, keeping the co-attention computation at lower cost than full $O(N^2)$ attention. Together these components are what the paper credits for the reported accuracy gain over the Conv4 and ResNet50 baselines.","core_discovery":"The paper's central claim is that its architecture, not a larger backbone, is what drives performance on Kvasir-v2. The feature extractor is Conv4 with 640 channels; tasks are drawn as N-way K-shot episodes, and at a randomly chosen layer the hidden representations of two tasks are interpolated as $\\tilde{H}^{cr}_{s,l} = \\lambda H^i_{s,l} + (1-\\lambda) H^j_{s,l}$ with $\\lambda \\sim \\mathrm{Beta}(\\alpha,\\beta)$, replacing both support and query representations. The self-correlational representation module computes a channel-wise Hadamard product within a neighbor window followed by convolutions, and the cross-correlational attention module builds a 4-dimensional cross-correlation tensor between query and support, refines it by convolutional matching, and derives co-attention maps. A bi-level routing attention then selects top-k key-value pairs from a coarse region-region affinity graph. In the paper's ablations, removing cross-correlational representation drops accuracy from 0.901 to 0.857, removing task interpolation drops it to 0.870, and replacing the routing attention with vanilla attention drops it to 0.895; these are presented as evidence for the contribution of each component.","pith_inferences":["If the reported class-disjointness is verified, the same pipeline transfers naturally to other low-data medical imaging domains where visually similar classes dominate, such as dermoscopy and laparoscopic video analysis; the pretraining ablations already show cross-domain gains from ISIC and Cholec80.","A stricter episodic benchmark on Hyper-Kvasir itself, holding out the eight classes that resemble the Kvasir-v2 labels and training on the remaining fifteen, would separate the few-shot machinery's contribution from any benefit of having seen related classes during fine-tuning.","The top-k parameter of the bi-level routing attention can be treated as a latency-accuracy dial for real-time endoscopy, since reducing k should lower computation further at an unmeasured accuracy cost."],"forward_implications":["The reported scores beat the paper's own Conv4-based ProtoNet baseline by 11 accuracy points, so the relational modules and task interpolation, rather than the backbone, are the source of the gain.","Removing the cross-correlational representation costs 4.4 accuracy points (0.901 to 0.857), the largest single ablation penalty, which points to inter-image transitions as the most important modeling choice.","Task interpolation outperforms MixUp by 2.3 points (0.901 vs 0.878), so synthesizing new tasks rather than mixing inputs is the more effective augmentation for endoscopic viewpoints.","The bi-level routing attention adds 0.6 points over vanilla attention while cutting inference time from 0.61 ms to 0.52 ms, indicating that the pruning mechanism is both accurate and faster.","Ablating ISIC 2018 lowers accuracy to 0.869 and ablating Cholec80 lowers it to 0.873, so cross-domain pretraining contributes to the final result beyond the few-shot architecture itself."],"supporting_citations":[{"why":"Supplies the task interpolation technique that is core to the data augmentation and to the 0.870 no-interpolation ablation.","marker":"[25]"},{"why":"Supplies the self- and cross-correlational embedding modules that constitute the relational core of the model.","marker":"[16]"},{"why":"Supplies the bi-level routing attention used to prune coarse affinity graphs and reduce attention cost.","marker":"[18]"},{"why":"MAML is a comparison baseline at 0.792 accuracy and represents the optimization-based few-shot approach.","marker":"[14]"},{"why":"ProtoNet is the Conv4-based metric baseline at 0.775 accuracy that the model directly improves on.","marker":"[15]"},{"why":"Defines the Kvasir-v2 dataset and its 8 classes, which the model is evaluated on.","marker":"[19]"},{"why":"Provides the 23-class Hyper-Kvasir dataset used for fine-tuning before the Kvasir-v2 test.","marker":"[20]"},{"why":"ResNet50 is the strongest CNN baseline in the comparison table at 0.812 accuracy.","marker":"[11]"},{"why":"Supplies the scaled dot-product attention formula used by the co-attention module and the Transformer baseline in the comparison.","marker":"[17]"},{"why":"MixUp provides the ablated augmentation alternative; replacing task interpolation with MixUp drops accuracy to 0.878.","marker":"[26]"}],"fun_headline_variants":["Few-shot network hits 90.1% on GI images","Relational embeddings boost few-shot GI classification","Task interpolation refines few-shot colonoscopy reads","90.1% accuracy with few endoscopic examples"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the eight Kvasir-v2 test classes never appeared in the Hyper-Kvasir images used for fine-tuning, yet the paper does not show that the two class lists are disjoint.","fun_headline_variants_meta":{"raw":{"variants":["Few-shot network hits 90.1% on GI images","Relational embeddings boost few-shot GI classification","Task interpolation refines few-shot colonoscopy reads","90.1% accuracy with few endoscopic examples"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000491,"raw_usage":{"total_tokens":2492,"prompt_tokens":1100,"completion_tokens":1392,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":716,"completion_tokens_details":{"reasoning_tokens":1331}},"tokens_in":716,"tokens_out":1392,"duration_ms":13015,"temperature":1.0,"reasoning_tokens":1331,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T12:12:47.904820+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compare the eight Kvasir-v2 class names (ulcerative colitis, polyps, normal cecum, normal pylorus, normal z-line, esophagitis, dyed resection margins, dyed lifted polyps) against the 23 Hyper-Kvasir classes; if any of these eight appears in the fine-tuning set, the reported 90.1% accuracy is not a measure of few-shot generalization. Then retrain on Hyper-Kvasir classes strictly excluding those eight and test on held-out images; if accuracy falls materially below 90.1%, the few-shot generalization claim is refuted.","supporting_citations":[{"cited_title":"Meta-Learning with Fewer Tasks through Task Interpolation","cited_arxiv_id":"2106.02695","evidence_quote":"Supplies the task interpolation technique that is core to the data augmentation and to the 0.870 no-interpolation ablation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the self- and cross-correlational embedding modules that constitute the relational core of the model."},{"cited_title":"BiFormer: Vision Transformer with Bi-Level Routing Attention","cited_arxiv_id":"2303.08810","evidence_quote":"Supplies the bi-level routing attention used to prune coarse affinity graphs and reduce attention cost."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"MAML is a comparison baseline at 0.792 accuracy and represents the optimization-based few-shot approach."},{"cited_title":"Prototypical net- works for few-shot learning,","cited_arxiv_id":null,"evidence_quote":"ProtoNet is the Conv4-based metric baseline at 0.775 accuracy that the model directly improves on."},{"cited_title":"Kvasir: A multi-class image dataset for computer aided gas- trointestinal disease detection,","cited_arxiv_id":null,"evidence_quote":"Defines the Kvasir-v2 dataset and its 8 classes, which the model is evaluated on."},{"cited_title":"HyperKvasir, a comprehensive multi-class image and video dataset for gastrointestinal endoscopy,","cited_arxiv_id":null,"evidence_quote":"Provides the 23-class Hyper-Kvasir dataset used for fine-tuning before the Kvasir-v2 test."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"ResNet50 is the strongest CNN baseline in the comparison table at 0.812 accuracy."}],"review_version":1}