{"id":"3a5cfb5b-b93e-46d6-805b-6de708971522","arxiv_id":"2505.05248","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A new augmentation that adds synthetic specular reflections to colonoscopy images improved YOLOv5 polyp detection mAP50 from 77.5% to 80.7% on the Harvard Dataverse dataset.","lead":"This paper proposes a data augmentation method that pastes synthetic white light reflections into colonoscopy training images to teach polyp detectors not to confuse reflections with polyps. On one public polyp dataset, a YOLOv5 model trained with 20% of images augmented this way improved mean average precision from 77.5% to 80.7%.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The +3.2 mAP50 gain is not yet established: the 20% replacement rate was selected on the test set and no variance or significance is reported; a held-out tuning protocol is needed before the central claim can be accepted.","rationale":"I considered whether the more load-bearing weakness is the realism and transfer of synthetic lights, which is the reader's stated weakest assumption. That concern is real but secondary: even if the lights are only approximately realistic, WLSR could still improve performance through generic regularization, and the paper's claim is empirical. The primary vulnerability is that the only quantitative evidence for the claim is a test-set-selected maximum without noise characterization. The paper deserves credit for a simple, clearly described method, a public dataset, and averaging over three seeds, but the absence of variance reporting and the use of the test split for model selection are internal experimental flaws, not disagreements with consensus. The reader's CONDITIONAL verdict already captures this; my analysis does not move the verdict, but it sharpens the required condition: a held-out tuning split and significance reporting. If the proposed check fails to reproduce the gain, the paper would need to be rejected; if it reproduces, the central claim would be substantially supported.","tokens_in":6313,"tokens_out":9538,"duration_ms":99840,"concrete_test":"Pre-register a tuning protocol: split the current training set into a tuning set and a train set. Select the replacement rate (10/20/50/100) using the tuning set only. Then train baseline and the selected WLSR model with at least 5 seeds on the train set and evaluate once on the original test set. Report per-seed mAP50, mean +/- SD, and a bootstrap or permutation confidence interval for the difference. If the gain is not outside the interval, or if the selected rate is not 20%, the claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3 reports WLSR replacement experiments at 100%, 10%, 20%, and 50%, then states that 20% produces the best performance (mAP50 80.7% vs 77.5% baseline). Because the same test split was used both to choose the rate and to report the headline result, the selected maximum is optimistically biased even if WLSR had no true effect. The paper says each model was trained three times with seeds 1, 2, and 3 and averaged, but Table 1 reports only the averages; no standard deviations, per-seed numbers, confidence intervals, or significance tests appear anywhere. With four configurations and three seeds, the observed 3.2-point gap could plausibly lie within run-to-run noise, especially since the 10%, 50%, and 100% configurations show much smaller gains (0.7, 1.6, and 0.9 points). The central claim therefore rests entirely on one comparison that has not been shown to be statistically reliable.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a data augmentation method, White Light Specular Reflection (WLSR), which adds synthetic specular reflections to colonoscopy training images to reduce false-positive polyp detections. The method first builds a bank of light patches cropped from real training images, applies geometric and intensity transformations, identifies prohibited regions (existing lights, polyps, black borders), and pastes one synthetic light into a random non-prohibited location using a sliding-window search. Experiments on the Harvard Dataverse polyp dataset with YOLOv5 report that training with 20% WLSR replacement improves mAP50 from 77.5% to 80.7%, and combining WLSR with standard augmentations yields 81.5% mAP50. The paper concludes that WLSR is an effective plug-in augmentation.","tokens_in":6507,"tokens_out":2465,"duration_ms":23998,"significance":"If the reported improvement is genuine, WLSR is a simple, domain-inspired augmentation that could help polyp detectors avoid a common failure mode, and it may be applicable to other endoscopic tasks with specular reflections. The method is intuitive and described in sufficient algorithmic detail to be reimplemented. However, the current statistical evidence is weak: the headline 3.2-point gain is the best of four tested replacement rates selected on the same test set, and no measures of variance are reported. The paper would be strengthened by a held-out tuning protocol, per-seed results with standard deviations, and possibly a significance test. The core idea has merit, but the evidence presented does not yet establish the claimed improvement.","major_comments":[{"comment":"The 20% replacement rate was selected as the best among 10%, 20%, 50%, and 100% after evaluating all configurations on the same test split. The headline 3.2% mAP50 gain is therefore the maximum of four comparisons rather than a pre-specified estimate, which introduces optimistic selection bias. Without a held-out tuning set or a correction for multiple comparisons, the central claim that WLSR at 20% replacement yields 80.7% mAP50 is not adequately supported. The authors should either pre-register the rate, tune it on a validation split and report test performance only for the chosen rate, or provide confidence intervals that account for the selection.","section":"Section 3, Table 1"},{"comment":"The paper reports only averaged metrics over three seeds (seeds 1, 2, and 3) and gives no standard deviations, per-seed values, confidence intervals, or significance tests. Given that the other replacement rates (10%, 50%, 100%) show much smaller gains (0.7, 1.6, and 0.9 percentage points) and deep learning training is typically noisy, the 3.2-point gap could plausibly lie within run-to-run variability. The authors should report the full distribution of results and ideally a paired significance test (e.g., a paired t-test or Wilcoxon signed-rank test over seeds) to support the claim.","section":"Section 3, Experimental Results and Table 1"},{"comment":"The generation of the light bank relies on a binary mask with 'a threshold range for shades of white' (Section 2.2), but the actual threshold values are not specified anywhere in the manuscript. This threshold is a free parameter that directly controls the appearance of the synthetic reflections and thus the difficulty of the augmentation. The authors should state the threshold range and, preferably, provide a visual or quantitative comparison between the distribution of generated lights and real reflections to support the claim of realism.","section":"Section 2.1"}],"minor_comments":[{"comment":"'0.90% improved in mAP50' should read '0.9 percentage points improvement in mAP50' (similarly '3.2% improvement' is a percentage-point change).","section":"Section 3, Experimental Results with 100% Replacement"},{"comment":"There is a typo: 'procedures;in this case' should be 'procedures; in this case'.","section":"Section 2.2"},{"comment":"The retry count n mentioned in 'the algorithm tries again n number of times' is never given a numeric value; please specify the value used in the experiments.","section":"Section 2.3"},{"comment":"There is a typo: 'equiped' should be 'equipped'.","section":"Section 3, Implementation Details"},{"comment":"In the introduction, 'two-stage detectors. such as Faster R-CNN' contains an erroneous period after 'detectors'.","section":"Section 1"},{"comment":"The sliding-window step size is described as the width/height of the cropped light image plus one, but it is not clear whether the overlap is intended or whether the step size should be exactly the width/height to avoid overlapping windows; clarify the design choice.","section":"Section 2.3"}],"recommendation":"major_revision","confidential_remarks":"The manuscript reports a plausible and domain-relevant augmentation idea, but the evaluation falls short of the standard needed to support the central claim. The selection of the 20% replacement rate on the test set, combined with the absence of any variance or significance reporting, is a load-bearing problem that can be fixed within the manuscript's scope. I would also encourage the authors to consider releasing code or a detailed pseudocode for the light-bank generation and placement, since the current description leaves some parameters unspecified (e.g., the white-mask threshold, the retry count n). The paper's fit to the journal is acceptable; the main concern is rigor, not novelty."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nHere's my read on the WLSR augmentation paper. The specific pipeline—crop real specular reflections from training images, distort them into a bank of 3,600 lights, mark prohibited areas (existing lights, polyp boxes, black borders), then slide a random light across each image and paste only into clean regions—is genuinely new as far as the cited prior art goes, and it targets a real failure mode: detectors mistaking specular highlights for polyps. The paper is easy to follow and the method is simple enough to reproduce from the description, modulo a few missing parameter values. Credit is due for testing several replacement rates and for checking that WLSR composes with standard augmentations.\n\nThe soft spot is the one the stress-test note flags, and it is real. The authors compared 10/20/50/100% replacement on the same test set and then reported the 20% result as the headline. That is test-set selection, and it biases the +3.2 mAP50 gain upward. The other rates gave 0.7, 1.6, and 0.9 points, so the spread across configurations is as large as the claimed effect. They averaged three seeds but give no standard deviations, per-seed numbers, confidence intervals, or significance tests. With four configurations and three seeds, the 3.2-point jump could easily be within run-to-run noise. Also minor: the white-mask threshold range is never stated, n retries for placement is unspecified, and the '0.90% improved' line is really 0.9 percentage points.\n\nThat said, the direction of the effect is consistent—every replacement rate improved mAP50 over baseline—so I would not claim the method does nothing. The paper just does not support the specific 3.2% number, and the authors' 'confirms our hypothesis' phrasing is too strong. The visual example is illustrative, not evidence.\n\nWho gets value: any group working on data augmentation for endoscopic detection. The method is plug-in, costs nothing at inference, and attacks a known source of false positives. It deserves a serious referee. For review, I would ask the authors to hold out a tuning split, report per-seed results and standard deviations, add a significance test or at least show the distribution, and release code plus the two missing parameters. Without that, the paper should only be accepted as a preliminary result, not as a definitive demonstration.\n\nMy call: send it to peer review; it is a legitimate incremental contribution with a methodological flaw that is fixable in revision.","headline":"A simple, new reflection-inpainting augmentation that probably helps a bit, but the headline gain is not statistically pinned down because the rate was chosen on the test set.","tokens_in":7057,"tokens_out":2011,"would_cite":false,"duration_ms":19304,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Pasting synthetic white-light reflections into colonoscopy training images helps polyp detectors stop confusing glare with polyps, raising a standard detection score from 77.5% to 80.7%.","keywords":["polyp detection","data augmentation","specular reflection","white light reflection","colonoscopy","YOLOv5","deep learning","false positives"],"falsifier":"Run the same 20%-replacement comparison on a colonoscopy test set recorded with a different endoscope model or light source, keeping the synthetic reflection bank fixed; if the mAP50 gain disappears while the bank still looks similar to training-set reflections, the augmentation is not transferring to real reflections.","tokens_in":6107,"feed_emoji":"🔦","tokens_out":8820,"duration_ms":82988,"temperature":0.7,"pith_summary":"The paper is trying to establish that adding artificial white-light specular reflections to colonoscopy training images makes deep-learning polyp detectors better at telling real glare from polyps. The idea is to give the model more chances to be confused during training so it learns to suppress those confusers at test time. In the main comparison, training a YOLOv5 detector with 20% of images replaced by WLSR-augmented versions raises the mean-average-precision score mAP50 from 77.5% to 80.7%, a 3.2-point gain. If the claim holds, data augmentation alone can reduce a specific class of false positives without new network architectures or manual relabeling.","feed_headline":"Polyp detectors gain 3.2 points when trained on synthetic glare","feed_subtitle":"Pasting artificial white-light reflections into 20% of colonoscopy images lifts mAP50 from 77.5% to 80.7%.","key_machinery":"The load-bearing mechanism is a three-stage data-augmentation pipeline. First, a Bank of Lights is created by cropping 300 real white-light reflection regions from the training set, adjusting their intensity shades, and applying flips, random scaling between 0.8 and 1.2, and random rotation between -30 and 30 degrees, yielding 3,600 patches. Second, prohibited areas are identified and colored orange: existing lights, polyp bounding boxes, and black borders in the outer 20% margins. Third, a sliding window moves across each image in steps of the patch width plus one and height plus one, records every top-left position whose covered area contains no orange pixels, and randomly places one artificial light at one such position. This ensures the added glare looks natural, does not occlude polyps, and does not deform existing reflections.","core_discovery":"The central claim is that deliberately increasing the number of white-light specular reflections in training images improves polyp detection, rather than hurting it. The paper builds a bank of 3,600 artificial light patches from real reflection regions, marks areas where a new light must not go (existing lights, polyps, and black borders), and pastes one synthetic light into each eligible training image via a sliding-window search. Training YOLOv5 with 20% WLSR-replaced images yields 80.7% mAP50 versus 77.5% with the original images, and recall rises from 63.6% to 71.5%. Combining WLSR with standard augmentations gives 81.5% mAP50, suggesting the augmentation is additive. The authors interpret the results as confirming that harder training scenarios, with more reflection opportunities to make mistakes, give the detector more chances to learn to distinguish polyps from specular highlights.","pith_inferences":["Editorial inference: because the paper only pastes one light per image and finds 20% replacement optimal, testing multiple lights per image and a finer sweep of replacement rates would reveal whether the gain saturates or has a dose-response ceiling.","Editorial inference: the same prohibited-area and sliding-window recipe could transfer to other endoscopic and laparoscopic domains with specular highlights, but the paper reports no evidence beyond the single colonoscopy dataset it uses.","Editorial inference: if the benefit comes mainly from adding bright, blob-shaped distractors, then simpler bright-ellipse or brightness-spike augmentations might capture part of the effect; comparing WLSR against such generic distractors would isolate what the realism of the reflection bank contributes.","Editorial inference: the method does not require a segmentation network, only bounding-box annotations and a white-mask threshold, so a natural test is whether the augmentation becomes even stronger when the reflection bank is built from a learned reflection detector rather than a fixed white threshold."],"forward_implications":["With a 20% replacement rate, WLSR raises mAP50 by 3.2 points over the original-image baseline, and recall improves by 7.9 points, so the augmentation changes detection behavior and not just confidence calibration.","WLSR stacks with ordinary augmentations: adding it to saturation, value, rotation, and flip augmentations raises mAP50 to 81.5%, so it can be plugged into existing training recipes.","Even 100% WLSR replacement still improves mAP50 slightly (78.4% versus 77.5%), which suggests the effect is not an artifact of mixing augmented and original images.","The visual example shows a model trained with WLSR no longer flags a specular reflection as a polyp, directly addressing one documented source of false positives in polyp detection."],"supporting_citations":[{"why":"Supplies the public colonoscopy dataset used for all training, validation, and test experiments.","marker":"[19]"},{"why":"Provides the YOLOv5 detector that the paper trains and evaluates, so the augmentation is tested on a real single-stage detection model.","marker":"[18]"},{"why":"Establishes the existing data-augmentation recipe for real-time polyp detection and identifies reflections and poor lighting as a false-positive source this paper targets.","marker":"[8]"},{"why":"Defines the YOLO single-stage detector family from which the experimental detector is drawn.","marker":"[12]"}],"fun_headline_variants":["Synthetic glare lifts polyp detection by 3.2 points","Adding fake reflections sharpens polyp detectors","Train on artificial light to spot more polyps","White-light reflection augmentation improves polyp finding"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that synthetic reflections cropped from the training images and pasted into non-prohibited areas are realistic enough to stand in for the real white-light reflections the detector will face at test time.","fun_headline_variants_meta":{"raw":{"variants":["Synthetic glare lifts polyp detection by 3.2 points","Adding fake reflections sharpens polyp detectors","Train on artificial light to spot more polyps","White-light reflection augmentation improves polyp finding"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000249,"raw_usage":{"total_tokens":1553,"prompt_tokens":951,"completion_tokens":602,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":567,"completion_tokens_details":{"reasoning_tokens":544}},"tokens_in":567,"tokens_out":602,"duration_ms":6120,"temperature":1.0,"reasoning_tokens":544,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T23:08:47.658429+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same 20%-replacement comparison on a colonoscopy test set recorded with a different endoscope model or light source, keeping the synthetic reflection bank fixed; if the mAP50 gain disappears while the bank still looks similar to training-set reflections, the augmentation is not transferring to real reflections.","supporting_citations":[{"cited_title":"You only look once: Unified, real-time object detection,","cited_arxiv_id":null,"evidence_quote":"Supplies the public colonoscopy dataset used for all training, validation, and test experiments."},{"cited_title":"Intracker: An integrated detector-tracker framework for cell detection and tracking,","cited_arxiv_id":null,"evidence_quote":"Provides the YOLOv5 detector that the paper trains and evaluates, so the augmentation is tested on a real single-stage detection model."},{"cited_title":"Colorectal cancer statistics, 2023,","cited_arxiv_id":null,"evidence_quote":"Establishes the existing data-augmentation recipe for real-time polyp detection and identifies reflections and poor lighting as a false-positive source this paper targets."}],"review_version":1}