{"id":"94504fc7-e0a2-46d2-9379-9cea189a4708","arxiv_id":"2506.17837","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"Time-contrastive prompt retrieval plus SAM2 video object segmentation yields strong FLARE 2022 Dice scores, but the reported gains are confounded by unequal fine-tuning and missing error bars.","lead":"This paper trains a self-supervised retriever on video frames so that, with a few labeled examples, a video segmentation model can segment new CT images by treating the examples plus the query as a short video. On the FLARE 2022 dataset the authors report Dice scores near 91% for images and 92% for videos, but the comparison is weakened by a tiny test set and by fine-tuning applied only to their method.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed 10.64% and 14.88% improvements are confounded by fine-tuning and an inconsistent improvement basis: compute fine-tuned baselines and verify the arithmetic of the claimed gains.","rationale":"The reader identified the fine-tuning confound as the weakest assumption, and the manuscript text supports this reading: Section 2.3 describes fine-tuning only for Temporal, Tables 1 and 2 list baselines without fine-tuning, and the abstract's improvement numbers are computed against unfine-tuned baselines. My independent check of the arithmetic shows the abstract's 10.64% and 14.88% figures are marginal/ambiguous: 10.64 is a percentage-point difference from 80.31 to 90.95, and 14.88 matches 92.45 minus 77.57 (SAM2), while the conclusion cites 14.88% without specifying which baseline. The no-fine-tune comparison (83.55 vs 80.31, Table 1) is a modest positive result, but with 10 test scans and no error bars it is not sufficient to support the headline claims. The method itself is plausible: treating top-K retrieved context frames plus the query as a synthetic video for SAM2 is a reasonable VOS-based ICL formulation, and the time-contrastive objective with a multipositive cross-entropy loss is a sensible extension of TCN. However, the central quantitative claims are not supported as stated because the comparison is inconsistent. The proposed concrete test directly settles the confound by fine-tuning baselines under the same recipe. I therefore agree with the reader's REJECT verdict at moderate confidence: the paper is promising but the evidence is insufficient as presented.","tokens_in":8353,"tokens_out":1805,"duration_ms":14715,"concrete_test":"Re-run Table 1 and Table 2 with the same fine-tuning protocol (Section 2.3: synthetic videos from each retriever's top-K contexts, 100 epochs, lr=1e-5, SAM2-B+) applied to at least the two strongest baselines, Foster et al. (DINOv2) for image segmentation and Medical SAM2 for video segmentation. Report per-organ Dice with per-scan standard deviations across the 10 test scans. If the fine-tuned baselines close the gap to within a few Dice points, the claimed 10.64/14.88 improvements do not survive; if the gap persists, the fine-tuning confound is resolved.","verdict_should_be":"REJECT","load_bearing_attack":"The central claim depends on comparing 'Temporal after fine-tuning' against baselines that are not fine-tuned. In Section 3.1, the no-fine-tune comparison is Temporal 83.55% vs. best baseline 80.31% (+3.24 points), and only after the fine-tuning stage described in Section 2.3 does Temporal reach 90.95% Dice, reported as a 10.64% improvement. The fine-tuning stage trains SAM2 on synthetic videos built from the same retriever's top-K contexts, so it changes the VOS backbone, not just the retriever. Baselines such as Foster et al. with CLIP, DINOv2, MedCLIP, and SAM2/Medical SAM2 receive no equivalent fine-tuning. If the same fine-tuning recipe improves those baselines by a comparable margin, the headline gain is attributable to the extra fine-tuning stage rather than to time-contrastive retrieval. Additionally, the reported improvement arithmetic is inconsistent: 90.95 - 80.31 = 10.64 (percentage points), while 92.45 - 77.83 = 14.62 and 92.45 - 77.57 = 14.88; the abstract's 14.88% is only achieved against the weaker SAM2 baseline, and Table 2's 92.23 vs 77.83 gives 14.4%, while the conclusion states 14.88% for video without specifying the baseline. The paper also provides no error bars for a 10-scan test set, so the 3.24-point no-fine-tune advantage and the much larger fine-tuned gaps are not shown to be statistically reliable.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Temporal, a self-supervised time-contrastive objective for pretraining a prompt retriever for visual in-context segmentation. Context images are arranged as frames of a synthetic video and segmented by a video object segmentation (VOS) model, SAM2. For image segmentation the retriever selects top-K annotated contexts; for video segmentation it selects keyframes, predicts keyframe masks with the same pipeline, and propagates them bidirectionally. The method is evaluated on MICCAI FLARE 2022 CT organ segmentation using 40 scans for context/pretraining and 10 scans for testing. The paper reports 90.95% Dice for image segmentation and 92.45% Dice for video segmentation after fine-tuning, claiming gains of 10.64 and 14.88 percentage points over non-fine-tuned baselines.","tokens_in":8707,"tokens_out":6527,"duration_ms":61960,"significance":"If the central comparisons were properly controlled, the contribution would be meaningful for medical few-shot segmentation: a domain-specific self-supervised retriever that supports variable context size and full-resolution images, a unified image/video inference pipeline, and a plausible pretraining objective with a multipositive contrastive loss. The no-fine-tune image result (83.55% vs. 80.31% Dice) is a positive but modest signal, and the authors provide a code link and a coherent method description. However, the headline gains are not established because the fine-tuned comparisons are uncontrolled and the reported gain arithmetic is internally inconsistent.","major_comments":[{"comment":"The headline improvements compare a fine-tuned Temporal against baselines that are not fine-tuned. The only controlled comparison is the no-fine-tune image row, where Temporal achieves 83.55% versus 80.31% for Foster et al. with DINOv2, a 3.24-point gap. The fine-tuned 90.95% result cannot be attributed to the time-contrastive retriever unless the same fine-tuning recipe is applied to the VOS-based baselines, because the procedure trains SAM2 on synthetic videos constructed by the same retriever and could improve any VOS-based ICL pipeline regardless of the retriever. Please add fine-tuned versions of the baselines and report the controlled comparison.","section":"§2.3 (Fine-tuning), §3.1, Tables 1–2"},{"comment":"The reported improvement arithmetic is inconsistent. For image segmentation, 90.95 − 80.31 = 10.64 percentage points, but the conclusion states a 9.23% increase. For video segmentation, 92.45 − 77.83 = 14.62, while 92.45 − 77.57 = 14.88; the abstract's 14.88% is only against the weaker SAM-2 baseline, and the row reaching 92.45% uses a manual mask prompt, whereas the automated fine-tuned row is 92.23% (14.4% above Medical SAM-2). The paper should specify the exact baseline for every reported gain and use 'percentage points' rather than '% improvement'.","section":"Abstract, §4, Table 2"},{"comment":"No error bars, confidence intervals, or significance tests are reported for any result, despite the test set consisting of only 10 scans (922 slices). The 3.24-point no-fine-tune advantage and the organ-level differences (e.g., Pancreas 79.53% vs. 49.10%) may be within patient-level or run-to-run variability. Please report per-scan variances, bootstrap intervals, or repeated-run statistics, and clarify what the '5 evals' for the random baseline actually are.","section":"§3.1, Tables 1–2"}],"minor_comments":[{"comment":"The negative-sum term in the denominator should be exp(ẑ_i^T ẑ_k / τ), not exp(ẑ_k^T ẑ_j / τ); as written, the loss does not match the stated contrastive objective.","section":"Eq. (5)"},{"comment":"The Zhang et al. grid-based row is reported as N/A, while the text states that the grid-based method achieves 'around 5%' Dice. Please either include the actual numbers or remove the row and adjust the text.","section":"Table 1"},{"comment":"The expressions '10.64% improvement' and '14.88% improvement' describe percentage-point differences, not relative improvements; use 'percentage points' to avoid overstating the gains.","section":"Abstract and §4"},{"comment":"The confidence score c_q used in confidence-based keyframe filtering is not defined. Please specify how the VOS model produces a confidence value for each keyframe prediction.","section":"§2.3, video segmentation"},{"comment":"The claimed ↑2.1% improvement for context size 5 is not shown in Table 1; only the size-10 comparison (89.65% vs. 90.95%) appears. Also, the validation protocol for setting λ=0.7 is not described, so it is unclear whether any of the 10 test scans were used for model selection.","section":"§3.1, Diversity-Aware Context Selection"},{"comment":"The word 'presever' appears to be a typo for 'preserve'.","section":"Introduction"}],"recommendation":"major_revision","confidential_remarks":"The core idea is plausible and the no-fine-tune comparison gives a small positive signal, but the paper currently overclaims because of the uncontrolled fine-tuning comparison and inconsistent gain arithmetic. Since these issues can be addressed with additional experiments and clarifications, I recommend major revision rather than rejection. If the authors cannot provide fine-tuned baselines or error bars, the claims should be substantially scaled back."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a reasonable idea with a real but modest no-fine-tune signal, undercut by a fine-tuning asymmetry that makes the headline numbers uninterpretable as stated.\n\nWhat's new: the specific combination of TCN-style time-contrastive pretraining with multipositive cross-entropy as the retriever for VOS-based ICL, plus diversity-aware keyframe selection for video. That combination is new, and the motivation is sensible—grid-based ICL loses resolution and limits context count, and domain-specific retrieval matters for medical images. The no-fine-tune comparison in Table 1 (83.55 vs 80.31 Dice) is a genuine, if small, positive signal, and the method description is clear enough to reproduce.\n\nThe load-bearing problem is the fine-tuning confound. The 90.95 and 92.45 numbers come after fine-tuning the SAM2 backbone on synthetic videos built by the same retriever, while all baselines are evaluated off-the-shelf. That is not apples-to-apples. If the same fine-tuning recipe lifts the baselines by a comparable margin, the claimed 10.64 and 14.88 point gains vanish. The paper needs fine-tuned baselines—at least Foster et al. with each retriever, and SAM2/MedSAM2—before any claim about retrieval quality can be assessed.\n\nThe arithmetic is messy too: 90.95 - 80.31 = 10.64 points, fine, but the conclusion says 9.23% for image; for video, 92.45 - 77.57 = 14.88 but against Medical SAM-2 it's 14.62, and Table 2 reports 92.23 fine-tuned with a 14.4% gap. The abstract's \"14.88% improvement\" is only true against the weaker SAM2 baseline. That inconsistency across abstract, results, and conclusion suggests the numbers were not checked. No error bars on a 10-scan test set makes the 3.24-point no-fine-tune edge statistically unverified, and the N/A grid baseline row is a gap but a minor one since they justify abandoning it.\n\nThe retriever itself is never evaluated directly, so it's unclear how much of the gain comes from retrieval vs. the fine-tuning stage.\n\nWho is this for? People working on medical few-shot segmentation and visual ICL. It deserves a serious referee because the idea is sound and the flaws are fixable with proper ablations and fine-tuned baselines. As submitted, I would not accept the headline claims. Recommend: send to peer review, but expect major revision—the authors must add fine-tuned baselines, a direct retriever evaluation, uncertainty quantification, and align the reported percentages.","headline":"Plausible combination of time-contrastive retrieval and VOS-based ICL, but the headline gains are confounded by an imbalanced fine-tuning procedure and inconsistent arithmetic.","tokens_in":9264,"tokens_out":2285,"would_cite":false,"duration_ms":19671,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Time-contrastive prompts lift CT organ Dice by up to 14.9 points","keywords":["time-contrastive learning","visual in-context learning","video object segmentation","prompt retrieval","medical image segmentation","self-supervised pretraining","CT organ segmentation","few-shot segmentation"],"falsifier":"Run the same fine-tuning protocol (100 epochs, SAM2-B+ backbone, synthetic videos built by each baseline's own retriever) on the VOS-based ICL baseline with CLIP, DINOv2, and MedCLIP retrieval as well as on SAM2 and Medical SAM2 with mask prompts, then re-evaluate on the same 10 FLARE test scans; if the gap narrows to a few Dice points, the time-contrastive objective is not the cause of the reported improvements.","tokens_in":8148,"feed_emoji":"🩻","tokens_out":7434,"duration_ms":62848,"temperature":0.7,"pith_summary":"Temporal, the method introduced here, claims that a prompt retriever pretrained with a time-contrastive objective—where nearby video frames count as matching examples and distant frames as non-matching—can choose the context images that make visual in-context learning work for organ segmentation. The paper casts in-context learning itself as a video object segmentation problem: selected context pairs plus the query are concatenated as frames of a synthetic video and passed to a VOS model (SAM2), which labels the query. On the MICCAI FLARE 2022 CT dataset it reports 90.95% Dice for image segmentation (10.64 points above its best baseline) and 92.45% Dice for video segmentation (14.88 points above baselines), with the largest jumps on organs like the pancreas. The deeper claim is that temporal proximity in unlabeled video is a sufficient signal for learning which examples to retrieve, removing the need for labeled retrieval training.","feed_headline":"Time-contrastive prompts lift CT organ Dice by up to 14.9 points","feed_subtitle":"A self-supervised retriever turns in-context learning into video object segmentation, no manual prompts needed.","key_machinery":"The load-bearing mechanism is the multi-positive time-contrastive loss, a self-supervised objective that trains an encoder so that temporally adjacent frames are pulled together and distant frames pushed apart; it extends Time-Contrastive Networks by replacing the triplet loss with a multi-positive cross-entropy computed over a binary label matrix, allowing multiple positives per anchor and cross-view positives via augmentation. The second half of the machinery is the formulation of ICL as video object segmentation: context pairs and the query are arranged as frames in a synthetic video, and a pretrained VOS model (SAM2) segments the final frame, which permits arbitrarily many full-resolution context images. For video inputs, the same retriever scores each frame, picks diverse keyframes, and propagates their masks bidirectionally to the entire sequence.","core_discovery":"The central discovery is that the context-retrieval problem for visual in-context learning can be solved by self-supervision from video: frames that are close in time are treated as positive pairs, frames far apart as negative pairs, and a ResNet-50 encoder trained with a multi-positive cross-entropy loss on this signal produces embeddings in which semantically similar medical images cluster. When a query arrives, the top similar training images are retrieved, concatenated with the query as a synthetic video, and segmented by the SAM2 VOS model without any manual prompt. The same retriever also scores every frame of a full video, selects temporally diverse keyframes that receive high-confidence mask predictions, and propagates those masks bidirectionally to cover the sequence. The paper attributes the large gains on FLARE 2022, especially on the pancreas, to this combination of time-contrastive retrieval and VOS-based inference.","pith_inferences":["One testable extension is to ablate the retriever: substituting random keyframes on the same fine-tuned VOS backbone would isolate how much of the video Dice gain comes from retrieval rather than propagation.","The paper's own design suggests the fine-tuning stage may be the dominant contributor; fine-tuning each baseline with the identical synthetic-video protocol would separate the retriever's contribution from the fine-tuning's.","The time-contrastive objective could generalize to other ICL modalities, such as using temporal order in surgical or ultrasound sequences to retrieve prompts for detection or registration.","An implicit consequence is that the retriever might transfer across datasets without retraining, since embeddings trained on FLARE videos could retrieve context for other CT tasks; the paper does not test this."],"forward_implications":["If the reported gains hold, in-context segmentation can run fully automatically on CT volumes, with no manual first-frame prompt for videos and only a handful of labeled slices for new organs.","The capacity for variable, full-resolution context sets removes a known ceiling of grid-based visual in-context learning, which should help tasks that require fine spatial detail.","Because the retriever learns from unlabeled video alone, the same pretraining signal is available in any temporally coherent domain, from ultrasound sweeps to endoscopic video.","The diversity-aware selection step indicates that context sets should be chosen for coverage as well as similarity, a design principle that transfers to any retrieval-based few-shot learner."],"supporting_citations":[{"why":"Introduced the time-contrastive sampling idea this objective builds on.","marker":"[16]"},{"why":"Formulated visual in-context learning as a VOS task, the direct foundation this paper extends with its own retriever.","marker":"[8]"},{"why":"Provided the SAM2 VOS model used for segmentation and as a video baseline.","marker":"[15]"},{"why":"Showed that context selection matters for visual ICL and supplied the CLIP-based retrieval baseline.","marker":"[24]"},{"why":"One of the retrieval baselines (DINOv2) that Temporal outperforms on FLARE 2022.","marker":"[13]"},{"why":"The standard CLIP embedding used for context retrieval in prior ICL work and a baseline here.","marker":"[14]"}],"fun_headline_variants":["Time-contrastive retriever: ICL for images and video, no manual prompts","Reframing in-context learning as video segmentation lifts Dice by 14.9","Self-supervised prompt retriever: no prompts, just time cues, better Dice","Time-contrastive video frames pick contexts, improving CT Dice by 14.9"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline improvements compare fine-tuned Temporal against baselines that were not fine-tuned; if the same synthetic-video fine-tuning boosts the baselines to a similar extent, the gains attributed to time-contrastive retrieval would largely disappear.","fun_headline_variants_meta":{"raw":{"variants":["Time-contrastive retriever: ICL for images and video, no manual prompts","Reframing in-context learning as video segmentation lifts Dice by 14.9","Self-supervised prompt retriever: no prompts, just time cues, better Dice","Time-contrastive video frames pick contexts, improving CT Dice by 14.9"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000528,"raw_usage":{"total_tokens":2565,"prompt_tokens":982,"completion_tokens":1583,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":598,"completion_tokens_details":{"reasoning_tokens":1489}},"tokens_in":598,"tokens_out":1583,"duration_ms":12502,"temperature":1.0,"reasoning_tokens":1489,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T19:00:05.312164+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same fine-tuning protocol (100 epochs, SAM2-B+ backbone, synthetic videos built by each baseline's own retriever) on the VOS-based ICL baseline with CLIP, DINOv2, and MedCLIP retrieval as well as on SAM2 and Medical SAM2 with mask prompts, then re-evaluate on the same 10 FLARE test scans; if the gap narrows to a few Dice points, the time-contrastive objective is not the cause of the reported improvements.","supporting_citations":[{"cited_title":"Flexible visual prompts for in-context learning in computer vision","cited_arxiv_id":"2312.06592","evidence_quote":"Formulated visual in-context learning as a VOS task, the direct foundation this paper extends with its own retriever."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provided the SAM2 VOS model used for segmentation and as a video baseline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"One of the retrieval baselines (DINOv2) that Temporal outperforms on FLARE 2022."}],"review_version":2}