{"id":"c6c01480-1f6f-4749-84db-510a55ca5858","arxiv_id":"2501.12331","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Cinepro fine-tunes MedSAM with an involvement-aware loss and cine-series augmentations, reporting 77.1% AUROC for prostate cancer detection in ultrasound cineloops.","lead":"This paper introduces Cinepro, a training method that adapts a medical image foundation model to detect prostate cancer in ultrasound video clips using only coarse pathology labels. If the gains hold outside this two-center dataset, it could make real-time biopsy guidance more accurate without needing pixel-level cancer annotations.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Section 2.4 does not state whether the 5-fold cross-validation is split by patient; with 10–12 cores per patient, core-level folds allow patient leakage that could inflate Cinepro's AUROC and make the headline superiority claim unreliable.","rationale":"The reader's weakest_assumption concerns the pathology-proportion target for the iMSE loss, and the reader's rationale also notes the missing patient-splitting detail. I agree the loss assumption matters, but the more fundamental and less fixable-by-rewording issue is the evaluation split. If folds are core-level, the test set contains cores from patients whose other cores were used for training; the model can memorize patient identity and inflate performance. This is a standard leakage failure mode. The paper's GitHub link makes the check straightforward. The abstract metric error should also be corrected, but it does not by itself change the ranking. Because the ambiguity is currently unresolved, the paper should not be accepted as-is; the reader's CONDITIONAL verdict is appropriate until the split unit is specified and, if necessary, the experiments are rerun. If the code inspection shows patient-level grouping, the concern would be resolved and the main results stand.","tokens_in":6507,"tokens_out":13580,"duration_ms":129931,"concrete_test":"Inspect the cross-validation code at https://github.com/mharmanani/cinepro to check whether the 5 folds are constructed by patient ID (e.g., GroupKFold or StratifiedGroupKFold) or randomly over cores. If not patient-stratified, rerun the full Table 2 comparison with patient-level grouping and record AUROC and balanced accuracy for Cinepro and the strongest baseline (MedSAM+iMSE+WS-Augs.). If the gap shrinks by more than one standard deviation or reverses, the central superiority claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing concern is the evaluation protocol. Section 2.4 says only that 'all models are trained and evaluated using a 5-fold cross-validation scheme'; it never states whether the folds are split by patient or by individual biopsy core. Each patient contributes 10–12 cores, so a core-level split places multiple cores from the same patient in both training and test. Under that protocol, the model can exploit patient- or transducer-specific cues (prostate shape, acoustic shadowing patterns, site-specific tissue properties) rather than learning generalizable cancer features. This would inflate the reported AUROC of 77.1 and the balanced accuracy of 71.9, and it would affect the comparison against the baselines in Table 2. The abstract's 'balanced accuracy of 83.8' is also a misreport (83.8 is the AUROC on the inv>0.35 subset), but that is a labeling error; the unstated split unit is a potential validity threat to every number in the paper.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript presents Cinepro, a training framework that adapts the MedSAM segmentation foundation model to prostate cancer detection in B-mode ultrasound cineloops. The method uses an 'involvement-aware' MSE loss that penalizes the difference between the mean model activation over the biopsy needle region and the pathologically reported percentage of cancer in the corresponding core, and a weak-strong augmentation scheme that combines a single frame with an averaged multi-frame 'strong' augmentation. The authors evaluate Cinepro against several in-house baselines (ROI-based iLR, UNet/SAM/MedSAM with masked CE or iMSE losses, with and without weak-strong augmentation) using 5-fold cross-validation on a private two-center dataset of 311 patients, reporting an AUROC of 77.1% and claiming superior performance over the baselines.","tokens_in":6731,"tokens_out":5728,"duration_ms":50233,"significance":"The idea of using quantitative pathology involvement as a regression target for a segmentation foundation model is a thoughtful approach to weak label noise, and the cine-series augmentation is a sensible use of temporal ultrasound data. The paper compares against a reasonable set of baselines and provides qualitative heatmaps. If the reported improvements are reliable, the approach would be a useful contribution to foundation-model fine-tuning for ultrasound. However, the present version contains internal numerical inconsistencies and an under-specified evaluation protocol, so the significance of the empirical claim is currently uncertain.","major_comments":[{"comment":"The abstract reports a balanced accuracy of 83.8%, and Section 3 states that Cinepro improves AUROC and balanced accuracy by +1.9% and +2.3% over the strongest baseline. According to Table 2, Cinepro's balanced accuracy is 71.9±1.6, not 83.8; the value 83.8 is the AUROC on the inv>0.35 subset. Also, compared with the strongest baseline (MedSAM+iMSE+WS-Augs), the improvements are +1.0 AUROC and +1.0 balanced accuracy, not +1.9/+2.3. These numbers must be corrected and the claims restated to match the table.","section":"Abstract and Section 3 (Results, first and last paragraphs)"},{"comment":"The 5-fold cross-validation is described only as 'all models are trained and evaluated using a 5-fold cross-validation scheme.' The split unit is not stated. Since each patient contributes 10–12 cores (Section 2.1), a core-level split would put multiple cores from the same patient in both training and test, allowing patient-specific or transducer-specific cues to inflate performance. The paper must specify whether the folds are patient-level or core-level, and if the latter, justify why this does not constitute leakage.","section":"Section 2.4 (Experiments)"},{"comment":"The loss function as written is mathematically inconsistent with the described behavior. The equation sums over all i,j ∈ [0,1024] the squared term (Ŷ[i,j]·1_{Ŷ[i,j]∈R} − inv)². For pixels outside R, the indicator makes the first term zero, so the contribution is inv², which would dominate the loss over the more than one million background pixels. The intended loss should be the mean of (Ŷ[i,j] − inv)² over pixels in R, e.g., using an indicator outside the square (or a sum restricted to R). Please correct the formula and confirm the implementation matches the intended definition.","section":"Section 2.2, Eq. (1)"},{"comment":"The iMSE loss is built on the assumption that the pathologically reported proportion of cancer in a biopsy core equals the spatial average of the model's pixel-level predictions over the needle region in the 2D ultrasound image. This assumption requires that the spatial distribution of cancer in the extracted core is faithfully represented in the 2D image plane, which is not self-evident given tissue deformation, needle placement, and slice thickness. The paper should acknowledge this assumption and provide supporting evidence (e.g., a scatter plot of predicted mean activation vs. reported involvement across cores) to justify the loss form.","section":"Section 2.2 (Involvement-aware training)"}],"minor_comments":[{"comment":"The text says 'we select the pixels in the needle region' for the final prediction, but the loss function defines R as the intersection of the needle region with the prostate mask. Please clarify which region is used for scoring and for the loss.","section":"Section 2.2, first paragraph"},{"comment":"The index ranges are inconsistent (text says i,j∈[1,1024] but the sum is written from 0 to 1024); please fix.","section":"Section 2.2, Eq. (1)"},{"comment":"The sentence 'The apply a confidence threshold, τ, to the output fθ(Xw)' contains a typo; it should be 'They apply' or 'We apply'.","section":"Section 2.3"},{"comment":"The hyperparameters τ, γ_w, and γ_s are described as tunable but their values or tuning procedure are not given; please provide them in the paper (or an appendix) rather than only in the GitHub repository.","section":"Section 2.3"},{"comment":"The claim that dual-augmented versions 'outperform their base counterparts ... by 6% and 1.1%' is ambiguous; specify whether these are absolute or relative differences and which metric is being referenced.","section":"Section 3 (Results)"},{"comment":"The exclusion of data containing motion and acoustic shadowing artifacts is mentioned but the number of excluded cineloops/cores is not reported; please quantify to allow assessment of selection bias.","section":"Section 2.1 (Preprocessing)"}],"recommendation":"major_revision","confidential_remarks":"The manuscript's strengths are the novelty of the involvement-aware loss and the cine-series augmentation. The main concerns are correctable but significant: the reported numbers do not match Table 2, the cross-validation split is unspecified, and the loss equation has a mathematical inconsistency. I recommend asking for a revision with corrected numbers, a clear description of the split, and a fixed loss expression."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper does what it says: a plausible, incremental extension of the authors' earlier ProstNFound work. The involvement-weighted MSE loss and cine-series augmentation together give a modest but consistent improvement over strong baselines in Table 2. But the abstract misreports the balanced accuracy, the text's improvement claims don't match the table, and Section 2.4 never states whether the 5-fold CV splits by patient or by core. With 10–12 cores per patient, a core-level split leaks patient identity across folds and would inflate the headline AUROC. If the split is patient-level, that concern disappears; the paper just needs to say so.\n\nWhat's actually new: regressing the mean predicted intensity inside the needle region to the pathology-reported involvement percentage (iMSE loss), and the cine-series weak-strong augmentation with confidence-weighted fusion. Both are reasonable, and the ablation chain (MaskCE→iMSE, translation→WS-Augs→cine-series) isolates their contributions. The multi-center dataset of 311 patients from two sites is a decent private collection, though no external validation.\n\nThe soft spots, in proportion. The missing split description is the load-bearing one; without it the reader cannot assess leakage. The abstract says 'balanced accuracy of 83.8%', but Table 2 shows 71.9±1.6; 83.8 is the AUROC on the inv>0.35 subset. The text's +1.9%/+2.3% improvement over the 'strongest baseline' doesn't match Table 2 (+1.0/+1.0 vs MedSAM+iMSE+WS-Augs; +2.3 matches no baseline). The iMSE loss assumes the pathology-reported percentage is a reliable spatial regression target, an untested assumption that nonetheless appears to work better than MaskCE. Exclusion of motion/shadowing artifacts is mentioned but not quantified.\n\nThis is for researchers working on weakly labeled ultrasound or foundation-model fine-tuning. It deserves a serious referee, but the authors should be asked to state the split unit, correct the abstract, and reconcile the improvement claims with the table. If the split is core-level, the central claim is unreliable; if patient-level, it's a solid if modest contribution.","headline":"Plausible incremental method; unstated patient-level split and a misreported balanced-accuracy metric are the key things to fix before trusting the headline numbers.","tokens_in":7267,"tokens_out":2896,"would_cite":true,"duration_ms":26634,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Cinepro claims that a segmentation foundation model, trained with a pathology-involvement loss and a multi-frame cine-series augmentation, detects prostate cancer in ultrasound cineloops at an AUROC of 77.1% and a balanced accuracy of…","keywords":["prostate cancer detection","ultrasound cineloops","foundation models","weakly supervised learning","involvement-aware loss","temporal augmentation","MedSAM","deep learning"],"falsifier":"Randomly permute the pathology involvement labels across cores (keeping the cancer/benign split fixed) and retrain Cinepro from the same initialization. If the AUROC remains near 77.1%, the involvement signal is not driving the gain; if it drops sharply, the label is load-bearing. A complementary check is to compute the Pearson correlation between the model's mean prediction in the needle region and the reported involvement on held-out cores, which should be substantially positive if the iMSE loss is teaching localization.","tokens_in":6338,"feed_emoji":"🩺","tokens_out":8385,"duration_ms":78773,"temperature":0.7,"pith_summary":"This paper claims that a segmentation foundation model can be repurposed for prostate cancer detection in ultrasound by training on the whole image rather than a small region, and by supervising it with the pathologist-reported percentage of cancer in each biopsy core rather than a binary label. The proposed framework, Cinepro, takes the first frame of a 200-frame cineloop with a light augmentation and the average of the remaining frames with heavy augmentation, shares weights between the two views, and optimizes an involvement-aware mean-squared error between the model's average activation in the needle region and the reported involvement. On a two-center dataset of 311 patients, the paper reports an AUROC of 77.1% and a balanced accuracy of 71.9%, beating prior ROI-based and whole-image baselines; the 83.8% figure quoted in the abstract is the AUROC on the subset of cores with more than 35% involvement. A sympathetic reader would care because whole-image, involvement-supervised training offers a path to better real-time biopsy guidance without pixel-level cancer annotations.","feed_headline":"Prostate cancer AUROC hits 77.1% via ultrasound video training","feed_subtitle":"The model averages 199 video frames and uses per-core cancer proportions as the training target.","key_machinery":"The central object is the involvement-aware mean-squared error (iMSE) loss: for each core, the model's 256x256 prediction mask is averaged inside the intersection of the manually selected needle region and the prostate mask, and that scalar is forced toward the pathology-reported cancer involvement percentage. The second mechanism is a weak-strong cine-series augmentation: a lightly augmented first frame and a strongly augmented average of the other 199 frames are processed by two shared-weight encoders and combined by a confidence-thresholded weighted average before the loss is computed. Together they convert a coarse per-core label into a spatially distributed training signal.","core_discovery":"On the paper's own terms, the central discovery is that a segmentation foundation model can be turned into a cancer detector for ultrasound without pixel-level labels by using the pathologist's per-core cancer involvement as a regression target and by training on whole cineloops with a weak-strong temporal augmentation. In a 5-fold cross-validation on two clinical centers, Cinepro reaches an AUROC of 77.1% and a balanced accuracy of 71.9%, with sensitivity of 78.1% at 60% specificity, surpassing the ROI-based iLR baseline and MaskCE/iMSE-trained UNet, SAM, and MedSAM variants. The authors report that the involvement-aware loss yields larger gains when paired with MedSAM than with UNet, and that the cine-series augmentation adds a further improvement, with the qualitative heatmaps tracking the reported 15-70% involvement range more closely than the fine-tuned MedSAM baseline. The abstract's balanced-accuracy figure of 83.8% corresponds in the results table to the AUROC on cores with more than 35% involvement, not to the table's balanced accuracy of 71.9%.","pith_inferences":["Editorial inference: The same involvement-regression loss, if it works as claimed, should transfer to other core-biopsy settings where pathology returns a proportion (e.g., tumor cellularity in breast or renal biopsy), provided the needle track is visible.","Editorial inference: Because the loss averages over a 2D needle region while the pathology core spans 18 mm of tissue, a volumetric ultrasound formulation would be a stricter test of whether the involvement signal or the temporal averaging is doing the heavy lifting.","Editorial inference: An ablation using fewer than 199 averaged frames would reveal whether the cine-series gain comes from denoising per se or from the particular temporal statistics of 3-5 seconds of transducer motion."],"forward_implications":["Whole-image foundation models outperform ROI-based and patch-based approaches on this task, so keeping full anatomical context is a better default for ultrasound prostate cancer detection.","Using iMSE instead of masked cross-entropy improves AUROC and sensitivity at matched specificity, suggesting that involvement supervision adds information beyond a binary benign/cancer label.","The cine-series augmentation yields further gains on top of iMSE, and the gain is larger for a generalist SAM than for MedSAM, suggesting the strategy is most useful when pre-training is weaker.","At 60% specificity Cinepro reaches 78.1% sensitivity, which is the operating point the paper highlights for biopsy guidance."],"supporting_citations":[{"why":"Supplies the MedSAM segmentation foundation model whose image encoder and mask decoder Cinepro adapts.","marker":"[4]"},{"why":"Provides the whole-image baseline (MaskCE) and the ProstNFound method that Cinepro extends and compares against.","marker":"[3]"},{"why":"Supplies the ROI-based iLR baseline that uses involvement-based label refinement; Cinepro claims to outperform it.","marker":"[10]"},{"why":"Provides the FixMatch weak-strong consistency idea that the cine-series augmentation strategy is loosely inspired by.","marker":"[8]"},{"why":"Extends weak-to-strong consistency to semantic segmentation, the basis for the dual-augmentation training loop.","marker":"[9]"},{"why":"Supplies the original SAM architecture used as an ablation baseline without the medical-domain pre-training.","marker":"[5]"},{"why":"Documents the multi-center ultrasound dataset and prior confidence-based detection approach from which Cinepro inherits the study setup.","marker":"[2]"}],"fun_headline_variants":["Ultrasound video + pathology ratios: 77.1% AUROC for prostate cancer","Weak-label cineloop training beats ROI methods in prostate cancer AI","Cinepro: Foundation model learns cancer from biopsy proportions","Whole-cineloop supervision lifts prostate cancer AUROC to 77.1%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the percentage of cancer measured in the extracted biopsy core corresponds to the average of what the ultrasound image shows inside the needle region, so training the model's mean activation to match that number teaches it where cancer actually is.","fun_headline_variants_meta":{"raw":{"variants":["Ultrasound video + pathology ratios: 77.1% AUROC for prostate cancer","Weak-label cineloop training beats ROI methods in prostate cancer AI","Cinepro: Foundation model learns cancer from biopsy proportions","Whole-cineloop supervision lifts prostate cancer AUROC to 77.1%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000296,"raw_usage":{"total_tokens":1741,"prompt_tokens":994,"completion_tokens":747,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":610,"completion_tokens_details":{"reasoning_tokens":665}},"tokens_in":610,"tokens_out":747,"duration_ms":7969,"temperature":1.0,"reasoning_tokens":665,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T17:15:37.978540+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Randomly permute the pathology involvement labels across cores (keeping the cancer/benign split fixed) and retrain Cinepro from the same initialization. If the AUROC remains near 77.1%, the involvement signal is not driving the gain; if it drops sharply, the label is load-bearing. A complementary check is to compute the Pearson correlation between the model's mean prediction in the needle region and the reported involvement on held-out cores, which should be substantially positive if the iMSE loss is teaching localization.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the MedSAM segmentation foundation model whose image encoder and mask decoder Cinepro adapts."},{"cited_title":"We observe that whole-image methods outperform the iLR baseline considerably, highlighting their advantage over ROI methods","cited_arxiv_id":null,"evidence_quote":"Provides the whole-image baseline (MaskCE) and the ProstNFound method that Cinepro extends and compares against."},{"cited_title":"& Wang, B","cited_arxiv_id":null,"evidence_quote":"Supplies the ROI-based iLR baseline that uses involvement-based label refinement; Cinepro claims to outperform it."},{"cited_title":"& Mousavi, P","cited_arxiv_id":null,"evidence_quote":"Provides the FixMatch weak-strong consistency idea that the cine-series augmentation strategy is loosely inspired by."},{"cited_title":"& Mousavi, P","cited_arxiv_id":null,"evidence_quote":"Extends weak-to-strong consistency to semantic segmentation, the basis for the dual-augmentation training loop."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the original SAM architecture used as an ablation baseline without the medical-domain pre-training."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Documents the multi-center ultrasound dataset and prior confidence-based detection approach from which Cinepro inherits the study setup."}],"review_version":1}