{"id":"cbb2b605-8d0e-4a45-9e2c-6cfacec7f6f1","arxiv_id":"2607.24703","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"A Dinomaly-style frozen DINOv3 encoder with noisy bottleneck and linear-attention decoder reaches 88% pixel AUROC and 95% frame specificity on uterine myoma MRI at 40.5 slices/s.","lead":"A frozen vision foundation model plus a light decoder flags pelvic MRI anomalies without labels, at scanner speed. It could let radiologists adapt scans on the fly when something looks off.","discovery_kind":"new_application","skeptic_critique":{"model":"moonshotai/kimi-k3","headline":"The headline 88.06% pixel AUROC is reported on the same D_test that Bayesian optimization (40 configs) directly maximized, so the central performance claim carries unquantified selection bias on a small test set (198 anomalous / 102 healthy slices).","rationale":"My read independently converges on the reader's weakest_assumption: test-set-driven HPO is the single most load-bearing methodological issue, because it directly contaminates the strongest_claim's headline number (88.06% pixel AUROC) on a small, single-slice-per-exam test set. I do not adjust the verdict because the reader already priced this in as CONDITIONAL with appropriate remedies (held-out HPO, CIs). I considered two alternative candidate concerns and judged them secondary: (1) the frame-level sensitivity of 11.61% undermines the clinical-utility framing — but the paper is unusually transparent about this trade-off, calibrates those thresholds leakage-free on healthy training data, and frames the tool as high-specificity decision support, so this is a limitation honestly stated rather than a hidden flaw; (2) the single-slice-per-exam selection and the 28×28 token grid limit sensitivity to small lesions — again acknowledged in §4/§5 as the primary limitation. The throughput claim (40.5 slices/s on an RTX 3080) is plausible for a frozen ViT-L/16 plus small decoder and is not load-bearing for correctness. The frozen-encoder design and the small spread across ablations provide some independent robustness evidence, so the realistic outcome of fixing the evaluation is a modest downward correction, not a collapse — consistent with CONDITIONAL rather than REJECT.","tokens_in":9108,"tokens_out":1450,"duration_ms":62238,"concrete_test":"Re-run the evaluation with a strict split: use a held-out portion of D_test (e.g., 100 anomalous + 51 healthy slices, stratified by examination) as the HPO set for the 40-configuration Bayesian optimization, then evaluate the single winning configuration once on the untouched remainder. Report bootstrap CIs on both sets. If the untouched-set AUROC falls below ~85% (outside the CI of the HPO-set estimate), the headline number is selection-inflated and the deployment claim should be restated with the corrected figure. As a cheap complement, estimate the max-selection bias by bootstrapping predictions from a single trained config, simulating the spread of 40 correlated configs, and measuring the expected gap between max and mean AUROC.","verdict_should_be":"UNCHANGED","load_bearing_attack":"§3.2 states plainly: \"Hyperparameters were selected via Bayesian optimization over 40 configurations directly maximizing pixel-level AUROC on D_test.\" The paper defends this by noting no gradient updates touch test data, but that misses the point: selecting the argmax of 40 noisy estimates on the same set used for reporting is classic adaptive analysis / selection bias, and no bound or correction is offered. The problem is amplified by the evaluation design in §3.1: one pre-selected slice per examination (the one containing annotated lesions), 198 anomalous and only 102 healthy slices. With n=198 anomalous slices, the standard error of an AUROC estimate is on the order of ±2-3 points, and taking the max over 40 correlated-but-noisy configs can add a further optimistic shift of comparable magnitude. So the headline \"88.06% AUROC meets real-time deployment requirements\" rests on a number whose true generalization value is unknown and plausibly several points lower. The same pattern recurs for the binary pixel metrics (τ_px chosen by maximizing Dice on D_test — acknowledged as an \"upper bound,\" which is honest but means accuracy 93.62% / sensitivity 38.49% are not operating characteristics). To be fair, the frame-level operating point (τ_ref, τ_slice calibrated only on healthy D_train slices) is leakage-free, and the resulting 11.61% sensitivity is reported transparently — but that also means the only unbiased numbers in the paper are the ones showing weak frame-level detection (AUROC 68.41%). The ablation deltas in Table 2 were also selected on D_test, so even relative comparisons inherit the bias. Notably, the architecture itself (frozen DINOv3 encoder, ~4-point robustness across ablations) suggests the true AUROC is likely in the mid-80s rather than collapsed, so the concern is about calibration of the headline number, not wholesale invalidation.","agreement_with_reader":"agree"},"referee_report":{"model":"moonshotai/kimi-k3","summary":"The manuscript adapts the Dinomaly unsupervised anomaly detection architecture to sagittal T2w pelvic MRI: a frozen DINOv3 ViT-L/16 encoder, a dropout-noised MLP bottleneck, and a Linear Attention decoder trained with a group-wise cosine reconstruction loss on 207 healthy volumes (plus 800 synthetic images). Per-token cosine distance between encoder and decoder features yields pixel-level anomaly maps; a frame-level score based on the spatial extent of supra-threshold pixels is calibrated on healthy training data only. On a curated subset of the Uterine Myoma Dataset (198 anomalous slices, 102 healthy slices) the authors report 88.06% pixel-level AUROC, 95.45% frame-level specificity at 11.61% sensitivity, and 40.5 slices/s inference, positioning the system as a real-time scanner-side decision support tool, with a comparison to a ResVAE baseline and an ablation study.","tokens_in":9516,"tokens_out":2812,"duration_ms":109212,"significance":"If the results hold up under a clean selection protocol, the work is a useful and timely contribution: unsupervised anomaly detection for pelvic MRI is underexplored, the evaluation is on a public dataset (UMD) and framed as a reproducible benchmark, and the system-level properties (frozen encoder, 40.5 slices/s on a mid-range GPU) are directly relevant to scanner-side deployment. Notable strengths: the frame-level thresholds are calibrated exclusively on healthy training data and thus provide a genuinely leakage-free operating point; the authors are transparent about the oracle nature of the pixel threshold; the ablation (Table 2) includes a data-scaling experiment and a ViT-S speed/accuracy trade-off; and the head-to-head with a reconstruction-based ResVAE, showing complementary sensitivity/specificity profiles, is informative for the field. The paper also reports its weak frame-level sensitivity openly rather than hiding it.","major_comments":[{"comment":"Hyperparameters were selected by Bayesian optimization over 40 configurations directly maximizing pixel-level AUROC on D_test, and the same D_test is then used to report the headline 88.06% AUROC. The defense offered (no gradient updates on test data; validation loss uncorrelated with detection, rho_S=+0.094) does not address selection bias: taking the argmax of 40 noisy performance estimates on the reporting set is adaptive analysis, and with n=198 anomalous / 102 healthy slices the per-config AUROC standard error is on the order of 2-3 points, so the optimistic shift from selection is plausibly of comparable magnitude. This is load-bearing because the abstract's central claim is this number. The fix is feasible within scope: carve a validation split from D_test (or use nested resampling), re-select hyperparameters on it, and report the selected model on the untouched remainder; alterna","section":"§3.2"},{"comment":"The abstract states the framework 'meets real-time clinical deployment requirements', but the only operating point free of test-set tuning — the frame-level score with thresholds calibrated exclusively on healthy D_train slices — achieves 11.61% sensitivity at 95.45% specificity, i.e., roughly 9 of 10 anomalous slices are not flagged. A scanner-side triage tool that misses ~88% of anomalous slices is a very different clinical proposition than the abstract conveys. Speed (40.5 slices/s) is necessary but not sufficient for the deployment claim. The abstract and §5 should be rewritten to state the actual operating characteristics, and the claim of clinical utility should be tied to the pixel-level maps as a localization aid on flagged/visibly suspicious slices rather than to frame-level detection. As written, the abstract cherry-picks the tuned pixel-level number and the high specificity wh","section":"Abstract and §5, with Table 1"},{"comment":"The pixel binarization threshold tau_px is set by maximizing Dice across D_test. The text calls this an 'upper bound', which is honest, but the resulting accuracy 93.62% / sensitivity 38.49% / specificity 95.10% / F1 23.87% are then presented in Table 1 as the paper's primary pixel-level metrics without any marker distinguishing them as oracle (per-dataset optimal-threshold) values. Table 1 should annotate these rows as upper-bound/oracle, and the comparison with ResVAE in §4 should state explicitly whether the ResVAE numbers use the same oracle-threshold protocol (the slightly different values quoted in the text — 87.85 vs. 88.06, 38.39 vs. 38.49, 95.02 vs. 95.10 — suggest the two may not be from identical evaluation runs).","section":"§3.3 and Table 1"},{"comment":"The evaluation uses one pre-selected slice per examination — the slice containing the annotated lesion — for the 198 anomalous cases, and healthy slices sampled at distance >=2 from lesions. Two concerns: (i) selecting the lesion-containing slice conditions on the annotation and removes the natural prevalence and difficulty distribution of full volumes, inflating both pixel AUROC and frame scores relative to whole-volume operation; (ii) anomalous and healthy slices come from the same UMD exams, so the frame-level task partly tests within-exam slice discrimination rather than detection of anomalous exams. No confidence intervals are reported anywhere despite the small sample. At minimum, report bootstrap CIs for Table 1 and add a whole-volume evaluation (score all slices, aggregate per exam) as a secondary result, or explicitly scope the claims to the curated per-slice benchmark.","section":"§3.1 (Evaluation Data)"}],"minor_comments":[{"comment":"Table 2 lists '8 Attention Heads' as an ablation, but §3.2 states the full model already uses eight attention heads; presumably the ablation varies head count from the base. Similarly '0 Decoder MLP Ratio' is unclear notation for what is presumably a variant MLP ratio. Please clarify the ablation axes.","section":"Table 2"},{"comment":"Numerical inconsistencies between §4 text and Table 1 for the ResVAE comparison: text gives 87.85/38.39/95.02 for the proposed method where Table 1 gives 88.06/38.49/95.10. Please reconcile.","section":"§4, Comparison with ResVAE Baseline"},{"comment":"Frame-level F1 is reported as '-' in Table 1; either compute it or state why it is not meaningful at this operating point.","section":"Table 1"},{"comment":"The over-precise hyperparameter values (dropout p=0.136, MLP ratio 2.097, lr 4.24e-4) are clearly artifacts of the Bayesian optimization; once the selection protocol is revised per the major comment, rounding or justifying these would improve readability.","section":"§3.2"},{"comment":"Fig. 3: the color legend (yellow/green/red) is described only in the caption; consider in-figure labeling. State in the caption whether the four cases were selected randomly or as representative/best cases.","section":"Fig. 3"},{"comment":"Only one baseline (ResVAE, from the authors' own prior work) is compared. Classical feature-embedding anomaly detection methods (e.g., PatchCore, PaDiM, or EfficientAD) are natural, strong, training-free-or-light baselines on frozen DINO features and would contextualize how much the decoder/bottleneck machinery adds over k-NN or Mahalanobis scoring on the frozen encoder. This is not blocking but would substantially strengthen the empirical claim.","section":"§4"},{"comment":"§3.1 first describes D_test as '198 anomalous volumes' and later '198 anomalous slices' (one per examination); please make the volume/exam/slice counts consistent throughout. Also state how many of the 102 healthy slices come from UMD vs. the 13 held-out low-field volumes.","section":"§3.1"},{"comment":"Latency (0.0247 s/slice, RTX 3080): report whether this includes preprocessing (resizing to 224x224, percentile scaling) and upsampling of the anomaly map, since scanner-side deployment would include these.","section":"§4"}],"recommendation":"major_revision","confidential_remarks":"The manuscript substantially overlaps with the authors' own prior arXiv preprint (ref. [8], arXiv:2602.06179), which addresses the same problem (unsupervised pelvic anomaly detection for real-time MRI) on apparently the same data. The incremental contribution here is the Dinomaly/DINOv3 architecture plus the head-to-head comparison. The authors should be asked to delineate the delta explicitly. The test-set hyperparameter selection is disclosed rather than hidden, which speaks in the authors' favor, but the headline numbers in the abstract are exactly the ones affected by it."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"Punchline: this is a careful application paper, not a new detector. They take Dinomaly (frozen DINOv3 ViT-L, noisy MLP bottleneck, linear-attention decoder), train only on healthy multi-source T2w pelvis (clinical + 0.55T research + radiologist-checked synthetic), crop to uterus, and ship scanner-side anomaly maps at ~40 slices/s with a ResVAE head-to-head on multi-site UMD slices.\n\nWhat is actually new is the packaging and the evaluation setting: healthy-only normative learning for an under-labeled pelvic/interventional niche, true vendor/site shift on UMD, extent-based frame score calibrated only on train healthy slices, and an honest qualitative failure mode (small/low-contrast lesions, bowel FPs). The complementary ResVAE comparison is useful—VAE is high-recall/diffuse, Dinomaly is high-specificity/semantic—and the ablations show the frozen encoder carries most of the load (~4-point swings). Throughput and the leakage-free frame operating point are real.\n\nThe soft spot that matters is evaluation hygiene, not the architecture story. §3.2 says Bayesian HPO over 40 configs directly maximized pixel AUROC on D_test; τ_px is also Dice-maxed on D_test. No gradients on test is not the same as unbiased reporting. On ~198 anomalous / 102 healthy pre-picked slices, that can inflate the 88.06% headline by a few points. They are transparent that binary pixel numbers are an upper bound, and the unbiased frame numbers are weaker (AUROC 68%, sensitivity 11.6% at 95% specificity)—so the “meets real-time clinical deployment” claim overreaches relative to detection power. Coarse 28×28 tokens are the stated limit; generalization past myomas/Nabothian cysts is unshown. Citations and math are standard and fine; no code/data release for the private train set.\n\nWho it is for: people building unsupervised MRI decision support or interventional pelvis pipelines. Worth a serious referee if they fix held-out HPO, CIs, and stop leading with test-tuned pixel metrics. I would engage the work, not treat the 88% as settled.","headline":"Solid domain adaptation of Dinomaly/DINOv3 for pelvic MRI UAD with real clinical framing, but the headline 88% AUROC was selected on the test set and frame-level recall is only ~12%.","tokens_in":10743,"tokens_out":580,"would_cite":false,"duration_ms":23738,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"A frozen vision transformer learns normal pelvic MRI from healthy scans alone and flags anomalies in real time without labels.","keywords":["Unsupervised Anomaly Detection","Pelvic MRI","Vision Transformers","Foundation Models","Real-time Imaging","DINOv3","Uterine Myoma"],"falsifier":"Retrain and retune only on a held-out healthy validation split, then measure pixel AUROC, frame sensitivity/specificity, and latency on a fresh multi-site pelvic MRI cohort that includes small low-contrast lesions and non-myoma pathologies never seen during tuning; a large drop below the reported 88% AUROC or collapse of useful frame flags would falsify the central performance claim.","tokens_in":10384,"feed_emoji":"🧲","tokens_out":943,"duration_ms":24341,"temperature":0.7,"pith_summary":"Pelvic MRI is hard to read in real time: motion, deformation, and rare heterogeneous disease make supervised labels impractical. This paper argues that an unsupervised reconstruction setup can learn what healthy pelvic anatomy looks like from routine cases only, then mark deviations as spatial anomaly maps at the scanner. The method freezes a large pretrained vision transformer, forces a noisy bottleneck so the decoder cannot simply copy the input, and scores anomalies by cosine distance between encoder and decoder tokens. On a curated uterine-myoma test set it reports 88% pixel-level AUROC, about 95% frame-level specificity, and 40.5 slices per second. If those numbers hold under real deployment, radiologists could get immediate localized flags and adapt protocols during the exam rather than after the fact.","feed_headline":"Pelvic MRI anomalies flagged in real time without labels","feed_subtitle":"Frozen vision transformer hits 88% pixel AUROC and 95% specificity at 40 slices per second from healthy scans only.","key_machinery":"Dinomaly-style reconstruction at representation level: a frozen DINOv3 encoder supplies stable normative features; a noisy MLP bottleneck blocks identity mapping; a Linear Attention decoder reconstructs grouped low- and high-level tokens; anomaly maps are per-token cosine distance between encoder and decoder features, reduced to frame scores by healthy-calibrated percentile thresholds.","core_discovery":"An adapted Dinomaly architecture—frozen DINOv3 ViT-L/16 encoder, dropout-noisy MLP bottleneck, and Linear Attention decoder—learns normative pelvic T2-weighted MRI from healthy volumes only and localizes anomalies via per-token cosine distance, reaching 88.06% pixel-level AUROC and 95.45% frame-level specificity at 40.5 slices/s on out-of-distribution uterine myoma data, which the authors present as meeting real-time clinical feedback needs.","pith_inferences":["If healthy-calibrated high-specificity thresholds stay trusted in the reading room, the tool is more likely to be used as a quiet second reader than as a high-recall screening net.","Cross-vendor OOD success on myomas does not yet imply the same maps will light up endometriosis, adenomyosis, or instrument artifacts without further healthy diversity.","Pairing this detector with a high-sensitivity generative branch could raise small-lesion recall without abandoning the frozen normative reference.","Real-time slice flagging at ~40 Hz is fast enough that the bottleneck for clinical impact shifts from compute to how radiologists act on the maps during acquisition."],"forward_implications":["Scanner-side spatial anomaly maps could support adaptive protocol changes during pelvic MRI without waiting for offline review.","Unsupervised healthy-only training removes the need to annotate rare heterogeneous pelvic adverse events for detection.","Reconstruction-based and generative detectors show complementary strengths, motivating fused high-specificity plus high-sensitivity systems.","Coarse 28×28 token grids remain the main barrier to small-lesion recall, so finer tokens or multi-scale scoring would be the next design lever.","The same frozen-encoder recipe is positioned for interventional pelvic MRI where labeled adverse events are scarce."],"fun_headline_variants":["Unsupervised pelvic MRI anomalies flagged in real time","Frozen DINOv3 detects pelvic anomalies at 40 slices/s","Label-free pelvic MRI anomaly maps at scanner speed","Dinomaly localizes uterine anomalies from healthy scans only","Real-time pelvic anomaly scores without any labels"],"cache_read_input_tokens":128,"weakest_assumption_plain":"That choosing hyperparameters by directly maximizing pixel AUROC on the same test set still gives an honest picture of how well the method will work on new patients and scanners.","fun_headline_variants_meta":{"raw":{"variants":["Unsupervised pelvic MRI anomalies flagged in real time","Frozen DINOv3 detects pelvic anomalies at 40 slices/s","Label-free pelvic MRI anomaly maps at scanner speed","Dinomaly localizes uterine anomalies from healthy scans only","Real-time pelvic anomaly scores without any labels"]},"model":"grok-4.5","effort":"low","cost_usd":0.00475,"raw_usage":{"total_tokens":1374,"prompt_tokens":819,"num_sources_used":0,"completion_tokens":65,"cost_in_usd_ticks":47504000,"prompt_tokens_details":{"text_tokens":819,"audio_tokens":0,"image_tokens":0,"cached_tokens":128},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":490,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":819,"tokens_out":65,"duration_ms":9191,"temperature":1.0,"reasoning_tokens":490,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-31T07:24:35.114749+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Retrain and retune only on a held-out healthy validation split, then measure pixel AUROC, frame sensitivity/specificity, and latency on a fresh multi-site pelvic MRI cohort that includes small low-contrast lesions and non-myoma pathologies never seen during tuning; a large drop below the reported 88% AUROC or collapse of useful frame flags would falsify the central performance claim.","supporting_citations":[],"review_version":1}