{"id":"ea655fcf-2841-4a7c-862a-f9d61e2b9317","arxiv_id":"2602.02124","paper_version":2,"verdict":"REJECT","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"A class-specific Mahalanobis-distance OOD detector on top of a DINOv2/LoRA segmentation model flags known and novel lesions in mouse liver WSIs, with 0.16% FNR and 0.35% FPR when the threshold is chosen on the test set.","lead":"This paper builds an AI system that scans mouse-liver microscope images, marks known types of tissue damage, and flags rare damage it was never trained on. It reports very few missed lesions, but its headline numbers are tuned on the test set, so real-world error rates are likely higher.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline FNR/FPR are selected by tuning p on the test set, so the reported 0.16%/0.35% are not an unbiased estimate of deployment performance.","rationale":"The reader's weakest assumption identifies exactly the most load-bearing concern: the operating point p is selected by minimizing BER on the test set, so the reported FNR/FPR are fitted to the evaluation data rather than predicted. This directly undermines the abstract's central numeric claim and the cross-method ranking in Table 1. I considered other potential issues—such as the macro-averaged FNR definition, the limited OOD classes (only apoptosis and artifacts), and the partial annotation strategy—but none is as directly tied to the headline claim as the test-set threshold selection. The paper's method is otherwise technically sound and the adaptive-threshold contribution is interesting, but the evaluation protocol is a fundamental flaw that justifies rejection or at least major revision. Therefore the reader's verdict should remain REJECT (UNCHANGED).","tokens_in":16331,"tokens_out":5112,"duration_ms":45181,"concrete_test":"Split the test set (or a subset of held-out studies) into two disjoint parts: a calibration set used solely to select p (e.g., by minimizing BER as done in the paper), and a true evaluation set. Report FNR/FPR on the evaluation set at the selected p. Repeat across several random splits or study-based folds. If the held-out FNR is substantially above 0.16% (or FPR above 0.35%), the reported numbers are biased by test-set fitting. Alternatively, fix p=0.99 (a reasonable deployment value chosen without looking at test data) and report the corresponding FNR/FPR; if the results degrade markedly, the central claim is not robust.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central quantitative claim (abstract: 'only 0.16% ... and 0.35% ...') is not a prediction of performance at a fixed operating point. In Results, the authors state: 'we select p from the evaluated range that minimizes the balanced error rate (BER), defined as the mean of FNR and FPR, on the test set.' For Adaptive-Maha+, p=0.996 is chosen to minimize BER on the test set, and the reported FNR=0.16±0.04 and FPR=0.35±0.03 are the values at that test-set-optimal p. Thus the headline numbers are the minimum of a BER curve over p∈[0.95,1.00), not the performance of a threshold chosen without seeing test labels. This is load-bearing because (i) the abstract presents these as achieved rates, implying a safety-first system with near-zero missed lesions; (ii) every method in Table 1 is evaluated at its own test-set-optimal p, so Table 1's ranking is confounded by selection bias; (iii) the standard errors across 5 models reflect only sampling of training randomness, not the variability of the p-selection procedure. Additionally, the abstract describes the selection as 'optimizing the false positive rate subject to a predefined constraint on the false negative rate', which does not match the actual BER-minimization on the test set. The method itself (DINOv2/LoRA segmentation + class-wise Mahalanobis OOD) is technically plausible and the adaptive-threshold idea is reasonable, but the evaluation protocol does not support the headline numeric claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a pixelwise anomaly-detection framework for H&E-stained mouse liver whole-slide images intended for preclinical toxicity screening. It combines a frozen DINOv2 ViT adapted with LoRA and a linear segmentation head, then computes class-wise Mahalanobis scores and adaptive per-class thresholds to separate known (in-distribution) anomalies from novel (out-of-distribution) anomalies. The authors contribute a manually annotated dataset from 742 WSIs across 44 studies, hold out apoptosis and artifact as OOD classes, and report FNR = 0.16% and FPR = 0.35% at p = 0.996 for Adaptive-Maha+. They also present a toxicology-study demonstration showing dose-dependent cytoplasmic vacuolation.","tokens_in":16694,"tokens_out":8755,"duration_ms":83988,"significance":"If the reported numbers held, this would be a practically important step: a single pixelwise map that segments nine lesion types and flags unseen morphologies, with near-zero missed pathological tissue, would be valuable for pathology workflow. The paper has strengths: a substantial annotated histopathology dataset, a sensible extension of Mahalanobis OOD detection to per-class thresholds, and a comprehensive comparison to seven post-hoc OOD methods under two thresholding strategies. However, the central quantitative claims are presently not supported because the operating point p is chosen on the test set and because the FNR metric is a per-class average under an 'any anomaly' definition. The value of the contribution is conditional on a corrected evaluation protocol.","major_comments":[{"comment":"The reported FNR=0.16% and FPR=0.35% are not estimates at a pre-specified operating point. The text states: for each method and threshold selection strategy, the authors select p from the evaluated range that minimizes the balanced error rate (BER), defined as the mean of FNR and FPR, on the test set. Thus the headline numbers are the minimum of a BER curve over p in [0.95,1.00) using test labels. Table 1 reports each method at its own test-set-optimal p, so the absolute rates and the cross-method ranking are optimized on the test set; the 5-seed standard errors do not include the variability of p selection. This directly undermines the abstract's claim that the system achieves these rates. Please re-select p on the training/validation set only, or report full BER/FNR/FPR curves with a fixed selection rule, and restate the headline numbers. The abstract's description of 'optimizing the f","section":"Results, 'Performance on the test set'"},{"comment":"The FNR definition counts any anomaly-as-anomaly prediction as a true positive and averages per-class FN rates: FNR = (1/(K+1)) * sum_i FN_i/(TP_i+FN_i). Therefore the abstract's 'only 0.16% of pathological tissue classified as healthy' is misleading: it is a mean over classes under a lenient 'anything but healthy' criterion, not the global fraction of pathological pixels, and cross-type confusions (e.g., necrosis predicted as inflammation) are treated as correct. The paper does disclose this, but the headline wording does not, and the safety-first 'never overlooking a lesion' claim relies on the lenient definition. Please report strict per-class FNR where a correct prediction requires the exact class, and the overall pixel-level healthy-vs-pathology rate, and adjust the abstract accordingly.","section":"Results, 'Unknown anomaly detection'; Abstract"},{"comment":"The general claim that rare pathologies are detected 'without training data' is supported by only two held-out classes, apoptosis and artifact, both from the same dataset and scanner. While apoptosis is a sensible near-OOD stress test, two classes do not license the broad conclusion about unseen morphologies. Please either restrict the claim to these two types or validate on additional held-out classes or studies. This is a load-bearing qualification for the abstract's 'rare pathologies' claim.","section":"Data splits; 'Detection of OOD anomalies on the whole-slide image'"}],"minor_comments":[{"comment":"The phrase 'Optimizing the false positive rate subject to a predefined constraint on the false negative rate' does not match the BER-minimization procedure described in the Results section; revise for consistency.","section":"Abstract"},{"comment":"The heading 'Averaging across spacial shifts' contains a typo: 'spacial' should be 'spatial'.","section":"Appendix A2"},{"comment":"The sentence 'Values marked with indicate improvement' appears to have a missing symbol; please fix the rendering.","section":"Table 1 and Table 2"},{"comment":"The confusion matrices in Fig. 10 are computed on the joint training and validation set, not the test set. This should be stated explicitly when the figure is referenced in the main text.","section":"Figure 10 / Appendix A3"},{"comment":"The t-SNE visualization is useful but is discussed only qualitatively. Consider adding an objective separability measure to support the claim that the OOD class occupies a near-OOD region.","section":"Figure 11 / Appendix A4"}],"recommendation":"major_revision","confidential_remarks":"The test-set p-selection issue is serious and needs a corrected evaluation before publication. If the authors can rerun with validation-based selection and the adjusted numbers remain in the same range, the paper could be a useful contribution. My recommendation is major_revision rather than outright rejection because the issue is fixable by re-running the evaluation with a valid threshold-selection protocol; however, if the corrected evaluation yields substantially worse FNR/FPR, rejection would be appropriate. The abstract's framing of the FNR metric should also be tightened."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should not quote the 0.16% FNR / 0.35% FPR as if they were deployment numbers. The authors select p on the test set to minimize balanced error rate, so the headline figures are the best achievable on that test set, not the performance of a threshold chosen without seeing test labels. That makes Table 1's ranking partly a selection artifact.\n\nWhat is genuinely useful: the pixelwise-annotated mouse-liver WSI dataset covering healthy tissue, nine lesion classes, and two held-out OOD classes; the unified segmentation-plus-OOD evaluation; and the class-specific threshold idea, which visibly reduces known anomalies being flagged as OOD. The validation on a real preclinical study with dose-dependent vacuolation is a nice practical sanity check. The method itself — DINOv2/LoRA segmentation with Mahalanobis OOD and ℓ2-normalized features — is technically plausible and the components are well chosen.\n\nThe soft spots are real but not fatal to the underlying idea. The test-set threshold selection is the load-bearing one: every method in Table 1 is reported at its own test-set-optimal p, so the cross-method comparison is confounded. The abstract phrasing ('optimizing FPR subject to a constraint on FNR') also does not match the actual BER minimization. Second, the FNR definition treats any anomaly-as-anomaly as a true positive, so 'pathological tissue classified as healthy' only means classified as the healthy class exactly; cross-type errors are not penalized. That is a deliberate safety-first choice, but it makes the headline number more lenient than it sounds. Third, OOD detection is validated on only two held-out classes (apoptosis and artifacts), which are very different in difficulty; behavior on other novel lesions is open. Finally, code and data are promised only after peer review, so reproducibility is currently zero.\n\nThis paper deserves a serious referee — the dataset alone is a contribution, and the adaptive-threshold idea is worth testing with a proper validation-based selection. But the reported numbers should not be taken at face value, and the authors should be required to fix the evaluation protocol before it is citable as a safety claim.\n\nWho this is for: researchers working on OOD detection in medical imaging, and computational pathology groups interested in preclinical toxicology screening. They will get value from the dataset and the analysis, but should re-analyze the results with a validation-based operating point.","headline":"The headline error rates are optimized on the test set, so don't quote them as deployment estimates; the dataset and adaptive-threshold idea are worth a look anyway.","tokens_in":17219,"tokens_out":2255,"would_cite":false,"duration_ms":20222,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A single pixelwise AI system can segment nine known liver lesion types and, without any training on them, flag rare pathologies as out-of-distribution, achieving a 0.16% false negative rate on pathological tissue.","keywords":["histopathology","anomaly detection","out-of-distribution detection","Mahalanobis distance","whole-slide images","semantic segmentation","preclinical toxicity screening","class-specific thresholds"],"falsifier":"Rerun the pipeline with the threshold parameter fixed using only the training and validation sets, then evaluate on an independent set of whole-slide images; if the false negative rate for pathological tissue exceeds the reported 0.16%, the safety claim fails. Alternatively, hold out a third lesion type (e.g., a subtle degeneration) with no training data and count how many of its pixels are flagged as out-of-distribution rather than healthy; if most are called healthy, the 'rare pathologies without training data' claim does not generalise beyond apoptosis and artifacts.","tokens_in":16180,"feed_emoji":"🔬","tokens_out":8235,"duration_ms":68203,"temperature":0.7,"pith_summary":"The paper claims that a single pixelwise AI system can do two jobs at once in preclinical drug-safety screening: it can segment a mouse-liver whole-slide image into healthy tissue and nine common lesion types, and it can also flag tissue appearances that fit none of those classes — including rare pathologies it never saw during training. The authors' central methodological move is class-specific anomaly thresholds: instead of drawing one global line that separates 'known' from 'unknown' tissue, they draw a separate line for each predicted class, because the natural variability of healthy liver and each lesion type is very different. With that adaptation they report that only 0.16% of pathological tissue is called healthy and 0.35% of healthy tissue is called pathological, while held-out single-cell apoptosis and staining artifacts are detected as unknown. If these numbers hold in prospective use, the system could relieve the pathologist bottleneck in toxicity studies, quantify dose-dependent effects at pixel level, and catch subtle lesions early enough to inform go/no-go decisions.","feed_headline":"AI screening misses just 0.16% of liver lesions in mouse slides","feed_subtitle":"Same map flags rare pathologies never seen in training, a step toward safer drug screening.","key_machinery":"The load-bearing component is class-aware Mahalanobis distance: after fitting a shared covariance and class-specific means to the ℓ2-normalized pixel features of every known tissue class, the anomaly score of a pixel is the negative Mahalanobis distance from its features to the mean of the class that the segmentation head predicted. Instead of a single global threshold, each class receives its own threshold set to the (1−p) quantile of that class's score distribution, which is what lets the system confirm rare in-distribution lesions as known while still flagging genuinely novel tissue as out-of-distribution. The remainder of the pipeline — a frozen pre-trained ViT with low-rank-adapted atte","core_discovery":"The central claim is that a Mahalanobis-distance anomaly score, computed per pixel from the features of a frozen pre-trained Vision Transformer adapted with low-rank updates, can serve simultaneously as a semantic segmentation of known tissue states and as an out-of-distribution detector for unknown ones. The authors' main departure from prior post-hoc OOD methods is to assign each predicted class its own acceptance threshold, defined as the (1−p) quantile of that class's score distribution, rather than thresholding one pooled score distribution. This matters because per-class score distributions in histology are strongly heterogeneous: a global threshold that keeps healthy tissue in-distrib","pith_inferences":["A natural next experiment the authors do not run: hold out a visually subtle third lesion class (rather than apoptosis and staining artifacts) to see whether the OOD recall remains near total; near-OOD lesions that resemble healthy cytoplasm are likely to be the hardest case.","The class-specific thresholding idea may transfer to other medical-imaging segmentation tasks where score distributions differ strongly by tissue compartment, such as cardiac MRI or dermatopathology, wherever a single global OOD threshold is known to be miscalibrated.","Because the reported operating point is selected on the test set, real deployment would need a pre-registered threshold protocol; without one, the safety margin a pathologist can rely on remains unknown.","If the dataset is made public, the same low-rank-adapted backbone could be benchmarked against newer histopathology foundation models, and the bottleneck would likely shift from architecture to annotation coverage."],"forward_implications":["A pathologist could receive one fused WSI map showing both the type of each known lesion and the locations that deserve extra scrutiny as potentially novel pathology.","In efficacy studies, non-target organs such as liver could be screened automatically for early toxicity signals, producing quantitative dose-response curves (e.g., vacuolated area per dose group) that mirror a pathologist's read.","Because the false negative rate is averaged per anomaly class, the metric itself enforces the safety-first requirement that no lesion type — however rare — may be silently missed.","The adaptive threshold selection can be applied on top of any post-hoc OOD score, so the principle is not tied to the Mahalanobis distance alone.","The framework can run as a secondary safety layer in exploratory toxicology studies, highlighting regions of interest and reducing the risk of missing subtle tissue alterations."],"fun_headline_variants":["AI liver screening: 0.16% of lesions missed, rare ones flagged","Class-aware AI catches novel liver toxicity with 0.16% miss rate","New AI flags unseen liver anomalies, misses only 0.16% of lesions","Per-class thresholds help AI detect known and novel liver damage","AI model spots novel liver dangers, misses only 0.16% of known ones"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The headline safety numbers are obtained by choosing the operating point p on the test set to minimize balanced error, so they are best-case values for this test set rather than unbiased estimates of a fixed, pre-specified operating point.","fun_headline_variants_meta":{"raw":{"variants":["AI liver screening: 0.16% of lesions missed, rare ones flagged","Class-aware AI catches novel liver toxicity with 0.16% miss rate","New AI flags unseen liver anomalies, misses only 0.16% of lesions","Per-class thresholds help AI detect known and novel liver damage","AI model spots novel liver dangers, misses only 0.16% of known ones"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000988,"raw_usage":{"total_tokens":4055,"prompt_tokens":799,"completion_tokens":3256,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":543,"completion_tokens_details":{"reasoning_tokens":3155}},"tokens_in":543,"tokens_out":3256,"duration_ms":20016,"temperature":1.0,"reasoning_tokens":3155,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T05:27:20.666083+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Rerun the pipeline with the threshold parameter fixed using only the training and validation sets, then evaluate on an independent set of whole-slide images; if the false negative rate for pathological tissue exceeds the reported 0.16%, the safety claim fails. Alternatively, hold out a third lesion type (e.g., a subtle degeneration) with no training data and count how many of its pixels are flagged as out-of-distribution rather than healthy; if most are called healthy, the 'rare pathologies without training data' claim does not generalise beyond apoptosis and artifacts.","supporting_citations":[],"review_version":1}