{"id":"8d5736eb-698a-4966-be93-4c47f8c6652d","arxiv_id":"2508.17667","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A CLIP-based framework with cross-scale visual fusion and entropy-gain pseudo-OOD generation improves medical out-of-distribution detection.","lead":"This paper proposes a vision-language framework that fuses multi-scale image patches to detect unknown diseases during medical diagnosis. It reports lower false-positive rates and higher AUROC than prior OOD detection methods on three medical imaging benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Entropy gain in Eq. (4) is computed on cross-scale fused embeddings, so the 'boundary patch' selection may reflect global context rather than local tissue; the ablation cannot rule out this confound.","rationale":"The reader's conditional verdict is well-founded: the entropy-gain heuristic is the least externally validated component, and the paper's ablations do not directly test whether selected patches correspond to lesion boundaries. My stress-test refines this into a more specific, technically checkable concern. Because the entropy values used in ΔH are computed after cross-scale fusion (Eq. 1), the parent term is contaminated by global image context, so a low-entropy parent may simply be a patch whose fused embedding is pulled toward a confident global prediction. Thus, even if the domain assumption about normal-tissue entropy holds for raw patches, the implemented criterion may not measure what the paper claims. The ablation against background selection cannot distinguish this context confound from genuine boundary selection. This concern does not overturn the empirical OOD results — the gains in Table 1 are large and internally consistent with the ablations — but it does mean the central mechanism is less secure than the narrative suggests. The proposed concrete test (pre-fusion selection comparison and, optionally, boundary IoU on ISIC-4) would settle whether the mechanism is real. Since the reader already assigned CONDITIONAL and this concern is a refinement rather than a demonstrated failure, I keep the verdict unchanged.","tokens_in":8541,"tokens_out":20649,"duration_ms":210672,"concrete_test":"Recompute the entropy gain for pseudo-OOD selection using the raw patch embeddings u1_i, u2_j (before the cross-scale fusion in Eq. 1) instead of \\uhat u1_i, \\uhat u2_j, while keeping all other components identical. Measure the Jaccard overlap of the top-K selected patches between the two variants. If overlap is low (<50%), the selection is dominated by fusion context, not local tissue; then retrain with the pre-fusion variant and compare Table 1. If the OOD metrics are unchanged, the 'lesion-boundary' mechanism is not the actual cause of the gain.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Section 2.2, the probabilities p1_i and p2_j that feed the entropy-gain criterion in Section 2.3 (ΔH(p2_j) = H(p2_j) − H(p1_j*)) are computed from the fused embeddings \\uhat u1_i and \\uhat u2_j of Eq. (1), not from the raw patch embeddings u1_i, u2_j. The parent term \\uhat u1_j* already contains the global embedding u0 added with cosine weight, so a parent patch that is actually normal tissue can have artificially low entropy whenever its local embedding aligns with the confidently-classified global image context. The '−H(p1_j*)' term is therefore not a clean signal that the parent contains a lesion; it may simply indicate that global context dominates the parent's prediction. Similarly, the child \\uhat u2_j is augmented with its parent, so the entropy gap H(p2_j) − H(p1_j*) mixes local tissue statistics with the different amounts of cross-scale context injected at the two scales. The ablation in Table 2 only compares ΔH selection against background-region selection (row 3 vs row 2); it does not control for this context confound. Consequently, the central mechanism claimed for hard pseudo-OOD generation — that selected patches are lesion-boundary regions — is not actually validated by the reported experiments. If the selection is instead driven by fusion context, the method's OOD gain could come from a different, unstated property of the selected embeddings, and the domain assumption about entropy gaps at lesion borders is not load-bearing as stated.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents HVL, a hierarchical vision-language framework for out-of-distribution (OOD) detection in medical images. It partitions each image into multi-scale patches, encodes them with a frozen CLIP image encoder, and fuses the patch embeddings in a coarse-to-fine manner by adding cosine-weighted context from lower-scale embeddings. Learnable biases are added to the text embeddings for scale-wise alignment, and patches whose softmax entropy is above the average are discarded as disease-irrelevant. For hard pseudo-OOD generation, high-scale patches with the largest entropy gain relative to their parent mid-scale patch are selected, then augmented with lower-scale embeddings, and trained with an outlier-exposure entropy-maximization loss. Classification is based on the averaged scale-wise prediction distributions, and the maximum softmax probability (MSP) is used for OOD detection at inference. Experiments on Skin40, NCT-CRC, and ISIC-4 report lower FPR95 and higher AUROC than all compared methods, with ablations and sensitivity analyses. The source code is made publicly available.","tokens_in":8878,"tokens_out":6437,"duration_ms":64380,"significance":"If the reported results are reproducible, HVL is a meaningful contribution: it targets the practically important setting of near-OOD detection in medical imaging; it provides a public implementation; it reports standard deviations and multiple ablations; and it consistently outperforms both uni-modal and VLM-based baselines on the three benchmarks. The cross-scale fusion and pseudo-OOD augmentation ideas are clearly described and the empirical story is coherent. The main caveats are that the mechanism attributed to the entropy-gain selection—identifying lesion-boundary patches—is not directly verified, and the concurrent GLAli method is cited but not compared, so the state-of-the-art claim should be treated with caution until those points are addressed.","major_comments":[{"comment":"The entropy-gain criterion ΔH(p2_j) = H(p2_j) - H(p1_j*) is computed from the fused embeddings \\hat u2_j and \\hat u1_j* defined in Eq. (1), not from the raw patch embeddings. Because \\hat u1_j* already contains the global embedding u0 added with cosine weighting, a parent patch of normal tissue can have artificially low entropy whenever global context aligns with a confidently classified class; the '-H(p1_j*)' term may then reflect the amount of injected context rather than the presence of lesion tissue. Similarly, \\hat u2_j contains its parent context, so the entropy gap mixes local tissue statistics with scale-dependent context injection. The ablation in Table 2 (row 3 versus row 2) only contrasts ΔH selection with background-region selection and does not control for this confound. Please add an experiment that computes the entropy-gain selection on unfused embeddings or with the fusion ablated, or that visualizes the selected patches, in order to validate the claimed boundary-region mechanism. Without such evidence, the central rationale for the hard pseudo-OOD generation strategy is not established.","section":"Section 2.3, Eq. (4) (with Eq. (1))"},{"comment":"The paper explicitly identifies GLAli [7] as concurrent work on VLM-based few-shot OOD detection with multi-scale image-text alignment, and the method description says 'Unlike GLAli, our method...', but GLAli is not included in the comparison in Table 1. Since the abstract and Section 3.2 claim superior OOD detection over existing methods, the absence of this directly related baseline makes the state-of-the-art claim incomplete. Please add the GLAli comparison under the same protocol, or state clearly why it cannot be evaluated.","section":"Section 3.1, Table 1"}],"minor_comments":[{"comment":"The threshold H^1 is defined as the sum Σ_i H(p1_i) but is described as the average entropy and used as a threshold ('higher than H^1'); this is internally inconsistent and should be corrected to (1/(n×n))Σ_i H(p1_i).","section":"Section 2.2"},{"comment":"The term 'AURCO' should be 'AUROC'.","section":"Section 3.2"},{"comment":"The row labels and the references to 'row 2 and row 4' in the text are difficult to follow; please explicitly name the configuration of each row in the caption or table body.","section":"Table 2"},{"comment":"The sentence 'such disease-relevant in general should not be used to recognize the disease category' is ungrammatical; presumably 'disease-irrelevant' patches should not be used.","section":"Section 2.2"},{"comment":"With n=2, the method encodes the original image, 4 mid-scale patches, and 16 high-scale patches, i.e., 21 forward passes per image; the statement that latency is 'nearly identical to same-backbone models' is surprising and should be substantiated with end-to-end timing measurements, not only memory usage.","section":"Section 3.2, Inference cost"},{"comment":"The caption notes that ISIC-4 results are omitted due to space constraints; please include them in the supplementary material for completeness.","section":"Figure 2"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The short version: this is a worthwhile applied paper. The cross-scale fusion of CLIP patch embeddings with cosine-weighted context (Eq. 1) is a genuine new twist, and the entropy-gain pseudo-OOD selection (Eqs. 4–5) is not something I've seen before. The reported gains are large and consistent across three medical benchmarks, especially on NCT-CRC (FPR95 37.59 vs. 50+ for the nearest baseline), with ID accuracy basically unchanged. The code is public, the baselines are reasonable, and standard deviations are reported. That is real empirical evidence and deserves credit.\n\nNow the soft spots. The stress-test note is correct and it matters. The probabilities p1_i and p2_j that feed the entropy-gain criterion are computed from the fused embeddings of Eq. (1), not from the raw patch embeddings. So the entropy gap H(p2_j) − H(p1_j*) mixes local tissue statistics with the different amounts of global context injected at the two scales. A parent mid-scale patch that is actually normal tissue can have artificially low entropy because the global image context dominates its fused embedding. The selected 'boundary' patches may therefore be selected for context reasons, not because they sit next to lesions. The ablation only compares against background-region selection; it does not control for this confound. That does not kill the method—the empirical gains stand regardless—but the paper's central mechanistic claim is unvalidated. A revision should add a control using unfused embeddings for selection, or a comparison against entropy-gain on raw patches.\n\nThe ablation table is also ambiguous. The rows are hard to map to the variants described in the text, and the Skin40 FPR95 comparison between the relevant rows seems to contradict the text's claim that replacing the proposed selection with background degrades performance. That needs a cleanup.\n\nMissing the concurrent baseline GLAli from Table 1 is a minor but real gap in the state-of-the-art claim, since the paper itself names it. The method's dependency on the heuristic that normal tissue is high-entropy and lesion borders produce a specific entropy gap is plausible but untested; the sensitivity study does not address it.\n\nOn the math and data side, I see no circularity. Metrics are computed on held-out OOD test data, and the pseudo-OOD samples come from training ID patches, which is standard self-training, not a leak. The citation pattern looks fine; GLAli is cited even though not compared.\n\nWho is this for? People working on VLM-based OOD detection in medical imaging, especially near-OOD settings. It is a niche applied contribution, not a scientific breakthrough, but it is solid and the improvements are practically meaningful.\n\nIf I were the editor, I would send this out for peer review. The empirical evidence is strong enough to deserve referee time, and the concerns are fixable in revision. Just don't take the boundary-selection story at face value until a control is run.","headline":"A competent applied OOD paper with a real new combination and strong reported gains, but the mechanism behind the pseudo-OOD selection is not actually validated and the ablation table is sloppy.","tokens_in":9384,"tokens_out":3651,"would_cite":false,"duration_ms":37708,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A multiscale vision-language model improves medical out-of-distribution detection by fusing global and local patch embeddings and training on hard boundary-region pseudo-OOD patches.","keywords":["out-of-distribution detection","vision-language models","medical image analysis","hierarchical learning","cross-scale visual fusion","pseudo-OOD generation","entropy gain","CLIP"],"falsifier":"A direct test would be to take a held-out medical dataset, run the pretrained CLIP encoder on normal tissue, lesion interior, and lesion-boundary patches, and inspect their softmax entropy distributions: if normal-tissue patches are not consistently the highest-entropy, or if boundary patches do not show a larger entropy jump than their containing mid-scale patch, then the selected hard pseudo-OOD patches in Eq. (4) are not actually boundary examples, and an ablation replacing the entropy-gain selector with random patch selection should produce comparable results.","tokens_in":8363,"feed_emoji":"🩺","tokens_out":13098,"duration_ms":109476,"temperature":0.7,"pith_summary":"The paper sets out to show that medical out-of-distribution (OOD) detection gets materially better when a vision-language model is trained hierarchically, using both global and multi-scale local patch embeddings, and when the pseudo-OOD training examples are drawn from the boundaries around lesions rather than from plain background regions. Unknown diseases are often visually close to known diseases, so the authors argue that background-based pseudo-OOD data is too easy and does not teach the model to reject near-OOD cases. Their framework, HVL, fuses lower-scale context into higher-scale patch embeddings and selects hard pseudo-OOD patches by an entropy-gain rule that identifies small patches sitting on the edge of a low-entropy lesion region. The claim, based on experiments on NCT-CRC, ISIC-4 and Skin40, is that HVL outperforms both uni-modal and VLM-based baselines in FPR95 and AUROC while keeping in-distribution classification accuracy essentially unchanged.","feed_headline":"Multiscale vision-language model catches unknown diseases better","feed_subtitle":"Cross-scale fusion plus boundary-region pseudo-OOD training beats prior methods on three public medical benchmarks.","key_machinery":"The load-bearing mechanism is a four-part training pipeline built on a frozen CLIP encoder. First, multi-scale inputs are created by upsampling the image and partitioning it into $n\\times n$ and $2n\\times 2n$ patches; the original image yields a global embedding $u^0$, while mid-scale and high-scale patches yield embeddings $\\{u^1_i\\}$ and $\\{u^2_j\\}$. Second, cross-scale visual fusion augments each higher-scale patch embedding with its containing lower-scale embedding weighted by cosine similarity (Eq. 1), so local patches inherit global context. Third, scale-wise text-side biases $b^0$ and $b^2$ (with the mid-scale bias their average) are added to the frozen text embeddings to produce per-scale class probabilities from softmax over cosine similarities (Eq. 2), and disease-irrelevant patches are filtered by an average-entropy threshold. Fourth, hard pseudo-OOD generation selects the top-K high-scale patches with highest entropy gain $\\Delta H(p^2_j) = H(p^2_j) - H(p^1_{j^*})$, fuses them with lower-scale embeddings (Eq. 4), and trains the model with the outlier-exposure loss (Eq. 5) that maximizes entropy on these pseudo-OOD examples. During inference, the three scale-wise prediction distributions are averaged and the maximum softmax probability is used as the OOD score.","core_discovery":"On its own terms, the paper's discovery is that the combination of cross-scale visual fusion and cross-scale hard pseudo-OOD generation is what makes a CLIP-based OOD detector competitive for medical near-OOD data. By upsampling and partitioning each image into global, mid-scale and high-scale patches and coupling their embeddings with cosine-weighted context, the model keeps fine detail while seeing the whole image; by selecting high-scale patches with the largest entropy gain relative to their containing mid-scale patch and fusing them with lower-scale information, the model trains against pseudo-OOD examples that sit close to real lesions. In the reported experiments, HVL reaches an average FPR95 of 56.05% and AUROC of 83.34% across the three benchmarks, and the ablations show that removing either the fusion or the entropy-gain pseudo-OOD mechanism degrades detection, while replacing the entropy-gain selector with background-region selection also hurts.","pith_inferences":["We infer that the entropy-gain heuristic is essentially a boundary-aware label-free segmentation signal: it tags the spatial transition from confidently classified lesion tissue to uncertain normal tissue, so the same idea could be adapted to non-medical near-OOD settings where foreground-background transitions carry semantic information (e.g., satellite imagery, defect inspection).","We infer that the method's sensitivity to the partition count $n$ (performance drops when $n>4$) points to an implicit optimal patch size related to typical lesion scale; a testable extension is to make $n$ or the patch grid adaptive per image, which the paper does not explore.","We infer that because all pseudo-OOD embeddings are mined from ID images alone, the framework could in principle be applied to unlabeled ID data or to a new clinical site without OOD labels, providing a data-efficient route to OOD-aware calibration before deployment.","We infer that the cosine-weighted fusion acts as a fixed spatial attention; replacing the hand-set cosine weights with learned attention or a graph over patches is a natural, untested variant that could either improve or simplify the method."],"forward_implications":["HVL's reported numbers on NCT-CRC, ISIC-4 and Skin40 imply that VLM-based OOD detection in medicine should be built on multi-scale visual inputs rather than single global or local embeddings.","The entropy-gain selection rule is shown to generalise from dermatology to histology, suggesting the boundary-patch pseudo-OOD signal transfers across medical imaging domains.","Because the method encodes original images and patches in parallel during inference, the paper reports latency nearly identical to same-backbone models and only modest memory overhead, making the approach practical for clinical deployment.","Ablations support that cross-scale fusion and hard pseudo-OOD generation contribute independently: each alone improves detection, and their combination gives the best FPR95 and AUROC."],"supporting_citations":[{"why":"Supplies the pretrained CLIP vision-language backbone that every compared method, including HVL, builds on.","marker":"[18]"},{"why":"Outlier exposure loss; the entropy-maximization objective that trains the model to reject pseudo-OOD embeddings.","marker":"[6]"},{"why":"LoCoOp baseline and representative local-patch pseudo-OOD method that HVL argues is insufficient for medical images because background regions differ too much from lesions.","marker":"[15]"},{"why":"SCT baseline, another background-region pseudo-OOD method used as comparison and as the replacement for the entropy-gain selector in ablations.","marker":"[23]"},{"why":"GalLop baseline, the strongest VLM competitor on ID accuracy, representing global-and-local prompt learning that HVL extends to cross-scale fusion.","marker":"[10]"},{"why":"Defines MSP, the maximum softmax score used as the OOD detection statistic during HVL inference.","marker":"[5]"},{"why":"The NCT-CRC pathology dataset used as one of the three OOD detection benchmarks.","marker":"[8]"},{"why":"The ISIC 2019 dermatology data underlying the ISIC-4 benchmark.","marker":"[21]"},{"why":"Defines the Skin-40 subset of SD-198 used as the skin-disease benchmark.","marker":"[12]"}],"fun_headline_variants":["Cross-scale fusion + pseudo-OOD training lifts medical OOD detection","Entropy-gain pseudo-OOD selection sharpens medical anomaly spotting","Hierarchical VLM with multi-scale cues outperforms on medical OOD","Coupling multi-scale visuals and hard pseudo-OOD beats medical baselines"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The pseudo-OOD signal rests on the assumption that, under the pretrained CLIP encoder, normal-tissue patches are more uniformly uncertain (higher softmax entropy) than disease patches, and that the small patch straddling a lesion's boundary has higher entropy than the larger mid-scale patch containing it; if normal tissue is confidently classified or lesion borders do not produce that entropy gap, the entropy-gain selector stops selecting hard boundary examples and the OOD gains are expected to shrink or vanish.","fun_headline_variants_meta":{"raw":{"variants":["Cross-scale fusion + pseudo-OOD training lifts medical OOD detection","Entropy-gain pseudo-OOD selection sharpens medical anomaly spotting","Hierarchical VLM with multi-scale cues outperforms on medical OOD","Coupling multi-scale visuals and hard pseudo-OOD beats medical baselines"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000194,"raw_usage":{"total_tokens":1318,"prompt_tokens":872,"completion_tokens":446,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":488,"completion_tokens_details":{"reasoning_tokens":369}},"tokens_in":488,"tokens_out":446,"duration_ms":5150,"temperature":1.0,"reasoning_tokens":369,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:01:35.812864+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct test would be to take a held-out medical dataset, run the pretrained CLIP encoder on normal tissue, lesion interior, and lesion-boundary patches, and inspect their softmax entropy distributions: if normal-tissue patches are not consistently the highest-entropy, or if boundary patches do not show a larger entropy jump than their containing mid-scale patch, then the selected hard pseudo-OOD patches in Eq. (4) are not actually boundary examples, and an ablation replacing the entropy-gain selector with random patch selection should produce comparable results.","supporting_citations":[{"cited_title":"In: MICCAI (2020)","cited_arxiv_id":null,"evidence_quote":"Defines the Skin-40 subset of SD-198 used as the skin-disease benchmark."},{"cited_title":"In: ICLR (2019)","cited_arxiv_id":null,"evidence_quote":"Outlier exposure loss; the entropy-maximization objective that trains the model to reject pseudo-OOD embeddings."},{"cited_title":"In: NeurIPS (2023)","cited_arxiv_id":null,"evidence_quote":"LoCoOp baseline and representative local-patch pseudo-OOD method that HVL argues is insufficient for medical images because background regions differ too much from lesions."},{"cited_title":"In: NeurIPS (2024)","cited_arxiv_id":null,"evidence_quote":"SCT baseline, another background-region pseudo-OOD method used as comparison and as the replacement for the entropy-gain selector in ablations."},{"cited_title":"In: ECCV (2024)","cited_arxiv_id":null,"evidence_quote":"GalLop baseline, the strongest VLM competitor on ID accuracy, representing global-and-local prompt learning that HVL extends to cross-scale fusion."},{"cited_title":"PLoS medicine (2019)","cited_arxiv_id":null,"evidence_quote":"The NCT-CRC pathology dataset used as one of the three OOD detection benchmarks."},{"cited_title":"Scientific data (2018)","cited_arxiv_id":null,"evidence_quote":"The ISIC 2019 dermatology data underlying the ISIC-4 benchmark."}],"review_version":2}