{"id":"9096c651-d1c6-46e6-80cc-e1382fe30d66","arxiv_id":"2501.06887","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"A proposed entropy-weighted gradient explainability method for CLIP-based skin lesion classification is described and compared visually to prior methods.","lead":"This paper proposes MedGrad E-CLIP, an explainability method that adds a weighted entropy term to gradient-based CLIP explanations for skin lesion images. The authors fine-tune CLIP on two dermoscopy datasets and compare resulting heatmaps to Grad-CAM and Grad E-CLIP, but provide only visual comparisons.","discovery_kind":"incremental","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'superior visual explanations' claim is unsupported: the entropy weight is never specified and explanation quality is evaluated only by eye on two images, with the paper itself deferring quantitative validation to future work.","rationale":"I agree with the Reader's weakest-assumption analysis and think it is even more decisive when combined with the missing reproducibility details. The strongest claimed contribution is a better explanation method, but the paper contains no experiment that measures explanation quality; the only support is subjective reading of two heatmaps. The manuscript itself acknowledges that quantitative validation is future work, which is an in-scope limitation that undercuts the conclusion's 'we showed.' An underspecified w_e would also prevent anyone from testing the method. A quantitative benchmark against clinical structure annotations is the natural check: if entropy weighting neither helps nor hurts, or if the method is not runnable, the central claim should not be accepted. None of this impugns the classification part of the paper; the concern is specifically the unsupported explainability claim.","tokens_in":8785,"tokens_out":4255,"duration_ms":42400,"concrete_test":"Run a quantitative explanation benchmark on PH² and Derm7pt images with clinician-labeled dermoscopic structures (e.g., pigment network, streaks, blue-whitish veil). Compute insertion/deletion AUC and saliency IoU or pointing-game accuracy for MedGrad E-CLIP, Grad E-CLIP, and Grad-CAM, with paired bootstrap confidence intervals over the full test set. In the same release, provide the exact local-entropy formula and disk radius used for w_e in Eq. (3), or the code. If MedGrad E-CLIP does not significantly outperform both baselines on the primary metric, the claim of 'superior visual explanations' is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's strongest claim, stated in the Conclusion, is that MedGrad E-CLIP 'provides superior visual explanations' and captures fine-grained details aligned with textual descriptions. That claim requires (1) a reproducible definition of the method and (2) evidence that it beats Grad E-CLIP and Grad-CAM on explanation quality. Neither requirement is met. In §3.4/§4, Eq. (3) defines H_i = ReLU(Σ_c w_c v_i w_e), but w_e is never given a formula; Section 4 only says local entropy is computed 'within the region covered by the disk around each pixel,' leaving radius, histogram binning, and normalization unspecified. The evaluation (Figures 5, 6) is qualitative inspection of two cases, with no quantitative metric connecting heatmaps to clinical ground truth. Section 6 explicitly defers 'quantitative assessments, such as insertion and deletion analysis, and statistical significance tests' to future work and concedes occasional 'over-explanation.' Thus the central 'superior' claim is unsupported. The underlying assumption—that higher local pixel-intensity entropy marks diagnostically significant structures—is itself untested; entropy can be high in hair, noise, or compression artifacts, and no dermoscopic annotation is used to rule this out.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MedGrad E-CLIP, a modification of the gradient-based explainability method Grad E-CLIP for CLIP-based skin lesion classification. The method replaces Grad E-CLIP's spatial importance with a locally computed entropy weight intended to highlight subtle, diagnostically relevant regions. The authors fine-tune CLIP on PH² and Derm7pt images paired with textual descriptions, report classification accuracy before and after fine-tuning, and qualitatively compare heatmaps produced by Grad-CAM, Grad E-CLIP, and MedGrad E-CLIP on two example cases (atypical nevus and melanoma). The paper concludes that MedGrad E-CLIP provides superior visual explanations and better alignment with textual descriptions.","tokens_in":9038,"tokens_out":4487,"duration_ms":47899,"significance":"If fully specified and rigorously validated, an entropy-weighted gradient explanation method for CLIP could be a useful contribution to interpretable medical image analysis, particularly for clinician-facing tools that need to connect visual evidence to diagnostic criteria. The paper addresses a real gap: gradient-based CLIP explanations are known to miss subtle features in medical images, and the idea of using local entropy to redistribute attention is plausible. The authors also make a concrete attempt to apply CLIP to two public dermoscopic datasets and compare explanation methods on the same model. However, the contribution as presented is not yet assessable: the method is incompletely specified, and the central claim of superiority is supported only by visual inspection of two images, with the paper itself deferring quantitative validation to future work. The manuscript in its current form does not meet the evidentiary standard needed to support the claimed level of trust and transparency.","major_comments":[{"comment":"The entropy weight w_e is never defined. Equation (3) defines H_i = ReLU(Σ_c w_c v_i w_e), but the manuscript does not specify how w_e is computed from local entropy, the disk radius used in the neighborhood, the number of gray-level bins, or the normalization procedure. Section 4 only says local entropy is computed 'within the region covered by the disk around each pixel,' which is insufficient for reproduction. Without a complete definition, the reader cannot determine what MedGrad E-CLIP actually computes, and the displayed heatmaps cannot be attributed to a reproducible algorithm.","section":"§3.4, Eq. (3)"},{"comment":"The central claim that MedGrad E-CLIP 'provides superior visual explanations' rests entirely on the visual inspection of two example images, one atypical nevus and one melanoma. No quantitative explanation metrics (such as insertion/deletion scores, pointing game, IoU against dermoscopic annotations, or a clinician user study) are reported, and Section 6 explicitly defers such assessments to future work. The comparison with Grad-CAM and Grad E-CLIP is therefore anecdotal; it does not provide evidence for the strong comparative conclusion stated in the Conclusion.","section":"§5.2, Figures 5 and 6"},{"comment":"The method rests on an unvalidated assumption that higher local pixel-intensity entropy marks diagnostically significant regions. This assumption can fail in dermoscopic images: hair, scale, compression artifacts, and lesion borders often have high local entropy without being clinically meaningful. The paper provides no test of this assumption against dermoscopic structure annotations or any external ground truth, so the heatmaps may highlight irrelevant texture or noise rather than the diagnostic features the method claims to capture.","section":"§3.4"},{"comment":"The classification results are reported as aggregate accuracy, F1, and loss on a single train/test split, with no comparison to standard skin-lesion classifiers, no per-class breakdown, no confidence intervals, and no explanation for the pre-training accuracy of 2.06% on a 17-class task. While classification performance is not the paper's main focus, the claim that the pipeline 'not only classifies... but also adds explainability' is not supported by a systematic evaluation, and the unusually low pre-fine-tuning accuracy suggests a possible label or prompting issue that should be clarified.","section":"§5.1, Table 2"}],"minor_comments":[{"comment":"The section heading contains a typo ('Prepration' should be 'Preparation'), and there are several grammatical errors such as 'These includes' and 'images are resized to 224x224 pixels to meet the input requirements of the image encoder' — the latter is acceptable but the sentence flow should be revised.","section":"§3.2"},{"comment":"The figure captions do not state which text prompt or class label was used to generate each row of heatmaps. Since the paper claims alignment with textual descriptions, each panel should specify the exact text input, and the captions should be self-contained.","section":"§5.2, Figures 5 and 6"},{"comment":"Equation (4) introduces fimg and ftext without defining them in terms of the earlier notation fI and fT. Use consistent notation for image and text features throughout.","section":"§3.3, Eq. (4)"},{"comment":"Several references are incomplete or inconsistent: reference [12] lists 'Seven-Point Checklist' as an author, and reference [15] cites a paper titled 'Deep-Lift' that is not the original DeepLIFT paper. Please correct the bibliography.","section":"References"},{"comment":"The paper states that the dataset comprises 17 distinct skin lesion types, but the description of PH² and Derm7pt does not explain how 17 classes are constructed from these two datasets or how overlapping cases are handled. Clarify the class definitions and the source of the text descriptions used for each class.","section":"§3.1 and §4"},{"comment":"The paragraph thanking referees is unusual in a submitted manuscript and should be removed; it also has no place in the final version before the acknowledgment section.","section":"§6"}],"recommendation":"reject","confidential_remarks":"The manuscript appears to be an early-stage draft. The core method is under-specified, the central claim of superior explainability is not quantitatively validated, and the paper's own future-work section concedes the absence of the required evidence. In my view, these issues are not local presentation problems but fundamental gaps in the current contribution, so I cannot recommend major revision as a path forward without a substantially new evaluation and a complete algorithmic specification."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take: the paper adds an entropy weighting to Grad E-CLIP for skin lesion explainability and claims 'superior visual explanations.' That claim has no quantitative support, and the method isn't fully specified. It's a plausible idea, but the evidence is a couple of heatmaps.\n\nWhat's genuinely useful: the authors identify a real limitation of Grad E-CLIP on medical images—tendency to focus on dominant regions while missing subtle dermoscopic features—and propose local pixel-intensity entropy as a reweighting. Training CLIP on PH2/Derm7pt with dermoscopic criteria text is a sensible pipeline, and the fine-tuned classifier shows reasonable accuracy (80%). The paper is honest about its limits: it acknowledges over-explanation cases and explicitly defers insertion/deletion analysis, statistical tests, and clinical trials to future work.\n\nThe soft spots are substantial. First, Eq. (3) defines the heatmap H_i with an entropy weight w_e that is never given a formula. Section 4 says only that local entropy is computed 'within the region covered by the disk around each pixel'—no radius, no binning, no normalization. I couldn't reproduce this from the paper. Second, the main evaluation is visual inspection of two examples (atypical nevus and melanoma). No insertion/deletion, no pointing game, no user study, no ground-truth annotations. That is not enough to support 'superior.' Third, the core assumption—that higher entropy marks diagnostically significant tissue—is untested; entropy can spike on hair or compression artifacts. The authors don't rule that out.\n\nThe citation practice is fine; they build on the original Grad E-CLIP and relevant medical CLIP literature. The circularity concern raised by the reader is not real; entropy weights come from image content, not from fitting a label.\n\nOverall: this is a workshop-level idea in its current form. The central claim is load-bearing and unsupported. If the authors specify the entropy computation, release code, and run a quantitative evaluation, it's worth a second look. As is, I would not send it to full peer review.","headline":"Plausible entropy-weighting extension to Grad E-CLIP, but the 'superior' claim rests on two eyeballed heatmaps and an underspecified formula.","tokens_in":9511,"tokens_out":3777,"would_cite":false,"duration_ms":35559,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MedGrad E-CLIP produces visual explanations for CLIP-based skin lesion classification that capture fine-grained diagnostic details that Grad E-CLIP and Grad-CAM miss.","keywords":["skin lesion classification","CLIP","explainability","entropy weighting","gradient-based explanations","dermoscopy","vision-language models","medical image analysis"],"falsifier":"A quantitative study in which MedGrad E-CLIP, Grad E-CLIP, and Grad-CAM heatmaps are scored against expert-annotated dermoscopic structures such as pigment network, streaks, or regression areas on PH2 or Derm7pt, using insertion/deletion or pointing-game metrics; if entropy-weighted heatmaps do not outperform the baselines on these metrics, the claimed superiority would be refuted.","tokens_in":8597,"feed_emoji":"🩺","tokens_out":7092,"duration_ms":61015,"temperature":0.7,"pith_summary":"The paper proposes MedGrad E-CLIP, an explainability method that replaces the spatial importance weights in gradient-based E-CLIP with weights derived from local pixel-intensity entropy, aiming to make CLIP-based skin lesion classification more transparent for clinicians. The authors train a CLIP model on dermoscopic images paired with textual diagnostic criteria from the PH2 and Derm7pt datasets, then use entropy-weighted gradients to highlight image regions aligned with specific descriptions. They claim the resulting heatmaps capture subtle, diagnostically significant features—such as light brown areas and symmetry in one axis—that Grad E-CLIP and Grad-CAM overlook. If correct, the method would give dermatologists a way to see which visual evidence drove an AI diagnosis, supporting trust and verification in AI-assisted skin cancer screening.","feed_headline":"Entropy-weighted gradients sharpen CLIP skin-lesion explanations","feed_subtitle":"MedGrad E-CLIP weights gradients by local pixel complexity to capture features Grad-CAM and Grad E-CLIP miss.","key_machinery":"The key mechanism is the entropy-weighted aggregation in Eq. (3): $H_i = \\operatorname{ReLU}\\left(\\sum_c w_c v_i w_e\\right)$, where $w_c$ is channel importance, $v_i$ is the pixel value at spatial location $i$, and $w_e$ is a weight derived from the local entropy of pixel intensities in a disk around each pixel. This replaces Grad E-CLIP's spatial importance term, so the explanation is driven by regions with high local complexity and information density rather than by the most gradient-activated regions. The paper also relies on the CLIP contrastive pipeline, which scores an image-text pair by cosine similarity between image and text embeddings, trained on PH2 and Derm7pt descriptions to align lesion images with diagnostic criteria.","core_discovery":"The central claim is that MedGrad E-CLIP provides superior visual explanations for CLIP-based skin lesion diagnosis by emphasizing fine-grained details. The paper argues that its weighted entropy mechanism, computed as local intensity entropy within a disk around each pixel and applied multiplicatively to gradient contributions, shifts attention toward regions with high pixel-level complexity and information density rather than merely the most activated areas. In qualitative comparisons on Atypical Nevus and Melanoma images, the method highlights features such as a slight light brown area outside the lesion and symmetry in one axis that the original Grad E-CLIP misses, while also aligning with textual descriptions like 'fully symmetric' and 'light brown color.' The same trained CLIP model improves test accuracy from 2.06% before training to 80.08% after training on the custom skin-lesion dataset, which the authors take as evidence that the model has learned meaningful image-text relationships that the explanations then expose.","pith_inferences":["A quantitative evaluation using insertion/deletion metrics or a pointing game against expert-annotated dermoscopic structures would test whether entropy weighting truly tracks diagnostic relevance; the paper itself lists this as future work.","Because entropy weighting responds to pixel-level variation, it could also amplify noise or texture artifacts in low-quality dermoscopic images, a risk the paper acknowledges as over-explanation in some cases.","The same entropy-weighting design could transfer to other medical imaging domains where text descriptions are paired with images, such as radiology reports, not just skin lesions.","The sizable accuracy jump from 2.06% to 80.08% likely reflects the model learning dataset-specific text-image associations rather than general dermatologic knowledge; calibration on external datasets would clarify this."],"forward_implications":["Clinicians could see which part of a lesion drove a CLIP classification, with heatmaps tied to specific textual diagnostic criteria such as symmetry, pigmentation, and streaks.","The method addresses known weaknesses of Grad E-CLIP, which struggles with physical and comparative attributes, by capturing features like full asymmetry and symmetry in one axis.","Training CLIP on skin-lesion image-text pairs lifts test accuracy from 2.06% to 80.08%, so the explainability gains come alongside a functional classifier.","The pipeline combines classification and explanation in one pass, so no separate post-hoc model is needed to produce visual justifications."],"supporting_citations":[{"why":"Introduces the gradient-based E-CLIP method that MedGrad E-CLIP extends and the primary baseline for comparison.","marker":"[36]"},{"why":"Introduces CLIP, the contrastive image-text model the pipeline trains and explains.","marker":"[23]"},{"why":"Provides the PH2 dermoscopic image database with structure criteria used as image-text training data.","marker":"[19]"},{"why":"Provides the Derm7pt dataset with 7-point checklist annotations used for image-text pairs.","marker":"[12]"},{"why":"Defines Grad-CAM, the alternative explainability baseline the paper compares against.","marker":"[26]"}],"fun_headline_variants":["Entropy-boosted CLIP reveals hidden skin lesion details","MedGrad E-CLIP: entropy gradients for clear skin AI","Weighted entropy makes CLIP see more in skin lesions","CLIP gets clearer with entropy-weighted gradients for skin lesions"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper assumes that local pixel-intensity entropy is a good proxy for diagnostically significant features, so that weighting gradients by entropy highlights clinically meaningful regions; it does not validate this assumption against clinical ground truth or quantitative explanation metrics.","fun_headline_variants_meta":{"raw":{"variants":["Entropy-boosted CLIP reveals hidden skin lesion details","MedGrad E-CLIP: entropy gradients for clear skin AI","Weighted entropy makes CLIP see more in skin lesions","CLIP gets clearer with entropy-weighted gradients for skin lesions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000836,"raw_usage":{"total_tokens":3641,"prompt_tokens":931,"completion_tokens":2710,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":547,"completion_tokens_details":{"reasoning_tokens":2648}},"tokens_in":547,"tokens_out":2710,"duration_ms":20099,"temperature":1.0,"reasoning_tokens":2648,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:48:46.232436+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A quantitative study in which MedGrad E-CLIP, Grad E-CLIP, and Grad-CAM heatmaps are scored against expert-annotated dermoscopic structures such as pigment network, streaks, or regression areas on PH2 or Derm7pt, using insertion/deletion or pointing-game metrics; if entropy-weighted heatmaps do not outperform the baselines on these metrics, the claimed superiority would be refuted.","supporting_citations":[{"cited_title":"Gradient-based visual explanation for transformer-based clip","cited_arxiv_id":null,"evidence_quote":"Introduces the gradient-based E-CLIP method that MedGrad E-CLIP extends and the primary baseline for comparison."},{"cited_title":"Ph 2-a dermoscopic image database for research and benchmarking","cited_arxiv_id":null,"evidence_quote":"Provides the PH2 dermoscopic image database with structure criteria used as image-text training data."},{"cited_title":"Skin lesion classification using multitask multimodal neural nets","cited_arxiv_id":null,"evidence_quote":"Provides the Derm7pt dataset with 7-point checklist annotations used for image-text pairs."}],"review_version":1}