{"id":"bf8f41a0-6886-4456-8eaa-f46be029e962","arxiv_id":"2505.03350","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"A text-guided CLIP-style model with a frozen BERT text encoder and cross-entropy alignment classifies focal liver lesions from multi-phase CT slices with about 79 percent average accuracy, outperforming CLIP and MedCLIP on a small in-house dataset.","lead":"A small medical imaging study applies a CLIP-style vision-language model, Liver-VLM, to classify four types of liver lesions from multi-phase CT scans. Using text prompts and a cross-entropy loss, it reports accuracy around 79 percent on an in-house 489-slice dataset, beating two general-purpose vision-language baselines.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 3 comparison to CLIP/MedCLIP appears to use zero-shot baselines; without same-protocol fine-tuning the outperformance claim is unsupported.","rationale":"Reader identified fair-comparison as weakest assumption; I agree and this is the most load-bearing issue. The reported baseline numbers cannot be interpreted without knowing whether CLIP/MedCLIP were fine-tuned. The degenerate per-class zeros strongly suggest zero-shot or frozen evaluation. Since Liver-VLM is trained on the target data, the only informative comparison would be against baselines trained or fine-tuned under identical conditions. If those baselines were fine-tuned and still underperformed, the claim would be credible; as reported, it is not. The same experiment would also need an image-only control to attribute the gain to text alignment rather than to the CE training of a reasonably strong supervised classifier. No other concern, such as dataset size, class imbalance, or lack of public code, is as directly load-bearing for the headline claim. These are not evidence of any methodological misconduct; they are missing experimental specifications. The appropriate verdict remains conditional: under current evidence, accept the internal comparisons but withhold the external 'outperforms CLIP/MedCLIP' claim until a same-protocol comparison is supplied.","tokens_in":6633,"tokens_out":5003,"duration_ms":53072,"concrete_test":"Run a controlled baseline evaluation on the same 3-fold splits and with the same training budget (200 epochs, AdamW lr 0.01, batch 32): fine-tune the CLIP ResNet50 and MedCLIP encoders using the identical cross-entropy/cosine-similarity protocol as Liver-VLM Model 2, with the image backbone initialized from the respective pre-trained weights. If either fine-tuned baseline achieves average accuracy within or above the 79.27±3.06% range, the headline superiority claim fails. Include one no-text control, ResNet18 with ImageNet init trained with the same CE objective on image embeddings only, to test whether the text branch is the source of the gain.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing premise of the paper is that Liver-VLM's supervised training outperforms CLIP and MedCLIP on MPCT-FLLs, but Section 4.3 never states the baseline protocol. The reported CLIP result (32.80% accuracy, 0.40 AUC, with FNH and HEM at 0%) and MedCLIP result (30.73%, 0.51 AUC, with FNH and HCC at 0%) are exactly what one would expect from evaluating off-the-shelf, zero-shot models on an out-of-domain CT dataset, not from models that received the same 200-epoch supervised training as Liver-VLM. Because Models 1 and 2 in the same table are trained with labels, the comparison conflates 'using the proposed objective and training procedure' with 'using the CLIP/MedCLIP backbone.' As a result, the abstract's claim that Liver-VLM 'outperforms both the standard CLIP and MedCLIP models' is not supported as stated. The omitted image-only control is the second half of the same problem: a ResNet18 with ImageNet initialization and a simple CE-trained linear head, under the same splits and hyperparameters, may already reach the 79.27% accuracy, which would mean the text branch contributes nothing measurable. The paper's internal Model1-vs-Model2 and ResNet18-vs-ResNet50 comparisons are useful, but the external comparison that forms the headline is insecure.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Liver-VLM, a CLIP-inspired model for classifying focal liver lesions (FLLs) in multi-phase CT images. The model uses a ResNet image encoder and a frozen BERT text encoder with class-level prompt embeddings, trained with a cross-entropy loss over cosine similarities. Experiments on the MPCT-FLLs dataset (489 slices, four lesion types) compare two variants (train from scratch and ImageNet fine-tuning) with ResNet50 and ResNet18 backbones against CLIP and MedCLIP baselines. The authors report that Liver-VLM outperforms CLIP and MedCLIP in accuracy and AUC, and that the lightweight ResNet18 backbone performs better than ResNet50 under limited-data conditions.","tokens_in":6925,"tokens_out":3532,"duration_ms":32610,"significance":"If the claims were fully supported, the work would offer a modest contribution to low-data medical image classification by showing that aligning image features with frozen text class prototypes can provide a strong inductive bias. The paper's internal comparisons (Model 1 vs. Model 2, ResNet50 vs. ResNet18) are clearly reported and internally consistent. However, the central external comparison against CLIP and MedCLIP is currently not credible because the baseline protocol is undisclosed and the reported baseline performances are characteristic of zero-shot evaluation rather than equivalent supervised training. The absence of an image-only control further prevents attribution of the gains to the text modality. The paper therefore does not yet substantiate its main claim.","major_comments":[{"comment":"The protocols for the CLIP and MedCLIP baselines are not specified. The authors state that a CLIP model with ResNet50 was used as a baseline and that MedCLIP was evaluated on the same dataset, but they do not state whether these models were fine-tuned on the MPCT-FLLs training folds, used in a zero-shot fashion, or adapted with any other procedure. The reported results—CLIP at 32.80% accuracy with 0% on FNH and HEM, and MedCLIP at 30.73% with 0% on FNH and HCC—are exactly what one would expect from frozen, off-the-shelf models on an out-of-domain CT dataset. Because the Liver-VLM models are trained for 200 epochs with labels, the comparison conflates the training procedure with the model architecture. The abstract claim that Liver-VLM 'outperforms both the standard CLIP and MedCLIP models' is therefore not supported unless the baselines received equivalent supervised fine-tuning under the same splits and hyperparameters.","section":"§4.3, Table 3"},{"comment":"No image-only baseline is included. The proposed method aligns image embeddings with text embeddings and optimizes with cross-entropy, but the paper never trains a standard ResNet18 or ResNet50 classifier with the same training procedure, data splits, and a simple linear head (or the same projection layer without text) to measure the contribution of the text branch. Without this control, the reported improvements over CLIP/MedCLIP may be entirely due to the ResNet backbone and the supervised training scheme, and the paper's central claim that the vision-language alignment improves small-sample classification is not established.","section":"§3, §4.3 (Tables 3 and 4)"},{"comment":"The data split protocol is ambiguous. The text says 'In each fold, one group was designated as the test set, while the other two were used for training,' but Table 1 lists a third 'Val' subset. The authors should clarify whether the validation group is used for model selection/early stopping, and if so, how it is incorporated into the 3-fold cross-validation. This matters for the reproducibility and fairness of all reported comparisons.","section":"§4.1, Table 1"},{"comment":"The claim that ResNet18 'enhances classification performance' over ResNet50 is based on differences between averages from only three folds, with standard deviations that overlap (e.g., ResNet50 Model 2: 74.08±4.33; ResNet18 Model 2: 79.27±3.06). The authors should report per-fold results and perform a paired statistical test (e.g., Wilcoxon signed-rank test) or otherwise demonstrate that the observed differences are not due to fold variance.","section":"§4.3, Tables 3 and 4"}],"minor_comments":[{"comment":"There is a typo in the abstract: 'pro-pose' should be 'propose', and 'classifiers' in 'which classifiers image' should be 'classifies'. The grammar in several places ('However, its reliance...', 'a detailed description of the proposed approach .') needs editing.","section":"Abstract and §1"},{"comment":"The prompt template 'a CT scan of tumors {label}' is described, but the exact prompt strings used for each class are not listed. Since prompt wording can materially affect zero-shot or text-encoder behavior, the authors should include the final prompts for Cyst, FNH, HCC, and HEM.","section":"§3.4"},{"comment":"Table 2 is titled 'Computation Environment' but lists no hyperparameters such as learning rate schedule, weight decay, or the number of warm-up steps. The training setup described in the text (200 epochs, batch size 32, learning rate 0.01, AdamW) is helpful, but additional details would improve reproducibility.","section":"§4.2, Table 2"},{"comment":"The standard deviations for per-class accuracies are large (e.g., HEM accuracy 63.15±25.12 for ResNet50 Model 2), yet the text discusses these differences as if they were meaningful. The authors should acknowledge the high variance and avoid over-interpreting per-class changes without statistical support.","section":"§4.3"},{"comment":"Reference [13] for MedCLIP is cited as a MICCAI 2022 paper, but the MedCLIP model has multiple versions (e.g., with ViT or CNN encoders). The authors should specify which MedCLIP checkpoint and variant was used and how it was obtained.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper appears to be a workshop-length submission with promising internal ablations but a flawed external comparison. The missing baseline protocol and the absence of an image-only control are fixable with additional experiments. I would encourage the editor to require those experiments before reconsidering the manuscript."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Main thing you should know: the paper reports a CLIP-style image-text alignment model for focal liver lesion classification, and the internal comparisons are clear, but the headline claim that it beats CLIP and MedCLIP is not supported because the baseline protocol is never stated and the numbers suggest zero-shot evaluation. That's a fixable flaw, not a fatal one.\n\nWhat's actually new: applying this known recipe (frozen BERT text encoder, cosine similarity, cross-entropy loss) to multiphase CT liver lesions, with simple prompt templates and class-name expansion. The paper compares training from scratch vs fine-tuning, and ResNet50 vs ResNet18. The best result is 79.27% accuracy and 0.91 AUC with fine-tuned ResNet18. The authors are honest that this is insufficient for clinical use.\n\nThe soft spots: the biggest is Table 3. CLIP gets 32.8% accuracy with FNH and HEM at 0%, and MedCLIP 30.7% with FNH and HCC at 0%. That pattern is what you'd expect from zero-shot or badly misadapted models, not from models trained the same way as Liver-VLM. The paper never says whether the baselines were fine-tuned, frozen, or zero-shot, so the outperformance claim conflates the proposed training procedure with the backbone. Second, there is no image-only control: a plain ResNet18 trained with cross-entropy might already reach around 79%, which would mean the text branch contributes nothing measurable. The authors should add that ablation. Also, the dataset is only 489 slices, in-house, and no code is released, so reproducibility is limited.\n\nOn the positive side, the paper is small and clearly written, the internal model variants are sensibly compared, and the conclusion that a lighter backbone helps in small-data settings is plausible. The citation pattern is fine, mostly relevant prior work, and there's no circular reasoning.\n\nWho should read it: people building classifiers for small medical imaging datasets, especially with multimodal text guidance. It's a modest but honest extension, not a breakthrough.\n\nRecommendation: send it to peer review, but require the authors to specify and rerun the CLIP/MedCLIP baselines under the same supervised protocol as Liver-VLM, and to report a no-text ablation. If the text branch adds no measurable value over a plain ResNet18, the contribution shrinks to a negative result, which is still informative but needs to be framed that way.","headline":"A modest, honest CLIP-style application to liver lesion classification whose internal comparisons are useful, but whose headline outperformance claim over CLIP/MedCLIP rests on an unstated and likely zero-shot baseline protocol.","tokens_in":7422,"tokens_out":1622,"would_cite":false,"duration_ms":15100,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A text-aligned vision-language model beats standard CLIP and MedCLIP on focal liver lesion CT classification.","keywords":["focal liver lesions","vision-language model","multimodal learning","multi-phase CT","cross-entropy alignment","CLIP","medical image classification","small-sample learning"],"falsifier":"Fine-tune CLIP and MedCLIP on the same 3-fold MPCT-FLLs splits, with an ImageNet-initialized ResNet backbone, the same optimizer, the same 200 epochs, and the same 'a CT scan of tumors {label}' prompt template. If either fine-tuned baseline reaches or exceeds $79.27\\%$ average accuracy or $0.91$ AUC, the paper's central claim that Liver-VLM outperforms them is falsified.","tokens_in":6451,"feed_emoji":"🩻","tokens_out":7260,"duration_ms":66600,"temperature":0.7,"pith_summary":"The paper proposes Liver-VLM, a vision-language model that classifies focal liver lesions from three-phase CT slices by aligning image features with class-specific text embeddings. It claims this text-guided alignment, trained with cross-entropy loss on cosine similarities, beats standard CLIP and MedCLIP on the MPCT-FLLs dataset, which contains only 489 annotated slices. The best variant, with an ImageNet-pretrained ResNet18 encoder and a frozen BERT text encoder, reaches $79.27\\%$ average accuracy and $0.91$ AUC. If the claim holds, it offers a practical route to useful medical classification where large annotated datasets are unavailable.","feed_headline":"Text-aligned CT model beats CLIP on liver lesions","feed_subtitle":"On 489 multiphase CT slices, ResNet18-based Liver-VLM reaches 79.3% accuracy and 0.91 AUC","key_machinery":"The load-bearing mechanism is cross-entropy alignment between image and text embeddings. Class names are expanded to full labels and placed in the prompt template 'a CT scan of tumors {label}', the frozen BERT text encoder turns each prompt into a class embedding, the image encoder output is projected by a fully connected layer to the same dimension, and the softmax of cosine similarities between the image and all class embeddings is trained with cross-entropy. The frozen text encoder and prompt template are what carry the class prior; the only trainable text-side component is the projection layer, so the added text guidance costs no extra computation at inference time.","core_discovery":"Liver-VLM is a CLIP-inspired framework in which a trainable image encoder (ResNet50 or ResNet18) embeds each three-phase CT slice and a frozen BERT text encoder embeds the class prompt 'a CT scan of tumors {label}'; a trainable projection layer brings the two modalities into a shared space, and cross-entropy loss over pairwise cosine similarities aligns each image with its true class text. The paper's central claim is that this class-level text alignment yields more discriminative features for focal liver lesions than standard CLIP or MedCLIP under annotation-scarce conditions. On the MPCT-FLLs dataset, CLIP and MedCLIP collapse to two classes and hover near chance, while every Liver-VLM variant classifies all four lesion types; the best configuration reaches $79.27\\pm3.06\\%$ average accuracy and $0.91\\pm0.03$ AUC. The authors also claim that a lighter ResNet18 backbone performs better than ResNet50 in this data-constrained setting, and they state the resulting accuracy is still insufficient for reliable clinical application.","pith_inferences":["Editorial inference: the reported comparison may be unfair if CLIP and MedCLIP were not fine-tuned; a same-protocol fine-tuned baseline is the decisive missing experiment.","Editorial inference: changing the prompt template could shift results, so the method's sensitivity to wording should be tested directly.","Editorial inference: because the 489 slices come from a single institution, external multi-center data are needed to test whether text-aligned embeddings generalize.","Editorial inference: adding a contrastive term over image pairs to the cross-entropy alignment might further separate classes whose text descriptions differ only slightly."],"forward_implications":["Class-level text conditioning can be applied to other small-sample medical imaging tasks with few labels.","A lightweight ResNet18 backbone should be preferred over ResNet50 when only a few hundred training slices are available.","Training from scratch with text alignment already outperforms the CLIP and MedCLIP baselines, so the text signal itself, not only ImageNet pretraining, drives the improvement.","The best model's $0.91$ AUC suggests acceptable ranking for triage, while its $79.27\\%$ accuracy is below what the paper considers clinically reliable."],"supporting_citations":[{"why":"Supplies the CLIP baseline and the contrastive image-text pretraining paradigm Liver-VLM adapts.","marker":"[10]"},{"why":"Supplies the MedCLIP baseline and the medical vision-language approach whose domain gap motivates the CT-specific model.","marker":"[13]"},{"why":"Supplies the frozen BERT text encoder that generates class embeddings from prompt templates.","marker":"[15]"},{"why":"Supplies the ResNet50 and ResNet18 image encoder backbones used in all variants.","marker":"[14]"},{"why":"Supplies the MPCT-FLLs dataset, the only experimental data on which training and evaluation are based.","marker":"[16]"},{"why":"Supplies the AdamW optimizer used in the training protocol.","marker":"[17]"}],"fun_headline_variants":["Liver-VLM bests CLIP and MedCLIP on focal liver lesions","ResNet18-powered VLM tops CLIP for liver CT classification","Text-aligned liver model reaches 79% accuracy, beats CLIP","Data-scarce liver lesion VLM outperforms CLIP with light backbone","Focal liver lesion VLM: text alignment lifts accuracy over CLIP"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the CLIP and MedCLIP baselines were evaluated under the same training protocol as Liver-VLM; the paper does not state whether those baselines were fine-tuned or used off the shelf, so the comparison may not be apples to apples.","fun_headline_variants_meta":{"raw":{"variants":["Liver-VLM bests CLIP and MedCLIP on focal liver lesions","ResNet18-powered VLM tops CLIP for liver CT classification","Text-aligned liver model reaches 79% accuracy, beats CLIP","Data-scarce liver lesion VLM outperforms CLIP with light backbone","Focal liver lesion VLM: text alignment lifts accuracy over CLIP"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000285,"raw_usage":{"total_tokens":1709,"prompt_tokens":1006,"completion_tokens":703,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":622,"completion_tokens_details":{"reasoning_tokens":606}},"tokens_in":622,"tokens_out":703,"duration_ms":6987,"temperature":1.0,"reasoning_tokens":606,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T23:53:27.726953+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Fine-tune CLIP and MedCLIP on the same 3-fold MPCT-FLLs splits, with an ImageNet-initialized ResNet backbone, the same optimizer, the same 200 epochs, and the same 'a CT scan of tumors {label}' prompt template. If either fine-tuned baseline reaches or exceeds $79.27\\%$ average accuracy or $0.91$ AUC, the paper's central claim that Liver-VLM outperforms them is falsified.","supporting_citations":[{"cited_title":"In: ICML 2021, pp","cited_arxiv_id":null,"evidence_quote":"Supplies the CLIP baseline and the contrastive image-text pretraining paradigm Liver-VLM adapts."},{"cited_title":"In: MICCAI 2022, LNCS, vol","cited_arxiv_id":null,"evidence_quote":"Supplies the MedCLIP baseline and the medical vision-language approach whose domain gap motivates the CT-specific model."},{"cited_title":"In: NAACL -HLT 2019, pp","cited_arxiv_id":null,"evidence_quote":"Supplies the frozen BERT text encoder that generates class embeddings from prompt templates."},{"cited_title":"In: CVPR 2016, pp","cited_arxiv_id":null,"evidence_quote":"Supplies the ResNet50 and ResNet18 image encoder backbones used in all variants."},{"cited_title":"In: Medical Physics, vol","cited_arxiv_id":null,"evidence_quote":"Supplies the MPCT-FLLs dataset, the only experimental data on which training and evaluation are based."},{"cited_title":"In: ICLR 2019","cited_arxiv_id":null,"evidence_quote":"Supplies the AdamW optimizer used in the training protocol."}],"review_version":1}