{"id":"1737907b-5ec2-49c3-938a-80ed035ad8f9","arxiv_id":"2505.10764","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"An explainability-based benchmark showing that surgical vision-language models often make correct predictions without attending to the relevant instruments or tissue.","lead":"SurgXBench adds explainability analysis to benchmarking vision-language models in robotic surgery, measuring whether model attention lands on surgical instruments and their interactions. On two laparoscopic datasets, surgical VLMs often classify correctly while focusing on weak contextual cues rather than the instruments, a pattern that standard accuracy metrics miss.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Cross-model AA/AC comparison confounds XAI method with model family; the headline 'weak cues' claim may be an artifact of Grad-CAM vs. attention rollout.","rationale":"The Pith Reader identified post-hoc XAI faithfulness as the weakest assumption. I agree, but locate the problem more specifically: the paper compares attention alignment across model families using different explanation methods. Surgical VLMs are ResNet-based and get Grad-CAM; general VLMs are ViT-based and get attention rollout. Differences in AA/AC can therefore arise from the spatial characteristics of the explanation method rather than from the models' actual decision evidence. This is load-bearing because Figure 3 is the primary quantitative support for the abstract's claim. The paper's own supplementary caveat (A.3.1) only addresses faithfulness for LVLMs, not this between-group confound. The occlusion analysis is a step in the right direction but lacks a random-occlusion control and is not applied across all compared models. I do not think this requires rejection: the benchmark, metrics, and qualitative observations are still useful contributions. However, the headline conclusion should remain conditional until a method-controlled comparison (or a method-agnostic causal probe) is provided. This does not change the reader's CONDITIONAL verdict.","tokens_in":13654,"tokens_out":6781,"duration_ms":70136,"concrete_test":"Recompute the instrument-classification alignment analysis with architecture/XAI method controlled. Add a ResNet-backbone general VLM (e.g., CLIP-RN50) and compute its AA/AC with the same Grad-CAM protocol used for SurgVLP/HecVL/PeskaVLP; provide attention-rollout AA/AC for the ViT models as a second method. If CLIP-RN50 with Grad-CAM yields AA/AC close to the surgical VLMs, the reported gap is an explanation-method artifact. Additionally, re-run the Sec. 5.4 occlusion experiment with a same-area random-occlusion control on the same frames; if random occlusion reduces IVT coverage or similarity scores as much as attention-based occlusion, the causal 'weak cue' interpretation does not follow.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that surgical VLMs 'rely on weak contextual cues rather than clinically relevant visual evidence' rests on the attention-alignment metrics AA/AC (Sec. 3.4.1, Fig. 3) and RAS (Sec. 3.4.2, Fig. 4). But the cross-model comparison is not method-controlled. Surgical VLMs (SurgVLP, HecVL, PeskaVLP) are ResNet-based and are explained with Grad-CAM (Eq. 6), while the general contrastive VLMs CLIP-ViT-B/32 and BLIP are ViT-based and are explained with gradient-weighted attention rollout (Eq. 7); LLaVA uses yet another Grad-CAM variant plus CLEANN. Grad-CAM and attention rollout have different spatial resolutions and biases, so lower AA/AC scores for the surgical group could reflect the explanation method (coarse ResNet Grad-CAM vs. patch-level ViT rollout) rather than weaker use of clinically relevant evidence. The paper's own caveat in Supp. A.3.1 concedes faithfulness limitations for LVLMs but does not address this between-group method confound. The occlusion study (Sec. 5.4) is more causal but lacks a random-occlusion control and is applied only to surgical VLMs; it cannot by itself establish the cross-model conclusion.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"SurgXBench introduces an explainability-oriented benchmark for zero-shot evaluation of six vision-language models (three surgical, three general) on surgical instrument classification (Cholec80BBox) and triplet recognition (CholecT45). The paper proposes attention-alignment metrics (AA, AC, RAS), uses RAFT with camera-motion correction to localize instrument-tissue interactions, applies Grad-CAM, gradient-weighted attention rollout, and CLEANN for visualization, and reports attention-based occlusion and causal-graph analyses. The central claim is that surgical VLMs, despite domain-specific training, often rely on weak contextual cues rather than clinically relevant visual evidence, and that general contrastive VLMs can localize instruments better even when their classifications are incorrect.","tokens_in":13925,"tokens_out":3268,"duration_ms":33898,"significance":"If the central finding is valid, the paper addresses an important gap: standard accuracy/F1 metrics can overstate the reliability of surgical VLMs, and the proposed benchmark would provide a useful template for trustworthiness evaluation in medical vision-language systems. The paper makes concrete contributions: a public benchmark with explainability metrics, automatic extraction of interaction regions via RAFT with camera-motion correction, qualitative causal graphs, and an occlusion-based counterfactual analysis. The authors are appropriately careful in some places, notably in acknowledging the softness of Grad-CAM interpretations for LVLMs in Supplementary A.3.1. However, the headline cross-model conclusion depends on contested post-hoc heatmap faithfulness and on a comparison that confounds explanation method with model family; the currently reported evidence is not yet strong enough to support the broad claim as stated.","major_comments":[{"comment":"The cross-model AA/AC comparison is confounded by explanation method. Surgical VLMs (ResNet backbones) are explained with Grad-CAM (Eq. 6), while CLIP and BLIP (ViT backbones) use gradient-weighted attention rollout (Eq. 7), and LLaVA uses a different Grad-CAM variant plus CLEANN. These methods have different spatial resolutions and biases, so lower AA/AC for the surgical group could reflect the explanation method rather than weaker use of clinically relevant evidence. The paper should provide a method-controlled comparison, for example by applying the same explanation method to all models where feasible, or by reporting results separately for each explanation method and showing that conclusions are invariant.","section":"§3.2, §3.4.1, Fig. 3"},{"comment":"The alignment scores are reported as point estimates or box plots without confidence intervals or statistical tests. Claims such as \"general contrastive learning VLMs outperform surgical VLMs in attention alignment\" rest on absolute differences that may be within sampling variability, especially given the small number of videos (5 for Cholec80BBox; 2 for the RAS analysis). The authors should report error bars, per-frame variability, and significance tests, and should include a sensitivity analysis over the hand-chosen thresholds τ, γ, δ, and top-k (Supp C.1) to show that the main conclusions do not depend on these choices.","section":"§5.1, §5.2, Fig. 3, Fig. 4"},{"comment":"The occlusion analysis lacks a random-occlusion control. Reporting that occluding high-attention regions removes IVT matches is only informative if compared with occluding randomly selected regions at the same area fraction; without such a control, the observed drop could be a generic effect of image occlusion. In addition, the analysis is applied to surgical VLMs but not to the general VLMs, so it cannot support the cross-model comparison that is central to the paper's conclusion. The authors should add a random-occlusion baseline and, if possible, apply the same counterfactual procedure to all model families.","section":"§5.4, Fig. 6, Supp E.2"},{"comment":"The paper explicitly concedes that Grad-CAM for LVLMs \"does not fully capture the underlying reasoning\" and serves only as a \"soft interpretability measure.\" Given that the central claim includes LLaVA and LVLMs, the causal graph analysis (two illustrative examples in Fig. 5) is too thin to carry quantitative weight. The authors should either derive quantitative metrics from CLEANN across a larger set of examples, or explicitly restrict the central claim to contrastive models and present the LVLM results as preliminary qualitative observations.","section":"Supp A.3.1, §5.3, Fig. 5"}],"minor_comments":[{"comment":"There is a typo: \"robustnees\" should be \"robustness.\"","section":"Introduction"},{"comment":"The phrase \"the only existing VLM benchmark in surgery is by by Rauet al.\" contains a duplicated \"by\".","section":"Related Work"},{"comment":"In the definition of Nk, \"appearnces\" should be \"appearances.\"","section":"Eq. (2)-(3) context"},{"comment":"The heading \"Triplet Recongition Task\" contains a misspelling; it should be \"Triplet Recognition Task.\"","section":"§4.3.2"},{"comment":"The phrase \"Addition results include pericison, recall and box plot\" should be \"Additional results include precision, recall and box plots.\"","section":"Supp E.1"},{"comment":"The paper uses \"attention\" to refer to heatmaps from Grad-CAM, which in ResNet backbones are gradient-based saliency maps rather than native attention; using a distinct term such as \"saliency\" or \"attribution\" would avoid confusion with true attention mechanisms.","section":"§3.2, Fig. 1"}],"recommendation":"major_revision","confidential_remarks":"The benchmark and metrics are a useful contribution, and the paper is generally within scope for the journal. However, the main cross-model conclusion rests on the faithfulness of post-hoc XAI and on a comparison that confounds explanation method with model family; this needs to be addressed with method-controlled experiments, uncertainty quantification, and a random-occlusion control. The authors' own caveat in Supp A.3.1 weakens the LVLM portion of the claims and should be integrated into the main text rather than left as a supplementary aside."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nSurgXBench is worth knowing about: it is the first surgical VLM benchmark that goes beyond accuracy and tries to measure whether model attention lands on clinically relevant regions. It introduces three alignment metrics (AA, AC, RAS), uses RAFT with camera-motion correction to approximate instrument-tissue interaction regions, and includes an occlusion test. That is genuinely useful scaffolding for the surgical AI community, and the authors deserve credit for being explicit about the limits of post-hoc XAI in Supplementary A.3.1.\n\nThe problem is that the central conclusion—surgical VLMs rely on weak contextual cues—is built on a comparison that is not method-controlled. The surgical models (ResNet backbones) are explained with Grad-CAM; the general contrastive models (ViT backbones) are explained with gradient-weighted attention rollout; LLaVA gets yet another Grad-CAM variant plus CLEANN. Grad-CAM and attention rollout have different spatial resolutions and different biases. So the lower AA/AC scores for the surgical group could be an artifact of the explanation method rather than evidence about what the model actually uses. That is a load-bearing confound, and the paper does not address it. The occlusion study is more causal, but it lacks a random-occlusion control and is only run on the surgical models, so it cannot carry the cross-model claim either.\n\nThere are smaller issues: the alignment metrics are reported as point estimates without error bars or significance tests; the thresholds (τ, γ, δ, top-k) are hand-chosen and no sensitivity analysis is given; no code or data release is mentioned. None of these are fatal, but they make the quantitative claims thinner than the text implies.\n\nStill, the benchmark itself is a real contribution. It frames the evaluation problem in a way that others can build on, and the proposed metrics will probably be reused even if the specific numbers change. The paper should go to peer review—the central claim needs to be re-analyzed with a controlled explanation-method setup, but the framework is worth refereeing.\n\nFor you: useful if you work on surgical VLM or on interpretability evaluation; otherwise a quick skim will do.","headline":"A useful explainability benchmark for surgical VLMs, but its headline 'weak cues' claim is confounded by different XAI methods across model groups.","tokens_in":14476,"tokens_out":2098,"would_cite":true,"duration_ms":19738,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Surgical vision-language models, even after surgical-video pretraining, often classify instruments correctly by attending to weak contextual cues rather than to the clinically relevant region.","keywords":["surgical vision-language models","explainable AI","instrument classification","triplet recognition","attention alignment","zero-shot evaluation","laparoscopic video","Grad-CAM"],"falsifier":"Run a controlled occlusion test on a held-out surgical video set: for each correct prediction, occlude the top 20% attention region and record how often the prediction survives. If correct predictions survive high-attention occlusion about as often as they survive occlusion of random patches, attention does not track decision evidence; if occlusion of top-attention regions reliably collapses accuracy while random occlusion does not, attention is diagnostic and the paper's central claim is false.","tokens_in":13457,"feed_emoji":"🩺","tokens_out":9278,"duration_ms":75734,"temperature":0.7,"pith_summary":"This paper claims that current vision-language models for surgery can produce correct instrument and action classifications while attending to the wrong parts of the image. It builds a benchmark that adds explainability analysis to standard accuracy metrics, scoring how much a model's attention overlaps with clinically relevant regions such as instrument boxes and instrument-tissue interaction areas. Across six general and surgical VLMs on two laparoscopic datasets, the paper finds that surgical models can beat general models on classification accuracy yet show worse attention alignment, meaning their correct answers are often not grounded in the visual evidence a clinician would use. If the claim holds, standard benchmarks overstate the reliability of surgical VLMs, and future training needs frame-level visual and reasoning supervision, not just video-level language alignment.","feed_headline":"Surgical AI often picks the right label by looking at the wrong thing","feed_subtitle":"Where a model looks, not just what it answers, shows why surgical accuracy overstates readiness.","key_machinery":"The load-bearing mechanism is post-hoc explanation heatmaps paired with region-overlap scores. Grad-CAM produces heatmaps for ResNet-based contrastive VLMs; gradient-weighted attention rollout does the same for CLIP's vision transformer; CLEANN builds causal graphs from token-level conditional-independence tests for the large vision-language model. The alignment metrics AA and AC threshold the heatmap at the top 20% of attention and measure overlap with ground-truth instrument boxes, either for the predicted class or for any annotated object. For triplet recognition, RAFT optical flow plus a pan/tilt/zoom/roll camera-motion correction generates binary masks of instrument-tissue interaction, and the RAS scores measure attention overlap with those masks plus instrument regions. These scores are what let the study distinguish correct-by-luck from correct-for-the-right-reason predictions.","core_discovery":"The central claim is that for surgical instrument and triplet recognition, a model can be right by coincidence: correct classification correlates poorly with attention on the clinically informative regions. On Cholec80BBox, contrastive general VLMs such as BLIP achieve higher alignment scores (bag TP_AC 66.9%, grasper 32.1%) than surgical VLMs, while surgical models such as PeskaVLP lead in F1 for several instruments (hook 0.7410, bipolar 0.7899, irrigator 0.7962) but have low attention overlap (hook TP_AA 6.6%, TP_AC 7.6%). Occluding the regions the surgical model attends to does not reliably remove the correct triplet from the top-5 predictions: 28.2% of previously IVT-matched frames lost their match, and the average similarity score dropped by 9.87. Causal-graph analysis of a large vision-language model shows text tokens such as surgical, image, and instrument contributing more directly than visual tokens, with visual tokens scattered outside clinically relevant regions. The paper interprets these patterns as evidence that surgical VLMs lean on weak contextual cues and that both visual grounding and explicit reasoning supervision are needed.","pith_inferences":["If heatmap faithfulness holds, a cheap safety check for surgical VLMs is to require a minimum alignment score before a prediction is shown to a surgeon; accuracy alone should not gate alerts.","The paper's logic suggests a testable division of labor: pair a general VLM that localizes well with a surgical VLM that classifies well, potentially outperforming either alone.","The occlusion protocol could itself become a benchmark measure: report the fraction of correct predictions that survive occlusion of their high-attention regions, since a high survival rate means the explained evidence is not what drives the model.","The RAFT-based motion-mask idea transfers to non-surgical video tasks where action location matters, such as assembly or manipulation, using the same camera-motion correction to isolate true interaction regions."],"forward_implications":["Standard classification metrics alone cannot certify a surgical VLM: a model can score well on F1 while its attention misses the instrument, so reliability reporting should include attention-alignment scores.","Surgical VLMs trained on clip-level video-language supervision need frame-level visual grounding, because weak supervision is the likely cause of reliance on contextual cues.","LVLMs used for surgery should be trained or prompted to reason from visual tokens rather than dominant text priors, since causal-graph evidence shows text tokens leading the prediction.","The explainability pipeline extends to other surgical tasks whenever clinically relevant cues can be defined, so phase recognition and tissue identification could use the same alignment scoring."],"supporting_citations":[{"why":"Supplies SurgVLP, the surgical contrastive VLM whose attention behavior is central to the benchmark.","marker":"[34]"},{"why":"Provides the only prior surgical VLM benchmark and the baseline that motivates explainability-based evaluation.","marker":"[22]"},{"why":"Supplies CLIP, a general contrastive VLM baseline and the image-text similarity evaluation method.","marker":"[21]"},{"why":"Supplies BLIP, a general contrastive VLM with the strongest attention alignment in the results.","marker":"[14]"},{"why":"Grad-CAM is the core heatmap method behind the AA, AC, and RAS alignment metrics.","marker":"[25]"},{"why":"Provides the weakly-supervised localization evaluation perspective that inspired the thresholded attention metrics.","marker":"[6]"},{"why":"CLEANN supplies the causal-graph analysis used to attribute LVLM predictions to visual versus text tokens.","marker":"[23]"},{"why":"RAFT optical flow builds the instrument-tissue interaction masks used for triplet recognition.","marker":"[27]"},{"why":"Supplies the CholecT45 dataset and the IVT/IV/IT triplet matching protocol used for action evaluation.","marker":"[19]"},{"why":"Supplies the Cholec80BBox instrument bounding boxes that serve as clinical ground truth for AA and AC.","marker":"[13]"}],"fun_headline_variants":["Surgical AI often right for wrong reasons, benchmark shows","Vision-language models in surgery: accurate but poorly grounded","Surgical VLMs: correct labels, misguided attention","Explainable benchmark exposes surgical AI's weak visual cues","Right answer, wrong focus: surgical AI's hidden flaw"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The conclusion collapses if post-hoc explanation heatmaps (Grad-CAM, attention rollout, CLEANN) do not faithfully reveal the visual evidence the model actually used; the paper itself notes that for large vision-language models Grad-CAM is only a soft interpretability measure.","fun_headline_variants_meta":{"raw":{"variants":["Surgical AI often right for wrong reasons, benchmark shows","Vision-language models in surgery: accurate but poorly grounded","Surgical VLMs: correct labels, misguided attention","Explainable benchmark exposes surgical AI's weak visual cues","Right answer, wrong focus: surgical AI's hidden flaw"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000209,"raw_usage":{"total_tokens":1445,"prompt_tokens":1020,"completion_tokens":425,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":636,"completion_tokens_details":{"reasoning_tokens":347}},"tokens_in":636,"tokens_out":425,"duration_ms":4666,"temperature":1.0,"reasoning_tokens":347,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:03:53.344423+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a controlled occlusion test on a held-out surgical video set: for each correct prediction, occlude the top 20% attention region and record how often the prediction survives. If correct predictions survive high-attention occlusion about as often as they survive occlusion of random patches, attention does not track decision evidence; if occlusion of top-attention regions reliably collapses accuracy while random occlusion does not, attention is diagnostic and the paper's central claim is false.","supporting_citations":[{"cited_title":"Systematic Evaluation of Large Vision-Language Models for Surgical Artificial Intelligence","cited_arxiv_id":"2504.02799","evidence_quote":"Provides the only prior surgical VLM benchmark and the baseline that motivates explainability-based evaluation."},{"cited_title":"Learning transferable visual models from natural language supervision","cited_arxiv_id":null,"evidence_quote":"Supplies CLIP, a general contrastive VLM baseline and the image-text similarity evaluation method."},{"cited_title":"Blip: Bootstrapping language-image pre- training for unified vision-language understanding and generation","cited_arxiv_id":null,"evidence_quote":"Supplies BLIP, a general contrastive VLM with the strongest attention alignment in the results."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the weakly-supervised localization evaluation perspective that inspired the thresholded attention metrics."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"CLEANN supplies the causal-graph analysis used to attribute LVLM predictions to visual versus text tokens."},{"cited_title":"Teed and J","cited_arxiv_id":null,"evidence_quote":"RAFT optical flow builds the instrument-tissue interaction masks used for triplet recognition."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the CholecT45 dataset and the IVT/IV/IT triplet matching protocol used for action evaluation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Cholec80BBox instrument bounding boxes that serve as clinical ground truth for AA and AC."}],"review_version":1}