{"id":"1da34cb6-bc35-4a62-bd50-87d9e079d9bd","arxiv_id":"2411.18101","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"ConcepPath aligns WSI patches to GPT-4-induced expert concepts plus learned concepts and uses a two-stage aggregation to outperform prior MIL methods on five histopathology classification tasks.","lead":"ConcepPath, a new method for analyzing whole slide pathology images, couples GPT-4-generated medical expert concepts with learnable concepts and a pathology vision-language model to classify cancer subtypes and HER2 status. It reports improvements over previous methods on five cancer tasks, with the largest gain in gastric cancer immunotherapy sensitivity classification.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported 'significant' gains rest on five-fold means with no significance testing and per-task tuning of the number of learnable concepts without a described nested validation; on the 26-positive EBV task this can fully explain the headline 6.23% AUC gain.","rationale":"The paper's components and ablations are reasonable, and the concept-alignment mechanism has some direct support (VLM comparison, concept maps). I therefore do not object to the method's plausibility. My concern is that the headline comparison, the part that must be true for 'significantly outperformed' to hold, is evaluated without statistical guarantees and with an ambiguous hyperparameter-selection protocol. The reader's weakest assumption about VLM alignment is a legitimate limitation and is explicitly acknowledged by the authors, but it is not the first thing that must be tested: if the evaluation is biased or underpowered, no amount of alignment quality can rescue the claim. The proposed test settles this because it directly probes whether the reported gain survives an unbiased protocol. Since the issue is addressable and the reader already conditioned the verdict, I recommend UNCHANGED rather than a new verdict.","tokens_in":16248,"tokens_out":7433,"duration_ms":73717,"concrete_test":"Use the released code to re-run STAD EBV vs. Others with strictly nested model selection: inside each of the five training folds, choose the number of learned concepts (2-12) on a validation split, then evaluate the chosen model once on the held-out test fold. Compute patient-level bootstrap 95% confidence intervals for the AUC difference against the best baseline and a paired permutation test over patients. If the 6.23% gain is not significant or disappears under nested selection, the 'significantly outperformed' claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim is 'ConcepPath significantly outperformed previous SOTA methods' (Abstract) and 'surpasses ... in both AUC and ACC across all assessed tasks' (Results). The support is mean AUC/ACC over patient-level five-fold cross-validation, with no confidence intervals, no paired tests, and no reported per-fold values. This matters most on STAD: the EBV vs. Others task has 26 positive cases, so each test fold contains about five EBV slides; the reported 6.23% AUC gain can be moved by a handful of patients. The paper's Training Details state that the number of learnable concepts is 'tuned from {2,4,6,8,10,12} for each target class,' and the Results report the best per-task value (e.g., 8 for EBV, 4 for NSCLC). If this tuning used the same test folds whose AUCs are then reported, the headline numbers are selected maxima rather than honest estimates, and the comparison against fixed-architecture baselines is biased. The 'TOP*' baseline is also reported as the higher of two implementations, adding favorable selection. This is more load-bearing than the frozen-VLM alignment caveat: even with imperfect text-image alignment, the adapters and learnable concepts could make the method work, but an invalid or underpowered evaluation would remove the evidence for the central claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ConcepPath, a weakly supervised whole-slide-image (WSI) classification framework that combines GPT-4-induced expert concepts from medical literature with learnable data-driven concepts, and aligns both to WSI patches through a frozen CLIP-based pathology vision-language model. The framework uses a two-stage concept-guided aggregation and slide-adapters before computing class-prompt similarities for prediction. The authors evaluate ConcepPath on five TCGA tasks (NSCLC subtyping, BRCA HER2 scoring, and three gastric cancer immunotherapy-related binary tasks) and report AUC/ACC improvements over seven baselines, including a 6.23% AUC gain on the EBV versus Others task. The paper also presents component ablations, comparisons of concept generation strategies, and qualitative interpretability heatmaps.","tokens_in":16425,"tokens_out":3001,"duration_ms":26245,"significance":"If the empirical claims hold, the paper makes a useful contribution by demonstrating that explicit human expert knowledge, induced via an LLM from medical literature, can be combined with learned concepts in a CLIP-based pathology VLM space to improve slide-level classification and provide interpretable similarity maps. The framework is novel in its decomposition of a WSI task into patch-level concept scoring subtasks, and the authors provide code, trained weights, and the induced concept lists, which are strengths for reproducibility. The component ablations (data-driven concepts, bag-level guidance, slide-adapters) are informative and support the design's internal logic. However, the central empirical claim of 'significantly outperformed' is not supported by appropriate statistical evidence, and the evaluation protocol has potential selection biases that must be addressed before the results can be accepted as stated.","major_comments":[{"comment":"The Abstract and Results claim that ConcepPath 'significantly outperformed' previous methods, but the manuscript reports only mean AUC/ACC over patient-level five-fold cross-validation without confidence intervals, per-fold values, or paired significance tests. On the EBV versus Others task, which has only 26 positive cases, the reported 6.23% AUC gain over the best baseline could be moved by a handful of patients. Please report per-fold results with confidence intervals and run appropriate statistical tests (e.g., DeLong's test or paired bootstrap) for the AUC comparisons, or temper the language to avoid the unsupported claim of significance.","section":"Results and Evaluation Protocol"},{"comment":"The Training Details state that the number of learned instance-level concepts was 'tuned from {2, 4, 6, 8, 10, 12} for each target class,' and Figure 3a reports the best per-task values (e.g., 4 for NSCLC, 8 for EBV). The manuscript does not specify whether this tuning was performed on the same test folds whose AUCs are then reported, nor does it describe a nested validation scheme. If the test folds were used to select the number, the reported results are selected maxima and the comparison against fixed-architecture baselines is biased. Please specify the tuning protocol clearly and report results for all candidate concept numbers for all tasks, ideally with nested cross-validation or a separate validation set.","section":"Training Details"},{"comment":"The TOP baseline is reported as the higher performance of the author's implementation and the authors' re-implementation. Selecting the higher of two implementations for a comparison baseline introduces favorable selection and is not a pre-specified protocol. Please report both implementations separately and either justify the choice of the higher one or use a single pre-registered implementation for the comparison.","section":"Results, Baseline Models"},{"comment":"The concept-guided aggregation (Eqs. 5–8) and final prediction (Eq. 13) rely on the assumption that cosine similarities between patch features Z and concept text embeddings C in the frozen CLIP-based pathology VLM space faithfully reflect the presence of morphological concepts. The Discussion acknowledges the domain shift between the encoder training data and downstream task data, but the paper provides no quantitative evidence about the reliability of this alignment for the target stains or cancer types. Please add an analysis that isolates this assumption, such as comparing concept similarity maps against pathologist region annotations, or reporting a variant that refines the alignment during training; otherwise it remains unclear whether the observed gains come from expert knowledge or from the learnable adapters and concepts.","section":"Methods, Eq. (5) and Discussion"}],"minor_comments":[{"comment":"The text contains a duplicated phrase: 'as as illustrated in Figure 2b' should read 'as illustrated in Figure 2b.'","section":"Comparison of Different Expert Concept Extraction Strategies"},{"comment":"The model name 'PatChat' appears twice in the Discussion and should be 'PathChat' to match the cited work.","section":"Discussion"},{"comment":"The caption contains a duplicated phrase: 'using different CLIP-based vision-language models as ConcepPath's basic component for aligning for aligning histopathology images' should have a single 'for aligning.'","section":"Figure 3c caption"},{"comment":"The caption of Figure 2a does not define 'TOP*'; the explanation is given only in the body text. Please add the definition to the caption for self-containment.","section":"Results, Figure 2a"}],"recommendation":"major_revision","confidential_remarks":"The paper's central contribution is promising and the framework is clearly described, but the current evaluation protocol is not sufficient to support the abstract's claim of significant outperformance. The per-task tuning of the number of learnable concepts without a described nested validation, the selection of the higher TOP implementation, and the absence of significance testing are load-bearing issues that must be fixed. These are fixable within the manuscript's scope by adding statistical comparisons, per-fold results, and a clear tuning protocol; hence I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: ConcepPath is a genuine synthesis—GPT-4-induced expert concepts from medical literature, learnable data-driven concepts, and a two-stage concept-guided aggregation built on a frozen pathology VLM. The combination is new and the ablations are informative. But the paper's central empirical claim, that ConcepPath 'significantly outperformed' SOTA, is not supported by the evidence as reported. The evaluation uses five-fold means with no confidence intervals or significance tests, and the number of learned concepts is tuned per class from {2,4,6,8,10,12} with the best values reported, without any description of nested validation. On the EBV vs. Others task, with only 26 positive cases, the headline 6.23% AUC gain over the best baseline could plausibly come from this selection. The 'TOP*' baseline is also the higher of two implementations, which adds a favorable bias.\n\nWhat's actually new: previous work (TOP) used language priors in a few-shot setting with direct LLM queries. ConcepPath instead induces concepts from curated literature, adds learnable concepts, and works in the full-training regime. The mutual-distinctive loss between concepts and the slide-adapters are reasonable components. The ablation study in Fig. 3 is a real strength: it isolates the contribution of induced vs. generated concepts, data-driven concepts, bag-level guidance, and adapters, and the results are consistent.\n\nThe frozen-VLM alignment worry is real but secondary. The authors flag the domain shift themselves. If the embedding space doesn't align morphology to text, the concept-guided aggregation loses its grounding, but the gains could still come from the adapters. That's a mechanism question, not an evaluation flaw.\n\nMy main problem is statistical. The phrase 'significantly outperformed' appears in the abstract without any test. The tuning of concept counts looks like it may have used the test folds. This is fixable: run a proper nested CV, report per-fold AUCs, add paired tests or at least CIs, and disclose the selection procedure. As it stands, the paper is a promising method with a credible mechanism but an overstated headline.\n\nI'd send it to peer review—the framework deserves referee time—but I'd ask for major revision on the evaluation. It's worth a reading-group debate.","headline":"A promising concept-based MIL framework with informative ablations, but the headline 'significant' gains are not backed by significance testing and the concept-count tuning may be selecting on the test folds.","tokens_in":17076,"tokens_out":2573,"would_cite":true,"duration_ms":22018,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"ConcepPath embeds GPT-4-induced expert concepts and learnable concepts into a frozen pathology vision-language space, claiming consistent gains over state-of-the-art weakly supervised methods on five TCGA whole-slide tasks.","keywords":["whole slide image analysis","multiple instance learning","vision-language model","concept-guided aggregation","GPT-4 expert concepts","weakly supervised classification","histopathology","immunotherapy response prediction"],"falsifier":"Freeze the training pipeline but replace the text descriptions of the expert concepts with semantically unrelated strings (or random word salad) while keeping the same number of learnable tokens; if AUC stays at the same level as the real-concept version, then the expert-knowledge alignment is not carrying the performance. A complementary check is to have pathologists score whether the concept similarity maps localize the morphology they name: if the maps for 'keratinization' do not highlight keratinizing tumor regions, the claimed interpretable grounding is unsupported.","tokens_in":15947,"feed_emoji":"🔬","tokens_out":5198,"duration_ms":43443,"temperature":0.7,"pith_summary":"ConcepPath is a weakly supervised multiple-instance-learning framework that injects human expert knowledge into whole-slide image analysis. Instead of training only from slide labels, it asks GPT-4 to induce disease-specific histopathology concepts from medical literature and adds learnable data-driven concepts, then aligns WSI patches to both concept groups through a frozen CLIP-based pathology vision-language model. A two-stage concept-guided aggregation turns patch-level concept scores into a slide-level prediction. On five TCGA tasks (lung cancer subtyping, breast HER2 scoring, and three gastric immunotherapy-sensitive subtyping tasks) the paper reports consistent gains over seven baselines, with the largest reported jump a 6.23% AUC improvement on EBV-positive versus other gastric cancers. The claim is that expert knowledge, when grounded in literature and aligned in a pathology VLM space, is complementary to data-driven features and improves both accuracy and interpretability.","feed_headline":"GPT-4 concepts steer whole-slide cancer analysis past seven baselines","feed_subtitle":"Aligning slide patches to expert and data-driven concepts gains up to 6.23% AUC on gastric cancer tasks.","key_machinery":"The load-bearing object is the concept set itself: each concept is a text description paired with a learnable prompt prefix, embedded by the text encoder of a frozen CLIP-based pathology vision-language model. Instance-level concepts (expert plus data-driven) are matched to patch features by cosine attention weights in the first aggregation stage; bag-level expert class prompts are matched to instance-level concepts in the second stage; slide-adapters blend adapted features with original features before final class-prompt similarity. A mutual-distinctiveness loss pushes data-driven concepts away from each other so they add information rather than duplicate it.","core_discovery":"The paper's central claim is that a WSI classifier can be built as a set of patch-level concept-scoring subtasks, where the concepts come from two complementary sources: expert concepts induced from medical literature by GPT-4 with manual verification, and purely learnable prompt vectors optimized on training data. In the shared embedding space of a frozen CLIP-based pathology vision-language model (QuiltNet by default, CONCH in supplementary), patch features are matched to concept embeddings, first stage aggregates patches into concept-specific bag features, second stage aggregates these into a bag representation weighted by each concept's correlation with bag-level class prompts, and slide-level prediction is the cosine similarity between adapted bag representations and class prompt embeddings. The paper reports that this two-stage, knowledge-augmented pipeline outperforms seven state-of-the-art MIL methods on all five evaluated tasks, with particularly large margins on the hardest gastric cancer immunotherapy-sensitive subtypes. It also shows that induced concepts beat directly queried concepts, and that learned concepts add 1.04–3.96 AUC points depending on task.","pith_inferences":["A natural extension the paper does not test is whether the same pipeline transfers to less common cancer types: the bottleneck would shift from model capacity to the quality and coverage of the medical literature used to induce concepts, so the approach is only as scalable as the concept-induction step.","If the 6.23% AUC gain on EBV vs. Others replicates in external cohorts, H&E-based screening could become a low-cost triage step for immune checkpoint inhibitor therapy in gastric cancer, reducing the need for IHC and PCR-based molecular tests.","Because the data-driven concepts are optimized prompts in the VLM space, they could be interpreted retroactively as candidate hypotheses for understudied disease mechanisms, provided the similarity maps are validated against expert annotations, which the paper leaves to future work.","The reported trade-off between expert and learned concepts suggests a selection rule the authors do not state: for tasks with sparse published knowledge, favor more learnable concepts; for well-characterized tasks, favor more expert concepts. This could be tested as an adaptive hyperparameter."],"forward_implications":["On the five tested TCGA tasks, ConcepPath reports higher AUC and ACC than seven baselines, with a 6.23-point AUC gain for EBV vs. Others gastric cancer.","Induced concepts outperform directly queried concepts, so the paper concludes that grounding LLM-generated knowledge in literature and manual review matters for downstream accuracy.","Adding learnable data-driven concepts raises AUC by 1.04 (NSCLC), 1.16 (BRCA), and 3.96 (EBV vs. Others) points, and the optimal number grows with task difficulty, suggesting a trade-off between prior and learned knowledge.","Removing the bag-level guidance or the slide-adapters drops performance (e.g., 4.11 AUC points on MSI vs. Others without bag-level guidance), so both stages are claimed to contribute.","Concept similarity maps localize expert concepts on the slide and, per the pathologist collaborator, match established morphology, giving post-hoc interpretability beyond attention maps."],"supporting_citations":[{"why":"Supplies the frozen CLIP-based pathology VLM (QuiltNet) that provides the shared embedding space for patches and concepts in the default setting.","marker":"[17]"},{"why":"Serves as the alternative pathology VLM (CONCH) in supplementary experiments to show the framework's generality across foundation models.","marker":"[14]"},{"why":"Defines the CLIP contrastive pretraining paradigm and the cosine-similarity prediction design that ConcepPath adapts for slide-level classification.","marker":"[15]"},{"why":"Provides the main weakly supervised baseline (CLAM) and the patch-extraction protocol, and its attention maps are compared in the visualization study.","marker":"[31]"},{"why":"The closest prior work using language priors for few-shot WSI classification; its reported limitations motivate the induction-based concept generation and data-driven concept learning.","marker":"[12]"},{"why":"Introduces the learnable prompt representation (CoOp) that is attached to each expert concept and class prompt in ConcepPath.","marker":"[39]"},{"why":"Basis of the slide-adapters used for residual-style feature blending before the final class-prompt prediction.","marker":"[40]"},{"why":"Defines the STAD binary task setup (EBV vs. Others, MSI vs. Others, EBV+MSI vs. Others) and the clinical motivation for immunotherapy-sensitive subtyping.","marker":"[26]"}],"fun_headline_variants":["GPT-4 expert concepts boost whole-slide cancer diagnostics","ConcepPath: GPT-4 concepts sharpen WSI analysis beyond SOTA","Aligning slides to GPT-4 concepts outperforms 7 baselines","GPT-4 knowledge concepts lift histopathology MIL accuracy","Expert concepts from GPT-4 beat seven MIL baselines on WSI"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole pipeline assumes that cosine similarity in the frozen pathology vision-language embedding space reliably indicates whether a medical concept is actually present in an image patch; if the embedding space does not align morphology to text for the target stains or tumor types, the concept guidance loses its meaning and the gains would come from the learnable adapters rather than from expert knowledge.","fun_headline_variants_meta":{"raw":{"variants":["GPT-4 expert concepts boost whole-slide cancer diagnostics","ConcepPath: GPT-4 concepts sharpen WSI analysis beyond SOTA","Aligning slides to GPT-4 concepts outperforms 7 baselines","GPT-4 knowledge concepts lift histopathology MIL accuracy","Expert concepts from GPT-4 beat seven MIL baselines on WSI"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000563,"raw_usage":{"total_tokens":2661,"prompt_tokens":927,"completion_tokens":1734,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":543,"completion_tokens_details":{"reasoning_tokens":1644}},"tokens_in":543,"tokens_out":1734,"duration_ms":10798,"temperature":1.0,"reasoning_tokens":1644,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T11:30:32.511822+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Freeze the training pipeline but replace the text descriptions of the expert concepts with semantically unrelated strings (or random word salad) while keeping the same number of learnable tokens; if AUC stays at the same level as the real-concept version, then the expert-knowledge alignment is not carrying the performance. A complementary check is to have pathologists score whether the concept similarity maps localize the morphology they name: if the maps for 'keratinization' do not highlight keratinizing tumor regions, the claimed interpretable grounding is unsupported.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Serves as the alternative pathology VLM (CONCH) in supplementary experiments to show the framework's generality across foundation models."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the CLIP contrastive pretraining paradigm and the cosine-similarity prediction design that ConcepPath adapts for slide-level classification."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the main weakly supervised baseline (CLAM) and the patch-extraction protocol, and its attention maps are compared in the visualization study."},{"cited_title":"The Rise of AI Language Pathologists: Exploring Two-level Prompt Learning for Few-shot Weakly-supervised Whole Slide Image Classification","cited_arxiv_id":"2305.17891","evidence_quote":"The closest prior work using language priors for few-shot WSI classification; its reported limitations motivate the induction-based concept generation and data-driven concept learning."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the learnable prompt representation (CoOp) that is attached to each expert concept and class prompt in ConcepPath."},{"cited_title":"& Ushiku, T","cited_arxiv_id":null,"evidence_quote":"Defines the STAD binary task setup (EBV vs. Others, MSI vs. Others, EBV+MSI vs. Others) and the clinical motivation for immunotherapy-sensitive subtyping."}],"review_version":1}