Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

A Versatile Pathology Co-pilot via Reasoning Enhanced Multimodal Large Language Model

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper presents SmartPath-R1, a single multimodal large language model that simultaneously handles ROI-level classification, detection, segmentation, and VQA along with whole-slide classification and VQA, reporting top average rank…

desk verdict A strong ROI-level pathology MLLM paper that overstates its WSI-level capability; the WSI results are almost certainly on tiles, not whole slides. read the letter →

arxiv 2507.17303 v2 pith:Y46AJAL4 submitted 2025-07-23 eess.IV cs.AIcs.CV

Zhe Xu , Ziyi Liu , Junlin Hou , Jiabo Ma , Cheng Jin , Yihui Wang , Zhixuan Chen , Zhengyu Zhang
show 8 more authors
Fuxiang Huang Zhengrui Guo Fengtao Zhou Yingxue Xu Xi Wang Ronald Cheong Kin Chan Li Liang Hao Chen
This is my paper
classification eess.IVcs.AIcs.CV
keywords multimodallargelanguagemodelcomputationalpathologywhole-slideimageanalysisreinforcementlearninggrouprelativepolicyoptimizationmixtureofexpertsvisualquestionansweringdiagnosticreasoning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that one multimodal large language model can act as a pathology co-pilot across both region-of-interest and whole-slide tasks, combining classification, detection, segmentation, and visual question answering in a single system. It claims that diagnostic reasoning can be learned through reinforcement rewards rather than expensive chain-of-thought annotations, and that the resulting model outperforms existing pathology and general vision-language models on a broad benchmark. A sympathetic reading takes these results as evidence that a unified, reasoning-enhanced model is a viable path toward practical AI assistance in pathology.

What carries the argument

The load-bearing mechanism is a two-stage training pipeline built on a Qwen2.5-VL base. Scale-dependent supervised fine-tuning adjusts image resolution and visual token budget per task, using 256 tokens for ROI-level tasks and 1024 tokens for WSI-level tasks. Task-aware GRPO then optimizes task-specific rewards for classification, detection, segmentation, and VQA, with a format reward enforcing structured <think></think> and <answer></answer> outputs; a mixture-of-experts LoRA mechanism routes different task families to specialized adapters, and segmentation masks are produced by feeding predicted boxes to the pre-trained MedSAM model.

What would settle it

Run SmartPath-R1 on genuine gigapixel whole-slide images with a documented tiling, aggregation, or downsampling pipeline and compare accuracy against the patch-level numbers; if accuracy collapses to tile-level performance or the model cannot process a real slide under the 1024-token cap, the WSI-level claim is not supported.

Watch

Extended reading notes

Core claim

The central discovery is that a single MLLM, trained with scale-dependent supervised fine-tuning followed by task-aware group relative policy optimization, can simultaneously solve ROI-level classification, detection, segmentation, VQA, and WSI-level classification and VQA. On 72 tasks, the model achieves an average rank of 1.1 and ranks first on 68 of them, while the strongest prior pathology reasoning model achieves an average rank of 2.9 and first place on 3 tasks. The paper further claims this is the only compared model that handles all six task types at once, and that the reinforcement stage improves reasoning without requiring procedural annotations.

Load-bearing premise

The load-bearing premise is that the inputs labeled whole-slide images carry true whole-slide diagnostic context; the reported training-sample counts suggest they may be tiles labeled by slide, and no tiling or aggregation pipeline is described.

Editorial extensions

If this is right

  • A single trained model can cover the main pathology task families, so a diagnostic assistant no longer needs separate systems for ROI analysis, detection, segmentation, and whole-slide interpretation.
  • Pathological reasoning can be elicited through reinforcement rewards instead of chain-of-thought annotations, lowering the cost of building explainable pathology models.
  • The task-aware reward design produces structured reasoning traces that make model outputs more inspectable and potentially easier for clinicians to verify.
  • The model's strong results on external validation sets suggest it can generalize to pathology datasets outside its training distribution.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the same two-stage recipe of scale-dependent supervised fine-tuning plus task-aware GRPO could transfer to other gigapixel medical imaging settings, since none of the components is pathology-specific, but the paper does not demonstrate this.
  • Editorial inference: if the WSI evaluation inputs are actually tiles labeled by slide rather than true whole-slide images, the practical route to genuine WSI diagnosis would require a tiling-and-aggregation wrapper around the model, not the raw 1024-token input described.
  • Editorial inference: the average-rank headline rewards breadth across many tasks; a deployment-focused study would need to weigh per-task accuracy and failure modes, which the paper does not provide.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces SmartPath-R1, a pathology multimodal large language model (MLLM) built on Qwen2.5-VL with a mixture-of-experts LoRA architecture, trained in two stages: scale-dependent supervised fine-tuning and task-aware GRPO with reward functions for classification, VQA, detection, and segmentation. The authors curate a large dataset of about 2.3M ROI samples and 188K WSI samples and evaluate on 72 tasks spanning ROI classification, detection, segmentation, VQA and WSI classification/VQA, reporting an average rank of 1.1 (first place in 68 tasks) against Qwen2.5-VL, LLaVA-Med, Quilt-LLaVA, and PathoR1, with SmartPath-SFT also appearing in most tables. The central claim is that a single MLLM can simultaneously address both ROI-level and WSI-level tasks while producing chain-of-thought-style diagnostic reasoning without explicit CoT supervision.

Significance. If the WSI-level evidence were properly established, this would be a notable step toward a unified pathology co-pilot, with a large curated training corpus and evaluation across many public datasets. Strengths include the use of public baseline models, explicit reward functions that avoid chain-of-thought annotation, and systematic reporting of confidence intervals and significance tests. However, the load-bearing WSI-level claim currently rests on an unverified and likely patch-based input protocol, and the absence of code/checkpoints and of specialist detection/segmentation baselines tempers the reproducibility and clinical significance of the comparison. The ROI-level results appear internally consistent and are the most defensible contribution at this stage.

major comments (4)
  1. [Sections 4.2, 4.6; Table 44] The description of WSI-level inputs is inconsistent with whole-slide processing. With patch size P=28 and maximum token count M=1024, the transformation T^{M,P} bounds an input to roughly 896x896 pixels, while diagnostic WSIs are gigapixel images. Table 44 reports 18,390 training samples for TCGA-LUAD and 38,427 for TCGA-BRCA, numbers that far exceed the number of diagnostic slides available in those TCGA projects; these samples must therefore be patches or tiles labeled with slide-level labels. No tiling, aggregation, downsampling, or multi-instance pooling pipeline is described anywhere in the methods. Consequently, the WSI-level results in Tables 34-43 and Figure 2d do not demonstrate whole-slide-image understanding, and the uniqueness claim in Figure 2a ('only SmartPath-R1 handles WSI') is overstated as presented. The authors must either provide the actual WSI processing pipeline, including any slide-level fusion mechanism, or explicitly re-scope the claims to patch-level inputs with slide-level labels.
  2. [Section 4.3; Table 44] There is a potential slide-level data leakage for the WSI-level tasks. The training and evaluation sample counts for the same TCGA project (e.g., TCGA-LUAD: 18,390 training samples vs. 191 evaluation samples; TCGA-BRCA: 38,427 vs. 452) strongly suggest that patches from the same slides could have been used in both training and test, because the number of samples exceeds the number of available whole-slide images. The paper states that overlapping samples were screened and removed, but it does not specify whether the split was performed at the slide or patient level. If the split is not slide-disjoint, the reported accuracies on WSI-level tasks are likely inflated by patch-level leakage. Please clarify the split protocol and, if necessary, re-evaluate with slide-disjoint training and test sets.
  3. [Section 2; Section 'WSI-Level VQA'; Table 39] There is an inconsistency in the number of WSI-level VQA tasks. Section 2 states 17 WSI-level VQA tasks, while the subsection 'WSI-Level VQA' states 13; the sum 8+14+14+10+9+17 = 72 matches the stated total of 72 tasks, but if the 13-task count is used the total becomes 68. The tables suggest 13 internal SlideInstruction tasks plus 4 BCNB tasks, which would reconcile the counts, but this should be stated explicitly. Additionally, Figure 2a marks the four baseline models as not handling WSI tasks (x entries for WSI CLS and WSI VQA), yet Tables 34-43 report their performance on those tasks; the figure caption should clarify that all models were evaluated on all tasks and that the check marks denote training coverage or native capability, not the set of evaluated tasks.
  4. [Sections 4.4-4.5; Figures 4-5] The detection and segmentation comparisons are mostly against other MLLMs plus the single foundation model BiomedParse; no specialized detectors (e.g., Mask R-CNN) or segmenters (e.g., U-Net, nnU-Net) are included. Since SmartPath-R1's segmentation masks are produced by MedSAM from the model's predicted bounding boxes, the segmentation comparison measures the quality of box proposals under a fixed promptable segmenter rather than end-to-end segmentation ability. To support the wording in Section 2 that SmartPath-R1 achieves 'state-of-the-art performance' in detection and segmentation, the authors should either include specialist baselines or temper the claim to 'state-of-the-art among evaluated MLLMs and BiomedParse.'
minor comments (5)
  1. [Section 4.3 (CCRCC)] The CCRCC dataset is modified from its original 6-class version to a 4-class version by excluding the background and heterogeneous-tissue classes. This post-hoc class filtering changes the task definition and should be disclosed prominently, with a discussion of its effect on comparability with previously published CCRCC results.
  2. [Table 35] Several confidence intervals appear internally inconsistent; for example, the TCGA-COAD row for SmartPath-R1 reports F1 = 0.977 with 95% CI (0.577, 0.940), where the upper bound is below the point estimate. Please verify all bootstrap or parametric CI computations across the extended tables.
  3. [Code Availability] The Code Availability statement says the code 'will be released' at a GitHub URL, but no code or model checkpoints are currently provided. For a methods-heavy paper, this limits reproducibility of the central results; please provide a working repository or state access conditions.
  4. [Section 2 and Figures 2-8] The ranking is described as being over five models (rank 1 to 5), but most tables include SmartPath-SFT as a sixth method. Please clarify whether the average rank of 1.1 is computed over the five named baselines or over all six methods, and report the rank of SmartPath-SFT in the latter case.
  5. [Table 44 and text] There are minor typographical inconsistencies, such as 'BRCAS' in Table 44 versus 'BRACS' elsewhere, and the WSI-level sample counts in Table 44 are labeled 'samples' without specifying whether they are whole-slide images, patches, or tiles, which is directly related to the major comment on WSI-level preprocessing.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reward/evaluation metric overlap is legitimate RL training, and the WSI preprocessing ambiguity is a validity risk, not a self-referential derivation.

full rationale

Score 0. I examined the claimed derivation chain: Eq. (1) defines SFT cross-entropy; Eqs. (2)-(4) define GRPO; Eqs. (5)-(10) define task rewards; metrics in Sec. 4.5 and Tables 3-43 are computed on held-out or external test sets. No reported number is a fitted parameter renamed as prediction, and no equation defines a result in terms of the quantity it claims to predict. The fact that Rdet uses AP50 and Rseg uses Dice while evaluation reports AP50/Dice is metric-aligned RL training, not circularity, because rewards are applied to training splits and the test evaluations are independent (the paper states it 'meticulously screened and removed any samples from the original test sets'). Self-citations (e.g., refs. 3-5, 10-17, 19, 24) are background or data references and are not load-bearing in the sense of importing an unverified uniqueness theorem or forbidding alternatives. The strongest concern is at Sec. 4.2/4.6 and Table 44: with M=1024 tokens and P=28, 'WSI-level' inputs are bounded to roughly 896x896 pixels, and Table 44 lists 18,390 TCGA-LUAD training samples—far more than available whole slides—so those samples are probably patches or tiles labeled by slide. That is an under-specified input/validity issue for the WSI claim, but it is not circular: no equation makes the WSI results equivalent to this preprocessing choice. Therefore no circular step can be exhibited under the hard rule.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central results rest on standard supervised and reinforcement learning empirical assumptions rather than new mathematical axioms. The paper introduces no new physical entities; the main unverified inputs are the token-budget assumption for WSI-level inputs, the reliability of public labels and GPT-4 generated reports, and the leakage screening procedure.

free parameters (6)
  • Max token count M = 256 (ROI), 1024 (WSI)
    Hand-chosen token budget controlling input resolution; directly determines how much slide context the model sees.
  • Patch size P = 28
    Vision encoder patch size used for resizing and tokenization.
  • LoRA rank and scale = rank=32, scale=128
    Configuration for the per-task LoRA experts.
  • GRPO group size G = 8
    Number of sampled outputs per question in the reinforcement learning objective.
  • KL and clip coefficients = beta=0.001, epsilon=0.2
    Hyperparameters stabilizing the GRPO policy update.
  • Learning rates = 1e-4 (SFT), 1e-5 (RL)
    Initial learning rates for the two fine-tuning stages.
assumptions (5)
  • domain assumption Held-out test splits are leakage-free after the authors' screening.
    Section 4.3 says overlapping samples were removed, but the screening procedure is not described; the central comparisons rely on this.
  • domain assumption MedSAM produces segmentation masks that faithfully reflect the model's predicted boxes.
    Equation (8) computes Dice(MedSAM(P), M); if MedSAM is inaccurate for a domain, the Dice score does not measure the MLLM's segmentation ability.
  • domain assumption Public dataset labels and GPT-4 curated WSI reports are reliable ground truth.
    The paper trains and evaluates against these labels; label errors would propagate into rewards and reported metrics.
  • domain assumption At most 1024 visual tokens per WSI capture slide-level diagnostic content.
    Section 4.2 sets M=1024 for WSI tasks; the validity of the WSI-level claims depends on whether this token budget preserves enough tissue context and whether inputs are true whole slides.
  • domain assumption Molecular and clinical features such as MSI and HER2 are inferable from H&E images.
    Datasets like CRC-MSI and BCNB assign slide-level molecular labels to images; the model is expected to learn these associations from visual input alone.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Versatile Pathology Co-pilot via Reasoning Enhanced Multimodal Large Language Model." pith.science (2026). https://pith.science/paper/Y46AJAL4

@misc{pith2026250717303,
  author       = {Pith},
  title        = {Pith review of: A Versatile Pathology Co-pilot via Reasoning Enhanced Multimodal Large Language Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Y46AJAL4}},
  note         = {Machine review of arXiv:2507.17303}
}
read the original abstract

Multimodal large language models (MLLMs) have emerged as powerful tools for computational pathology, offering unprecedented opportunities to integrate pathological images with language context for comprehensive diagnostic analysis. These models hold particular promise for automating complex tasks that traditionally require expert interpretation of pathologists. However, current MLLM approaches in pathology demonstrate significantly constrained reasoning capabilities, primarily due to their reliance on expensive chain-of-thought annotations. Additionally, existing methods remain limited to simplex application of visual question answering (VQA) at the region-of-interest (ROI) level, failing to address the full spectrum of diagnostic needs such as ROI classification, detection, segmentation, whole-slide-image (WSI) classification and VQA in clinical practice. In this study, we present SmartPath-R1, a versatile MLLM capable of simultaneously addressing both ROI-level and WSI-level tasks while demonstrating robust pathological reasoning capability. Our framework combines scale-dependent supervised fine-tuning and task-aware reinforcement fine-tuning, which circumvents the requirement for chain-of-thought supervision by leveraging the intrinsic knowledge within MLLM. Furthermore, SmartPath-R1 integrates multiscale and multitask analysis through a mixture-of-experts mechanism, enabling dynamic processing for diverse tasks. We curate a large-scale dataset comprising 2.3M ROI samples and 188K WSI samples for training and evaluation. Extensive experiments across 72 tasks validate the effectiveness and superiority of the proposed approach. This work represents a significant step toward developing versatile, reasoning-enhanced AI systems for precision pathology.

Figures

Figures reproduced from arXiv: 2507.17303 by the authors.

Figure 1
Figure 1. Overview of the SmartPath-R1. a. SmartPath-R1 is a versatile pathology co-pilot that can simultaneously address both ROI-level and WSI-level tasks across various anatomical regions while demonstrating robust pathological reasoning capability. b. We curate a dataset comprising 2,295,095 ROI-level and 188,154 WSI-level samples, enabling comprehensive model training and evaluation. c. Performance evaluation of MLLMs ac… view at source ↗
Figure 2
Figure 2. Comprehensive Comparison of MLLMs across 72 Tasks. a. Task types evaluated by different MLLMs. CLS, DET, SEG, and VQA represent classification, detection, segmentation, and visual question answering, respectively. b. Average rank of MLLMs across 72 downstream tasks. The solid line and the dashed line represent the median and the mean, respectively. c. Ranking order of MLLMs across 46 ROI-level tasks. d. Ranking orde… view at source ↗
Figure 3
Figure 3. Performance of MLLMs on ROI-Level Classification Tasks. a. Average performance of MLLMs based on accuracy (ACC) and F1 across 8 ROI-level classification tasks. b-i. Model performance on specific tasks. * represents external validation datasets. Error bars represent 95% CI. The box limits represent the standard error. P-values are computed using a Wilcoxon signed-rank two-sided test28. Additional results are shown in… view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Performance of MLLMs on ROI-Level Detection Tasks. a. Average performance of MLLMs based on average precision at IoU threshold 0.5 (AP50) across 14 ROI-level detection tasks. b-o. Model performance on specific tasks. * represents external validation datasets. Error bar…
Figure 5
Figure 5. Figure 5: Performance of MLLMs on ROI-Level Segmentation Tasks. For MLLMS, masks are generated by using the predicted objects’ coordinates as prompts for the pre-trained MedSAM model. a. Average performance of MLLMs based on Dice score across 14 ROI-level segmentation tasks. b-o…
Figure 6
Figure 6. Figure 6: Performance of MLLMs on ROI-Level VQA Tasks. a. Average performance of MLLMs based on accuracy (ACC) across 10 ROI VQA tasks. b-h. Model performance on specific tasks. Error bars represent 95% CI. The box limits represent the standard error. P-values are computed using…
Figure 7
Figure 7. Figure 7: Performance of MLLMs on WSI-Level Classification Tasks. a. Average performance of MLLMs based on accuracy (ACC) and F1 across 9 WSI-level classification tasks. b-e. Model performance on specific tasks. * represents external validation datasets. Error bars represent 95%…
Figure 8
Figure 8. Figure 8: Performance of MLLMs on WSI-Level VQA Tasks. a. Average performance of MLLMs based on accuracy (ACC) across 13 WSI-level VQA tasks. b-r. Model performance on specific tasks. * represents external validation datasets. Error bars represent 95% CI. The box limits represen…
Figure 9
Figure 9. Figure 9: Performance of MLLMs on ROI-Level Detection Tasks. a. Average performance of MLLMs based on average precision at IoU threshold 0.3 (AP30) across 14 ROI-level detection tasks. b-o. Model performance on specific tasks. * represents external validation datasets. Error bar…
Figure 10
Figure 10. Figure 10: Performance of MLLMs on ROI-Level Detection Tasks. a. Average performance of MLLMs based on average precision at IoU threshold 0.7 (AP70) across 14 ROI-level detection tasks. b-o. Model performance on specific tasks. * represents external validation datasets. Error ba…
Figure 11
Figure 11. Figure 11: Performance of MLLMs on ROI-Level Segmentation Tasks. a-c. Average performance of MLLMs based on average Dice Score across 14 ROI-level segmentation tasks. d-q. Model performance on specific tasks. * represents external validation datasets. The violin plots visualize …

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Towards Reliable Stain Transfer: An Iterative Data-Model Co-Optimization Framework Based on Multimodal Expert-Guided Assessment

    cs.CV 2026-07 conditional novelty 7.0 of 10

    DMCoStain iteratively refines stain-transfer training data by using a pathology-savvy vision-language model to select high-quality generated IHC images, achieving state-of-the-art H&E-to-IHC translation on six biomarkers.

  2. Enhancing Pathological VLMs with Cross-scale Reasoning

    cs.CV 2026-06 unverdicted novelty 7.0 of 10

    Presents Scale-VQA benchmark for cross-scale pathology VQA and RL-trained ScaleReasoner-R1 model that reaches SOTA on the new benchmark plus existing single-scale tasks.

Reference graph

Works this paper leans on

55 extracted references · 39 canonical work pages · cited by 2 Pith papers

  1. [1]

    He, Y .et al.Foundation model for advancing healthcare: challenges, opportunities and future directions.IEEE Rev. Biomed. Eng.(2024)

    work page 2024

  2. [2]

    neural information processing systems 36, 37995–38017 (2023)

    Ikezogwo, W.et al.Quilt-1m: One million image-text pairs for histopathology.Adv. neural information processing systems 36, 37995–38017 (2023)

    work page 2023

  3. [3]

    Hou, J.et al.Self-explainable ai for medical image analysis: A survey and new outlooks.arXiv preprint arXiv:2410.02331 (2024)

    arXiv 2024

  4. [4]

    & Chen, H

    Hou, J., Xu, J., Feng, R. & Chen, H. Qmix: Quality-aware learning with mixed noise for robust retinal disease diagnosis. IEEE Transactions on Med. Imaging(2025)

    work page 2025

  5. [5]

    & Heng, P.-A

    Chen, H., Qi, X., Yu, L. & Heng, P.-A. Dcan: deep contour-aware networks for accurate gland segmentation. InProceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2487–2496 (2016)

    work page 2016

  6. [6]

    J.et al.Towards a general-purpose foundation model for computational pathology.Nat

    Chen, R. J.et al.Towards a general-purpose foundation model for computational pathology.Nat. Medicine30, 850–862 (2024)

    2024

  7. [7]

    Huang, Z., Bianchi, F., Yuksekgonul, M., Montine, T. J. & Zou, J. A visual–language foundation model for pathology image analysis using medical twitter.Nat. medicine29, 2307–2316 (2023)

    2023

  8. [8]

    & Gao, X

    Huang, F., Zhang, L., Zhou, Y . & Gao, X. Adversarial and isotropic gradient augmentation for image retrieval with text feedback.IEEE Transactions on Multimed.25, 7415–7427 (2022)

    work page 2022

Show all 55 references
  1. [9]

    Medicine30, 1081–1091 (2024)

    Mormont, R.et al.A visual-language foundation model for computational pathology.Nat. Medicine30, 1081–1091 (2024)

    2024

  2. [10]

    & Xue, H

    Xu, Z., Chen, D., Wei, K., Deng, C. & Xue, H. Hisa: Hierarchically semantic associating for video temporal grounding. IEEE Transactions on Image Process.31, 5178–5188 (2022)

    2022

  3. [11]

    & Deng, C

    Xu, Z., Wei, K., Yang, X. & Deng, C. Point-supervised video temporal grounding.IEEE Transactions on Multimed.25, 6121–6131 (2022). 12.Lu, M. Y .et al.A visual-language foundation model for computational pathology.Nat. Medicine30(2024)

    2022

  4. [13]

    Xu, Y .et al.A multimodal knowledge-enhanced whole-slide pathology foundation model.arXiv preprint arXiv:2407.15362 (2024)

    2024 arXiv

  5. [14]

    & Liu, W

    Xu, Z., Wei, K., Yang, E., Deng, C. & Liu, W. Bilateral relation distillation for weakly supervised temporal action localization.IEEE Transactions on Pattern Analysis Mach. Intell.45, 11458–11471 (2023)

    2023

  6. [15]

    & Deng, C

    Xu, Z., Wei, K., Yang, X. & Deng, C. Exploiting intrinsic multilateral logical rules for weakly supervised natural language video localization. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 4511–4521 (2024)

    2024

  7. [16]

    & Zhang, L

    Huang, F., Song, S. & Zhang, L. Gradient harmonization in unsupervised domain adaptation.IEEE Transactions on Pattern Analysis Mach. Intell.(2024)

    2024

  8. [17]

    & Song, S

    Huang, F., Zhang, L., Fu, X. & Song, S. Dynamic weighted combiner for mixed-modal image retrieval. InProceedings of the AAAI Conference on Artificial Intelligence, vol. 38, 2303–2311 (2024). 18.Xiang, J.et al.A vision–language foundation model for precision oncology.Nature1–10...

    2024 arXiv

  9. [20]

    & Lee, Y

    Liu, H., Li, C., Li, Y . & Lee, Y . J. Improved baselines with visual instruction tuning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 26296–26306 (2024). 21.Liu, H., Li, C., Wu, Q. & Lee, Y . J. Visual instruction tuning.Adv. Neural Inf...

    2024

  10. [23]

    S., Ikezogwo, W

    Seyfioglu, M. S., Ikezogwo, W. O., Ghezloo, F., Krishna, R. & Shapiro, L. Quilt-llava: Visual instruction tuning by extracting localized narratives from open-source histopathology videos. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 1318...

    2024

  11. [24]

    In Proceedings of the AAAI Conference on Artificial Intelligence, vol

    Sun, Y .et al.Pathasst: A generative foundation ai assistant towards artificial general intelligence of pathology. In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, 5034–5042 (2024). 25.Bai, S.et al.Qwen2. 5-vl technical report.arXiv preprint arXiv:250...

    2024 arXiv

  12. [26]

    Neural Inf

    Li, C.et al.Llava-med: Training a large language-and-vision assistant for biomedicine in one day.Adv. Neural Inf. Process. Syst.36, 28541–28564 (2023)

    2023

  13. [27]

    28.Demšar, J

    Zhang, W.et al.Patho-r1: A multimodal reinforcement learning-based pathology expert reasoner.arXiv preprint arXiv:2505.11404(2025). 28.Demšar, J. Statistical comparisons of classifiers over multiple data sets.J. Mach. learning research7, 1–30 (2006). 29.Ma, J.et al.Segment any...

    2025

  14. [31]

    Jordan, M. I. & Jacobs, R. A. Hierarchical mixtures of experts and the em algorithm.Neural computation6, 181–214 (1994)

    1994

  15. [32]

    A., Jordan, M

    Jacobs, R. A., Jordan, M. I., Nowlan, S. J. & Hinton, G. E. Adaptive mixtures of local experts.Neural computation3, 79–87 (1991)

    1991

  16. [33]

    Guo, D.et al.Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948(2025)

    2025 arXiv

  17. [34]

    & Klimov, O

    Schulman, J., Wolski, F., Dhariwal, P., Radford, A. & Klimov, O. Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347(2017)

    2017 arXiv

  18. [35]

    & Brück, O

    Brummer, O., Pölönen, P., Mustjoki, S. & Brück, O. Computational textural mapping harmonises sampling variation and reveals multidimensional histopathological fingerprints.Br. J. Cancer129, 683–695 (2023)

    2023

  19. [36]

    & Jin, M

    Zhu, C., Chen, W., Peng, T., Wang, Y . & Jin, M. Hard sample aware noise robust learning for histopathology image classification.IEEE transactions on medical imaging41, 881–894 (2021)

    2021

  20. [37]

    N.et al.Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer

    Kather, J. N.et al.Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer. Nat. medicine25, 1054–1056 (2019)

    2019

  21. [38]

    Heal.5, e265–e275 (2023)

    Tolkach, Y .et al.Artificial intelligence for tumour tissue detection and histological regression grading in oesophageal adenocarcinomas: a retrospective algorithm development and validation study.The Lancet Digit. Heal.5, e265–e275 (2023)

    2023

  22. [39]

    oncology11, 806603 (2022)

    Abousamra, S.et al.Deep learning-based mapping of tumor infiltrating lymphocytes in whole slide images of 23 types of cancer.Front. oncology11, 806603 (2022)

    2022

  23. [40]

    Saltz, J.et al.Spatial organization and molecular correlation of tumor-infiltrating lymphocytes using deep learning on pathology images.Cell reports23, 181–193 (2018)

    2018

  24. [41]

    A.et al.Unitopatho, a labeled histopathological dataset for colorectal polyps classification and adenoma dysplasia grading

    Barbano, C. A.et al.Unitopatho, a labeled histopathological dataset for colorectal polyps classification and adenoma dysplasia grading. In2021 IEEE International Conference on Image Processing (ICIP), 76–80 (IEEE, 2021)

    2021

  25. [42]

    A., Oliveira, L

    Spanhol, F. A., Oliveira, L. S., Petitjean, C. & Heutte, L. A dataset for breast cancer histopathological image classification. Ieee transactions on biomedical engineering63, 1455–1462 (2015). 43.Komura, D.et al.Universal encoding of pan-cancer histology by deep texture repres...

    2015

  26. [44]

    Amgad, M.et al.Structured crowdsourcing enables convolutional segmentation of histology images.Bioinformatics35, 3461–3467 (2019)

    2019

  27. [45]

    Graham, S.et al.Conic challenge: Pushing the frontiers of nuclear detection, segmentation, classification and counting. Med. image analysis92, 103047 (2024)

    2024

  28. [46]

    image analysis52, 199–211 (2019)

    Graham, S.et al.Mild-net: Minimal information loss dilated network for gland instance segmentation in colon histology images.Med. image analysis52, 199–211 (2019)

    2019

  29. [47]

    image analysis80, 102485 (2022)

    Da, Q.et al.Digestpath: A benchmark dataset with challenge review for the pathological detection and segmentation of digestive-system.Med. image analysis80, 102485 (2022). 23/42

    2022

  30. [48]

    Image Analysis94, 103155 (2024)

    Aubreville, M.et al.Domain generalization across tumor types, laboratories, and species—insights from the 2022 edition of the mitosis domain generalization challenge.Med. Image Analysis94, 103155 (2024)

    2024

  31. [49]

    Salvi, M.et al.A hybrid deep learning approach for gland segmentation in prostate histopathological images.Artif. Intell. Medicine115, 102076 (2021)

    2021

  32. [50]

    A., Molina, R

    Silva-Rodríguez, J., Colomer, A., Sales, M. A., Molina, R. & Naranjo, V . Going deeper through the gleason scoring scale: An automatic end-to-end system for histology prostate grading and cribriform pattern detection.Comput. methods programs biomedicine195, 105637 (2020)

    2020

  33. [51]

    & Walter, T

    Naylor, P., Laé, M., Reyal, F. & Walter, T. Segmentation of nuclei in histopathology images by deep regression of the distance map.IEEE transactions on medical imaging38, 448–459 (2018)

    2018

  34. [52]

    arXiv preprint arXiv:2204.06455(2022)

    Han, C.et al.Wsss4luad: Grand challenge on weakly-supervised tissue semantic segmentation for lung adenocarcinoma. arXiv preprint arXiv:2204.06455(2022)

    2022 arXiv

  35. [53]

    image analysis35, 489–502 (2017)

    Sirinukunwattana, K.et al.Gland segmentation in colon histology images: The glas challenge contest.Med. image analysis35, 489–502 (2017)

    2017

  36. [54]

    Amgad, M.et al.Nucls: A scalable crowdsourcing approach and dataset for nucleus classification and segmentation in breast cancer.GigaScience11, giac037 (2022)

    2022

  37. [55]

    In European Conference on Computer Vision, 56–73 (Springer, 2024)

    Sun, Y .et al.Pathmmu: A massive multimodal expert-level benchmark for understanding and reasoning in pathology. In European Conference on Computer Vision, 56–73 (Springer, 2024)

    2024

  38. [56]

    & Xie, P

    He, X., Zhang, Y ., Mou, L., Xing, E. & Xie, P. Pathvqa: 30000+ questions for medical visual question answering.arXiv preprint arXiv:2003.10286(2020)

    2020 arXiv

  39. [57]

    InProceedings of the Computer Vision and Pattern Recognition Conference, 5134–5143 (2025)

    Chen, Y .et al.Slidechat: A large vision-language assistant for whole-slide pathology image understanding. InProceedings of the Computer Vision and Pattern Recognition Conference, 5134–5143 (2025). 58.Weinstein, J. N.et al.The cancer genome atlas pan-cancer analysis project.Na...

    2025

  40. [59]

    Brancati, N.et al.Bracs: A dataset for breast carcinoma subtyping in h&e histology images.Database2022, baac093 (2022)

    2022

  41. [60]

    E.et al.Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer.Jama318, 2199–2210 (2017)

    Bejnordi, B. E.et al.Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer.Jama318, 2199–2210 (2017)

    2017

  42. [61]

    Bandi, P.et al.From detection of individual metastases to classification of lymph node status at the patient level: the camelyon17 challenge.IEEE transactions on medical imaging38, 550–560 (2018)

    2018

  43. [62]

    J.et al.The cptac data portal: a resource for cancer proteomics research.J

    Edwards, N. J.et al.The cptac data portal: a resource for cancer proteomics research.J. proteome research14, 2707–2713 (2015)

    2015

  44. [63]

    oncology11, 759007 (2021)

    Xu, F.et al.Predicting axillary lymph node metastasis in early breast cancer using deep learning on primary tumor biopsy slides.Front. oncology11, 759007 (2021)

    2021

  45. [64]

    methods22, 166–176 (2025)

    Zhao, T.et al.A foundation model for joint segmentation, detection and recognition of biomedical objects across nine modalities.Nat. methods22, 166–176 (2025)

    2025

  46. [65]

    InAdvances in Neural Information Processing Systems, 8026–8037 (2019)

    Paszke, A.et al.Pytorch: An imperative style, high-performance deep learning library. InAdvances in Neural Information Processing Systems, 8026–8037 (2019). 66.Kingma, D. P. & Ba, J. Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980(2014)

    2019 arXiv

  47. [67]

    Signal Process

    Sitnik, D.et al.A dataset and a methodology for intraoperative computer-aided diagnosis of a metastatic colon cancer in a liver.Biomed. Signal Process. Control.66, 102402 (2021). Extended Data 24/42 Table 2.Classes of Different Datasets for ROI-Level Classification.* represent...

    2021

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.