REVIEW 4 major objections 4 minor 3 cited by
WSI-Agents: A Collaborative Multi-Agent System for Multi-Modal Whole Slide Image Analysis
T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read The paper claims that a multi-agent system with task allocation, consistency checks, knowledge-base validation, and consensus scoring beats every single whole-slide MLLM and medical-agent baseline on the WSI-Bench and WSI-VQA benchmarks.
desk verdict A plausible and useful multi-agent orchestration for WSI analysis, with internally consistent gains but missing error bars, undisclosed verification weights, and an underexamined consensus-score assumption. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the classifier verification score $\phi_c = \phi_a \cdot \phi_b$: the consensus agent extracts a cancer type from a candidate response, compares it with the outputs of three slide-level pathology foundation models to obtain $\phi_a$, and then weights that agreement by $\phi_b$, the pairwise agreement among the three classifiers, treated as a confidence factor. This score, together with the internal consistency score $\phi_l$ and the knowledge-base score $\phi_k$, forms the total verification score that selects the best response. The model zoo, five whole-slide MLLMs, supplies the candidate answers that the verification layer ranks.
What would settle it
Build a test set of whole-slide cases whose true diagnoses are rare subtypes missing from the training vocabulary of the three foundation classifiers; if correct responses systematically receive lower verification scores than confidently wrong ones on those cases, the consensus-multiplier premise would be refuted.
Extended reading notes
Core claim
On its own terms, the central claim is that a wrapper, not a retrained model, can make existing whole-slide MLLMs competitive on morphology analysis, diagnosis, treatment planning, report generation, and visual question answering. The paper attributes the gain to its verification mechanism: the summarizing agent picks the candidate response with the highest total score $\phi_{\text{total}} = w_1\phi_l + w_2\phi_k + w_3\phi_c$, where $\phi_l$ is internal consistency, $\phi_k$ is knowledge-base agreement, and $\phi_c = \phi_a \cdot \phi_b$ multiplies the agreement between the response's cancer type and three foundation classifiers by those classifiers' agreement with one another. The ablation table, where removing each agent lowers average performance, is offered as evidence that every module carries weight.
Load-bearing premise
The load-bearing premise is that inter-classifier agreement among the three foundation models is a trustworthy proxy for correctness, so the verification scores can rank the true answer first.
Editorial extensions
If this is right
- Existing whole-slide MLLMs can be upgraded by adding an orchestration and verification layer without changing their weights.
- The multi-agent design should transfer to new pathology questions by swapping in a differently tailored model zoo and knowledge base.
- Because the system selects and merges answers rather than training one model, it may be a faster route to multi-task accuracy in domains where expert models already exist.
- If the reported ablation results are representative, both internal consistency checks and external knowledge checks are necessary; dropping either erodes the gains.
Reading between the lines
- The consensus-multiplier principle could generalize to other diagnostic domains: wherever several reliable classifiers and one free-text language model coexist, agreement-weighted selection is a cheap reliability signal.
- A direct comparison against simple majority voting over the same five-model zoo would separate the contribution of verification from the contribution of ensembling; the paper does not report that control.
- Task-dependent weights for $w_1,w_2,w_3$ would likely improve results, since diagnosis may lean more on classifier consensus while report generation may lean more on internal consistency and textbook knowledge.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. WSI-Agents is a collaborative multi-agent system for whole-slide-image analysis. A task-allocation agent assigns each query to one of four expert agents, which query a zoo of five WSI multimodal large language models (MLLMs) to produce initial responses. A verification mechanism scores each response with an internal-consistency score (Eq. (1)), a knowledge-base fact score, and a consensus score (Eq. (2)) built from agreement with TITAN, CONCH, and Prism. A summarizing agent selects the highest-scoring response and refines it with reasoning agents until a majority endorses it. Experiments on WSI-Bench, report generation, and WSI-VQA report gains over WSI-LLaVA, Quilt-LLaVA, WSI-VQA, and Med-Agents/MDAgents; an ablation on WSI-Bench shows each agent contributes.
Significance. If the empirical results are reproducible, the paper makes a useful systems contribution: it shows that a wrapper around off-the-shelf WSI MLLMs and foundation-model classifiers, coordinated by simple task allocation, verification, and summarization mechanisms, can outperform every single WSI MLLM and medical-agent baseline tested on two public benchmarks. The paper's strengths include public source code, the use of public benchmarks, and a module-level ablation that is consistent with the qualitative claim that each component matters. The verification score is a selection heuristic rather than a circular construction, so the circularity concern is not central; however, the label-alignment assumptions, unreported aggregation weights, and single-point scores mean the magnitude of the claimed gains is not yet fully established.
major comments (4)
- [§2.3, Eq. (2)] The consensus verification score phi_c = phi_a * phi_b is load-bearing because it is a multiplicative factor in the total score that selects the final response, but the paper does not specify how the extracted cancer type T is matched to the label spaces of TITAN, CONCH, and Prism, nor what those label spaces are. If a correct MLLM response names a subtype that the three classifiers do not emit, or names a diagnosis phrased differently from the classifier labels, then phi_a will be low and a correct response will be outranked. Please publish the label vocabulary for each foundation model, the exact string/normalization mapping from T to labels, and an analysis of how often the ground-truth labels in WSI-Bench and WSI-VQA are representable by all three classifiers.
- [§2.4] The total verification score is phi_total = w1 phi_l + w2 phi_k + w3 phi_c, and the summarizing agent selects the response with the highest score and terminates when more than half of the reasoning agents endorse the summary, but the manuscript never reports w1, w2, w3, the number M of candidate MLLMs selected per task, or the endorsement threshold in absolute terms. These parameters determine the reported outputs and are needed both for reproducibility and for assessing how sensitive the 0.703 and 0.600 headline numbers are to the verification weights; please report them and include a sensitivity analysis over the weights.
- [§3, Tables 1-3] All benchmark numbers are reported as single-point accuracies with no number of runs, seeds, confidence intervals, or significance tests. The WSI-VQA headline is 0.600 versus 0.550 for WSI-LLaVA, a 5-point difference whose statistical meaning is unclear without variance information, and several WSI-Bench subcategory gaps are smaller than 0.02. Please report repeated-run statistics with the sampling temperature used by the LLM agents, or explicitly state that the protocol is deterministic and justify the single-run protocol.
- [§1 and §3, Tables 1-3] The introduction motivates the work by stating that MLLMs underperform task-specific foundation models by 15-30% on comparable diagnostic tasks, and the contribution is framed as improving task-specific accuracy while retaining versatility. Tables 1-3 nonetheless contain no task-specific baselines such as TITAN, CONCH, or Prism classifications on the diagnosis questions, so the reader cannot tell whether WSI-Agents closes or even reduces the gap to specialized models. Adding such classifier baselines for the diagnosis subtasks would directly test the paper's central motivation.
minor comments (4)
- [§2.2, Eq. (1)] Equation (1) divides by N(N-1)/2, which is undefined when a response contains fewer than two claims; please state the convention used for N less than 2.
- [Table 5] The alignment between the binary columns in Table 5 and the agent abbreviations listed in the caption is unclear; please clarify which column is removed in each row. Also, the Avg of 0.637 in the full row differs from the overall Avg of 0.703 in Table 1; please state explicitly that Table 5 averages the four task-category averages including report generation.
- [§3, report generation] The report generation evaluation says Acc uses WSI-Precision, but the extraction and matching procedure is not described in enough detail to be reproduced; please define WSI-Precision formally.
- [Throughout] There are several citation and typographical inconsistencies: Table 3 lists Med-Agents [29] while the text cites [15] for Med-Agents, the Fig. 2 caption contains 'Responce', Table 1 has a 'T reat. Plan.' header, and the text alternates between 'Quilt-LLAVA' and 'Quilt-LLaVA'. Please proofread these items.
Circularity Check
No significant circularity: WSI-Agents is an empirical selection/verification wrapper; benchmark gains are measured, not constructed.
full rationale
The paper's derivation chain is not circular. WSI-Agents generates multiple candidate responses by querying an MLLM model zoo, scores them with heuristic internal-consistency (Eq. 1), knowledge-base, and classifier-agreement (Eq. 2) verification, and selects/summarizes the best. The reported benchmark numbers (WSI-Bench, WSI-VQA, report generation) are empirical outcomes compared against external metrics, not quantities that are equal to the inputs by construction. The consensus score phi_c = phi_a * phi_b is a selection heuristic whose reliability is an empirical assumption, not a definitional replication of the answer. No fitted parameter is renamed as a prediction, and no load-bearing claim reduces to a self-citation. The dual use of WSI-LLaVA as both a zoo member and a baseline is a legitimate ensemble/comparison design: WSI-Agents can select WSI-LLaVA's response or improve upon it, and beating it is a meaningful, non-identical result. The mild concerns about label-space alignment of TITAN/CONCH/Prism and unreported verification weights are correctness/robustness risks, not circularity.
Assumptions & free parameters
free parameters (3)
- Verification weights w1, w2, w3 =
The paper does not report values.
- Number of candidate MLLMs selected per task =
Not stated in the paper.
- Reasoning agent endorsement threshold =
Threshold is more than half of reasoning agents.
assumptions (4)
- domain assumption The pathology knowledge base assembled from PathologyOutlines and WHO texts is complete and accurate enough to score factual correctness.
- domain assumption The three WSI foundation models provide a reliable external reference for cancer type.
- domain assumption The model zoo contains at least one sufficiently accurate candidate answer for each question.
- domain assumption Resizing whole slide images to 1024 by 1024 thumbnails preserves diagnostically meaningful morphology for all tasks.
Cite this review
Pith. "Pith review of WSI-Agents: A Collaborative Multi-Agent System for Multi-Modal Whole Slide Image Analysis." pith.science (2026). https://pith.science/paper/INDRQPNV
@misc{pith2026250714680,
author = {Pith},
title = {Pith review of: WSI-Agents: A Collaborative Multi-Agent System for Multi-Modal Whole Slide Image Analysis},
year = {2026},
howpublished = {\url{https://pith.science/paper/INDRQPNV}},
note = {Machine review of arXiv:2507.14680}
}
read the original abstract
Whole slide images (WSIs) are vital in digital pathology, enabling gigapixel tissue analysis across various pathological tasks. While recent advancements in multi-modal large language models (MLLMs) allow multi-task WSI analysis through natural language, they often underperform compared to task-specific models. Collaborative multi-agent systems have emerged as a promising solution to balance versatility and accuracy in healthcare, yet their potential remains underexplored in pathology-specific domains. To address these issues, we propose WSI-Agents, a novel collaborative multi-agent system for multi-modal WSI analysis. WSI-Agents integrates specialized functional agents with robust task allocation and verification mechanisms to enhance both task-specific accuracy and multi-task versatility through three components: (1) a task allocation module assigning tasks to expert agents using a model zoo of patch and WSI level MLLMs, (2) a verification mechanism ensuring accuracy through internal consistency checks and external validation using pathology knowledge bases and domain-specific models, and (3) a summary module synthesizing the final summary with visual interpretation maps. Extensive experiments on multi-modal WSI benchmarks show WSI-Agents's superiority to current WSI MLLMs and medical agent frameworks across diverse tasks.
Figures
Forward citations
Cited by 3 Pith papers
-
PathView-Bench: Can Multimodal Large Language Models Achieve Fine-grained Multiscale Understanding of Pathology Images?
A 308k-sample dual-FOV pathology benchmark finds that current MLLMs remain weak at fine-grained multiscale visual grounding even when high-level slide QA looks correct.
-
Evaluating Agentic Bioinformatics through Function, Evidence, and Validation
Agentic bioinformatics systems mostly demonstrate planning and tool execution but rarely prospective empirical validation, so the paper argues evaluation should center on inspectable workflow trajectories (FEV) rather...
-
The Path to Self-Evolving Clinical Systems: Scaling Medical Agents from Assistance to Autonomy
Medical agents should be scaled mainly by richer clinical environments and self-evolution loops, not parameter growth alone, under a three-level autonomy taxonomy.
Reference graph
Works this paper leans on
-
[1]
Bosman, F.T., Carneiro, F., Hruban, R.H., Theise, N.D.: WHO classification of tumours of the digestive system. No. Ed. 4 (2010)
work page 2010
-
[2]
The Lancet Oncology21(2), 233–241 (2020)
Bulten, W., Pinckaers, H., Van Boven, H., Vink, R., De Bel, T., Van Ginneken, B., van der Laak, J., Hulsbergen-van de Kaa, C., Litjens, G.: Automated deep-learning system for gleason grading of prostate cancer using biopsies: a diagnostic study. The Lancet Oncology21(2), 233–241 (2020)
work page 2020
-
[3]
In: Diag- nostic Biomedical Signal and Image Processing Applications with Deep Learning Methods, pp
Çayır, S., Darbaz, B., Solmaz, G., Yazıcı, Ç., Kusetogulları, H., Tokat, F., Iheme, L.O.,Bozaba,E.,Tekin,E.,Özsoy,G.,etal.:Patch-basedapproachestowholeslide histologic grading of breast cancer using convolutional neural networks. In: Diag- nostic Biomedical Signal and Image Processing Applications with Deep Learning Methods, pp. 103–118. Elsevier (2023)
work page 2023
-
[4]
In: In- ternational Conference on Medical Image Computing and Computer-Assisted In- tervention
Chen, P., Li, H., Zhu, C., Zheng, S., Shui, Z., Yang, L.: Wsicaption: Multiple instance generation of pathology reports for gigapixel whole-slide images. In: In- ternational Conference on Medical Image Computing and Computer-Assisted In- tervention. pp. 546–556. Springer (2024)
2024
-
[5]
In: European Conference on Com- puter Vision
Chen, P., Zhu, C., Zheng, S., Li, H., Yang, L.: Wsi-vqa: Interpreting whole slide images by generative visual question answering. In: European Conference on Com- puter Vision. pp. 401–417. Springer (2024)
2024
-
[6]
arXiv preprint arXiv:2410.11761 (2024)
Chen, Y., Wang, G., Ji, Y., Li, Y., Ye, J., Li, T., Zhang, B., Pei, N., Yu, R., Qiao, Y., et al.: Slidechat: A large vision-language assistant for whole-slide pathology image understanding. arXiv preprint arXiv:2410.11761 (2024)
arXiv 2024
-
[7]
Scientific Reports 11(1), 8894 (2021)
Ciga, O., Xu, T., Nofech-Mozes, S., Noy, S., Lu, F.I., Martel, A.L.: Overcoming the limitations of patch-based learning to detect cancer in whole slide images. Scientific Reports 11(1), 8894 (2021)
work page 2021
-
[8]
arXiv preprint arXiv:2411.19666 (2024) 10 Authors Suppressed Due to Excessive Length
Ding, T., Wagner, S.J., Song, A.H., Chen, R.J., Lu, M.Y., Zhang, A., Vaidya, A.J., Jaume, G., Shaban, M., Kim, A., et al.: Multimodal whole slide foundation model for pathology. arXiv preprint arXiv:2411.19666 (2024) 10 Authors Suppressed Due to Excessive Length
arXiv 2024
Show all 32 references
-
[9]
arXiv preprint arXiv:2502.02673 (2025)
Fallahpour, A., Ma, J., Munim, A., Lyu, H., Wang, B.: Medrax: Medical reasoning agent for chest x-ray. arXiv preprint arXiv:2502.02673 (2025)
2025 arXiv
-
[10]
In: International Conference on Medical Image Computing and Computer- Assisted Intervention
Guo, Z., Ma, J., Xu, Y., Wang, Y., Wang, L., Chen, H.: Histgen: Histopathology report generation via local-global feature encoding and cross-modal context inter- action. In: International Conference on Medical Image Computing and Computer- Assisted Intervention. pp. 189–199. S...
2024
-
[11]
In: CVPR
Hashimoto, N., Fukushima, D., Koga, R., Takagi, Y., Ko, K., Kohno, K., Nakaguro, M., Nakamura, S., Hontani, H., Takeuchi, I.: Multi-scale domain- adversarial multiple-instance cnn for cancer subtype classification with unanno- tated histopathological images. In: CVPR. pp. 3852...
2020
-
[12]
In: Pro- ceedings of the IEEE conference on computer vision and pattern recognition
Hou, L., Samaras, D., Kurc, T.M., Gao, Y., Davis, J.E., Saltz, J.H.: Patch-based convolutional neural network for whole slide tissue image classification. In: Pro- ceedings of the IEEE conference on computer vision and pattern recognition. pp. 2424–2433 (2016)
2016
-
[13]
Scientific reports 11(1), 11579 (2021)
Khened, M., Kori, A., Rajkumar, H., Krishnamurthi, G., Srinivasan, B.: A gener- alized deep learning framework for whole-slide image segmentation and analysis. Scientific reports 11(1), 11579 (2021)
2021
-
[14]
arXiv preprint arXiv:2404.15155 (2024)
Kim, Y., Park, C., Jeong, H., Chan, Y.S., Xu, X., McDuff, D., Lee, H., Ghassemi, M., Breazeal, C., Park, H.W.: Mdagents: An adaptive collaboration of llms in medical decision making. arXiv preprint arXiv:2404.15155 (2024)
2024 arXiv
-
[15]
arXiv preprint arXiv:2407.02483 (2024)
Li, B., Yan, T., Pan, Y., Luo, J., Ji, R., Ding, J., Xu, Z., Liu, S., Dong, H., Lin, Z., et al.: Mmedagent: Learning to use medical tools with multi-modal agent. arXiv preprint arXiv:2407.02483 (2024)
2024 arXiv
-
[16]
arXiv preprint arXiv:2305.19118 (2023)
Liang, T., He, Z., Jiao, W., Wang, X., Wang, Y., Wang, R., Yang, Y., Shi, S., Tu, Z.: Encouraging divergent thinking in large language models through multi-agent debate. arXiv preprint arXiv:2305.19118 (2023)
2023 arXiv
-
[17]
arXiv preprint arXiv:2412.02141 (2024)
Liang, Y., Lyu, X., Ding, M., Chen, W., Zhang, J., Ren, Y., He, X., Wu, S., Yang, S., Wang, X., et al.: Wsi-llava: A multimodal large language model for whole slide image. arXiv preprint arXiv:2412.02141 (2024)
2024 arXiv
-
[18]
arXiv preprint arXiv:2412.01605 (2024)
Liu, J., Wang, W., Ma, Z., Huang, G., SU, Y., Chang, K.J., Chen, W., et al.: Med- chain:Bridgingthegapbetweenllmagentsand clinicalpracticethroughinteractive sequential benchmarking. arXiv preprint arXiv:2412.01605 (2024)
2024
-
[19]
arXiv preprint arXiv:2308.03688
Liu, X., Yu, H., Zhang, H., Xu, Y., Lei, X., Lai, H., Gu, Y., Ding, H., Men, K., et al.: Agentbench: Evaluating llms as agents. arXiv preprint arXiv:2308.03688
-
[20]
Nature Medicine30(3), 863–874 (2024)
Lu, M.Y., Chen, B., Williamson, D.F., Chen, R.J., Liang, I., Ding, T., Jaume, G., Odintsov, I., Le, L.P., Gerber, G., et al.: A visual-language foundation model for computational pathology. Nature Medicine30(3), 863–874 (2024)
2024
-
[21]
In: CVPR
Lu, M.Y., Chen, B., Zhang, A., Williamson, D.F., Chen, R.J., Ding, T., Le, L.P., Chuang, Y.S., Mahmood, F.: Visual language pretrained multiple instance zero- shot transfer for histopathology images. In: CVPR. pp. 19764–19775 (2023)
2023
-
[22]
https://github.com/microsoft/autogen (2023), 2025-02-27
Microsoft: Autogen. https://github.com/microsoft/autogen (2023), 2025-02-27
2023
-
[23]
Histopathology76(2), 182 (2019)
Nagtegaal, I.D., Odze, R.D., Klimstra, D., Paradis, V., Rugge, M., Schirmacher, P., Washington, K.M., Carneiro, F., Cree, I.A., et al.: The 2019 who classification of tumours of the digestive system. Histopathology76(2), 182 (2019)
2019
-
[24]
PathologyOutlines: PathologyOutlines.com (2001), 2025-02-27
2001
-
[25]
arXiv preprint arXiv:2307.07924 (2023)
Qian, C., Cong, X., Yang, C., Chen, W., Su, Y., Xu, J., Liu, Z., Sun, M.: Commu- nicative agents for software development. arXiv preprint arXiv:2307.07924 (2023)
2023 arXiv
-
[26]
In: Proceedings of the IEEE/CVF CVPR (2024) WSI-Agents: A Multi-Agent System for Whole Slide Image Analysis 11
Seyfioglu, M.S., Ikezogwo, W.O., Ghezloo, F., Krishna, R., Shapiro, L.: Quilt- llava: Visual instruction tuning by extracting localized narratives from open-source histopathology videos. In: Proceedings of the IEEE/CVF CVPR (2024) WSI-Agents: A Multi-Agent System for Whole Sli...
2024
-
[27]
arXiv preprint arXiv:2405.10254 (2024)
Shaikovski, G., Casson, A., Severson, K., Zimmermann, E., Wang, Y.K., et al.: Prism: A multi-modal generative foundation model for slide-level histopathology. arXiv preprint arXiv:2405.10254 (2024)
2024 arXiv
-
[28]
Histopathology77(2) (2020)
Tan, P.H., Ellis, I., Allison, K., Brogi, E., Fox, S.B., Lakhani, S., Lazar, A.J., Morris, E.A., Sahin, A., Salgado, R., et al.: The 2019 who classification of tumours of the breast. Histopathology77(2) (2020)
2020
-
[29]
arXiv preprint arXiv:2311.10537 (2023)
Tang, X., Zou, A., Zhang, Z., Li, Z., Zhao, Y., Zhang, X., Cohan, A., Gerstein, M.: Medagents:Largelanguagemodelsascollaboratorsforzero-shotmedicalreasoning. arXiv preprint arXiv:2311.10537 (2023)
2023 arXiv
-
[30]
Computerized Medical Imaging and Graphics87, 101838 (2021)
Tripathi, S., Singh, S.K., Lee, H.K.: An end-to-end breast tumour classification model using context-based patch modelling–a bilstm approach for image classifi- cation. Computerized Medical Imaging and Graphics87, 101838 (2021)
2021
-
[31]
arXiv preprint arXiv:2401.16107 (2024)
Wang, H., Zhao, S., Qiang, Z., Xi, N., Qin, B., Liu, T.: Beyond direct diagno- sis: Llm-based multi-specialist agent consultation for automatic diagnosis. arXiv preprint arXiv:2401.16107 (2024)
2024 arXiv
-
[32]
Nature pp
Xu, H., Usuyama, N., Bagga, J., Zhang, S., Rao, R., Naumann, T., Wong, C., Gero, Z., González, J., Gu, Y., et al.: A whole-slide foundation model for digital pathology from real-world data. Nature pp. 1–8 (2024)
2024
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.