Pith. sign in

REVIEW 3 major objections 4 minor 52 references

Advancing Generalizable Tumor Segmentation with Anomaly-Aware Open-Vocabulary Attention Maps and Frozen Foundation Diffusion Models

T0 review · 3 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The internal representations of a frozen medical foundation diffusion model can be repurposed into zero-shot tumor segmenters across organs and modalities.

desk verdict A well-constructed zero-shot tumor segmentation paper with large reported gains; the MAISI/MSD training overlap is a real caveat but not fatal, and the paper deserves a serious referee. read the letter →

arxiv 2505.02753 v1 pith:UUBKWAKV submitted 2025-05-05 cs.CV

Yankai Jiang , Peng Zhang , Donglin Yang , Yuan Tian , Hai Lin , Xiaosong Wang This is my paper · ORCID
classification cs.CV
keywords zero-shottumorsegmentationfrozendiffusionmodelsopen-vocabularyattentionmapsanomalydetectionpseudo-healthyimagesynthesislatentspaceinpainting3Dmedicalimagingtext-prompt
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper aims to show that a frozen medical foundation diffusion model, trained to generate 3D CT and MRI anatomy, already carries enough spatial knowledge to segment tumors it was never trained to segment. The proposed framework, DiffuGTS, builds anomaly-aware open-vocabulary attention maps from the model's internal VAE features and text prompts naming a normal organ or an abnormal finding, then refines those maps by inpainting the suspected tumor region to a pseudo-healthy version and comparing the two at pixel and feature levels. The authors report large zero-shot margins over prior text-guided and anomaly-detection methods across seven tumor categories, four datasets, and both CT and MRI, including a 21-point Dice similarity gain when moving from CT training to MRI test data. If the claim holds, a single frozen generative model could serve as a parameter-efficient backbone for segmenting unseen diseases without retraining.

What carries the argument

The load-bearing machinery is the anomaly-aware open-vocabulary attention (AOVA) map: cross-attention matrices between pixel-level queries from the frozen VAE encoder's multi-scale features and text attribution keys projected from normal/organ and abnormal/disease prompts are resized and summed across feature levels to localize anomalies in a category-free, text-prompt-driven way. A second mechanism is training-free latent space inpainting: the suspected tumor region, enlarged by a coefficient, is re-sampled from the frozen diffusion model while known healthy regions are copied from the input latent, and the resulting pseudo-healthy image is compared with the original through pixel-level and feature-level residuals that are combined and thresholded into a final mask. Only lightweight adapters, MLP projections, and attention layers are trained; the diffusion model and the VAE stay frozen.

What would settle it

Rerun the leave-one-out and cross-modality benchmarks with the same foundation diffusion model trained on a corpus that excludes MSD, and compare results on MSD test cases against results on datasets never used in pretraining; if the large margins over prior zero-shot and anomaly-detection baselines mostly evaporate on the non-overlapping MSD cases while persisting elsewhere, the apparent generalization is driven by pretraining overlap rather than by the attention-map and inpainting machinery.

Watch

Extended reading notes

Core claim

The central discovery is that the internal representations of a frozen medical foundation diffusion model—specifically the multi-scale features of its 3D variational autoencoder—can be repurposed as zero-shot semantic cues for tumor localization, and that the same frozen model can then be used to clean up those cues. Text embeddings of normal and abnormal prompts are projected into attribution keys that attend over the VAE's pixel features, producing anomaly-aware open-vocabulary attention maps that mark tumor candidates without a preset category list. A training-free latent inpainting step replaces the candidate region by sampling from the diffusion model, producing a pseudo-healthy volume that preserves non-tumor anatomy; subtracting the original volume from this pseudo-healthy version at both pixel and latent-feature levels yields the final segmentation. On leave-one-out and cross-modality tests, the authors report that this combination outperforms existing zero-shot segmentation and anomaly-detection methods, with the largest gains coming from the mask-refinement stage rather than from the attention maps alone.

Load-bearing premise

The load-bearing premise, stated in the supplementary material's data-leakage analysis rather than proven by experiment, is that the foundation model's pretraining on partly overlapping public evaluation datasets does not leak useful appearance or label information into the variational-autoencoder features or the pseudo-healthy images; if that overlap helps the scores, the zero-shot claim is overstated.

Editorial extensions

If this is right

  • A new tumor type can be added at test time by writing a text prompt; no training examples or labels for it are needed.
  • Cross-modality generalization is a direct corollary: the framework passes from CT-trained attention maps to MRI tumor segmentation with the same weights, with much larger gains than text-only baselines.
  • The method is parameter-efficient in the sense that only small adapter and projection modules are trained, although inference FLOPs are high because the generative backbone runs at test time.
  • Mask quality improves because the pseudo-healthy comparison is anchored to the input anatomy; replacing direct organ synthesis with inpainting preserves healthy regions and sharpens tumor boundaries.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the same attention-plus-inpainting recipe transfers to other generative foundation models with a latent anatomy prior, it could convert any such model into an anomaly segmenter without retraining it.
  • The reported efficiency is a trade-off: the method trades high inference FLOPs for few trainable parameters, so distilling the frozen model's features into a compact segmenter is a natural next step that the paper leaves open.
  • The cleanest test of genuine zero-shot generalization would be to compare performance on evaluation datasets that overlap the foundation model's pretraining with performance on non-overlapping datasets, since the paper's supplementary data-leakage discussion argues, but does not experimentally demonstrate, that overlap leaks no label information.
  • A wider taxonomy of anomalies—cysts, hemorrhage, inflammation, or imaging artifacts—could be probed with the same category-free attention maps, because the method does not rely on a closed training category list.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes DiffuGTS, a framework for zero-shot tumor segmentation that repurposes the frozen VAE encoder and diffusion decoder of MAISI, a medical foundation diffusion model. It first builds anomaly-aware open-vocabulary attention (AOVA) maps by aligning multi-scale VAE features with text prompts for normal and abnormal categories, then refines the resulting masks through training-free latent-space inpainting that synthesizes pseudo-healthy images, followed by pixel-level and feature-level residual learning. The method is evaluated on MSD, KiTS23, BraTS23, and an in-house MRI dataset under leave-one-out and cross-modality zero-shot settings, reporting consistent improvements over SAM-based, vision-language, and anomaly-detection baselines. Ablations support the contribution of each component, and the authors release code.

Significance. If the reported results hold, the paper makes a substantive contribution: it demonstrates that a frozen medical foundation diffusion model can serve as a highly parameter-efficient backbone for zero-shot tumor segmentation across organs and modalities, and it introduces a novel mask-refinement mechanism via latent-space inpainting that goes beyond simple residual maps. The work is well motivated, the ablations are informative, and the experiments cover a broad set of tumor types and both CT and MRI. The authors also ship code and provide detailed dataset documentation. However, the central empirical claim depends on the absence of leakage from MAISI's pretraining on MSD, and the paper asserts statistical significance without reporting the supporting intervals or p-values; these issues need to be resolved before the zero-shot superiority claim can be accepted.

major comments (3)
  1. [Supplementary Material F; Tables 1 and 2] The central zero-shot claim is primarily established on MSD datasets, and MAISI was pretrained on MSD. The rebuttal in Supp. F argues that the VAE was trained for reconstruction and that the repaint paradigm is out-of-distribution, but it provides no control experiment. If MAISI's features have memorized MSD-specific tumor appearances, the AOVA maps and residual refinement could exploit that memory, converting an allegedly zero-shot task into near-duplicate retrieval. This is load-bearing because five of six categories in Table 1 and MSD-Brain in Table 2 come from MSD. Please add a control that does not rely on argument, for example: (a) evaluate the same tumor categories on non-overlapping datasets, or (b) train a probe to check whether the frozen VAE or the diffusion model behaves differently on MSD versus non-MSD images, or (c) retrain MAISI on a disjoint corpus. Without such a control, the headline claim that DiffuGTS 'surpasses current state-of-the-art models across multiple zero-shot settings' is not independently verified.
  2. [Section 4, Evaluation Metrics] The paper states that '95% CIs were calculated, and a p-value cutoff of less than 0.05 was used to define statistical significance,' but no confidence intervals, p-values, or significance tests appear anywhere in the main text or supplementary material. This makes the assertion of statistical significance unsupported and prevents the reader from judging whether the reported margins over baselines are meaningful. Please either report the CIs and the results of the significance testing, or remove the claim. This is a load-bearing issue because the abstract and conclusions rely on 'surpassing' state-of-the-art methods.
  3. [Section 4, Competing Methods and Baselines] The prompt setup for SAM-based baselines is unspecified. The text says these methods 'need manual or automatic-generated prompts during testing' but does not state what prompts were used for SAM 2, SaLIP, and H-SAM, or whether prompts were derived from ground-truth annotations. If the baselines received weaker prompts than the text prompts used by DiffuGTS, the comparison would be biased. Please specify the exact prompt protocol (e.g., number of points, boxes, or masks, and whether they come from ground truth) for each SAM-based baseline, or explicitly acknowledge the limitation in the comparison.
minor comments (4)
  1. [Section 3.1, Textual Prompt Composition] There is a typo in the prompt template: 'A nomal CT scan/MRI of {organ name}' should read 'A normal CT scan/MRI of {organ name}'.
  2. [Table 3 caption] The caption contains 'competting methods'; this should be 'competing methods'.
  3. [Section 3.2, Eq. (3)] The notation for the noise schedule is confusing: the text defines α as a product over timesteps using β_s, but the equation uses α_t in the Gaussian distribution. Clarify the relationship between α_t, α, and β_s to avoid ambiguity.
  4. [Supplementary Material D] The robustness analysis with misleading prompts is useful, but it would be stronger if it also reported quantitative results (e.g., false-positive rates) rather than only showing that no foreground mask is predicted in two examples.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: DiffuGTS trains on seen categories and tests on held-out categories; the acknowledged MAISI/MSD overlap is a data-leakage risk, not a definitional reduction.

full rationale

No load-bearing circular step is present. The AOVA maps (Sec. 3.1, Eq. 1) are cross-attention products of frozen MAISI VAE features and frozen text embeddings, trained with L_ano, L_sim, and L_dice on seen categories and then evaluated on held-out categories in leave-one-out and cross-modality settings; no test-time prediction is defined as a fitted parameter of the training set. The mask-refinement stage (Sec. 3.2) synthesizes a pseudo-healthy volume conditioned on the initial AOVA mask and derives final masks from pixel- and feature-level residuals; the final mask is not equal by construction to the input mask, and the ablations in Tables 5, 6, 9, and 10 show that changing the refinement and residual components changes results, which would not happen if the output were identical to the input by definition. Self-citations are present but non-load-bearing: ZePT [22] and Malenia [23] are baselines and related-work comparisons, not premises of the method, and MAISI [16] is a released frozen foundation model used as a backbone rather than an imported uniqueness or ansatz claim. The acknowledged overlap between MAISI's training data and the MSD evaluation sets (Supp. F) is a potential data-contamination or soundness threat, not a circular reduction: the paper's equations do not feed MSD labels into the predictions, and the absence of a disjoint-data control is a soundness limitation rather than a circularity. The score of 2 reflects only the minor non-load-bearing self-citations, not any definitional equivalence or fitted-input-as-prediction step.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the frozen MAISI model's pretraining being a reusable source of anatomy knowledge, and on the assumption that the reconstruction/inpainting setup yields pseudo-healthy images whose residuals mark tumors. The hyperparameters above are hand-set defaults; the mask-enlargement beta and temperature tau are not reported, which leaves some tuning freedom unspecified.

free parameters (5)
  • residual ratio alpha = 0.1
    Controls how much original MAISI feature is preserved in the adapter; set by default in Sec. 3.1.
  • mask enlargement coefficient beta = not reported
    Enlarges the AOVA tumor mask before latent inpainting to cover boundaries (Sec. 3.2); value is not stated.
  • residual weighting beta1, beta2 = 0.5, 0.5
    Weights for combining pixel-level and feature-level residual maps at inference (Sec. 3.2).
  • contrastive temperature tau = not reported
    Temperature in the CLIP-style contrastive loss L_sim (Sec. 3.1); affects alignment sharpness.
  • anomaly score threshold th = 0.5
    Used to binarize the image-level anomaly prediction in L_ano (Sec. 3.1).
assumptions (5)
  • domain assumption MAISI VAE features encode sufficient anatomical knowledge for zero-shot tumor localization across organs and modalities
    Core hypothesis; motivates Sec. 3.1 and is tested via ablations in Tab. 4.
  • domain assumption The frozen text encoder Clinical-BERT produces embeddings for arbitrary disease names that align with the image feature space after contrastive training
    Underwrites the open-vocabulary claim; only seven categories and fixed templates are evaluated.
  • domain assumption RePaint-style latent inpainting preserves non-tumor regions while generating pseudo-healthy tissue in the masked region
    Needed for the residual maps in Sec. 3.2 to highlight tumors; the paper provides qualitative evidence but no quantitative fidelity metric.
  • domain assumption The leave-one-out and unseen-modality protocols measure zero-shot generalization
    Standard in the field, but the MAISI overlap (Supp. F) weakens the independence of the MSD-based tests.
  • standard math Statistical significance claim relies on standard hypothesis tests
    The paper states 95% CIs and p<0.05 were used but does not report the test statistics.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Advancing Generalizable Tumor Segmentation with Anomaly-Aware Open-Vocabulary Attention Maps and Frozen Foundation Diffusion Models." pith.science (2026). https://pith.science/paper/UUBKWAKV

@misc{pith2026250502753,
  author       = {Pith},
  title        = {Pith review of: Advancing Generalizable Tumor Segmentation with Anomaly-Aware Open-Vocabulary Attention Maps and Frozen Foundation Diffusion Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UUBKWAKV}},
  note         = {Machine review of arXiv:2505.02753}
}
read the original abstract

We explore Generalizable Tumor Segmentation, aiming to train a single model for zero-shot tumor segmentation across diverse anatomical regions. Existing methods face limitations related to segmentation quality, scalability, and the range of applicable imaging modalities. In this paper, we uncover the potential of the internal representations within frozen medical foundation diffusion models as highly efficient zero-shot learners for tumor segmentation by introducing a novel framework named DiffuGTS. DiffuGTS creates anomaly-aware open-vocabulary attention maps based on text prompts to enable generalizable anomaly segmentation without being restricted by a predefined training category list. To further improve and refine anomaly segmentation masks, DiffuGTS leverages the diffusion model, transforming pathological regions into high-quality pseudo-healthy counterparts through latent space inpainting, and applies a novel pixel-level and feature-level residual learning approach, resulting in segmentation masks with significantly enhanced quality and generalization. Comprehensive experiments on four datasets and seven tumor categories demonstrate the superior performance of our method, surpassing current state-of-the-art models across multiple zero-shot settings. Codes are available at https://github.com/Yankai96/DiffuGTS.

Figures

Figures reproduced from arXiv: 2505.02753 by the authors.

Figure 1
Figure 1. We propose DiffuGTS, a novel framework that utilizes and extends the capabilities of a frozen medical foundational diffusion model for advanced zero-shot tumor segmentation across various anatomical regions and imaging modalities. (a) DiffuGTS employs de￾scriptions of both normal and abnormal categories to generate open-vocabulary text-attribution attention maps (b) for anomaly segmentation through cross-modal featu… view at source ↗
Figure 2
Figure 2. The architecture of DiffuGTS for generalizable tumor segmentation. through a Multi-Layer Perception (MLP), adapting them to be compatible with the image features in terms of dimension size and thus enhances computational efficiency. Then, the text embeddings e are projected using the key projections Wl K to generate attribution keys Kl = Wl K(e) ∈ R N×Cl . Similarly, we project the image features Fl using the query … view at source ↗
Figure 3
Figure 3. Synthesizing pseudo-healthy images directly using [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Qualitative visualizations of zero-shot segmentation results on MSD [ [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Visualization illustrating how utilizing pixel-level and [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Qualitative visualizations of zero-shot segmentation results on BraTS23 [ [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 7
Figure 7. Figure 7: The illustration of the one-step reverse process of the inpainting strategy. [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: How the model processes misleading text prompts. [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

52 extracted references · 33 canonical work pages

  1. [1]

    The brain tumor segmentation (brats) chal- lenge 2023: glioma segmentation in sub-saharan africa pa- tient population (brats-africa)

    Maruf Adewole, Jeffrey D Rudie, Anu Gbdamosi, Oluyemisi Toyobo, Confidence Raymond, Dong Zhang, Olubukola Omidiji, Rachel Akinola, Mohammad Abba Suwaid, Adaobi Emegoakor, et al. The brain tumor segmentation (brats) chal- lenge 2023: glioma segmentation in sub-saharan africa pa- tient population (brats-africa). ArXiv, 2023. 5, 6, 7, 1, 2

    work page 2023

  2. [2]

    Test-time adaptation with salip: A cascade of sam and clip for zero-shot medical image seg- mentation

    Sidra Aleem, Fangyijie Wang, Mayug Maniparambil, Eric Arazo, Julia Dietlmeier, Kathleen Curran, Noel EO’ Con- nor, and Suzanne Little. Test-time adaptation with salip: A cascade of sam and clip for zero-shot medical image seg- mentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 5184– 5193, 2024. 2, 6

    work page 2024

  3. [3]

    The medical segmentation decathlon.Nature communications, 13(1):4128, 2022

    Michela Antonelli, Annika Reinke, Spyridon Bakas, Key- van Farahani, Annette Kopp-Schneider, Bennett A Landman, Geert Litjens, Bjoern Menze, Olaf Ronneberger, Ronald M Summers, et al. The medical segmentation decathlon.Nature communications, 13(1):4128, 2022. 1, 5, 6, 7, 2

    work page 2022

  4. [4]

    Guided reconstruction with conditioned dif- fusion models for unsupervised anomaly detection in brain mris

    Finn Behrendt, Debayan Bhattacharya, Robin Mieling, Lennart Maack, Julia Kr ¨uger, Roland Opfer, and Alexan- der Schlaefer. Guided reconstruction with conditioned dif- fusion models for unsupervised anomaly detection in brain mris. arXiv preprint arXiv:2312.04215, 2023. 3

    arXiv 2023

  5. [5]

    Diffusion models with implicit guidance for medical anomaly detection

    Cosmin I Bercea, Benedikt Wiestler, Daniel Rueckert, and Julia A Schnabel. Diffusion models with implicit guidance for medical anomaly detection. In International Conference on Medical Image Computing and Computer-Assisted Inter- vention, pages 211–220. Springer, 2024. 3, 5, 6, 7, 8

    work page 2024

  6. [6]

    Cancerunit: Towards a single unified model for ef- fective detection, segmentation, and diagnosis of eight major cancers using a large collection of ct scans

    Jieneng Chen, Yingda Xia, Jiawen Yao, Ke Yan, Jianpeng Zhang, Le Lu, Fakai Wang, Bo Zhou, Mingyan Qiu, Qihang Yu, et al. Cancerunit: Towards a single unified model for ef- fective detection, segmentation, and diagnosis of eight major cancers using a large collection of ct scans. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion,...

    work page 2023

  7. [7]

    Towards generaliz- able tumor synthesis

    Qi Chen, Xiaoxi Chen, Haorui Song, Zhiwei Xiong, Alan Yuille, Chen Wei, and Zongwei Zhou. Towards generaliz- able tumor synthesis. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 11147–11158, 2024. 1, 3

    work page 2024

  8. [8]

    Berdiff: Conditional bernoulli diffusion model for medical image segmentation

    Tao Chen, Chenhui Wang, and Hongming Shan. Berdiff: Conditional bernoulli diffusion model for medical image segmentation. In International conference on medical image computing and computer-assisted intervention , pages 491–

    work page

Show all 52 references
  1. [9]

    Masked-attention mask transformer for universal image segmentation

    Bowen Cheng, Ishan Misra, Alexander G Schwing, Alexan- der Kirillov, and Rohit Girdhar. Masked-attention mask transformer for universal image segmentation. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1290–1299, 2022. 8

    2022

  2. [10]

    Unleashing the potential of sam for medical adaptation via hierarchi- cal decoding

    Zhiheng Cheng, Qingyue Wei, Hongru Zhu, Yan Wang, Liangqiong Qu, Wei Shao, and Yuyin Zhou. Unleashing the potential of sam for medical adaptation via hierarchi- cal decoding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 3511– 352...

    2024

  3. [11]

    Diffusion trans- former u-net for medical image segmentation

    G Jignesh Chowdary and Zhaozheng Yin. Diffusion trans- former u-net for medical image segmentation. In In- ternational conference on medical image computing and computer-assisted intervention , pages 622–631. Springer,

  4. [12]

    Latentpaint: Image inpainting in latent space with diffusion models

    Ciprian Corneanu, Raghudeep Gadde, and Aleix M Mar- tinez. Latentpaint: Image inpainting in latent space with diffusion models. In Proceedings of the IEEE/CVF Win- ter Conference on Applications of Computer Vision , pages 4334–4343, 2024. 5

    2024

  5. [13]

    De- coupling zero-shot semantic segmentation

    Jian Ding, Nan Xue, Gui-Song Xia, and Dengxin Dai. De- coupling zero-shot semantic segmentation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11583–11592, 2022. 8

    2022

  6. [14]

    Training like a medical resident: Context-prior learning toward universal medical image segmentation

    Yunhe Gao. Training like a medical resident: Context-prior learning toward universal medical image segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 11194–11204, 2024. 1

    2024

  7. [15]

    Scal- ing open-vocabulary image segmentation with image-level labels

    Golnaz Ghiasi, Xiuye Gu, Yin Cui, and Tsung-Yi Lin. Scal- ing open-vocabulary image segmentation with image-level labels. In European Conference on Computer Vision, pages 540–557. Springer, 2022. 8

    2022

  8. [16]

    Maisi: Medical ai for synthetic imaging

    Pengfei Guo, Can Zhao, Dong Yang, Ziyue Xu, Vish- wesh Nath, Yucheng Tang, Benjamin Simon, Mason Belue, Stephanie Harmon, Baris Turkbey, et al. Maisi: Medical ai for synthetic imaging. arXiv preprint arXiv:2409.11169,

  9. [17]

    Accelerating dif- fusion models via pre-segmentation diffusion sampling for medical image segmentation

    Xutao Guo, Yanwu Yang, Chenfei Ye, Shang Lu, Bo Peng, Hua Huang, Yang Xiang, and Ting Ma. Accelerating dif- fusion models via pre-segmentation diffusion sampling for medical image segmentation. In 2023 IEEE 20th Interna- tional Symposium on Biomedical Imaging (ISBI), pages 1–5...

    2023

  10. [18]

    The kits21 chal- lenge: Automatic segmentation of kidneys, renal tumors, and renal cysts in corticomedullary-phase ct

    Nicholas Heller, Fabian Isensee, Dasha Trofimova, Resha Tejpaul, Zhongchen Zhao, Huai Chen, Lisheng Wang, Alex Golts, Daniel Khapun, Daniel Shats, et al. The kits21 chal- lenge: Automatic segmentation of kidneys, renal tumors, and renal cysts in corticomedullary-phase ct. arXi...

    2023 arXiv

  11. [19]

    Label-free liver tumor segmentation

    Qixin Hu, Yixiong Chen, Junfei Xiao, Shuwen Sun, Jieneng Chen, Alan L Yuille, and Zongwei Zhou. Label-free liver tumor segmentation. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 7422–7432, 2023. 1

    2023

  12. [20]

    Adapting visual-language models for generalizable anomaly detection in medical im- ages

    Chaoqin Huang, Aofan Jiang, Jinghao Feng, Ya Zhang, Xin- chao Wang, and Yanfeng Wang. Adapting visual-language models for generalizable anomaly detection in medical im- ages. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 11375–11385,

  13. [21]

    nnu-net: a self-configuring 9 method for deep learning-based biomedical image segmen- tation

    Fabian Isensee, Paul F Jaeger, Simon AA Kohl, Jens Pe- tersen, and Klaus H Maier-Hein. nnu-net: a self-configuring 9 method for deep learning-based biomedical image segmen- tation. Nature methods, 18(2):203–211, 2021. 8

    2021

  14. [22]

    Zept: Zero-shot pan-tumor segmentation via query-disentangling and self-prompting

    Yankai Jiang, Zhongzhen Huang, Rongzhao Zhang, Xiao- fan Zhang, and Shaoting Zhang. Zept: Zero-shot pan-tumor segmentation via query-disentangling and self-prompting. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 11386–11397, 202...

    2024

  15. [23]

    Unleashing the potential of vision-language pre- training for 3d zero-shot lesion segmentation via mask- attribute alignment

    Yankai Jiang, Wenhui Lei, Xiaofan Zhang, and Shaoting Zhang. Unleashing the potential of vision-language pre- training for 3d zero-shot lesion segmentation via mask- attribute alignment. arXiv preprint arXiv:2410.15744, 2024. 1, 2, 3, 6, 8

    2024 arXiv

  16. [24]

    Diffusion adver- sarial representation learning for self-supervised vessel seg- mentation

    Boah Kim, Yujin Oh, and Jong Chul Ye. Diffusion adver- sarial representation learning for self-supervised vessel seg- mentation. arXiv preprint arXiv:2209.14566, 2022. 3

    2022 arXiv

  17. [25]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 4015–4026, 2023. 2, 6

    2023

  18. [26]

    Self-supervised diffusion model for anomaly segmentation in medical imaging

    Komal Kumar, Snehashis Chakraborty, and Sudipta Roy. Self-supervised diffusion model for anomaly segmentation in medical imaging. In International Conference on Pat- tern Recognition and Machine Intelligence, pages 359–368. Springer, 2023. 5

    2023

  19. [27]

    Open-vocabulary semantic segmentation with mask-adapted clip

    Feng Liang, Bichen Wu, Xiaoliang Dai, Kunpeng Li, Yinan Zhao, Hang Zhang, Peizhao Zhang, Peter Vajda, and Diana Marculescu. Open-vocabulary semantic segmentation with mask-adapted clip. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pag...

    2023

  20. [28]

    Clip-driven universal model for organ segmentation and tumor detection

    Jie Liu, Yixiao Zhang, Jie-Neng Chen, Junfei Xiao, Yongyi Lu, Bennett A Landman, Yixuan Yuan, Alan Yuille, Yucheng Tang, and Zongwei Zhou. Clip-driven universal model for organ segmentation and tumor detection. In Proceedings of the IEEE/CVF International Conference on Compute...

    2023

  21. [29]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017. 6

    2017 arXiv

  22. [30]

    Repaint: Inpainting using denoising diffusion probabilistic models

    Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11461–11471, 2022. 5

    2022

  23. [31]

    V-net: Fully convolutional neural networks for volumetric medical image segmentation

    Fausto Milletari, Nassir Navab, and Seyed-Ahmad Ahmadi. V-net: Fully convolutional neural networks for volumetric medical image segmentation. In 2016 fourth international conference on 3D vision (3DV), pages 565–571. Ieee, 2016. 4, 5

    2016

  24. [32]

    A threshold selection method from gray-level histograms

    Nobuyuki Otsu et al. A threshold selection method from gray-level histograms. Automatica, 11(285-296):23–27,

  25. [33]

    Freeseg: Unified, universal and open-vocabulary image segmentation

    Jie Qin, Jie Wu, Pengxiang Yan, Ming Li, Ren Yuxi, Xue- feng Xiao, Yitong Wang, Rui Wang, Shilei Wen, Xin Pan, et al. Freeseg: Unified, universal and open-vocabulary image segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages...

    2023

  26. [34]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  27. [35]

    Ambiguous medical image segmentation using diffusion models

    Aimon Rahman, Jeya Maria Jose Valanarasu, Ilker Haci- haliloglu, and Vishal M Patel. Ambiguous medical image segmentation using diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11536–11546, 2023. 3

    2023

  28. [36]

    Sam 2: Segment anything in images and videos

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman R¨adle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.00714, 2024. 6

    2024 arXiv

  29. [37]

    Zero-shot medical image segmentation based on sparse prompt using finetuned sam

    Tal Shaharabany and Lior Wolf. Zero-shot medical image segmentation based on sparse prompt using finetuned sam. In Medical Imaging with Deep Learning, 2024. 2

    2024

  30. [38]

    Learning fixed points in gener- ative adversarial networks: From image-to-image transla- tion to disease detection and localization

    Md Mahfuzur Rahman Siddiquee, Zongwei Zhou, Nima Tajbakhsh, Ruibin Feng, Michael B Gotway, Yoshua Ben- gio, and Jianming Liang. Learning fixed points in gener- ative adversarial networks: From image-to-image transla- tion to disease detection and localization. In Proceedings o...

    2019

  31. [39]

    Tassilo Wald, Saikat Roy, Gregor Koehler, Nico Disch, Max- imilian Rouven Rokuss, Julius Holzschuh, David Zimmerer, and Klaus Maier-Hein. Sam. md: Zero-shot medical image segmentation capabilities of the segment anything model. In Medical Imaging with Deep Learning, short pape...

  32. [40]

    To- talsegmentator: robust segmentation of 104 anatomic struc- tures in ct images

    Jakob Wasserthal, Hanns-Christian Breit, Manfred T Meyer, Maurice Pradella, Daniel Hinck, Alexander W Sauter, Tobias Heye, Daniel T Boll, Joshy Cyriac, Shan Yang, et al. To- talsegmentator: robust segmentation of 104 anatomic struc- tures in ct images. Radiology: Artificial In...

  33. [41]

    Diffusion models for medical anomaly detection

    Julia Wolleb, Florentin Bieder, Robin Sandk ¨uhler, and Philippe C Cattin. Diffusion models for medical anomaly detection. In International Conference on Medical image computing and computer-assisted intervention, pages 35–45. Springer, 2022. 3, 5, 6, 7, 8

    2022

  34. [42]

    Diffusion models for implicit image segmentation ensembles

    Julia Wolleb, Robin Sandk ¨uhler, Florentin Bieder, Philippe Valmaggia, and Philippe C Cattin. Diffusion models for implicit image segmentation ensembles. In International Conference on Medical Imaging with Deep Learning, pages 1336–1348. PMLR, 2022. 3

    2022

  35. [43]

    Medsegdiff: Medical image segmentation with diffusion probabilistic model

    Junde Wu, Rao Fu, Huihui Fang, Yu Zhang, Yehui Yang, Haoyi Xiong, Huiying Liu, and Yanwu Xu. Medsegdiff: Medical image segmentation with diffusion probabilistic model. In Medical Imaging with Deep Learning , pages 1623–1639. PMLR, 2024

    2024

  36. [44]

    Medsegdiff-v2: Diffusion-based medical im- age segmentation with transformer

    Junde Wu, Wei Ji, Huazhu Fu, Min Xu, Yueming Jin, and Yanwu Xu. Medsegdiff-v2: Diffusion-based medical im- age segmentation with transformer. In Proceedings of the 10 AAAI Conference on Artificial Intelligence , pages 6030– 6038, 2024. 3

    2024

  37. [45]

    Freetumor: Advance tumor segmentation via large-scale tu- mor synthesis

    Linshan Wu, Jiaxin Zhuang, Xuefeng Ni, and Hao Chen. Freetumor: Advance tumor segmentation via large-scale tu- mor synthesis. arXiv preprint arXiv:2406.01264, 2024. 1, 3

    2024 arXiv

  38. [46]

    Anoddpm: Anomaly detection with de- noising diffusion probabilistic models using simplex noise

    Julian Wyatt, Adam Leach, Sebastian M Schmon, and Chris G Willcocks. Anoddpm: Anomaly detection with de- noising diffusion probabilistic models using simplex noise. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 650–656, 2022. 3

    2022

  39. [47]

    Zero- shot 3d segmentation of abdominal organs in ct scans us- ing segment anything model 2: Adapting video track- ing capabilities for 3d medical imaging

    Yosuke Yamagishi, Shouhei Hanaoka, Tomohiro Kikuchi, Takahiro Nakao, Yuta Nakamura, Yukihiro Nomura, Soichiro Miki, Takeharu Yoshikawa, and Osamu Abe. Zero- shot 3d segmentation of abdominal organs in ct scans us- ing segment anything model 2: Adapting video track- ing capabil...

    2024 arXiv

  40. [48]

    Clinical-bert: Vision-language pre-training for radiograph diagnosis and reports generation

    Bin Yan and Mingtao Pei. Clinical-bert: Vision-language pre-training for radiograph diagnosis and reports generation. In Proceedings of the AAAI Conference on Artificial Intelli- gence, pages 2982–2990, 2022. 3

    2022

  41. [49]

    Uniseg: A prompt-driven universal segmenta- tion model as well as a strong representation learner

    Yiwen Ye, Yutong Xie, Jianpeng Zhang, Ziyang Chen, and Yong Xia. Uniseg: A prompt-driven universal segmenta- tion model as well as a strong representation learner. In In- ternational Conference on Medical Image Computing and Computer-Assisted Intervention , pages 508–518. Springer,

  42. [50]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3836–3847, 2023. 5

    2023

  43. [51]

    One model to rule them all: Towards universal segmentation for medical im- ages with text prompts

    Ziheng Zhao, Yao Zhang, Chaoyi Wu, Xiaoman Zhang, Ya Zhang, Yanfeng Wang, and Weidi Xie. One model to rule them all: Towards universal segmentation for medical im- ages with text prompts. arXiv preprint arXiv:2312.17183 ,

    arXiv

  44. [2023]

    #~𝑞Masklnput 𝑥! 𝑥!

    1 11 Advancing Generalizable Tumor Segmentation with Anomaly-Aware Open-Vocabulary Attention Maps and Frozen Foundation Diffusion Models Supplementary Material A. Dataset Details Our study utilizes datasets encompassing tumors across 7 diseases and 6 organs, derived from both ...

    2023

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.