Pith. sign in

REVIEW 4 major objections 6 minor 44 references

VOILA: Complexity-Aware Universal Segmentation of CT images by Voxel Interacting with Language

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read VOILA replaces the fully connected classification head with voxel-text cosine similarity, cutting trainable parameters to 6.44M while matching or exceeding larger universal CT segmentation baselines and transferring across datasets…

desk verdict Voxel-text contrastive head for CT segmentation is a neat idea, but the paper's central efficiency claim is not backed by measurements; referee with a request for real FLOPs/timing and stronger baselines. read the letter →

arxiv 2501.03482 v1 pith:CT3737HE submitted 2025-01-07 cs.CV

Zishuo Wan , Yu Gao , Wanyuan Pang , Dawei Ding This is my paper · ORCID
classification cs.CV
keywords universalCTsegmentationvoxel-textcontrastivelearningcosinesimilarityclassificationcomplexity-awaresamplingGaussianmixturecomplexityconditionalvariationalauto-encodermedicalimagevision-languagemodel
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes VOILA, a universal CT segmentation model that classifies each voxel by measuring cosine similarity between the voxel's learned representation and text tokens describing organ names, rather than using a fully connected classification head. The authors argue this design makes the model lightweight and dataset-agnostic: after training on one multi-organ dataset, VOILA transfers to other CT datasets without any fine-tuning, and with only 6.44M trainable parameters it matches or exceeds larger baselines such as nnU-Net, UNETR++, and nnFormer on several benchmarks. To keep training efficient, they add a Complexity-Aware Sampling module that learns a heatmap of voxel difficulty from a Gaussian mixture fit to the model's own confidence, then focuses each iteration on the hardest voxels. The paper reports that this sampling both speeds convergence and improves edge-region accuracy, and that transfer without fine-tuning often improves Dice over per-dataset retraining on smaller datasets.

What carries the argument

The load-bearing objects are the voxel-text cosine similarity classifier and the complexity-aware sampler. Voxel tokens $v \in \mathbb{R}^C$ and text tokens $t_i \in \mathbb{R}^C$ are projected into the same space (dimension reduced from $C$ to $M=32$); the prediction is $\arg\max_i \cos(v, t_i)$, trained with a temperature-scaled cross-entropy loss inspired by InfoNCE plus a voxel-wise F1 loss. The CAS module assumes each voxel's classification complexity is drawn from a mixture of $g$ univariate Gaussians; a conditional VAE with the CT image as conditioning input reconstructs a pseudo-heatmap $H \in \mathbb{R}^{D\times H\times W}$ that ranks voxel difficulty. At each training step it samples the $K$ highest-complexity voxels plus $nK$ uniformly random ones (to avoid a self-reinforcing loop where the sampler only sees what it already found easy), cutting the complexity to $\Omega(M,K) = KCM + NCM + KMN$, with $K$ the sampled count and $N$ the number of classes.

What would settle it

Run VOILA on a dataset with synthetic ground-truth difficulty, such as a phantom where known boundary regions are intentionally corrupted by noise, and check whether the CAS-selected 10% of voxels concentrate on those known hard regions during training; if the overlap with the known hard set is no better than random, the complexity model is not doing what the paper claims.

Watch

Extended reading notes

Core claim

VOILA's central claim is that voxel-wise segmentation can be reformulated as a retrieval problem: instead of learning a final linear layer that maps features to class logits, the model projects voxel tokens and text tokens into a shared space and assigns each voxel the class whose text token has the highest cosine similarity. The authors show that this voxel-text interaction, combined with a voxel-wise F1 loss and enriched text prompts that mention neighboring structures, mitigates the class imbalance intrinsic to CT volumes and improves generalization because the text encoder is frozen and the prompts are dataset-agnostic. A Complexity-Aware Sampling module further reduces computation by training a conditional variational auto-encoder on pseudo-heatmaps built from the model's own confidence ranking; the CVAE then generates complexity heatmaps that select the 10% most informative voxels per iteration. Across seven public CT datasets and 400 training epochs, VOILA reaches a Dice of 92.1 on TotalSegmentator v2 with 6.44M trainable parameters, outperforms the three baselines in several multi-class settings, and, when trained on TotalSegmentator v2 only, transfers to six other datasets without fine-tuning, in some cases beating its own per-dataset retrained models.

Load-bearing premise

The load-bearing premise is that a voxel's segmentation difficulty can be modeled as a Gaussian mixture and that the model's own confidence ranking, which trains the heatmap, identifies the genuinely hardest voxels rather than amplifying the model's current blind spots.

Editorial extensions

If this is right

  • With the fully connected head replaced by cosine similarity, the number of trainable parameters no longer grows linearly with the number of classes, so adding new segmentable organs costs almost nothing in parameters.
  • Training on a high-class-count dataset such as TotalSegmentator v2 yields a model that can be applied to other CT datasets without retraining; the paper shows this transfer often keeps or improves Dice and surface-distance metrics.
  • The complexity-aware sampling reduces the voxel-level computation during training to a small fraction of full-volume processing while focusing on boundaries and hard-to-segment regions; sampling ratio 0.1 gives the best Dice in ablations.
  • Because the text encoder is frozen and class names enter as prompts, the same trained voxel encoder could in principle be re-phrased to segment unseen categories by swapping the text prompts, without changing network weights.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the Gaussian-mixture complexity model is trained on pseudo-labels from the model's own confidence; if the initial confidence is systematically biased toward a particular structure, the sampler can over-attend to that structure and neglect other hard regions. A testable extension is to seed the heatmap with a small set of ground-truth boundary voxels and measure whether the advant
  • Beyond the paper, the comparison between zero-shot transfer and per-dataset retraining is uneven because the retrained models are trained for the same fixed 400 epochs; a fairer test would compare against per-dataset models trained to convergence, which the paper does not do.
  • Beyond the paper, the method's open-vocabulary nature is not pushed; one could prompt the frozen text encoder with class names never seen in training (e.g., 'fracture' or 'calcification') and probe the voxel encoder's zero-shot behavior, which would reveal whether the shared space truly separates generic anatomical concepts or only the organ names used during training.
  • Beyond the paper, the CAS module's heatmaps are trained with reconstruction loss on the model's own complexity ranking; this self-supervision loop is broken only by random oversampling, so the method's efficiency claim ultimately depends on how informative the random oversample can be — a point the paper only partially analyzes.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes VOILA, a universal CT segmentation framework in which voxel tokens are classified by cosine similarity to text prompts instead of by a fully connected layer. The architecture consists of a convolutional voxel encoder with an FPN neck, a frozen CLIP text encoder, a voxel-language interaction module using a cross-entropy loss and a voxel-wise F1 loss, and a Complexity-Aware Sampling (CAS) module. CAS trains a conditional variational autoencoder to generate a pseudo heatmap from the model's own per-voxel confidence ranking, then samples a fraction of voxels weighted toward predicted-hard regions. Experiments are reported on seven public CT datasets, with comparisons to nnU-Net, UNETR++, and nnFormer, together with a transfer experiment from TotalSegmentator-v2 to six datasets without fine-tuning. The core claims are that VOILA achieves competitive or improved Dice with fewer trainable parameters and lower training cost, and that it generalizes across datasets without fine-tuning.

Significance. If the claims are substantiated, VOILA would be a useful contribution: a lightweight universal CT segmenter with released code, a genuinely voxel-wise contrastive formulation, and a self-supervised hard-voxel sampling mechanism that could transfer to other dense prediction tasks. The idea of replacing a fully connected classification head with text-anchored cosine similarity is timely and relevant to the universal-segmentation literature. However, the paper as written does not yet provide enough empirical or analytical support for the efficiency claim, and the comparisons are too narrow and statistically thin to establish the claimed advantage over existing universal models. The code release and the clear description of the CAS self-loop are strengths that should be acknowledged in revision.

major comments (4)
  1. [Method: Computational Complexity; Abstract; Conclusion] The claim of 'reduced computational cost during training' is not supported. Equations (2)-(5) count only the cosine-similarity/classification operations (DHW·C·N, KCM, etc.), but they omit the cost of the voxel encoder and FPN, which must produce a dense token for every voxel, the CVAE heatmap generation over the full volume, and the nK random oversampling described in 'Avoiding Self-Loop'. No FLOPs, wall-clock training time, GPU memory, or throughput figures are reported anywhere. As written, the efficiency advantage is an assertion, not an empirical or analytical result. Please either provide end-to-end training-cost measurements or explicitly scope the claim to the classifier head.
  2. [Experiments: Table 1; Table 3] The empirical comparison is too thin to support the central performance and generalization claims. Tables 1-4 report single-run average Dice scores with no standard deviations and no significance tests, so differences of 1-2 Dice points (e.g., VOILA vs nnU-Net on BTCV, or VOILA vs random sampling in Table 3) cannot be distinguished from noise. Moreover, only three baselines are compared, and the most directly related recent universal model -- CLIP-Driven Universal Model (Liu et al., 2023), discussed in the Related Work -- is not benchmarked. Please add multiple runs with variance reporting, significance tests where meaningful, and a comparison against CLIP-Driven Universal Model or another universal segmentation baseline.
  3. [Method: Complexity-Aware Self-Supervised Sampling and Avoiding Self-Loop] The CAS module has a self-referential training target: the pseudo heatmap is generated from the model's own confidence ranking, and this heatmap then selects which voxels are trained on. The paper acknowledges this in 'Avoiding Self-Loop' and mitigates it with random oversampling at ratio n, but no experiment varies n or measures how sensitive the results are to this choice. Table 3 compares CAS with random sampling at a single ratio (0.1) and with a single run, so the robustness of the hard-voxel sampling benefit is not established. Please add an ablation over n (including n=0 and large n) and, if possible, a diagnostic showing whether the learned heatmap tracks true error regions rather than merely the model's current confidence bias.
  4. [Experiments: Table 2] The 'Evaluation without Fine-tuning' experiment is not a controlled zero-shot comparison with other universal models. The left column trains on Ts-v2 (117 classes) and evaluates on each target dataset, while the right column trains and evaluates separately on each target dataset; these conditions differ in both training data and label space, so the differences cannot be attributed to the segmentation head or the CAS module alone. The claim of 'significant generalizability' is stronger than what the table shows: for WORD, Ab-1K, and AMOS the fine-tuned Dice is higher, and no significance testing is reported for the NSD/HD95 improvements. Please compare zero-shot transfer against an existing universal model under identical training data and report statistical significance.
minor comments (6)
  1. [Method: Equation (4)] The notation in the voxel-wise F1 loss is unclear: N is used both for the total number of classes in Equation (1) and for 'only the foreground classes' in Equation (4), and the summation indices for TP/FP/FN are not precisely defined. Please define the sets over which these quantities are computed.
  2. [Method: Voxel-Language Interaction] The exact text prompt templates are not specified beyond 'This region represents the {label}' and 'Neighboring structures include {STR1}, {STR2} ...'. For reproducibility, please provide the full set of prompts and the procedure for choosing neighboring structures.
  3. [Experiments: Table 3] The table format for the ablation is ambiguous: the checkmark columns do not clearly indicate which rows use the voxel-language interaction versus a fully connected layer, and the Sampling column mixes CAS, Random, and a no-sampling case. Please use explicit row labels or a clearer legend.
  4. [Experiments: Table 4] The entry 'N/A' for the no-sampling case is confusing; consider replacing it with 'No sampling' or '1.0 (full)' and clarifying that the reported Dice is the average score on Ts-v2.
  5. [Experiments: Figure 6] The F1 loss curves are described as showing 'two notable stepwise decreases', but the figure caption does not identify which curve corresponds to which method or which loss (training or validation). Please add a legend and define the plotted quantity.
  6. [General] The manuscript contains several typographical and stylistic issues: 'V oxel' in the title and abstract, 'the proposed VOILA is capable to achieve', and 'methods' in the conclusion. A careful proofread is needed.

Circularity Check

1 steps flagged · score 4.0 of 10

CAS complexity heatmap is constructed from the model's own confidence ranking, making the 'hard voxel' selection self-referential; core segmentation results are still benchmarked independently.

  1. self definitional [Method, 'Complexity-Aware Self-Supervised Sampling' -> 'Self-Supervised Training' and 'Complexity-Aware Sampling' paragraphs]
    "Assuming that all voxel tokens in the image have interacted with the text tokens, we can obtain a classification confidence for each voxel. By sorting the voxels based on this confidence, we derive a complexity order, which also reflects uncertainty. If we assign values to the voxels in descending order from 1 to 0 and then smooth this map with a Gaussian filter, we obtain a heatmap that reflects the complexity."

    The heatmap that drives voxel selection is literally a smoothed rank transform of the model's own per-voxel confidence. Sampling then chooses 'the K voxels with the highest complexity', i.e. the voxels the model currently finds least confident. The target of the CVAE is therefore defined by the model's own outputs, so the claim that CAS 'focuses on region hard to segment' is true by construction rather than by an independent complexity measure. The paper itself acknowledges the resulting 'self-reinforcing loop' and adds nK random oversampled voxels as a corrective; this confirms that the unmodified CAS selection criterion is a self-referential training target.

full rationale

The segmentation and generalization claims are supported by external benchmarks (Tables 1-2) and by ablations that compare CAS against random sampling and against no sampling (Table 3), so the central claim has independent empirical content. The only genuine circular element is the self-supervised CAS objective: complexity is defined as a smoothed inverse ranking of the model's own confidence, and sampled training voxels are then selected by that same ranking; this is a self-referential target. The paper's 'Avoiding Self-Loop' paragraph admits the loop and mitigates it with random oversampling, and the ablation shows CAS outperforms random sampling at the same ratio, which is a non-circular empirical result. The abstract/conclusion claim of 'reduced ... computational cost during training' is not circularity but an unsupported assertion: Eqs. (2)-(5) count only the classification/attention operations and omit encoder, FPN, full-resolution CVAE heatmap, and oversampling costs, and no end-to-end FLOPs, wall-time, memory, or throughput numbers are reported. There is no load-bearing self-citation: the paper cites no prior work by the same authors as the basis of its method. Accordingly, score 4 reflects one substantial self-referential training loop while the core benchmarked results remain independently grounded.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

No new physical or categorical entities are introduced; the pseudo heatmap is an internal training signal rather than an invented entity. The central claim rests on the free parameters and assumptions listed above, especially the Gaussian-mixture complexity model and the self-generated heatmap target.

free parameters (6)
  • InfoNCE temperature tau = not reported
    In Eq. (1), tau scales the cosine similarities in the softmax; it is a standard contrastive hyperparameter and is never specified in the text.
  • Number of Gaussian components g = not reported
    The CVAE models voxel complexity as a mixture of g univariate Gaussians; the value of g is not stated.
  • KL loss weight lambda = not reported
    Eq. (6) combines reconstruction loss with lambda times the KL divergence; lambda is not quantified.
  • Voxel sampling ratio = 0.1
    The CAS module samples 10% of voxels; Table 4 sweeps this ratio, so it is a tuned hyperparameter that controls the compute/accuracy trade-off.
  • Oversampling ratio n = 2
    In the self-loop avoidance strategy, nK voxels are uniformly sampled in addition to CAS-selected voxels; n is set to 2 without sensitivity analysis.
  • Token dimension M = 32
    Voxel and text tokens are reduced to dimension 32 before interaction; this manually chosen dimension drives the complexity reduction in Eq. (3).
assumptions (5)
  • ad hoc to paper Classification complexity of each voxel follows a mixture of g univariate Gaussian distributions.
    Stated in the Complexity-Aware Self-Supervised Sampling section; this distributional assumption is the foundation of the CVAE and has no independent empirical support.
  • ad hoc to paper A CVAE can predict complexity heatmaps from CT images using a self-generated target.
    The pseudo heatmap is created by ranking the model's own confidence, smoothing, and reconstructing; the paper provides no external validation that these heatmaps reflect true segmentation difficulty.
  • domain assumption Frozen CLIP text embeddings transfer to medical organ concepts.
    The text encoder is a pre-trained CLIP model trained on natural images; the paper assumes its text embeddings provide useful class prototypes for CT organs.
  • domain assumption Cosine similarity in the aligned voxel-text space yields correct classification.
    The method classifies each voxel by the argmax of cosine similarity with text tokens; this assumes the contrastive loss sufficiently aligns the two modalities.
  • ad hoc to paper Sampling voxels by predicted complexity without bias improves learning.
    The paper's core efficiency claim depends on complexity-guided sampling capturing informative voxels; the random oversampling mitigation is added precisely because this premise is fragile.

how reviews work

0 comments
Cite this review

Pith. "Pith review of VOILA: Complexity-Aware Universal Segmentation of CT images by Voxel Interacting with Language." pith.science (2026). https://pith.science/paper/CT3737HE

@misc{pith2026250103482,
  author       = {Pith},
  title        = {Pith review of: VOILA: Complexity-Aware Universal Segmentation of CT images by Voxel Interacting with Language},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CT3737HE}},
  note         = {Machine review of arXiv:2501.03482}
}
read the original abstract

Satisfactory progress has been achieved recently in universal segmentation of CT images. Following the success of vision-language methods, there is a growing trend towards utilizing text prompts and contrastive learning to develop universal segmentation models. However, there exists a significant imbalance in information density between 3D images and text prompts. Moreover, the standard fully connected layer segmentation approach faces significant challenges in handling multiple classes and exhibits poor generalizability. To address these challenges, we propose the VOxel Interacting with LAnguage method (VOILA) for universal CT image segmentation. Initially, we align voxels and language into a shared representation space and classify voxels on the basis of cosine similarity. Subsequently, we develop the Voxel-Language Interaction framework to mitigate the impact of class imbalance caused by foreground-background discrepancies and variations in target volumes. Furthermore, a Complexity-Aware Sampling method is proposed to focus on region hard to segment, achieved by generating pseudo-heatmaps from a trainable Gaussian mixture distribution. Our results indicate the proposed VOILA is capable to achieve improved performance with reduced parameters and computational cost during training. Furthermore, it demonstrates significant generalizability across diverse datasets without additional fine-tuning.

Figures

Figures reproduced from arXiv: 2501.03482 by the authors.

Figure 1
Figure 1. Overview of VOILA. (a) The overall workflow of VOILA. When taking CT images and text prompts as inputs, the [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Cosine similarities of text tokens extracted for the [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. (a) The pseudo heatmap generated by the CVAE [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Example results for heatmap generated by the [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: The visual comparison of 3 methods on Totoalsegmentator-v2. [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: The voxel-wise F1 loss curves during training. [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 27 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Beyond the Cranial Vault Segmentation Challenge. 2015. https://www.synapse.org/Synapse:syn3193805/wiki/89480. Accessed: 2024-08-11

    work page 2015

  4. [4]

    B.; Vorontsov, E.; and et al

    Bilic, P.; Christ, P.; Li, H. B.; Vorontsov, E.; and et al. 2023. The Liver Tumor Segmentation Benchmark (LiTS). Medical Image Analysis, 84: 102680

    work page 2023

  5. [5]

    I.; Ortiz, J

    Butoi, V. I.; Ortiz, J. J. G.; Ma, T.; Sabuncu, M. R.; Guttag, J.; and Dalca, A. V. 2023. UniverSeg: Universal Medical Image Segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 21438--21451

    work page 2023

  6. [6]

    Cao, H.; Wang, Y.; Chen, J.; Jiang, D.; Zhang, X.; Tian, Q.; and Wang, M. 2023. Swin-Unet: Unet-Like Pure Transformer for Medical Image Segmentation. In Karlinsky, L.; Michaeli, T.; and Nishino, K., eds., Computer Vision -- ECCV 2022 Workshops, 205--218. Cham: Springer Nature Switzerland. ISBN 978-3-031-25066-8

    work page 2023

  7. [7]

    L.; and Zhou, Y

    Chen, J.; Lu, Y.; Yu, Q.; Luo, X.; Adeli, E.; Wang, Y.; Lu, L.; Yuille, A. L.; and Zhou, Y. 2021. TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation. arXiv:2102.04306

    arXiv 2021

  8. [8]

    X.; Zheng, H.; Han, Y.; Qiu, X.; and Huang, G

    Chng, Y. X.; Zheng, H.; Han, Y.; Qiu, X.; and Huang, G. 2024. Mask Grounding for Referring Image Segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 26573--26583

    work page 2024

Show all 44 references
  1. [9]

    Gao, Y.; Xia, W.; Hu, D.; Wang, W.; and Gao, X. 2024. DeSAM: Decoupled Segment Anything Model for Generalizable Medical Image Segmentation. arXiv:2306.00499

    2024 arXiv

  2. [10]

    Gu, X.; Lin, T.-Y.; Kuo, W.; and Cui, Y. 2022. Open-vocabulary Object Detection via Vision and Language Knowledge Distillation. In International Conference on Learning Representations

    2022

  3. [11]

    R.; and Xu, D

    Hatamizadeh, A.; Nath, V.; Tang, Y.; Yang, D.; Roth, H. R.; and Xu, D. 2022. Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images. In Crimi, A.; and Bakas, S., eds., Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, ...

    2022

  4. [12]

    Huang, Z.; Wang, H.; Deng, Z.; Ye, J.; Su, Y.; Sun, H.; He, J.; Gu, Y.; Gu, L.; Zhang, S.; and Qiao, Y. 2023. STU-Net: Scalable and Transferable Medical Image Segmentation Models Empowered by Large-Scale Supervised Pre-training. arXiv:2304.06716

    2023 arXiv

  5. [13]

    F.; Kohl, S

    Isensee, F.; Jaeger, P. F.; Kohl, S. A.; Petersen, J.; and Maier-Hein, K. H. 2021. nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nature methods, 18(2): 203--211

    2021

  6. [14]

    Ji, Y.; Bai, H.; Yang, J.; Ge, C.; Zhu, Y.; Zhang, R.; Li, Z.; Zhang, L.; Ma, W.; Wan, X.; et al. 2022. AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentation. arXiv preprint arXiv:2206.08023

    2022 arXiv

  7. [15]

    H.; Cho, J.; Choi, S

    Lee, G.-E.; Kim, S. H.; Cho, J.; Choi, S. T.; and Choi, S.-I. 2023. Text-Guided Cross-Position Attention for Segmentation: Case of Medical Image. In Greenspan, H.; Madabhushi, A.; Mousavi, P.; Salcudean, S.; Duncan, J.; Syeda-Mahmood, T.; and Taylor, R., eds., Medical Image Co...

    2023

  8. [16]

    Q.; Belongie, S.; Koltun, V.; and Ranftl, R

    Li, B.; Weinberger, K. Q.; Belongie, S.; Koltun, V.; and Ranftl, R. 2022. Language-driven Semantic Segmentation. In International Conference on Learning Representations

    2022

  9. [17]

    Liu, J.; Zhang, Y.; Chen, J.-N.; Xiao, J.; Lu, Y.; A Landman, B.; Yuan, Y.; Yuille, A.; Tang, Y.; and Zhou, Z. 2023. CLIP-Driven Universal Model for Organ Segmentation and Tumor Detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 21152--21164

    2023

  10. [18]

    N.; Wang, G.; and Zhang, S

    Luo, X.; Liao, W.; Xiao, J.; Chen, J.; Song, T.; Zhang, X.; Li, K.; Metaxas, D. N.; Wang, G.; and Zhang, S. 2022. WORD: A large scale dataset, benchmark and clinical applicable study for abdominal organ segmentation from CT image. Medical Image Analysis, 82: 102642

    2022

  11. [19]

    Ma, J.; Zhang, Y.; Gu, S.; Zhu, C.; Ge, C.; Zhang, Y.; An, X.; Wang, C.; Wang, Q.; Liu, X.; Cao, S.; Zhang, Q.; Liu, S.; Wang, Y.; Li, Y.; He, J.; and Yang, X. 2022. AbdomenCT-1K: Is Abdominal Organ Segmentation a Solved Problem? IEEE Transactions on Pattern Analysis and Machi...

    2022

  12. [20]

    Moor, M.; Banerjee, O.; Abad, Z. S. H.; Krumholz, H. M.; Leskovec, J.; Topol, E. J.; and Rajpurkar, P. 2023. Foundation models for generalist medical artificial intelligence. Nature, 616(7956): 259--265

    2023

  13. [21]

    Mu, N.; Kirillov, A.; Wagner, D.; and Xie, S. 2022. SLIP: Self-supervision Meets Language-Image Pre-training. In Avidan, S.; Brostow, G.; Ciss \'e , M.; Farinella, G. M.; and Hassner, T., eds., Computer Vision -- ECCV 2022, 529--544. Cham: Springer Nature Switzerland. ISBN 978...

    2022

  14. [22]

    H.; Lao, Q.; and Li, K

    Qin, Z.; Yi, H. H.; Lao, Q.; and Li, K. 2023. Medical Image Understanding with Pretrained Vision Language Models: A Comprehensive Study. In The Eleventh International Conference on Learning Representations

    2023

  15. [23]

    W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; Krueger, G.; and Sutskever, I

    Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; Krueger, G.; and Sutskever, I. 2021. Learning Transferable Visual Models From Natural Language Supervision. In Meila, M.; and Zhang, T., eds., Proceedings of...

    2021

  16. [24]

    Ramesh, A.; Dhariwal, P.; Nichol, A.; Chu, C.; and Chen, M. 2022. Hierarchical Text-Conditional Image Generation with CLIP Latents. arXiv:2204.06125

    2022 arXiv

  17. [25]

    Rao, Y.; Zhao, W.; Chen, G.; Tang, Y.; Zhu, Z.; Huang, G.; Zhou, J.; and Lu, J. 2022. DenseCLIP: Language-Guided Dense Prediction With Context-Aware Prompting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 18082--18091

    2022

  18. [26]

    Ronneberger, O.; Fischer, P.; and Brox, T. 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation. In Navab, N.; Hornegger, J.; Wells, W. M.; and Frangi, A. F., eds., Medical Image Computing and Computer-Assisted Intervention -- MICCAI 2015, 234--241. Cham: Spri...

    2015

  19. [27]

    B.; Lu, L.; Liu, J.; and Summers, R

    Roth, H.; Farag, A.; Turkbey, E. B.; Lu, L.; Liu, J.; and Summers, R. M. 2016. Data From Pancreas-CT (Version 2) [Data set]. https://doi.org/10.7937/K9/TCIA.2016.tNB1kqBU. Accessed: 2024-08-11

    2016 doi

  20. [28]

    M.; Maaz, M.; Rasheed, H.; Khan, S.; Yang, M.-H.; and Khan, F

    Shaker, A. M.; Maaz, M.; Rasheed, H.; Khan, S.; Yang, M.-H.; and Khan, F. S. 2024. UNETR++: Delving into Efficient and Accurate 3D Medical Image Segmentation. IEEE Transactions on Medical Imaging, 1--1

    2024

  21. [29]

    Shin, G.; Xie, W.; and Albanie, S. 2022. ReCo: Retrieve and Co-segment for Zero-shot Transfer. In Koyejo, S.; Mohamed, S.; Agarwal, A.; Belgrave, D.; Cho, K.; and Oh, A., eds., Advances in Neural Information Processing Systems, volume 35, 33754--33767. Curran Associates, Inc

    2022

  22. [30]

    Singh, A.; Hu, R.; Goswami, V.; Couairon, G.; Galuba, W.; Rohrbach, M.; and Kiela, D. 2022. FLAVA: A Foundational Language and Vision Alignment Model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 15638--15650

    2022

  23. [31]

    Ulrich, C.; Isensee, F.; Wald, T.; Zenk, M.; Baumgartner, M.; and Maier-Hein, K. H. 2023. MultiTalent: A Multi-dataset Approach to Medical Image Segmentation. In Greenspan, H.; Madabhushi, A.; Mousavi, P.; Salcudean, S.; Duncan, J.; Syeda-Mahmood, T.; and Taylor, R., eds., Med...

    2023

  24. [32]

    van den Oord, A.; Li, Y.; and Vinyals, O. 2019. Representation Learning with Contrastive Predictive Coding. arXiv:1807.03748

    2019 arXiv

  25. [33]

    Wang, Z.; Lu, Y.; Li, Q.; Tao, X.; Guo, Y.; Gong, M.; and Liu, T. 2022 a . CRIS: CLIP-Driven Referring Image Segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 11686--11695

    2022

  26. [34]

    Wang, Z.; Wu, Z.; Agarwal, D.; and Sun, J. 2022 b . M ed CLIP : Contrastive Learning from Unpaired Medical Images and Text. In Goldberg, Y.; Kozareva, Z.; and Zhang, Y., eds., Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 3876--3887. A...

    2022

  27. [35]

    T.; Pradella, M.; Hinck, D.; Sauter, A

    Wasserthal, J.; Breit, H.-C.; Meyer, M. T.; Pradella, M.; Hinck, D.; Sauter, A. W.; Heye, T.; Boll, D. T.; Cyriac, J.; Yang, S.; et al. 2023. TotalSegmentator: Robust Segmentation of 104 Anatomic Structures in CT Images. Radiology: Artificial Intelligence, 5(5)

    2023

  28. [36]

    Ye, Y.; Xie, Y.; Zhang, J.; Chen, Z.; and Xia, Y. 2023. UniSeg: A Prompt-Driven Universal Segmentation Model as Well as A Strong Representation Learner. In Greenspan, H.; Madabhushi, A.; Mousavi, P.; Salcudean, S.; Duncan, J.; Syeda-Mahmood, T.; and Taylor, R., eds., Medical I...

    2023

  29. [37]

    Yi, H.; Qin, Z.; Lao, Q.; Xu, W.; Jiang, Z.; Wang, D.; Zhang, S.; and Li, K. 2023 a . Towards General Purpose Medical AI: Continual Learning Medical Foundation Model. arXiv:2303.06580

    2023 arXiv

  30. [38]

    Yi, M.; Cui, Q.; Wu, H.; Yang, C.; Yoshie, O.; and Lu, H. 2023 b . A Simple Framework for Text-Supervised Semantic Segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 7071--7080

    2023

  31. [39]

    Yu, J.; Wang, Z.; Vasudevan, V.; Yeung, L.; Seyedhosseini, M.; and Wu, Y. 2022. CoCa: Contrastive Captioners are Image-Text Foundation Models. arXiv:2205.01917

    2022 arXiv

  32. [40]

    Zhang, J.; Xie, Y.; Xia, Y.; and Shen, C. 2021. DoDNet: Learning To Segment Multi-Organ and Tumors From Multiple Partially Labeled Datasets. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 1195--1204

    2021

  33. [41]

    Zhang, K.; and Liu, D. 2023. Customized Segment Anything Model for Medical Image Segmentation. arXiv:2304.13785

    2023 arXiv

  34. [42]

    H.; Zhou, L.; Dai, X.; Yuan, L.; Li, Y.; and Gao, J

    Zhong, Y.; Yang, J.; Zhang, P.; Li, C.; Codella, N.; Li, L. H.; Zhou, L.; Dai, X.; Yuan, L.; Li, Y.; and Gao, J. 2022. RegionCLIP: Region-Based Language-Image Pretraining. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 16793--16803

    2022

  35. [43]

    Zhou, H.-Y.; Guo, J.; Zhang, Y.; Han, X.; Yu, L.; Wang, L.; and Yu, Y. 2023 a . nnFormer: Volumetric Medical Image Segmentation via a 3D Transformer. IEEE Transactions on Image Processing, 32: 4036--4045

    2023

  36. [44]

    Zhou, Z.; Lei, Y.; Zhang, B.; Liu, L.; and Liu, Y. 2023 b . ZegCLIP: Towards Adapting CLIP for Zero-Shot Semantic Segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 11175--11185

    2023

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.