Pith. sign in

REVIEW 4 major objections 5 minor 83 references

Fine-Grained Image-Text Correspondence with Cost Aggregation for Open-Vocabulary Part Segmentation

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that separating object-level and part-level cost volumes, with a compositional loss and DINO structural guidance, sets a new state of the art for open-vocabulary part segmentation.

desk verdict PartCATSeg is a strong empirical extension of CAT-Seg/PartCLIPSeg with clean ablations and code, but the headline gains rest on one protocol with thin baselines; the compositional-loss mechanism deserves a targeted stress test before I'd trust the numbers fully. read the letter →

arxiv 2501.09688 v2 pith:O7TPRGBC submitted 2025-01-16 cs.CV

Jiho Choi , Seonho Lee , Minhyun Lee , Seungho Lee , Hyunjung Shim This is my paper · ORCID
classification cs.CV
keywords open-vocabularypartsegmentationcostaggregationimage-textcorrespondencecompositionallossDINOstructuralguidancezero-shotCLIPembeddingsharmonicIoU
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Open-vocabulary part segmentation aims to label fine-grained object parts for categories never seen during training, but part-level image-text alignment is weak and models lack structural understanding of how parts relate to objects. This paper proposes PartCATSeg, which builds on cost aggregation to explicitly compute and refine separate cost volumes for objects and parts, then fuses them into object-specific part costs. It adds a compositional loss that enforces the inductive bias that parts collectively compose their object, compensating for scarce part annotations, and injects DINO features as structural guidance during aggregation. The paper reports large improvements over prior methods on Pascal-Part-116, ADE20K-Part-234, and PartImageNet, with the largest gains on unseen classes. If these results hold, PartCATSeg provides a new baseline for robust generalization in open-vocabulary part segmentation.

What carries the argument

The central object is the disentangled cost aggregation architecture: two separate image-text cost volumes, one for object class names and one for part class names, are each refined by spatial- and class-aggregation transformers before being concatenated and projected into an object-specific part cost volume that is refined again. The compositional loss is the key identity: at each pixel, the softmax distribution over object classes is compared by Jensen-Shannon divergence to the distribution obtained by summing part-class probabilities according to a fixed mapping from each part to its object, injecting the prior that parts compose the whole. Structural guidance is the third mechanism: DINO's self-supervised features are fed into the query and key of the spatial aggregation transformers, supplying geometric and boundary information that complements CLIP's semantic alignment.

What would settle it

Take a trained PartCATSeg model and randomly permute the part-to-object mapping used in the compositional loss during fine-tuning while keeping all other components fixed; if the harmonic-mean IoU on unseen parts does not drop, the compositional loss's claimed mechanism is not the source of the gains. Alternatively, train on a dataset with artificially incomplete part annotations (e.g., dropping a subset of part masks) and compare with and without the compositional loss.

Watch

Extended reading notes

Core claim

PartCATSeg establishes that disentangling the image-text matching signal by object and part levels, rather than pooling them into one cost volume, markedly improves fine-grained part alignment. Three mechanisms carry the result: separate spatial and class aggregation transformers refine an object cost volume and a part cost volume independently; a projection fuses the refined features into an object-specific part cost volume aligned with names like 'bird's head'; and a compositional loss forces the softmax distribution over object classes to match the sum of part-class probabilities mapped through a fixed part-to-object correspondence. DINO features are appended to the cost volume during spatial aggregation, providing geometric structure that helps delineate part boundaries. On the three main benchmarks the method reports harmonic-mean IoU gains of 15.10, 12.81, and 27.79 points over the second-best method in the Pred-All setting, with the largest relative improvements on unseen part classes.

Load-bearing premise

The compositional loss assumes a fixed, correct mapping from each part class to its object class; if training masks omit some parts, part classes are shared across objects, or the mapping is mis-specified, the enforced part-to-object consistency could distort rather than sharpen part predictions.

Editorial extensions

If this is right

  • If the reported gains replicate, cost aggregation becomes a validated mechanism for fine-grained open-vocabulary tasks beyond whole-object segmentation.
  • The compositional loss provides a template for leveraging scarce part annotations by transferring supervision from object-level signals to part-level predictions.
  • DINO features, already known for semantic correspondence, are shown to be directly useful inside an image-text cost volume for improving part boundaries.
  • The Pred-All evaluation numbers suggest that full part segmentation without any object-mask oracle is closer to being practical, since unseen-class performance is no longer far below seen-class performance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The fixed part-to-object mapping in the compositional loss could likely be replaced by a learned or probabilistic mapping, which may extend the method to datasets where part classes are shared across object categories.
  • The disentangled cost-volume design might transfer to other fine-grained recognition problems, such as attribute segmentation or human parsing, where object context and fine-grained labels have a similar part-whole hierarchy.
  • A testable extension is to apply the same three mechanisms to instance-level part segmentation by combining them with an off-the-shelf open-vocabulary instance segmentation module, which the authors themselves flag as future work.
  • The reported ablation suggests that the structural guidance is more useful at the part level than the object level, implying that the benefit comes from intra-object geometry rather than background separation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. PartCATSeg extends CAT-Seg's cost-aggregation machinery to open-vocabulary part segmentation. The method computes separate object-level and part-level image-text cost volumes (Sec. 3.3), fuses them into an object-specific part cost volume (Sec. 3.4), injects DINOv2 features as structural guidance into spatial aggregation (Sec. 3.6), and trains with an auxiliary compositional loss that forces the softmax distribution over object classes to equal the part-class distribution aggregated through a fixed part-to-object mapping M (Sec. 3.5, Eqs. 13-14). Experiments on Pascal-Part-116, ADE20K-Part-234, and PartImageNet report large gains over prior methods in both Pred-All and Oracle-Obj settings, especially on unseen classes, with component-level ablations for the compositional loss and structural guidance.

Significance. If the reported numbers are accurate, PartCATSeg is a clear advance for zero-shot part segmentation: the Pred-All h-IoU gains of 15.1, 12.8, and 27.8 points over the strongest available baseline on the three main benchmarks are large and qualitatively visible in the supplied figures. The strengths are the clear architectural decomposition, the release of code, the honest limitations section, and the component-level ablations that separately attribute gains to cost aggregation, DINO guidance, and the compositional loss. The main novelty is moderate--the framework is a direct extension of the authors' earlier PartCLIPSeg and borrows CAT-Seg's cost aggregation--but the empirical margin is substantial. The central risk is that the largest gains come from the compositional loss, whose premise (complete, one-to-one part-to-object coverage at every pixel) is only approximately true in the benchmarks; this needs targeted experiments before the specific mechanism can be considered established.

major comments (4)
  1. [Sec. 3.5, Eqs. (11)-(14)] The compositional loss is applied at every spatial location and equates PObj(i) with the sum of part probabilities over M^{-1}(o). The paper never states whether CObj includes a background/void class. If it does not, background pixels have an arbitrary object-class distribution, and Lcomp will push part probabilities to match it, encouraging hallucinated parts. If it does, Eq. (13) assigns zero aggregated mass to the background class because no part maps to it, while PObj may carry large background mass, so the equality cannot hold except by suppressing the background. Moreover, the benchmarks do not annotate all parts in every image (e.g., a side-view aeroplane on Pascal-Part-116 can have no visible engine), so the premise 'parts collectively compose the object' is violated for many pixels. The paper should specify the class set, mask the loss to pixels that have a valid part annotation (or to a foreground object mask), and report the fraction of pixels affected. Without this, the gradient from Lcomp may distort part confidences on unseen classes rather than sharpen them.
  2. [Sec. 4.3, Table 5] The ablation of Lcomp toggles the loss on/off and compares softmax vs L1 normalization, but it does not test the sensitivity of Eq. (13) to the fixed mapping M or to incomplete part annotations. The mapping M is described as 'predefined' and is never justified or varied; part classes that are shared across multiple object classes or part sets that only partially cover an object would change the aggregated target. Since the largest reported gains are on unseen classes (Tables 1-3), it is important to ablate (a) a masked version of Lcomp that ignores unannotated pixels, (b) a random or perturbed mapping M as a negative control, and (c) per-image part-coverage statistics. This would determine whether the loss sharpens genuine part-object composition or simply suppresses low-confidence predictions.
  3. [Sec. 4.3, Table 6] In the structural guidance ablation, applying DINO guidance to both TSA_Obj and TSA_Part (the full model) yields seen mIoU 52.62, which is 3.66 points lower than TSA_Part alone (56.28), while unseen mIoU rises from 36.67 to 40.51. The text states that applying guidance at both levels gives 'further improvements, especially in unseen classes,' but it does not mention the clear regression on seen classes. This trade-off is important because it indicates the object-level guidance may hurt the very classes used for training, and it is not explained. If the full model is used, the paper should discuss the reason or adjust the design; otherwise, the claim that both-level guidance is beneficial is unsupported.
  4. [Sec. 4.2, Table 2 and general] The main results tables report single-run numbers without standard deviations or significance tests, and on ADE20K-Part-234 Pred-All the only existing baseline is PartCLIPSeg, whose seen mIoU is only 14.15. This makes the 12.81-point h-IoU claim for that benchmark rest on a single comparison point and on one trial. The difference between the two Lcomp variants in Table 5 (2.49 points in Pred-All h-IoU) may be within run-to-run noise. I ask the authors to provide at least three seeds with mean +/- std for the main tables and for the two ablation tables, and to add any available Pred-All numbers for PartGLEE and VLPart on ADE20K-Part-234, or to qualify the 'state of the art' claim for that setting.
minor comments (5)
  1. [Sec. 3.4, Eq. (8)] The symbol FObj-Part is used both for the concatenation-projection output and, later in the same paragraph, for the convolved object-aware cost feature; please use distinct names (e.g., F^init and F^conv) to avoid confusion.
  2. [Sec. 4.3, Table 5] The L1-normalization variant is not defined in the text; please give the exact form of Lcomp-L1 (e.g., normalization by the sum of absolute values rather than softmax) so the comparison is reproducible.
  3. [Sec. 2 and Sec. 4.1] There is a duplicate citation '[8, 8]' in the list after 'Part Segmentation', and the dataset name is inconsistently spelled as 'PascalPart' in Section 4.1.
  4. [Appendix F.4] The main text says the OOD split has 109 training/19 validation/30 test classes, but Table A7 lists only 109 base and 19 novel object classes; please reconcile the numbers and specify which 30 classes are used for test.
  5. [Sec. 4.3, final paragraph] The comparison 'applying structural guidance at the part level yields more h-IoU increases than at the object level' should also state that the object-level-only row already includes the object-specific part-level guidance, per the table note, otherwise the attribution is unclear.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reported gains rest on held-out benchmarks and an auxiliary regularizer, not on fitting to the target or on self-referential definitions.

full rationale

PartCATSeg's derivation chain is self-contained. The disentangled cost aggregation (Eqs. 4-10) is an architectural extension of CAT-Seg's cost volume; the compositional loss (Eqs. 11-14) is a training-time regularizer that enforces a part-to-object consistency prior, not a quantity that is later reported as a prediction. The held-out novel classes are not used in fitting; the paper follows the standard OVPS split and reports h-IoU on Pascal-Part-116, ADE20K-Part-234, and PartImageNet against external baselines (ZSSeg+, CLIPSeg, CAT-Seg, PartGLEE). The only self-citation with methodological weight is the Pred-All protocol from the authors' prior PartCLIPSeg [14]; that protocol is a benchmark definition, not an unverified theorem, and the Oracle-Obj results against non-self baselines provide independent support for the central SOTA claim. Incrementalism relative to PartCLIPSeg is a novelty concern, not a circularity: no equation reduces to its own input, no fitted parameter is renamed as a prediction, and no load-bearing premise depends solely on a self-citation. Therefore the paper's empirical claims are not circular.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim depends on standard CLIP and DINOv2 feature assumptions, plus two paper-specific inductive biases: the compositional prior and the fixed part-to-object mapping. There are no newly invented physical or conceptual entities. The loss weights and training hyperparameters are fitted on the validation set, and the DINO feature source is unspecified.

free parameters (5)
  • lambda_Obj, lambda_Part, lambda_comp = 1.0, 1.0, 1.0
    Tuned on the training set (Appendix B.5) to balance the disentanglement and compositional losses; results may shift with different weights and no sensitivity analysis is shown.
  • learning rate = 1e-4
    From CAT-Seg; chosen by standard practice, affects convergence but not the method's logic.
  • training iterations = 20,000
    Checkpoint selection uses best validation performance; the reported numbers depend on this schedule.
  • batch size = 8
    Standard choice; affects optimization but not the core claim.
  • DINOv2 feature source for structural guidance
    The paper does not state which layer or spatial resolution of DINOv2 is concatenated in Eq. 16; this is an unspecified design choice that could affect the gains.
assumptions (5)
  • domain assumption The CLIP embedding space supports meaningful image-text alignment for object-specific part phrases such as 'cat's paw'.
    The cost-volume approach (Eqs. 1, 4, 9) assumes frozen CLIP text and image embeddings carry usable part-level semantics; Figure 2 motivates this but also shows part-level alignment is weak, so the method amplifies a weak signal.
  • domain assumption DINOv2 features encode local spatial structure and inter-part geometry that is useful when injected into spatial aggregation.
    Section 3.6 and the PCA visualizations in Figure 6 are the only support; there is no quantitative analysis of what DINO encodes for parts.
  • ad hoc to paper Parts collectively compose the object, expressed as equality of the object-class distribution and the aggregated part-class distribution in Lcomp.
    This is an inductive bias introduced for training (Eqs. 13-14); it is not derived and may be violated when object pixels have no annotated part.
  • ad hoc to paper The mapping M from part classes to object classes is known, fixed, and complete for all training classes.
    Section 3.5 defines M deterministically from the class list; if a part name is shared across objects or the mapping is incomplete, the compositional loss is mis-specified.
  • domain assumption Dataset splits and baseline numbers inherited from PartCLIPSeg and OV-PARTS are correct.
    The evaluation relies on splits described in Appendix F and numbers reproduced from references [14, 70]; the paper does not re-run all baselines under identical code.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Fine-Grained Image-Text Correspondence with Cost Aggregation for Open-Vocabulary Part Segmentation." pith.science (2026). https://pith.science/paper/O7TPRGBC

@misc{pith2026250109688,
  author       = {Pith},
  title        = {Pith review of: Fine-Grained Image-Text Correspondence with Cost Aggregation for Open-Vocabulary Part Segmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/O7TPRGBC}},
  note         = {Machine review of arXiv:2501.09688}
}
read the original abstract

Open-Vocabulary Part Segmentation (OVPS) is an emerging field for recognizing fine-grained parts in unseen categories. We identify two primary challenges in OVPS: (1) the difficulty in aligning part-level image-text correspondence, and (2) the lack of structural understanding in segmenting object parts. To address these issues, we propose PartCATSeg, a novel framework that integrates object-aware part-level cost aggregation, compositional loss, and structural guidance from DINO. Our approach employs a disentangled cost aggregation strategy that handles object and part-level costs separately, enhancing the precision of part-level segmentation. We also introduce a compositional loss to better capture part-object relationships, compensating for the limited part annotations. Additionally, structural guidance from DINO features improves boundary delineation and inter-part understanding. Extensive experiments on Pascal-Part-116, ADE20K-Part-234, and PartImageNet datasets demonstrate that our method significantly outperforms state-of-the-art approaches, setting a new baseline for robust generalization to unseen part categories.

Figures

Figures reproduced from arXiv: 2501.09688 by the authors.

Figure 1
Figure 1. The proposed PartCATSeg exploits part-level and object-level image-text correspondence using cost aggregation, enhancing [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 3
Figure 3. Challenges of Current OVPS. (a, b) Prediction results of state-of-the-art OVSS [13] and OVPS [14] methods. Due to a lack of structural understanding, these methods often produce in￾correct OVPS predictions, such as predicting a “sheep’s neck” as being larger than its “torso” or placing a “bird’s beak” and a “bird’s neck” at the tip of its “tail”. objects. As shown in [PITH_FULL_IMAGE:figures/full_fig_p002_3.png] view at source ↗
Figure 4
Figure 4. The Overall Architecture of PartCATSeg. dex of the image patch and n ∈ R T denotes the index of the text token corresponding to category classes T. Cost Aggregation. The computed cost volume is refined through a process known as cost aggregation [7, 11, 13, 29, 30], which employs self-attention modules [65]. This pro￾cess consists of two main components: the spatial aggre￾gation transformer (T SA) and the class aggr… view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: Compositional Loss. The loss function Lcomp guides learning by ensuring that the aggregated class distribution of ob￾ject parts aligns closely with the overall class distribution of the object, enhancing the consistency between part-level and object￾level representatio…
Figure 6
Figure 6. Figure 6: Visualization of Frozen DINO PCA results. (a) With￾out mask: The PCA visualization shows that DINO can separate object features from the background. (b) With mask: highlights consistent feature values across structurally similar parts of the objects, indicating strong …
Figure 7
Figure 7. Figure 7: Qualitative evaluation of zero-shot part segmentation on Pascal-Part-116 in the [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

83 extracted references · 52 canonical work pages

  1. [3]

    Paco: a novel procrustes application to co- phylogenetic analysis

    Juan Antonio Balbuena, Ra ´ul M ´ıguez-Lozano, and Isabel Blasco-Costa. Paco: a novel procrustes application to co- phylogenetic analysis. PloS one, 8(4):e61048, 2013. 1, 3

    2013

  2. [1]

    Label-embedding for image classification

    Zeynep Akata, Florent Perronnin, Zaid Harchaoui, and Cordelia Schmid. Label-embedding for image classification. IEEE transactions on pattern analysis and machine intelli- gence, 38(7):1425–1438, 2015. 1

    2015

  3. [2]

    Evaluation of output embeddings for fine-grained image classification

    Zeynep Akata, Scott Reed, Daniel Walter, Honglak Lee, and Bernt Schiele. Evaluation of output embeddings for fine-grained image classification. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 2927–2936, 2015. 1

    2015

  4. [4]

    Zero-shot semantic segmentation.Advances in Neural Information Processing Systems, 32, 2019

    Maxime Bucher, Tuan-Hung Vu, Matthieu Cord, and Patrick P´erez. Zero-shot semantic segmentation.Advances in Neural Information Processing Systems, 32, 2019. 1

    work page 2019

  5. [5]

    Emerg- ing properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers. In Pro- ceedings of the IEEE/CVF international conference on com- puter vision, pages 9650–9660, 2021. 2, 6

    work page 2021

  6. [6]

    Rethinking atrous convolution for semantic image segmentation

    Liang-Chieh Chen. Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv:1706.05587, 2017. 1, 3

    arXiv 2017

  7. [7]

    Costformer: Cost transformer for cost aggregation in multi-view stereo

    Weitao Chen, Hongbin Xu, Zhipeng Zhou, Yang Liu, Baigui Sun, Wenxiong Kang, and Xuansong Xie. Costformer: Cost transformer for cost aggregation in multi-view stereo. arXiv preprint arXiv:2305.10320, 2023. 3, 4

    arXiv 2023

  8. [8]

    Detect what you can: Detecting and representing objects using holistic mod- els and body parts

    Xianjie Chen, Roozbeh Mottaghi, Xiaobai Liu, Sanja Fi- dler, Raquel Urtasun, and Alan Yuille. Detect what you can: Detecting and representing objects using holistic mod- els and body parts. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 1971–1978,

    1971

Show all 83 references
  1. [9]

    Per- pixel classification is not all you need for semantic segmen- tation

    Bowen Cheng, Alex Schwing, and Alexander Kirillov. Per- pixel classification is not all you need for semantic segmen- tation. Advances in neural information processing systems , 34:17864–17875, 2021. 1

    2021

  2. [10]

    Masked-attention mask transformer for universal image segmentation

    Bowen Cheng, Ishan Misra, Alexander G Schwing, Alexan- der Kirillov, and Rohit Girdhar. Masked-attention mask transformer for universal image segmentation. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1290–1299, 2022. 1, 3

    2022

  3. [11]

    Cats: Cost aggre- gation transformers for visual correspondence

    Seokju Cho, Sunghwan Hong, Sangryul Jeon, Yunsung Lee, Kwanghoon Sohn, and Seungryong Kim. Cats: Cost aggre- gation transformers for visual correspondence. Advances in Neural Information Processing Systems , 34:9011–9023,

  4. [12]

    Cats++: Boosting cost aggregation with convolutions and transformers

    Seokju Cho, Sunghwan Hong, and Seungryong Kim. Cats++: Boosting cost aggregation with convolutions and transformers. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(6):7174–7194, 2022. 3

    2022

  5. [13]

    Cat-seg: Cost aggregation for open-vocabulary semantic segmentation

    Seokju Cho, Heeseong Shin, Sunghwan Hong, Seungjun An, Seungjun Lee, Anurag Arnab, Paul Hongsuck Seo, and Seungryong Kim. Cat-seg: Cost aggregation for open-vocabulary semantic segmentation. arXiv preprint arXiv:2303.11797, 2023. 2, 3, 4, 6, 7, 8, 1, 9, 10

    2023 arXiv

  6. [14]

    Understanding multi-granularity for open-vocabulary part segmentation

    Jiho Choi, Seonho Lee, Seungho Lee, Minhyun Lee, and Hyunjung Shim. Understanding multi-granularity for open-vocabulary part segmentation. arXiv preprint arXiv:2406.11384, 2024. 1, 2, 3, 4, 6, 7, 8, 5, 9, 10

    2024 arXiv

  7. [15]

    Unsupervised part discovery from con- trastive reconstruction

    Subhabrata Choudhury, Iro Laina, Christian Rupprecht, and Andrea Vedaldi. Unsupervised part discovery from con- trastive reconstruction. Advances in Neural Information Pro- cessing Systems, 34:28104–28118, 2021. 3

    2021

  8. [16]

    Histograms of oriented gra- dients for human detection

    Navneet Dalal and Bill Triggs. Histograms of oriented gra- dients for human detection. In 2005 IEEE computer soci- ety conference on computer vision and pattern recognition (CVPR’05), pages 886–893. Ieee, 2005. 3

    2005

  9. [17]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009. 6, 5

    2009

  10. [18]

    Open- vocabulary universal image segmentation with maskclip

    Zheng Ding, Jieke Wang, and Zhuowen Tu. Open- vocabulary universal image segmentation with maskclip. arXiv preprint arXiv:2208.08984, 2022. 3

    2022 arXiv

  11. [19]

    Generalized jensen- shannon divergence loss for learning with noisy labels

    Erik Englesson and Hossein Azizpour. Generalized jensen- shannon divergence loss for learning with noisy labels. Advances in Neural Information Processing Systems , 34: 30284–30297, 2021. 5

    2021

  12. [20]

    De- vise: A deep visual-semantic embedding model

    Andrea Frome, Greg S Corrado, Jon Shlens, Samy Bengio, Jeff Dean, Marc’Aurelio Ranzato, and Tomas Mikolov. De- vise: A deep visual-semantic embedding model. Advances in neural information processing systems, 26, 2013. 1

    2013

  13. [21]

    Scal- ing open-vocabulary image segmentation with image-level labels

    Golnaz Ghiasi, Xiuye Gu, Yin Cui, and Tsung-Yi Lin. Scal- ing open-vocabulary image segmentation with image-level labels. In European Conference on Computer Vision, pages 540–557. Springer, 2022. 3

    2022

  14. [22]

    Open- vocabulary object detection via vision and language knowl- edge distillation

    Xiuye Gu, Tsung-Yi Lin, Weicheng Kuo, and Yin Cui. Open- vocabulary object detection via vision and language knowl- edge distillation. arXiv preprint arXiv:2104.13921, 2021. 1, 3

    2021 arXiv

  15. [23]

    Aˆ 3: Accelerating attention mechanisms in neural networks with approximation

    Tae Jun Ham, Sung Jun Jung, Seonghak Kim, Young H Oh, Yeonhong Park, Yoonho Song, Jung-Hun Park, Sanghee Lee, Kyoung Park, Jae W Lee, et al. Aˆ 3: Accelerating attention mechanisms in neural networks with approximation. In2020 IEEE International Symposium on High Performance C...

    2020

  16. [24]

    Global knowledge calibration for fast open-vocabulary segmentation

    Kunyang Han, Yong Liu, Jun Hao Liew, Henghui Ding, Ji- ajun Liu, Yitong Wang, Yansong Tang, Yujiu Yang, Jiashi 9 Feng, Yao Zhao, et al. Global knowledge calibration for fast open-vocabulary segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Visio...

    2023

  17. [25]

    Partimagenet: A large, high- quality dataset of parts

    Ju He, Shuo Yang, Shaokang Yang, Adam Kortylewski, Xi- aoding Yuan, Jie-Neng Chen, Shuai Liu, Cheng Yang, Qi- hang Yu, and Alan Yuille. Partimagenet: A large, high- quality dataset of parts. In European Conference on Com- puter Vision, pages 128–145. Springer, 2022. 1, 2, 3, 6, 5, 7

    2022

  18. [26]

    Compositor: Bottom-up clustering and compositing for robust part and object segmentation

    Ju He, Jieneng Chen, Ming-Xian Lin, Qihang Yu, and Alan L Yuille. Compositor: Bottom-up clustering and compositing for robust part and object segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11259–11268, 2023. 3

    2023

  19. [27]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 2

    2016

  20. [28]

    Mask r-cnn

    Kaiming He, Georgia Gkioxari, Piotr Doll ´ar, and Ross Gir- shick. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 2961–2969, 2017. 1, 3

    2017

  21. [29]

    Cost aggregation with 4d convolutional swin transformer for few-shot segmentation

    Sunghwan Hong, Seokju Cho, Jisu Nam, Stephen Lin, and Seungryong Kim. Cost aggregation with 4d convolutional swin transformer for few-shot segmentation. In European Conference on Computer Vision , pages 108–126. Springer,

  22. [30]

    Unifying feature and cost aggregation with transformers for semantic and visual correspondence

    Sunghwan Hong, Seokju Cho, Seungryong Kim, and Stephen Lin. Unifying feature and cost aggregation with transformers for semantic and visual correspondence. In The Twelfth International Conference on Learning Representa- tions, 2024. 3, 4

    2024

  23. [31]

    Fast cost-volume filtering for visual correspondence and beyond

    Asmaa Hosni, Christoph Rhemann, Michael Bleyer, Carsten Rother, and Margrit Gelautz. Fast cost-volume filtering for visual correspondence and beyond. IEEE transactions on pattern analysis and machine intelligence , 35(2):504–511,

  24. [32]

    Scops: Self-supervised co-part segmentation

    Wei-Chih Hung, Varun Jampani, Sifei Liu, Pavlo Molchanov, Ming-Hsuan Yang, and Jan Kautz. Scops: Self-supervised co-part segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 869–878, 2019. 3

    2019

  25. [33]

    Scaling up visual and vision-language representa- tion learning with noisy text supervision

    Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representa- tion learning with noisy text supervision. In International conference on machine learning, pages 4904–4916. PMLR,

  26. [34]

    Salad: Part-level latent diffusion for 3d shape gen- eration and manipulation

    Juil Koo, Seungwoo Yoo, Minh Hieu Nguyen, and Minhyuk Sung. Salad: Part-level latent diffusion for 3d shape gen- eration and manipulation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 14441– 14451, 2023. 2

    2023

  27. [35]

    Language-driven semantic seg- mentation

    Boyi Li, Kilian Q Weinberger, Serge Belongie, Vladlen Koltun, and Ren ´e Ranftl. Language-driven semantic seg- mentation. arXiv preprint arXiv:2201.03546, 2022. 3

    2022 arXiv

  28. [36]

    Mask dino: Towards a unified transformer-based framework for object detection and segmentation

    Feng Li, Hao Zhang, Huaizhe Xu, Shilong Liu, Lei Zhang, Lionel M Ni, and Heung-Yeung Shum. Mask dino: Towards a unified transformer-based framework for object detection and segmentation. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pa...

    2023

  29. [37]

    Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In Interna- tional conference on machine learning, pages 12888–12900. PMLR, 2022. 2, 3

    2022

  30. [38]

    Partglee: A foundation model for recognizing and pars- ing any objects

    Junyi Li, Junfeng Wu, Weizhi Zhao, Song Bai, and Xiang Bai. Partglee: A foundation model for recognizing and pars- ing any objects. arXiv preprint arXiv:2407.16696, 2024. 1, 2, 3, 4, 7

    2024 arXiv

  31. [39]

    Clip surgery for better explainability with enhancement in open- vocabulary tasks

    Yi Li, Hualiang Wang, Yiqun Duan, and Xiaomeng Li. Clip surgery for better explainability with enhancement in open- vocabulary tasks. arXiv preprint arXiv:2304.05653 , 2023. 2

    2023 arXiv

  32. [40]

    Open-vocabulary semantic segmentation with mask-adapted clip

    Feng Liang, Bichen Wu, Xiaoliang Dai, Kunpeng Li, Yinan Zhao, Hang Zhang, Peizhao Zhang, Peter Vajda, and Diana Marculescu. Open-vocabulary semantic segmentation with mask-adapted clip. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pag...

    2023

  33. [41]

    Divergence measures based on the shannon en- tropy

    Jianhua Lin. Divergence measures based on the shannon en- tropy. IEEE Transactions on Information theory, 37(1):145– 151, 1991. 5

    1991

  34. [42]

    Editgan: High-precision semantic image editing

    Huan Ling, Karsten Kreis, Daiqing Li, Seung Wook Kim, Antonio Torralba, and Sanja Fidler. Editgan: High-precision semantic image editing. Advances in Neural Information Processing Systems, 34:16331–16345, 2021. 2

    2021

  35. [43]

    Sift flow: Dense correspondence across scenes and its applications

    Ce Liu, Jenny Yuen, and Antonio Torralba. Sift flow: Dense correspondence across scenes and its applications. IEEE transactions on pattern analysis and machine intelligence , 33(5):978–994, 2010. 3

    2010

  36. [44]

    Se- mantic correspondence as an optimal transport problem

    Yanbin Liu, Linchao Zhu, Makoto Yamada, and Yi Yang. Se- mantic correspondence as an optimal transport problem. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 4463–4472, 2020. 3

    2020

  37. [45]

    Open-vocabulary segmentation with semantic-assisted calibration

    Yong Liu, Sule Bai, Guanbin Li, Yitong Wang, and Yansong Tang. Open-vocabulary segmentation with semantic-assisted calibration. arXiv preprint arXiv:2312.04089, 2023. 3

    2023 arXiv

  38. [46]

    3d part guided image editing for fine-grained object understanding

    Zongdai Liu, Feixiang Lu, Peng Wang, Hui Miao, Liangjun Zhang, Ruigang Yang, and Bin Zhou. 3d part guided image editing for fine-grained object understanding. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11336–11345, 2020. 2

    2020

  39. [47]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012–10022, 2021. 4, 1

    2021

  40. [48]

    Fully convolutional networks for semantic segmentation

    Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. In Pro- ceedings of the IEEE conference on computer vision and pat- tern recognition, pages 3431–3440, 2015. 1 10

    2015

  41. [49]

    Decoupled weight decay regularization

    I Loshchilov. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017. 1

    2017 arXiv

  42. [50]

    Image segmenta- tion using text and image prompts

    Timo L ¨uddecke and Alexander Ecker. Image segmenta- tion using text and image prompts. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7086–7096, 2022. 3, 7, 2

    2022

  43. [51]

    Wordnet: a lexical database for english

    George A Miller. Wordnet: a lexical database for english. Communications of the ACM, 38(11):39–41, 1995. 5

    1995

  44. [52]

    Hyperpixel flow: Semantic correspondence with multi-layer neural features

    Juhong Min, Jongmin Lee, Jean Ponce, and Minsu Cho. Hyperpixel flow: Semantic correspondence with multi-layer neural features. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision , pages 3395–3404,

  45. [53]

    Learning to compose hypercolumns for visual correspon- dence

    Juhong Min, Jongmin Lee, Jean Ponce, and Minsu Cho. Learning to compose hypercolumns for visual correspon- dence. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceed- ings, Part XV 16, pages 346–363. Springer, 2020. 3

    2020

  46. [54]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 2, 3, 6, 1

    2023 arXiv

  47. [55]

    To- wards open-world segmentation of parts

    Tai-Yu Pan, Qing Liu, Wei-Lun Chao, and Brian Price. To- wards open-world segmentation of parts. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15392–15401, 2023. 2

    2023

  48. [56]

    Computational optimal transport: With applications to data science.Foundations and Trends® in Machine Learning, 11(5-6):355–607, 2019

    Gabriel Peyr ´e, Marco Cuturi, et al. Computational optimal transport: With applications to data science.Foundations and Trends® in Machine Learning, 11(5-6):355–607, 2019. 3

    2019

  49. [57]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  50. [58]

    Neighbourhood con- sensus networks

    Ignacio Rocco, Mircea Cimpoi, Relja Arandjelovi ´c, Akihiko Torii, Tomas Pajdla, and Josef Sivic. Neighbourhood con- sensus networks. Advances in neural information processing systems, 31, 2018. 3

    2018

  51. [59]

    U- net: Convolutional networks for biomedical image segmen- tation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U- net: Convolutional networks for biomedical image segmen- tation. In Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, pa...

    2015

  52. [60]

    Pwc-net: Cnns for optical flow using pyramid, warping, and cost volume

    Deqing Sun, Xiaodong Yang, Ming-Yu Liu, and Jan Kautz. Pwc-net: Cnns for optical flow using pyramid, warping, and cost volume. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 8934–8943,

  53. [61]

    Going denser with open-vocabulary part segmentation

    Peize Sun, Shoufa Chen, Chenchen Zhu, Fanyi Xiao, Ping Luo, Saining Xie, and Zhicheng Yan. Going denser with open-vocabulary part segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 15453–15465, 2023. 1, 2, 3, 6, 7, 8

    2023

  54. [62]

    Parts and wholes in face recognition

    James W Tanaka and Martha J Farah. Parts and wholes in face recognition. The Quarterly Journal of Experimental Psychology Section A, 46(2):225–245, 1993. 2

    1993

  55. [63]

    Glu- net: Global-local universal network for dense flow and corre- spondences

    Prune Truong, Martin Danelljan, and Radu Timofte. Glu- net: Global-local universal network for dense flow and corre- spondences. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6258–6268,

  56. [64]

    Pdisconet: Semantically consistent part discovery for fine-grained recognition

    Robert van der Klis, Stephan Alaniz, Massimiliano Mancini, Cassio F Dantas, Dino Ienco, Zeynep Akata, and Diego Mar- cos. Pdisconet: Semantically consistent part discovery for fine-grained recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision,...

    2023

  57. [65]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017. 4

    2017

  58. [66]

    Optimal transport: old and new

    C ´edric Villani et al. Optimal transport: old and new . Springer, 2009. 3

    2009

  59. [67]

    In- structpart: Affordance-based part segmentation from lan- guage instruction

    Zifu Wan, Yaqi Xie, Ce Zhang, Zhiqiu Lin, Zihan Wang, Si- mon Stepputtis, Deva Ramanan, and Katia P Sycara. In- structpart: Affordance-based part segmentation from lan- guage instruction. In AAAI-2024 Workshop on Public Sector LLMs: Algorithmic and Sociotechnical Design, 2024. 2

    2024

  60. [68]

    Pyramid vision transformer: A versatile backbone for dense prediction without convolutions

    Wenhai Wang, Enze Xie, Xiang Li, Deng-Ping Fan, Kaitao Song, Ding Liang, Tong Lu, Ping Luo, and Ling Shao. Pyramid vision transformer: A versatile backbone for dense prediction without convolutions. In Proceedings of the IEEE/CVF international conference on computer vision , p...

    2021

  61. [69]

    Instancediffusion: Instance- level control for image generation

    Xudong Wang, Trevor Darrell, Sai Saketh Rambhatla, Ro- hit Girdhar, and Ishan Misra. Instancediffusion: Instance- level control for image generation. arXiv preprint arXiv:2402.03290, 2024. 2

    2024 arXiv

  62. [70]

    Ov-parts: Towards open- vocabulary part segmentation

    Meng Wei, Xiaoyu Yue, Wenwei Zhang, Shu Kong, Xi- hui Liu, and Jiangmiao Pang. Ov-parts: Towards open- vocabulary part segmentation. Advances in Neural Informa- tion Processing Systems, 36, 2024. 1, 2, 3, 6, 7, 8, 4, 5, 9, 10

    2024

  63. [71]

    Semantic projection network for zero-and few-label semantic segmentation

    Yongqin Xian, Subhabrata Choudhury, Yang He, Bernt Schiele, and Zeynep Akata. Semantic projection network for zero-and few-label semantic segmentation. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8256–8265, 2019. 3

    2019

  64. [72]

    Sed: A simple encoder-decoder for open-vocabulary semantic segmentation

    Bin Xie, Jiale Cao, Jin Xie, Fahad Shahbaz Khan, and Yanwei Pang. Sed: A simple encoder-decoder for open-vocabulary semantic segmentation. arXiv preprint arXiv:2311.15537, 2023. 3

    2023 arXiv

  65. [73]

    Open-vocabulary panop- tic segmentation with text-to-image diffusion models

    Jiarui Xu, Sifei Liu, Arash Vahdat, Wonmin Byeon, Xiao- long Wang, and Shalini De Mello. Open-vocabulary panop- tic segmentation with text-to-image diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 2955–2966, 2023. 3

    2023

  66. [74]

    A simple baseline for open- vocabulary semantic segmentation with pre-trained vision- 11 language model

    Mengde Xu, Zheng Zhang, Fangyun Wei, Yutong Lin, Yue Cao, Han Hu, and Xiang Bai. A simple baseline for open- vocabulary semantic segmentation with pre-trained vision- 11 language model. In European Conference on Computer Vi- sion, pages 736–753. Springer, 2022. 1, 3, 7, 2

    2022

  67. [75]

    Homerobot: Open-vocabulary mobile manipulation

    Sriram Yenamandra, Arun Ramachandran, Karmesh Yadav, Austin Wang, Mukul Khanna, Theophile Gervet, Tsung-Yen Yang, Vidhi Jain, Alexander William Clegg, John Turner, et al. Homerobot: Open-vocabulary mobile manipulation. arXiv preprint arXiv:2306.11565, 2023. 2

    2023 arXiv

  68. [76]

    C2fnas: Coarse- to-fine neural architecture search for 3d medical image seg- mentation

    Qihang Yu, Dong Yang, Holger Roth, Yutong Bai, Yixiao Zhang, Alan L Yuille, and Daguang Xu. C2fnas: Coarse- to-fine neural architecture search for 3d medical image seg- mentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 4126...

    2020

  69. [77]

    Convolutions die hard: Open-vocabulary seg- mentation with single frozen convolutional clip.Advances in Neural Information Processing Systems, 36, 2024

    Qihang Yu, Ju He, Xueqing Deng, Xiaohui Shen, and Liang- Chieh Chen. Convolutions die hard: Open-vocabulary seg- mentation with single frozen convolutional clip.Advances in Neural Information Processing Systems, 36, 2024. 3

    2024

  70. [78]

    Open-vocabulary object detection using captions

    Alireza Zareian, Kevin Dela Rosa, Derek Hao Hu, and Shih- Fu Chang. Open-vocabulary object detection using captions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14393–14402, 2021. 3

    2021

  71. [79]

    Open vocabulary scene parsing

    Hang Zhao, Xavier Puig, Bolei Zhou, Sanja Fidler, and Anto- nio Torralba. Open vocabulary scene parsing. InProceedings of the IEEE International Conference on Computer Vision , pages 2002–2010, 2017. 1, 3

    2002

  72. [80]

    Scene parsing through ade20k dataset

    Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, and Antonio Torralba. Scene parsing through ade20k dataset. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 633–641,

  73. [81]

    Extract free dense labels from clip

    Chong Zhou, Chen Change Loy, and Bo Dai. Extract free dense labels from clip. In European Conference on Com- puter Vision, pages 696–712. Springer, 2022. 1, 3

    2022

  74. [82]

    Learning to prompt for vision-language models

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-language models. In- ternational Journal of Computer Vision, 130(9):2337–2348,

  75. [83]

    person” (b) “person’s eye

    Ziqin Zhou, Yinjie Lei, Bowen Zhang, Lingqiao Liu, and Yi- fan Liu. Zegclip: Towards adapting clip for zero-shot se- mantic segmentation. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 11175–11185, 2023. 3 12 Contents A . Limitati...

    2023

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.