Pith. sign in

REVIEW 4 major objections 5 minor 54 references

LGD: Leveraging Generative Descriptions for Zero-Shot Referring Image Segmentation

T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Zero-shot referring image segmentation improves when a multimodal language model rewrites the referring expression into attribute and surrounding descriptions, and CLIP similarity scores are linearly combined to pick the mask.

desk verdict A clean incremental pipeline that likely works, but the SOTA claim is under-supported because the two fusion weights are tuned per benchmark and no one measures the proposal-recall ceiling. read the letter →

arxiv 2504.14467 v2 pith:ENXUEW7X submitted 2025-04-20 cs.CV

Jiachen Li , Qing Xie , Renshu Gu , Jinyu Xu , Yongjian Liu , Xiaohan Yu This is my paper · ORCID
classification cs.CV
keywords zero-shotreferringimagesegmentationexpressionmulti-modallargelanguagemodelvision-languageCLIPSAMvisual-textmatchingpromptdesign
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that zero-shot referring image segmentation (locating and masking an object from a free-form phrase without any training) can be made more accurate by enriching the phrase before matching. Instead of matching the raw referring expression directly to candidate masks, LGD asks a multi-modal large language model to generate two extra texts: an attribute description of the referent and a surrounding description naming nearby distractors. These three texts are encoded with CLIP and turned into three cosine-similarity scores, and the mask with the highest weighted sum wins. If the claim holds, the method reports gains on RefCOCO, RefCOCO+, and RefCOCOg over prior zero-shot techniques with no training and only two tuned weights.

What carries the argument

The load-bearing machinery is the pair of hand-designed prompts: the attribute prompt asks the MLLM to output a caption of the form “A photo of <object> (attribute)”, and the surrounding prompt asks for “A photo of <object> surrounded by (entities)”. Noun phrases from the surrounding description are extracted with a dependency parser and treated as negative samples, making $S_{\mathrm{sur}}$ a penalty for looking like a distractor. The three CLIP scores are fused with fixed weights, $\alpha=0.5$ on RefCOCO and RefCOCO+ and $0.3$ on RefCOCOg, with $\beta=1$ everywhere, and the maximum of the resulting sum selects the mask. Every component except the two weights is frozen, so the method is a pure inference-time combination of an MLLM, CLIP, and a mask proposal network.

What would settle it

On a random sample of RefCOCO/RefCOCO+/RefCOCOg images, run SAM alone and Grounding DINO + SAM and record whether any proposal overlaps the ground-truth mask by more than 0.5 IoU, then compare that set with LGD's failures. If the failures coincide with missing proposals, the score combination is not the limiting factor; if LGD fails on images where a good proposal exists, the fusion itself is the bottleneck.

Watch

Extended reading notes

Core claim

LGD's central claim is that a linear score combination $S = S_{\mathrm{van}} + \alpha S_{\mathrm{att}} + \beta S_{\mathrm{sur}}$ followed by $M = \arg\max_m S(m)$ is enough to beat existing zero-shot referring image segmentation methods on all three standard benchmarks. $S_{\mathrm{van}}$ is the CLIP cosine similarity between the instance-level visual feature and the vanilla referring expression, $S_{\mathrm{att}}$ is the similarity to a generated attribute description, and $S_{\mathrm{sur}}$ is the negative mean similarity to noun phrases of surrounding objects. The paper reports maximum improvements of 9.97% in oIoU and 11.29% in mIoU over the previous BMS baseline, with consistent improvements over TAS as well. The ablation shows that the full three-score combination performs best, while adding only $S_{\mathrm{att}}$ can hurt on RefCOCOg because its referring expressions already carry the key attributes.

Load-bearing premise

The whole method presupposes that a good mask for the referent object is already present in the candidate set produced by the mask generator; if SAM or Grounding DINO + SAM never proposes a mask covering the target, no combination of scores can select it, so LGD's accuracy is capped by the recall of that proposal generator.

Editorial extensions

If this is right

  • Reported gains appear on every split of all three datasets, with the largest margins on RefCOCO and RefCOCO+, where referring expressions are shortest and least informative.
  • The ablation implies both generated terms are needed: $S_{\mathrm{sur}}$ helps on every dataset, while $S_{\mathrm{att}}$ alone lowers RefCOCOg performance.
  • Using an open-set detector before SAM changes the proposal set and produces the largest improvements over BMS, so proposal quality and score quality compound.
  • Because nothing is trained, the same pipeline can be pointed at any new image and phrase at inference time, limited only by the frozen components.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • My inference: the paper's reported ceiling is set by proposal recall, and LGD inherits every failure of SAM or Grounding DINO + SAM; a recall-vs-failure analysis would reveal how much headroom remains.
  • My inference: the dataset-dependent optimal $\alpha$ suggests an adaptive scheme where the model estimates how informative the raw expression already is and down-weights generated descriptions on expressions that are already detailed.
  • My inference: the same description-enrichment step should transfer to CLIP-based referring expression comprehension (boxes instead of masks) and to other frozen vision-language encoders, which the paper does not test.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes LGD, a training-free zero-shot referring image segmentation framework. Given an image and a referring expression, LGD uses an MLLM (LLaVA-v1.6-Vicuna-7B) with two hand-designed prompts to generate an attribute description of the referent and a surrounding description of contextual objects. CLIP text embeddings of these descriptions and of the original expression are compared with CLIP image embeddings of mask proposals produced by SAM (optionally with Grounding DINO). Three cosine-similarity scores, S_att, S_sur, and S_van, are combined linearly as S = S_van + αS_att + βS_sur, and the proposal with the highest score is output. Experiments on RefCOCO, RefCOCO+, and RefCOCOg report oIoU and mIoU improvements over prior zero-shot RIS methods, including TAS and BMS, with the largest gains when Grounding DINO is added.

Significance. If the reported results are robust and properly validated, the paper makes a useful contribution: it demonstrates that MLLM-generated attribute and surrounding descriptions can improve CLIP-based region-text matching for zero-shot referring image segmentation without any training. The ablation study shows that each of the three scores contributes, and the use of surrounding-object noun phrases as negative samples is a sensible extension of the TAS and BMS lines of work. The main limitations are that the headline numbers rely on per-dataset hyperparameter tuning, the method is bounded by the recall of the off-the-shelf proposal generator, and no statistical validation is provided; these issues currently weaken the strength of the state-of-the-art claim.

major comments (4)
  1. [Section 4.1, Eq. (10)] The hyperparameters α and β are tuned per benchmark: β=1 for all datasets, α=0.5 for RefCOCO and RefCOCO+, and α=0.3 for RefCOCOg, selected via the sensitivity analyses in Figs. 3–5. Because the reported SOTA numbers use these per-dataset values, the evaluation is not fully a-priori and may give LGD an advantage over baselines that did not receive the same tuning. Please state explicitly whether α was chosen using only validation splits, provide results with a single fixed α across all datasets, or otherwise justify the per-dataset tuning protocol.
  2. [Section 3.2, Eq. (11)] The final mask is always selected by argmax over the proposal set M produced by SAM or DINO+SAM, so the reported oIoU and mIoU are upper-bounded by the recall of that proposal generator. The paper does not report proposal recall, the oracle ceiling (best IoU achievable among proposals), or any analysis of failure cases where the referent is absent from M. Without this information, the claimed improvements over TAS and BMS cannot be fully attributed to the proposed scoring mechanism, because they may partly reflect the quality of the proposal set. Please add a recall/oracle analysis and discuss failure cases.
  3. [Section 3.2, Eq. (8)] The procedure for selecting negative noun phrases from the surrounding description is underspecified: the paper says CLIP is used to identify phrases 'semantically irrelevant to the referent object' but gives no threshold, criterion, or constraint on the number of negative samples. This is a free parameter that directly affects S_sur and the final result. Please specify the selection rule, report sensitivity to it, and state how the case |Tsur|=0 is handled, since Eq. (8) divides by |Tsur|.
  4. [Table 1] No error bars, confidence intervals, or multiple-run statistics are reported. Several claimed improvements are small (e.g., 0.69% mIoU on RefCOCOg versus TAS), and the MLLM generation process may be stochastic, so single-run numbers are not sufficient to establish SOTA-level claims. Please report means and standard deviations over at least a few runs, or otherwise clarify that the MLLM inference is deterministic and the reported numbers are reproducible.
minor comments (5)
  1. [Section 4.3] The heading 'Sensitive toward α and β' should be 'Sensitivity to α and β'.
  2. [Section 3.4] There is a typo in 'α andβ represent adaptive hyberparameters' — 'hyberparameters' should be 'hyperparameters'.
  3. [Figure 6] The comparison of different MLLMs is presented only as a figure; please also provide the numerical values in a table or in the caption for reproducibility.
  4. [Section 3.1] In the prompt templates, placeholders like '<input>', '<object>', '(attribute)', and '(entities)' should be clearly marked as placeholders rather than literal strings.
  5. [Section 3.2] The notation Tsur is used both for the full surrounding description and, after extraction, for the set of negative noun phrases; please use distinct notation to avoid confusion.

Circularity Check

1 steps flagged · score 6.0 of 10

The reported SOTA is partly in-sample: α and β are tuned on the same Val/TestA/TestB splits used in Table 1, while the generative-description core is independent.

  1. fitted input called prediction [Section 3.4, Eq. (10); Section 4.1 Implementation Details; Section 4.3 Sensitivity analysis (Figs. 3–5)]
    "S = S van +αS att +βS sur. whereα andβ represent adaptive hyberparameters, respectively. ... We set β = 1 for all datasets, α = 0.5 for RefCOCO and RefCOCO+, and α = 0.3 for RefCOCOg."

    The two coefficients in the final score are not derived a priori; they are chosen by sweeping α and β on the Val, TestA, and TestB splits of the same three benchmarks (Figs. 3–5), and the SOTA numbers in Table 1 are then reported using those in-sample values. Thus the headline 'new state-of-the-art' is partly a tuned evaluation: the reported oIoU/mIoU are the result of Eq. (10) with hyperparameters selected to maximize performance on the exact splits being reported, rather than a parameter-free zero-shot prediction. The MLLM-description mechanism remains an independent contribution, so the circularity is partial rather than total.

full rationale

There is no definitional circularity, no load-bearing self-citation, and no imported uniqueness theorem; the method relies on frozen CLIP, SAM, and LLaVA components and external benchmarks. The one substantive circularity is that the two hyperparameters α and β in the final score are tuned on the same Val/TestA/TestB partitions that are later reported as SOTA, making the quantitative headline in-sample for those coefficients. The proposal-recall ceiling noted by the skeptic is a real correctness risk, but it is not a form of circularity: it concerns external component quality rather than the derivation reducing to its inputs. Because the central contribution (generative attribute and surrounding descriptions plus three matching scores) has independent content, the score is 6 rather than higher.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The method introduces no new physical or learned entities; it assembles off-the-shelf components (CLIP, SAM, LLaVA, Spacy). The main introduced degrees of freedom are the two combination weights (tuned per benchmark) and the unspecified negative-sample filtering rule, both of which affect the reported results.

free parameters (3)
  • alpha (weight for S_att) = 0.5 for RefCOCO and RefCOCO+, 0.3 for RefCOCOg
    Chosen per dataset via sensitivity analysis (Figs. 3-5); final results use these tuned values.
  • beta (weight for S_sur) = 1.0 for all datasets
    Determined from sensitivity analysis on the benchmarks, reported in Section 4.1 and Figs. 3-5.
  • Negative sample relevance threshold = Not reported
    The paper says CLIP identifies noun phrases 'semantically irrelevant' to the referent, but no threshold or selection rule is specified; this is a hidden free parameter.
assumptions (4)
  • domain assumption CLIP similarity is a valid ranking signal for region-text matching.
    All three scores S_att, S_sur, and S_van rely on CLIP cosine similarity between instance-level visual features and text features; if CLIP cannot rank regions correctly, the whole method fails. Assumed in Section 3.3.
  • domain assumption The MLLM-generated attribute and surrounding descriptions are accurate and beneficial.
    The method depends on LLaVA producing captions that correctly identify the referent's attributes and surrounding objects; incorrect generated text would mislead the scores. Assumed in Section 3.1 and used throughout.
  • domain assumption SAM (or Grounding DINO+SAM) proposals cover the referent object.
    The final selection is argmax over the proposal set M; if the referent is not in M, the method cannot find it. Stated in Section 3.2 and Eq. 11.
  • domain assumption Spacy noun phrase extraction followed by CLIP relevance filtering yields valid negative samples.
    The negative samples in S_sur are chosen by extracting noun phrases from Tsur and filtering with CLIP; the reliability of this filtering is not validated. Described in Section 3.2.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LGD: Leveraging Generative Descriptions for Zero-Shot Referring Image Segmentation." pith.science (2026). https://pith.science/paper/ENXUEW7X

@misc{pith2026250414467,
  author       = {Pith},
  title        = {Pith review of: LGD: Leveraging Generative Descriptions for Zero-Shot Referring Image Segmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ENXUEW7X}},
  note         = {Machine review of arXiv:2504.14467}
}
read the original abstract

Zero-shot referring image segmentation aims to locate and segment the target region based on a referring expression, with the primary challenge of aligning and matching semantics across visual and textual modalities without training. Previous works address this challenge by utilizing Vision-Language Models and mask proposal networks for region-text matching. However, this paradigm may lead to incorrect target localization due to the inherent ambiguity and diversity of free-form referring expressions. To alleviate this issue, we present LGD (Leveraging Generative Descriptions), a framework that utilizes the advanced language generation capabilities of Multi-Modal Large Language Models to enhance region-text matching performance in Vision-Language Models. Specifically, we first design two kinds of prompts, the attribute prompt and the surrounding prompt, to guide the Multi-Modal Large Language Models in generating descriptions related to the crucial attributes of the referent object and the details of surrounding objects, referred to as attribute description and surrounding description, respectively. Secondly, three visual-text matching scores are introduced to evaluate the similarity between instance-level visual features and textual features, which determines the mask most associated with the referring expression. The proposed method achieves new state-of-the-art performance on three public datasets RefCOCO, RefCOCO+ and RefCOCOg, with maximum improvements of 9.97% in oIoU and 11.29% in mIoU compared to previous methods.

Figures

Figures reproduced from arXiv: 2504.14467 by the authors.

Figure 1
Figure 1. An illustration demonstrating the inherent ambiguity and diversity of free-form referring expressions. (a) The referring expression describes the clothing of the referent object without explicitly mentioning the man wearing a white shirt. (b) The referring expression highlights the action of the referent object rather than explicitly describing the woman making the call. (c) The referring expression conveys the regi… view at source ↗
Figure 2
Figure 2. The pipeline of LGD. We construct the attribute prompt and surrounding prompt by combining instructions and referring expressions. Given the input image and the different prompts, the MLLMs gener￾ates the attribute description and surrounding description. After extracting features from the image and text using CLIP, we calculate three visual-text matching scores and obtain the most relevant mask by linearly combinin… view at source ↗
Figure 3
Figure 3. Sensitive toward α and β on RefCOCO. characterization of the referent object. However, when α becomes too large, perfor￾mance starts to decline, due to an overemphasis on crucial attributes, which causes other important information to be overlooked. Similarly, increasing β demonstrates that surrounding descriptions help distinguish the referent object from irrelevant ob￾jects. However, excessively large values of β … view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Sensitive toward α and β on RefCOCO+. (a) oIoU on RefCOCOg Val(u) (b) oIoU on RefCOCOg Test(u) (c) oIoU on RefCOCOg Test(g) (d) mIoU on RefCOCOg Val(u) (e) mIoU on RefCOCOg Test(u) (f) mIoU on RefCOCOg Test(g) [PITH_FULL_IMAGE:figures/full_fig_p018_4.png]
Figure 5
Figure 5. Figure 5: Sensitive toward α and β on RefCOCOg. 17 [PITH_FULL_IMAGE:figures/full_fig_p018_5.png]
Figure 6
Figure 6. Figure 6: Effects of Different MLLMs. RefCOCO and RefCOCO+, while a smaller value α = 0.3 works better on RefCOCOg, possibly because of the increased complexity of its expressions. These results confirm that combining attribute and surrounding descriptions in a balanced way is e…
Figure 7
Figure 7. Figure 7: Qualitative results of different methods. From left to right are the Ground Truth and the vi￾sualization results of Cropping, Global-Local CLIP (SAM), TAS, and LGD. The last column presents the referring expression associated with the referent object, along with attrib…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

54 extracted references · 45 canonical work pages

  1. [1]

    Zhang, M

    Y . Zhang, M. A. Mazurowski, Convolutional neural networks rarely learn shape for semantic segmentation, Pattern Recognition 146 (2024) 110018

    work page 2024

  2. [2]

    Jiang, X

    J. Jiang, X. He, X. Zhu, W. Wang, J. Liu, Cgvit: Cross-image groupvit for zero- shot semantic segmentation, Pattern Recognition 164 (2025) 111505

    work page 2025

  3. [3]

    M. Liao, W. Li, C. Yin, Y . Jin, Y . Peng, Concept-guided domain generalization for semantic segmentation, Pattern Recognition 164 (2025) 111550

    work page 2025

  4. [4]

    Zheng, X

    H. Zheng, X. Lin, H. Liang, B. Zhou, Y . Liang, Region-aware mutual rela- tional knowledge distillation for semantic segmentation, Pattern Recognition 161 (2025) 111319

    work page 2025

  5. [5]

    Ye-Bin, D

    M. Ye-Bin, D. Choi, Y . Kwon, J. Kim, T.-H. Oh, Eninst: Enhancing weakly- supervised low-shot instance segmentation, Pattern Recognition 145 (2024) 109888

    work page 2024

  6. [6]

    Ouyang, Z

    W. Ouyang, Z. Xu, J. Xu, Q. Wang, Y . Xu, Mixingmask: A contour-aware ap- proach for joint object detection and instance segmentation, Pattern Recognition 155 (2024) 110620

    work page 2024

  7. [7]

    C. Wang, G. Wang, Q. Zhang, P. Guo, W. Liu, X. Wang, Openinst: A simple query-based method for open-world instance segmentation, Pattern Recognition 153 (2024) 110570

    work page 2024

  8. [8]

    T. Chu, W. Cai, Q. Liu, Learning panoptic segmentation through feature discrim- inability, Pattern Recognition 122 (2022) 108240

    work page 2022

Show all 54 references
  1. [9]

    Liang, B

    F. Liang, B. Wu, X. Dai, K. Li, Y . Zhao, H. Zhang, P. Zhang, P. Vajda, D. Marculescu, Open-vocabulary semantic segmentation with mask-adapted clip, in: 2023 IEEE /CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, pp. 7061–7070. 21

    2023

  2. [10]

    J. Chen, Y . Shen, J. Gao, J. Liu, X. Liu, Language-based image editing with recurrent attentive models, in: 2018 IEEE /CVF Conference on Computer Vision and Pattern Recognition, 2018, pp. 8721–8729

    2018

  3. [11]

    J. Gu, E. Stefani, Q. Wu, J. Thomason, X. Wang, Vision-and-language navigation: A survey of tasks, methods, and future directions, in: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (V olume 1: Long Papers), 2022, pp. 7606–7623

    2022

  4. [12]

    Marino, M

    K. Marino, M. Rastegari, A. Farhadi, R. Mottaghi, Ok-vqa: A visual question answering benchmark requiring external knowledge, in: 2019 IEEE /CVF Con- ference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 3190– 3199

    2019

  5. [13]

    Z. Wang, Y . Lu, Q. Li, X. Tao, Y . Guo, M. Gong, T. Liu, Cris: Clip-driven refer- ring image segmentation, in: 2022 IEEE /CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 11676–11685

    2022

  6. [14]

    H. Ding, C. Liu, S. Wang, X. Jiang, Vlt: Vision-language transformer and query generation for referring segmentation, IEEE Transactions on Pattern Analysis and Machine Intelligence 45 (6) (2023) 7900–7916

    2023

  7. [15]

    Z. Yang, J. Wang, Y . Tang, K. Chen, H. Zhao, P. H. Torr, Lavt: Language-aware vision transformer for referring image segmentation, in: 2022 IEEE /CVF Con- ference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 18134– 18144

    2022

  8. [16]

    J. Tang, G. Zheng, C. Shi, S. Yang, Contrastive grouping with transformer for re- ferring image segmentation, in: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, pp. 23570–23580

    2023

  9. [17]

    D. Kim, N. Kim, C. Lan, S. Kwak, Shatter and gather: Learning referring image segmentation with text supervision, in: 2023 IEEE/CVF International Conference on Computer Vision (ICCV), 2023, pp. 15501–15511. 22

    2023

  10. [18]

    J. Lee, S. Lee, J. Nam, S. Yu, J. Do, T. Taghavi, Weakly supervised refer- ring image segmentation with intra-chunk and inter-chunk consistency, in: 2023 IEEE/CVF International Conference on Computer Vision (ICCV), 2023, pp. 21813–21824

    2023

  11. [19]

    Q. Dai, S. Yang, Curriculum point prompting for weakly-supervised referring image segmentation, in: 2024 IEEE /CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024, pp. 13711–13722

    2024

  12. [20]

    Radford, J

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al., Learning transferable visual models from natural language supervision, in: International conference on machine learning, PMLR, 2021, pp. 8748–8763

    2021

  13. [21]

    Kirillov, E

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Lo, P. Doll´ar, R. Girshick, Segment anything, in: 2023 IEEE/CVF International Conference on Computer Vision (ICCV), 2023, pp. 3992–4003

    2023

  14. [22]

    X. Wang, Z. Yu, S. De Mello, J. Kautz, A. Anandkumar, C. Shen, J. M. Alvarez, Freesolo: Learning to segment objects without annotations, in: 2022 IEEE /CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 14156–14166

    2022

  15. [23]

    S. Yu, P. H. Seo, J. Son, Zero-shot referring image segmentation with global- local context features, in: 2023 IEEE /CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, pp. 19456–19465

    2023

  16. [24]

    Y . Suo, L. Zhu, Y . Yang, Text augmented spatial aware zero-shot referring im- age segmentation, in: Findings of the Association for Computational Linguistics: EMNLP 2023, 2023, pp. 1032–1043

    2023

  17. [25]

    W. Li, C. Pang, W. Nie, H. Tian, A.-A. Liu, Bidirectional mask selection for zero- shot referring image segmentation, IEEE Transactions on Circuits and Systems for Video Technology (2024) 1–1. 23

    2024

  18. [26]

    J. Li, D. Li, S. Savarese, S. Hoi, Blip-2: Bootstrapping language-image pre- training with frozen image encoders and large language models, in: International conference on machine learning, PMLR, 2023, pp. 19730–19742

    2023

  19. [27]

    S. Sun, R. Li, P. Torr, X. Gu, S. Li, Clip as rnn: Segment countless visual concepts without training endeavor, in: 2024 IEEE /CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024, pp. 13171–13182

    2024

  20. [28]

    J. Li, G. Shakhnarovich, R. A. Yeh, Adapting clip for phrase localization without further training, arXiv preprint arXiv:2204.03647 (2022)

    2022 arXiv

  21. [29]

    X. Liu, S. Huang, Y . Kang, H. Chen, D. Wang, Vgdi ffzero: Text-to-image dif- fusion models can be zero-shot visual grounders, in: ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2024, pp. 2765–2769

    2024

  22. [30]

    G. Feng, Z. Hu, L. Zhang, H. Lu, Encoder fusion network with co-attention em- bedding for referring image segmentation, in: 2021 IEEE /CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp. 15501–15510

    2021

  23. [31]

    H. Liu, C. Li, Q. Wu, Y . J. Lee, Visual instruction tuning, Advances in neural information processing systems 36 (2024)

    2024

  24. [32]

    D. Zhu, J. Chen, X. Shen, X. Li, M. Elhoseiny, Minigpt-4: Enhancing vision- language understanding with advanced large language models, arXiv preprint arXiv:2304.10592 (2023)

    2023 arXiv

  25. [33]

    X. Lai, Z. Tian, Y . Chen, Y . Li, Y . Yuan, S. Liu, J. Jia, Lisa: Reasoning segmenta- tion via large language model, in: Proceedings of the IEEE /CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 9579–9589

    2024

  26. [34]

    Z. Xia, D. Han, Y . Han, X. Pan, S. Song, G. Huang, Gsva: Generalized segmen- tation via multimodal large language models, in: Proceedings of the IEEE /CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 3858–3869. 24

    2024

  27. [35]

    Z. Ren, Z. Huang, Y . Wei, Y . Zhao, D. Fu, J. Feng, X. Jin, Pixellm: Pixel reason- ing with large multimodal model, in: Proceedings of the IEEE /CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024, pp. 26374–26383

    2024

  28. [36]

    Chen, W.-H

    Y .-C. Chen, W.-H. Li, C. Sun, Y .-C. F. Wang, C.-S. Chen, Sam4mllm: Enhance multi-modal large language model for referring expression segmentation, in: Eu- ropean Conference on Computer Vision, Springer, 2025, pp. 323–340

    2025

  29. [37]

    Zhang, X

    T. Zhang, X. Li, H. Fei, H. Yuan, S. Wu, S. Ji, C. C. Loy, S. Yan, Omg- llava: Bridging image-level, object-level, pixel-level reasoning and understand- ing, arXiv preprint arXiv:2406.19389 (2024)

    2024 arXiv

  30. [38]

    K. Zhou, J. Yang, C. C. Loy, Z. Liu, Learning to prompt for vision-language models, International Journal of Computer Vision 130 (9) (2022) 2337–2348

    2022

  31. [39]

    K. Zhou, J. Yang, C. C. Loy, Z. Liu, Conditional prompt learning for vision- language models, in: 2022 IEEE /CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), 2022, pp. 16795–16804

    2022

  32. [40]

    J. Ding, N. Xue, G. Xia, D. Dai, Decoupling zero-shot semantic segmentation, in: 2022 IEEE /CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 11573–11582

    2022

  33. [41]

    C. Jia, Y . Yang, Y . Xia, Y .-T. Chen, Z. Parekh, H. Pham, Q. Le, Y .-H. Sung, Z. Li, T. Duerig, Scaling up visual and vision-language representation learning with noisy text supervision, in: International conference on machine learning, PMLR, 2021, pp. 4904–4916

    2021

  34. [42]

    J. Jang, C. Kong, D. Jeon, S. Kim, N. Kwak, Unifying vision-language represen- tation space with single-tower transformer, in: Proceedings of the AAAI Confer- ence on Artificial Intelligence, V ol. 37, 2023, pp. 980–988

    2023

  35. [43]

    Z. Sun, Y . Fang, T. Wu, P. Zhang, Y . Zang, S. Kong, Y . Xiong, D. Lin, J. Wang, Alpha-clip: A clip model focusing on wherever you want, in: 2024 IEEE /CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024, pp. 13019–13029. 25

    2024

  36. [44]

    Honnibal, M

    M. Honnibal, M. Johnson, An improved non-monotonic transition system for de- pendency parsing, in: Proceedings of the 2015 conference on empirical methods in natural language processing, 2015, pp. 1373–1378

    2015

  37. [45]

    V . K. Nagaraja, V . I. Morariu, L. S. Davis, Modeling context between objects for referring expression understanding, in: Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Pro- ceedings, Part IV 14, Springer, 2016, pp. 792–807

    2016

  38. [46]

    Kazemzadeh, V

    S. Kazemzadeh, V . Ordonez, M. Matten, T. Berg, Referitgame: Referring to ob- jects in photographs of natural scenes, in: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), 2014, pp. 787–798

    2014

  39. [47]

    J. Mao, J. Huang, A. Toshev, O. Camburu, A. L. Yuille, K. Murphy, Generation and comprehension of unambiguous object descriptions, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 11–20

    2016

  40. [48]

    R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, D. Batra, Grad- cam: visual explanations from deep networks via gradient-based localization, In- ternational journal of computer vision 128 (2020) 336–359

    2020

  41. [49]

    C. Zhou, C. C. Loy, B. Dai, Extract free dense labels from clip, in: European Conference on Computer Vision, Springer, 2022, pp. 696–712

    2022

  42. [50]

    L. Xu, M. H. Huang, X. Shang, Z. Yuan, Y . Sun, J. Liu, Meta compositional referring expression segmentation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 19478–19487

    2023

  43. [51]

    Eiras, K

    F. Eiras, K. Oksuz, A. Bibi, P. H. Torr, P. K. Dokania, Segment, select, cor- rect: A framework for weakly-supervised referring segmentation, arXiv preprint arXiv:2310.13479 (2023)

    2023 arXiv

  44. [52]

    S. Liu, Z. Zeng, T. Ren, F. Li, H. Zhang, J. Yang, Q. Jiang, C. Li, J. Yang, H. Su, et al., Grounding dino: Marrying dino with grounded pre-training for open-set object detection, in: European Conference on Computer Vision, Springer, 2025, pp. 38–55. 26

    2025

  45. [53]

    H. Liu, C. Li, Y . Li, Y . J. Lee, Improved baselines with visual instruction tuning, in: Proceedings of the IEEE /CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 26296–26306

    2024

  46. [54]

    J. Bai, S. Bai, Y . Chu, Z. Cui, K. Dang, X. Deng, Y . Fan, W. Ge, Y . Han, F. Huang, et al., Qwen technical report, arXiv preprint arXiv:2309.16609 (2023). 27

    2023 arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.