Pith. sign in

REVIEW 4 major objections 5 minor 82 references

Deformable Attentive Visual Enhancement for Referring Segmentation Using Vision-Language Model

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SegVLM claims that adding deformable convolutions, SE blocks, residual shortcuts, and a hybrid RAF loss to a CLIP-based baseline lifts referring segmentation accuracy by more than 10 IoU points on PhraseCut.

desk verdict SegVLM is a plausible engineering extension of CRIS, but the final model's precision scores disagree across Table 1 and Table 2, so the headline gain is not verifiable as written. read the letter →

arxiv 2505.19242 v1 pith:I3UAGCDN submitted 2025-05-25 cs.CV

Alaa Dalaq , Muzammil Behzad This is my paper
classification cs.CV
keywords referringimagesegmentationvision-languagemodelCLIPdeformableconvolutionsqueeze-and-excitationhybridlossPhraseCutdatasetcross-modalalignment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a CLIP-based referring segmentation model can be made substantially more accurate on hard, compositional queries by adding three inexpensive modules to its projection head: deformable convolutions, squeeze-and-excitation blocks, and residual shortcuts, trained with a hybrid loss it calls RAF. The authors build SegVLM on the CRIS baseline and report 53.87 mean IoU on the PhraseCut test set, up from 43.57 for the baseline and slightly above MDETR ENB3. The ablation study says RAF loss contributes the largest single gain, with deformable convolutions plus residuals and then SE blocks adding further improvements at every IoU threshold. If the numbers hold, the practical message is that a 63-million-parameter model with modest modifications can match or beat systems that are far heavier on language-guided segmentation.

What carries the argument

The load-bearing mechanism is the projection module inserted between the fused cross-modal features and the segmentation head, together with the RAF training objective. In that module, a deformable convolution learns offsets $\Delta p$ and applies $\phi(\mathbf{x}, \Delta p)$ so the receptive field adapts to object shape; a squeeze-and-excitation block computes channel weights $s = \sigma(W_2 \, \delta(W_1 \, \mathrm{GAP}(\mathbf{x}_{dcf})))$ and rescales $\mathbf{x}_{dcf} = s \odot \mathbf{x}_{dcf}$; a residual shortcut $\mathbf{x}_{res} = F_{residual}(U(\mathbf{x}))$ is added to preserve fine spatial detail. The RAF loss combines binary cross-entropy, focal loss, and adaptive dice loss as $\mathcal{L}_{RAF} = \lambda_1 \mathcal{L}_{BCE} + \lambda_2 \mathcal{L}_{focal} + \lambda_3 \mathcal{L}_{AdaptiveDice}$, with pixel weights $\alpha_i = |p_i - y_i|$ or $(1-p_i)^\gamma$ to emphasize uncertain and misclassified pixels.

What would settle it

Run the official CRIS code on PhraseCut with the paper's training protocol (50 epochs, batch 64, Adam at 1e-4, 416x416 input); the claimed +10.30 IoU gain is genuine only if this baseline lands near 43.57 and if removing the RAF loss from SegVLM produces roughly the 4.7-point drop reported in Table 2.

Watch

Extended reading notes

Core claim

SegVLM's central claim is that a carefully placed set of visual enhancement modules and a composite loss can overcome CRIS's failure mode on the PhraseCut dataset, which has long compositional expressions and fine object distinctions. On the PhraseCut test set, SegVLM reaches 53.87 IoU, 58.23 Prec@50, 40.81 Prec@70, and 12.64 Prec@90, and its precision at each IoU threshold is higher than the listed comparison models that report those values. The ablation in Table 2 shows a monotone climb from the 43.57-IoU baseline to 48.25 with RAF loss, to 51.34 with deformable convolutions and residual connections added, and to 53.87 with the SE block. The authors present the improvements as complementary: the loss handles class imbalance and hard-to-classify pixels, deformable convolutions adapt to irregular object boundaries, residual connections preserve fine spatial structure, and SE recalibration emphasizes task-relevant channels.

Load-bearing premise

The paper assumes its re-implementation of CRIS under its training protocol is faithful, so the 10.30-point IoU gap in Table 2 is attributable to the new modules rather than to a weaker-than-standard baseline.

Editorial extensions

If this is right

  • The ablation order in Table 2 implies the three components are complementary, since each one raises IoU on top of the previous additions and no single module accounts for the full 10.30-point gain.
  • The rise in Prec@90 from 4.10 to 10.30 implies the modifications improve boundary precision, not merely coarse overlap.
  • Because the changes sit in the projection module and the loss rather than in the CLIP encoders, the recipe should transfer to other CLIP-based referring segmentation pipelines unchanged.
  • The reported 40-45 FPS at 416x416 input with about 63 million parameters implies the accuracy gain does not require a slower or much larger model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the cross-dataset claims on COCO and VOC rest on hand-written prompts with no reported metrics, so a fixed prompt template and a quantitative IoU evaluation would turn that observation into a testable claim.
  • Beyond the paper: the reported segmentation of a bottle's reflection in Fig. 8 suggests the projector may be learning a shape-agnostic objectness signal; a targeted probe that measures mask overlap on mirrored regions across many images could tell whether this behavior is systematic.
  • Beyond the paper: the RAF loss contributes the largest single step in the ablation (about 4.7 IoU points), which makes it a plausible drop-in component for other imbalanced dense-prediction tasks such as medical or remote-sensing segmentation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The manuscript proposes SegVLM, an extension of the CRIS model for referring image segmentation. SegVLM augments CRIS with squeeze-and-excitation blocks, deformable convolutions with residual connections, and a referring-aware fusion (RAF) loss that combines binary cross-entropy, focal loss, and an adaptive Dice term. The central empirical claim is a PhraseCut test IoU of 53.87 against 43.57 for the CRIS baseline, with each added component producing monotone gains in Table 2 and the final model outperforming several prior methods in Table 1. The paper also presents qualitative generalization examples on COCO and VOC. The main evidence, however, is internally inconsistent: the same final model is assigned different precision values in the two central tables, and the CRIS baseline is not validated against the official CRIS result or any reproducibility artifacts.

Significance. If the empirical claims were reliable, SegVLM would offer a simple and lightweight recipe for improving CRIS-style referring segmentation on PhraseCut by combining known architectural modules (SE, deformable convolution, residual connections) with a composite loss. The ablation design is clear and the monotone IoU progression in Table 2 is encouraging. However, the paper ships no code, checkpoints, or raw prediction files; Table 1 and Table 2 report different precision numbers for the identical final model; and the baseline is not compared with the published CRIS performance on PhraseCut. As submitted, the main quantitative conclusion is not verifiable, and the novelty is incremental because the individual components are established in prior work.

major comments (4)
  1. [§5.1, Tables 1 and 2] The identical final SegVLM configuration is reported with different numbers in the two central tables: Table 1 gives IoU 53.87, Prec@50 58.23, Prec@70 40.81, and Prec@90 12.64, while Table 2 gives IoU 53.87, P@50 57.10, P@70 36.00, and P@90 10.30. Because Table 1 supports the state-of-the-art comparison and Table 2 is the sole evidence for the incremental-ablation claim, at least one set of numbers is not reproducible from the described protocol. No code, checkpoints, or prediction files are provided, so a reader cannot determine which is correct. This internal inconsistency is load-bearing and blocks acceptance of the paper's main empirical claim.
  2. [§1.2 and §4.2] The paper states that CRIS is used as the baseline, but it neither cites nor reports the official CRIS result on PhraseCut. The reported baseline IoU of 43.57 is far below the external methods listed in Table 1 (MDETR ENB3 at 53.70 and GROUNDHOG at 54.50) and, by the standards of the referring-segmentation literature, appears substantially lower than a properly trained CRIS. Without a validation of the baseline implementation against the original CRIS numbers, the +10.30 IoU gain in Table 2 cannot be attributed to the proposed modules; it may reflect an under-trained or otherwise weakened baseline. This is central to the paper's claim that SegVLM improves CRIS.
  3. [§5.1, Table 1] The text claims that SegVLM "outperforms all baselines across precision metrics," but Table 1 lists GROUNDHOG with only an IoU value of 54.50, which is higher than SegVLM's 53.87, and no precision entries are provided for GROUNDHOG. The state-of-the-art comparison is therefore incomplete, and the stated claim is directly contradicted by the table as printed. The authors should supply GROUNDHOG's precision metrics or substantially qualify the claim.
  4. [§3.2] The architecture section promises a "language-modulated dynamic convolution module that generates instance-specific convolution kernels conditioned on the input referring expression," but no such module is defined in §3.3 or shown in Fig. 2, and no ablation isolates it. Either this module is part of SegVLM and must be specified with equations and implementation details, or it is a leftover description that should be removed. In its current form, the architecture cannot be reproduced from the text.
minor comments (5)
  1. [§5.1] The sentence "the inclusion of the RAF loss improves IoU by nearly 6%" is not supported by Table 2, where the RAF row is 48.25 versus 43.57 for the baseline, an improvement of 4.68 IoU points rather than nearly 6 points.
  2. [§3.4, Eq. (7)] The two proposed forms for α_i are not justified, and the notation "E(ε)" in the text should be "ε." In particular, α_i = |p_i − y_i| vanishes at correctly classified pixels, which appears opposite to the intended emphasis on uncertain or misclassified pixels, so the choice needs clarification.
  3. [§4.2] The phrase "gradient clipping with clipnorm set to 0 (prevents exploding gradients)" is ambiguous, because a clip norm of zero typically disables clipping; the actual clipping value and its effect should be stated precisely.
  4. [§2.3 and References] There is a duplicated phrase "most recent contemporary models, most recent contemporary models" in §2.3, references [80] and [81] are duplicates of the same GROUNDHOG paper, and the CLIP reference appears twice as [14] and [39].
  5. [§4.3] The text contains the typo "distibuted" for "distributed," and the reported inference speed of 40-45 FPS is not accompanied by hardware, batch size, or input-resolution details needed to interpret it.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper's claims are empirical benchmark comparisons of known components, with a reproducibility inconsistency but no derivation step that reduces to its own inputs.

full rationale

SegVLM's central claims are empirical performance numbers on PhraseCut: the proposed model combines external components (deformable convolution, SE blocks, residual connections) and a loss that is explicitly written as a weighted sum of BCE, Focal, and adaptive Dice losses. There is no derivation in which an output quantity is defined in terms of the quantity it is claimed to predict. The RAF loss is introduced by explicit equations (Eqs. 4-8) and then evaluated on a held-out test set, so any measured improvement is an empirical result rather than a tautology. The ablations in Table 2 are incremental training configurations, not fitted parameters renamed as predictions. The paper contains no load-bearing self-citation: references to CRIS, CLIP, deformable convolution, and V-Net are all external works with independent content, and no 'uniqueness theorem' from the authors is invoked to force a choice. The residual, deformable, and SE modules are used as architectural components, not as conclusions derived from the paper's own premises. One substantive flaw is flagged that is relevant to correctness but not to circularity: the same final SegVLM configuration is reported with different precision scores in Table 1 and Table 2 (e.g., Prec@70 40.81 vs. 36.00; Prec@90 12.64 vs. 10.30). This internal inconsistency prevents verification of the main quantitative conclusion, and the unreported lambda/alpha/gamma weights for the RAF loss are a reporting gap, but neither is a circular reduction. The claim 'each component contributes consistent improvements' is supported (if at all) by independent training runs against external baselines, so under the stated rules the honest finding is no circularity.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central result rests on standard components (deformable convolution, SE, residual connections, BCE/Focal/Dice losses) taken from prior work, plus a custom loss with five undisclosed hyperparameters. No new physical or conceptual entities are introduced. The main uncharged load is the fidelity of the CRIS baseline reimplementation and the selection of loss weights.

free parameters (3)
  • RAF loss weights lambda1, lambda2, lambda3 = not specified
    Eq. (8) weights the BCE, Focal, and Adaptive Dice terms; values are never given, so the exact objective is unreproducible and the reported gains could depend on tuned weights.
  • Focal loss parameters alpha, gamma = not specified
    Eq. (5) sets alpha in (0,1) and gamma > 0 but no values are reported; these control the hard-example weighting that motivates the RAF loss.
  • Adaptive Dice reweighting alpha_i choice = |p_i - y_i| or (1-p_i)^gamma
    Eq. (7) offers two alternative definitions of the pixel weight but the paper does not say which is used or whether both were tried.
assumptions (4)
  • domain assumption The CRIS baseline is faithfully re-implemented and its CLIP-based alignment transfers to PhraseCut.
    Stated in Sections 1.2 and 4.2; the +10.30 IoU delta in Table 2 is computed against this baseline, and no code or official CRIS PhraseCut number is provided to confirm it.
  • standard math Deformable convolution (Eq. 1) and squeeze-and-excitation (Eq. 2) behave as in the cited/alluded prior work and improve projection features.
    The formulas are taken from the literature (deformable conv cited as [71]; the SE block is standard but the original SE paper is not cited), and the paper assumes these modules help without an independent mechanism.
  • domain assumption The three losses in RAF (Eq. 8) can be combined linearly with positive weights without degrading optimization.
    The paper provides no analysis or sensitivity study for lambda1, lambda2, lambda3; the weights are unstated, so this assumption is untested.
  • domain assumption The LMDB conversion and resizing of PhraseCut masks preserve pixel-level label alignment.
    Section 4.1 describes conversion and resizing; no verification (e.g., visualization or intersection checks) is reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Deformable Attentive Visual Enhancement for Referring Segmentation Using Vision-Language Model." pith.science (2026). https://pith.science/paper/I3UAGCDN

@misc{pith2026250519242,
  author       = {Pith},
  title        = {Pith review of: Deformable Attentive Visual Enhancement for Referring Segmentation Using Vision-Language Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/I3UAGCDN}},
  note         = {Machine review of arXiv:2505.19242}
}
read the original abstract

Image segmentation is a fundamental task in computer vision, aimed at partitioning an image into semantically meaningful regions. Referring image segmentation extends this task by using natural language expressions to localize specific objects, requiring effective integration of visual and linguistic information. In this work, we propose SegVLM, a vision-language model that incorporates architectural improvements to enhance segmentation accuracy and cross-modal alignment. The model integrates squeeze-and-excitation (SE) blocks for dynamic feature recalibration, deformable convolutions for geometric adaptability, and residual connections for deep feature learning. We also introduce a novel referring-aware fusion (RAF) loss that balances region-level alignment, boundary precision, and class imbalance. Extensive experiments and ablation studies demonstrate that each component contributes to consistent performance improvements. SegVLM also shows strong generalization across diverse datasets and referring expression scenarios.

Figures

Figures reproduced from arXiv: 2505.19242 by the authors.

Figure 1
Figure 1. Overview of SegVLM Vision-Language Segmentation. Given the input image [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. The SegVLM architecture for referring image segmentation comprises an im [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Ablation study showing the impact of each proposed enhancement on IoU, [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Precision across IoU thresholds (50–90). SegVLM consistently achieves higher [PITH_FULL_IMAGE:figures/full_fig_p015_4.png]
Figure 5
Figure 5. Figure 5: Incremental improvements in IoU and precision scores (P@50–P@90) at each [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 6
Figure 6. Figure 6: Qualitative evaluation of segmentation results across different model configura [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]
Figure 7
Figure 7. Figure 7: Qualitative results demonstrating the generalization capability of SegVLM on [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: Segmentation result demonstrating the model’s ability to capture fine-grained [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

82 extracted references · 63 canonical work pages

  1. [1]

    J. Long, E. Shelhamer, T. Darrell, Fully convolutional networks for se- mantic segmentation, in: Proceedings of the IEEE conference on com- puter vision and pattern recognition, 2015, pp. 3431–3440

    2015

  2. [2]

    J. Ma, Y. He, F. Li, L. Han, C. You, B. Wang, Segment anything in medical images, Nature Communications 15 (2024) 654.doi:10.1038/ s41467-024-44824-z. URLhttps://www.nature.com/articles/s41467-024-44824-z

    2024

  3. [3]

    W. Hao, J. Wang, H. Lu, A real-time semantic segmentation method based on transformer for autonomous driving, Computers, Materials & Continua 81 (3) (2024) 4419–4433.doi:10.32604/cmc.2024.055478. URLhttps://www.techscience.com/cmc/v81n3/59019

    work page arXiv 2024

  4. [4]

    Yu, Y.-D

    C.-C. Yu, Y.-D. Chen, H.-Y. Cheng, C.-L. Jiang, Semantic segmentation of satellite images for landslide detection using foreground-aware and multi-scale convolutional attention mechanism, Sensors 24 (20) (2024) 6539.doi:10.3390/s24206539. URLhttps://www.mdpi.com/1424-8220/24/20/6539

    work page doi:10.3390/s24206539 2024

  5. [5]

    Zhang, Y

    X. Zhang, Y. Zhu, L. Chen, P. Duan, M. Zhou, Augmented reality navigation method based on image segmentation and sensor tracking registration technology, Scientific Reports 14 (2024) 15281.doi:10. 1038/s41598-024-65204-z. URLhttps://www.nature.com/articles/s41598-024-65204-z

    work page 2024

  6. [6]

    L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, A. L. Yuille, Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs, IEEE Transactions on Pat- tern Analysis and Machine Intelligence 40 (4) (2017) 834–848

    work page 2017

  7. [7]

    C. Liu, H. Ding, X. Jiang, Gres: Generalized referring expression seg- mentation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023

    work page 2023

  8. [8]

    T. Yan, Y. Xu, Z. Zhao, H. Wang, W. Xie, Y. Liu, Y. Tian, Mmnet: Multi-mask network for referring image segmentation, arXiv preprint arXiv:2305.14969 (2023). 19

    work page Pith review arXiv 2023

Show all 82 references
  1. [9]

    Z. Li, M. Wang, J. Mei, Y. Liu, Mail: A unified mask-image-language trimodal network for referring image segmentation, arXiv preprint arXiv:2111.10747 (2021)

    2021 arXiv

  2. [10]

    H. Ding, C. Liu, S. Wang, X. Jiang, Vision-language transformer and query generation for referring segmentation, arXiv preprint arXiv:2108.05565 (2021)

    2021 arXiv

  3. [11]

    Y. Cho, H. Yu, K. Kong, S.-J. Kang, Metris: Multi-expressions for transformer-based referring image segmentation, in: International Con- ference on Learning Representations (ICLR), 2025

    2025

  4. [12]

    L. Xiao, X. Yang, F. Peng, Y. Wang, C. Xu, Oneref: Unified one-tower expression grounding and segmentation with mask referring modeling, arXiv preprint arXiv:2410.08021 (2024)

    2024 arXiv

  5. [13]

    M. K. Reza, A. Prater-Bennette, M. S. Asif, Mmsformer: Multimodal transformer for material and semantic segmentation, IEEE Open Jour- nal of Signal Processing 4 (2023) 123–135.doi:10.1109/OJSP.2023. 3291234

    2023 doi

  6. [14]

    Radford, J

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, I. Sutskever, Learning transferable visual models from natural language supervision, in: Proceedings of the 38th International Conference on Machine Learn-...

    2021 arXiv

  7. [15]

    R. Hu, M. Rohrbach, T. Darrell, Segmentation from natural language expressions, in: European Conference on Computer Vision (ECCV), Springer, 2016, pp. 108–124

    2016

  8. [16]

    T. Ruan, Y. Wang, Z. Zhang, Y. Li, Seem: Segment everything every- where all at once, arXiv preprint arXiv:2304.06718 (2023). URLhttps://arxiv.org/abs/2304.06718

    2023 arXiv

  9. [17]

    J. Li, D. Li, C. Xiong, S. C. H. Hoi, Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation, in: Proceedings of the 39th International Conference on Machine Learn- ing (ICML), 2022. URLhttps://arxiv.org/abs/2201.12086 20

    2022 arXiv

  10. [18]

    T. Wang, X. Zhang, H. Zhao, Cris: Clip-driven referring image seg- mentation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 11686–11695

    2022

  11. [19]

    L. Yu, P. Poirson, S. Yang, A. C. Berg, T. L. Berg, Modeling context in referring expressions, in: European Conference on Computer Vision (ECCV), 2016

    2016

  12. [20]

    J. Mao, J. Huang, A. Toshev, O. Camburu, A. Yuille, K. Murphy, Generation and comprehension of unambiguous object descriptions, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016

    2016

  13. [21]

    Z. Wu, L. Wang, Z. Jiang, Y. Xiong, D. Lin, Phrasecut: Language- basedimagesegmentationinthewild, in: ProceedingsoftheIEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 10219–10229

    2020

  14. [22]

    Liu, et al., Referring image segmentation using text supervision, in: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), IEEE, 2023, pp

    Z. Liu, et al., Referring image segmentation using text supervision, in: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), IEEE, 2023, pp. 12345–12354

    2023

  15. [23]

    Alalyani, N

    R. Alalyani, N. Krishnaswamy, Multimodal referring expression gen- eration in interactive virtual agents, in: International Conference on Human-Computer Interaction, Springer, 2024, pp. 3–22

    2024

  16. [24]

    Alalyani, N

    R. Alalyani, N. Krishnaswamy, Scmre: A corpus for generative ai in multimodal human-computer interaction, in: International Conference on Human-Computer Interaction, Springer, 2024, pp. 23–39

    2024

  17. [25]

    J. Tang, G. Zheng, C. Shi, S. Yang, Contrastive grouping with transformer for referring image segmentation, in: Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, pp. 23570–23580. doi:10.1109/CVPR52729.2023.02257. URLhttps://opena...

    2023

  18. [26]

    Shridhar, D

    M. Shridhar, D. Hsu, Interactive visual grounding of referring expres- sions for human-robot interaction, in: Proceedings of Robotics: Science and Systems (RSS), 2018

    2018

  19. [27]

    R. Paul, R. C. Arkin, S. Chernova, Grounding spatio-semantic referring expressions for human-robot interaction, in: 2020 IEEE/RSJ Interna- tional Conference on Intelligent Robots and Systems (IROS), IEEE, 2020, pp. 2328–2335

    2020

  20. [28]

    L. Yu, Z. Lin, X. Shen, J. Yang, X. Lu, M. Bansal, T. L. Berg, Mattnet: Modular attention network for referring expression comprehension, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 1307–1315

    2018

  21. [29]

    H. Liu, C. Li, Q. Wu, Y. J. Lee, Visual instruction tuning, in: A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, S. Levine (Eds.), Advances in Neural Information Processing Systems, Vol. 36, Curran Associates, Inc., 2023, pp. 34892–34916. URLhttps://proceedings.neurips.cc...

    2023

  22. [30]

    S. Tong, E. Brown, P. Wu, S. Woo, M. Middepogu, S. C. Akula, J. Yang, S. Yang, A. Iyer, X. Pan, A. Wang, R. Fergus, Y. LeCun, S. Xie, Cambrian-1: A fully open, vision-centric exploration of multimodal llms, in: A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tom- c...

    2024

  23. [31]

    K. Yan, Z. Wang, L. Ji, Y. Wang, N. Duan, S. Ma, Voila-a: Aligning vision-language models with user's gaze attention, in: A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, C. Zhang (Eds.), Advances in Neural Information Processing Systems, Vol. 37, Curran As...

    2024

  24. [32]

    W. Kim, B. Son, I. Kim, Vilt: Vision-and-language transformer with- out convolution or region supervision, in: M. Meila, T. Zhang (Eds.), 22 Proceedings of the 38th International Conference on Machine Learning, Vol. 139 of Proceedings of Machine Learning Research, PMLR, 2021, ...

    2021

  25. [33]

    Radford, J

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, I. Sutskever, Learning transferable visual models from natural language supervision, in: M. Meila, T. Zhang (Eds.), Proceedings of the 38th International C...

    2021

  26. [34]

    Z. Yang, Z. Gan, J. Wang, X. Hu, F. Ahmed, Z. Liu, Y. Lu, L. Wang, Unitab: Unifying text and box outputs for grounded vision-language modeling, in: European Conference on Computer Vision (ECCV), Springer, 2022, pp. 521–539

    2022

  27. [35]

    W. Wang, H. Bao, L. Dong, J. Bjorck, Z. Peng, Q. Liu, K. Aggarwal, O. K. Mohammed, S. Singhal, S. Som, F. Wei, Image as a foreign lan- guage: Beit pretraining for all vision and vision-language tasks, arXiv preprint arXiv:2208.10442 (2022)

    2022 arXiv

  28. [36]

    J. Bai, S. Bai, S. Yang, S. Wang, S. Tan, P. Wang, J. Lin, C. Zhou, J. Zhou, Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond, arXiv preprint arXiv:2308.12966 (2023)

    2023 arXiv

  29. [37]

    J. Li, D. Li, S. Savarese, S. Hoi, BLIP-2: Bootstrapping language-image pre-training with frozen image encoders and large language models, in: A. Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabato, J. Scarlett (Eds.), Proceedings of the 40th International Conference on Mac...

    2023

  30. [38]

    D. Zhu, J. Chen, X. Shen, X. Li, M. Elhoseiny, Minigpt-4: Enhancing vision-language understanding with advanced large language models, 23 in: Proceedings of the International Conference on Learning Represen- tations (ICLR), 2024. URLhttps://openreview.net/forum?id=1tZbq88f27

    2024

  31. [39]

    Radford, J

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, I. Sutskever, Learning transferable visual models from natural language supervision, in: Proceedings of the 38th International Conference on Machine Learn-...

    2021

  32. [40]

    S.-A. Liu, Y. Zhang, Z. Qiu, H. Xie, Y. Zhang, T. Yao, Caris: Context-aware referring image segmentation, in: Proceedings of the 31st ACM International Conference on Multimedia, MM ’23, Associa- tion for Computing Machinery, New York, NY, USA, 2023, p. 779–788. doi:10.1145/358...

    2023

  33. [41]

    not specified, Referring expression comprehension using language adaptive dynamic subnetworks, in: Proceedings of the AAAI Confer- ence on Artificial Intelligence, Vol

    A. not specified, Referring expression comprehension using language adaptive dynamic subnetworks, in: Proceedings of the AAAI Confer- ence on Artificial Intelligence, Vol. 37, 2023, pp. 13780–13800

    2023

  34. [42]

    Hemanthage, H

    B. Hemanthage, H. Bilen, P. Bartie, C. Dondrup, O. Lemon, Recant- former: Referring expression comprehension with varying numbers of targets, in: Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP), Association for Computa- tional Lin...

    2024

  35. [43]

    G. Yang, J. Li, Y. S. Wang, W. Zhang, S. C. H. Hoi, Videococa: Video contrastive captioners are temporal learners, in: Advances in Neural Information Processing Systems (NeurIPS), 2023

    2023

  36. [44]

    J. Mei, A. Piergiovanni, J.-N. Hwang, W. Li, Slvp: Self-supervised language-video pre-training for referring video object segmentation, in: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACV Workshops), 2024

    2024

  37. [45]

    Zou, Z.-Y

    X. Zou, Z.-Y. Dou, J. Yang, Z. Gan, L. Li, C. Li, X. Dai, H. Behl, J. Wang, L. Yuan, N. Peng, L. Wang, Y. J. Lee, J. Gao, Generalized de- coding for pixel, image, and language, in: Proceedings of the IEEE/CVF ConferenceonComputerVisionandPatternRecognition(CVPR),2023. 24

    2023

  38. [46]

    J. Wang, D. Chen, Z. Wu, C. Luo, L. Zhou, Y. Zhao, Y. Xie, C. Liu, Y.- G. Jiang, L. Yuan, Omnivl: One foundation model for image-language and video-language tasks, in: Advances in Neural Information Process- ing Systems (NeurIPS), 2022

    2022

  39. [47]

    W. Dai, J. Li, D. Li, A. M. H. Tiong, J. Zhao, W. Wang, B. Li, P. Fung, S. Hoi, Instructblip: Towards general-purpose vision-language models with instruction tuning, in: Advances in Neural Information Processing Systems (NeurIPS), 2023

    2023

  40. [48]

    Hadsell, S

    R. Hadsell, S. Chopra, Y. LeCun, Dimensionality reduction by learning an invariant mapping, in: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), Vol. 2, 2006, pp. 1735–1742.doi:10.1109/CVPR.2006.100

    2006 doi

  41. [49]

    T. Chen, S. Kornblith, M. Norouzi, G. Hinton, A simple framework for contrastive learning of visual representations, in: H. D. III, A. Singh (Eds.), Proceedings of the 37th International Conference on Machine Learning, Vol.119ofProceedingsofMachineLearningResearch, PMLR, 2020,...

    2020

  42. [50]

    Y. Ci, C. Lin, L. Bai, W. Ouyang, Fast-moco: Boost momentum-based contrastive learning with combinatorial patches, in: Computer Vision – ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXVI, Springer-Verlag, Berlin, Heidel- berg, 2...

    2022 doi

  43. [51]

    K. He, H. Fan, Y. Wu, S. Xie, R. Girshick, Momentum contrast for unsupervised visual representation learning, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2020, pp. 9729–9738

    2020

  44. [53]

    Zhang, C

    Y. Zhang, C. Liu, Y. Zhou, W. Wang, Q. Ye, X. Ji, Beyond instance dis- crimination: Relation-aware contrastive self-supervised learning, Trans. Multi. 26 (2024) 4628–4640.doi:10.1109/TMM.2023.3324588. URLhttps://doi.org/10.1109/TMM.2023.3324588

    2024

  45. [54]

    Zhang, T

    J. Zhang, T. Lin, Y. Xu, K. Chen, R. Zhang, Relational contrastive learning for scene text recognition, in: Proceedings of the 31st ACM International Conference on Multimedia, MM ’23, Association for Com- puting Machinery, New York, NY, USA, 2023, p. 5764–5775.doi: 10.1145/358...

    2023

  46. [55]

    P. O. Pinheiro, A. Almahairi, R. Y. Benmalek, F. Golemo, A. Courville, Unsupervised learning of dense visual representations, in: Advances in Neural Information Processing Systems (NeurIPS), 2020. URLhttps://arxiv.org/abs/2011.05499

    2020 arXiv

  47. [56]

    X. Wang, R. Zhang, C. Shen, T. Kong, L. Li, Dense contrastive learning for self-supervised visual pre-training, in: Proceedings of the IEEE/CVF ConferenceonComputerVisionandPatternRecognition(CVPR),2021, pp. 3023–3032.doi:10.1109/CVPR46437.2021.00302

    2021

  48. [57]

    R. Li, K. Li, Y.-C. Kuo, M. Shu, X. Qi, X. Shen, J. Jia, Referring image segmentation via recurrent refinement networks, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 5745–5753

    2018

  49. [58]

    C. Liu, Z. Lin, X. Shen, J. Yang, X. Lu, A. Yuille, Recurrent multi- modal interaction for referring image segmentation, in: Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017, pp. 1271–1280

    2017

  50. [59]

    Margffoy-Tuay, J

    E. Margffoy-Tuay, J. C. Pérez, E. Botero, P. Arbeláez, Dynamic mul- timodal instance segmentation guided by natural language queries, in: Proceedings of the European Conference on Computer Vision (ECCV), Springer, 2018, pp. 630–645

    2018

  51. [60]

    Hochreiter, J

    S. Hochreiter, J. Schmidhuber, Long short-term memory, Neural Com- putation 9 (8) (1997) 1735–1780.doi:10.1162/neco.1997.9.8.1735. 26

    1997 doi

  52. [61]

    J. Long, E. Shelhamer, T. Darrell, Fully convolutional networks for se- mantic segmentation, in: Proceedings of the IEEE Conference on Com- puter Vision and Pattern Recognition (CVPR), 2015, pp. 3431–3440

    2015

  53. [62]

    K. He, G. Gkioxari, P. Dollár, R. Girshick, Mask r-cnn, in: Proceed- ings of the IEEE International Conference on Computer Vision (ICCV), 2017, pp. 2961–2969

    2017

  54. [63]

    G. Luo, Y. Zhou, X. Sun, L. Cao, C. Wu, C. Deng, R. Ji, Multi-task collaborative network for joint referring expression comprehension and segmentation, in: Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), 2020, pp. 10034–10043

    2020

  55. [64]

    Zhang, Y

    M. Zhang, Y. Liu, X. Yin, H. Yue, J. Yang, Risam: Referring im- age segmentation via mutual-aware attention features, arXiv preprint arXiv:2311.15727 (2023). URLhttps://arxiv.org/abs/2311.15727

    2023 arXiv

  56. [65]

    Z. Yang, J. Wang, Y. Tang, K. Chen, H. Zhao, P. H. Torr, Lavt: Language-aware vision transformer for referring image segmentation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2022, pp. 17247–17256.doi: 10.1109/CVPR52688.2...

    2022

  57. [66]

    Ouyang, H

    S. Ouyang, H. Wang, S. Xie, Z. Niu, R. Tong, Y.-W. Chen, L. Lin, Slvit: Scale-wise language-guided vision transformer for referring image segmentation, in: Proceedings of the 32nd International Joint Confer- ence on Artificial Intelligence (IJCAI), International Joint Conferen...

    2023

  58. [67]

    L. Ye, M. Rochan, Z. Liu, Y. Wang, Cross-modal self-attention network for referring image segmentation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2019, pp. 10502–10511

    2019

  59. [68]

    Z. Hu, G. Feng, J. Sun, L. Zhang, H. Lu, Bi-directional relationship in- ferring network for referring image segmentation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2020, pp. 4424–4433. 27

    2020

  60. [69]

    Y. Yan, X. He, S. Chen, S. Lu, J. Liu, Fuse & calibrate: A bi- directional vision-language guided framework for referring image seg- mentation, arXiv preprint arXiv:2405.11205 (2024). URLhttps://arxiv.org/abs/2405.11205

    2024 arXiv

  61. [70]

    Shang, Z

    C. Shang, Z. Song, H. Qiu, L. Wang, F. Meng, H. Li, Prompt-guided bidirectional deep fusion network for referring image segmentation, Neu- rocomputing (2024).doi:10.1016/j.neucom.2024.01.123

    2024 doi

  62. [71]

    J. Dai, H. Qi, Y. Xiong, Y. Li, G. Zhang, H. Hu, Y. Wei, Deformable convolutional networks, in: Proceedings of the IEEE International Con- ference on Computer Vision (ICCV), 2017, pp. 764–773

    2017

  63. [72]

    Goodfellow, Y

    I. Goodfellow, Y. Bengio, A. Courville, Deep Learning, MIT Press, 2016. URLhttps://www.deeplearningbook.org/

    2016

  64. [73]

    T.-Y. Lin, P. Goyal, R. Girshick, K. He, P. Dollár, Focal loss for dense object detection, in: Proceedings of the IEEE international conference on computer vision, 2017, pp. 2980–2988

    2017

  65. [74]

    Milletari, N

    F. Milletari, N. Navab, S.-A. Ahmadi, V-net: Fully convolutional neural networks for volumetric medical image segmentation, in: 2016 Fourth International Conference on 3D Vision (3DV), IEEE, 2016, pp. 565–571

    2016

  66. [75]

    Chu, Howard chu on lightning memory-mapped database, IEEE Soft- ware 36 (6) (2019) 96–100.doi:10.1109/MS.2019.2936273

    H. Chu, Howard chu on lightning memory-mapped database, IEEE Soft- ware 36 (6) (2019) 96–100.doi:10.1109/MS.2019.2936273. URLhttps://doi.org/10.1109/MS.2019.2936273

    2019

  67. [76]

    Zhang, K

    H. Zhang, K. Dana, J. Shi, Z. Zhang, X. Wang, A. Tyagi, A. Agrawal, Context encoding for semantic segmentation, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 7151–7160

    2018

  68. [77]

    Pascanu, T

    R. Pascanu, T. Mikolov, Y. Bengio, On the difficulty of training recur- rent neural networks, in: International Conference on Machine Learning (ICML), 2013, pp. 1310–1318

    2013

  69. [78]

    K. He, X. Zhang, S. Ren, J. Sun, Delving deep into rectifiers: Surpassing human-level performance on imagenet classification, in: Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2015, pp. 1026–1034. 28

    2015

  70. [79]

    Kamath, M

    A. Kamath, M. Singh, Y. LeCun, G. Synnaeve, I. Misra, N. Carion, Mdetr–modulateddetectionforend-to-endmulti-modalunderstanding, in: Proceedings of the IEEE/CVF International Conference on Com- puter Vision (ICCV), 2021, pp. 1760–1770

    2021

  71. [80]

    Zhang, Z

    Y. Zhang, Z. Ma, X. Gao, S. Shakiah, Q. Gao, J. Chai, Ground- hog: Grounding large language models to holistic segmentation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024, pp. 14227–14238.doi:10.1109/ CVPR52733.2024.01349

    2024

  72. [81]

    Zhang, Z

    Y. Zhang, Z. Ma, X. Gao, S. Shakiah, Q. Gao, J. Chai, Groundhog: Grounding large language models to holistic segmentation, in: Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024, pp. 14227–14238

    2024

  73. [82]

    Everingham, L

    M. Everingham, L. Van Gool, C. K. Williams, J. Winn, A. Zisserman, The pascal visual object classes (voc) challenge, International Journal of Computer Vision (IJCV) 88 (2) (2010) 303–338

    2010

  74. [83]

    T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, C. L. Zitnick, Microsoft coco: Common objects in context, in: European Conference on Computer Vision (ECCV), Springer, 2014, pp. 740–755. 29

    2014

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.