Pith. sign in

REVIEW 4 major objections 5 minor 60 references

SynRES: Towards Referring Expression Segmentation in the Wild via Synthetic Data

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Synthetic triplets boost referring segmentation by up to 3.8 gIoU on wild-scene queries.

desk verdict Useful benchmark and a plausible synthetic-data pipeline, but the validation-set fine-tuning footnote threatens the headline numbers; must be resolved before trusting the gains. read the letter →

arxiv 2505.17695 v1 pith:HJUXWN3W submitted 2025-05-23 cs.LG cs.AIcs.CV

Dong-Hee Kim , Hyunjee Song , Donghyun Kim This is my paper · ORCID
classification cs.LGcs.AIcs.CV
keywords referringexpressionsegmentationsyntheticdatagenerationbenchmarklargemultimodalmodelsattribute-richqueriesmulti-targetdomainshiftaugmentation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that current referring expression segmentation (RES) benchmarks are too easy: queries are short, carry few attributes, or single out targets with distinctive counts, so models never learn to reason over long attribute-heavy expressions or to find several objects that share attributes. To expose this gap, the authors build WildRES, a 724-image benchmark with single-target queries that often contain over 11 attributes and multi-target queries with shared, non-distinctive attributes, drawn from COCO plus crowd, driving, and robotics scenes. They then claim that SynRES, an automatic pipeline generating densely paired synthetic image-expression-mask triplets, lets existing large multimodal model (LMM) based RES models such as LISA and GSVA improve by up to 2.0 gIoU on WildRES-ID and 3.8 gIoU on WildRES-DS while retaining or improving performance on classic RES benchmarks. The central assertion is that carefully aligned and augmented synthetic data can substitute for expensive manual annotation when the missing capability is compositional reasoning rather than vocabulary breadth.

What carries the argument

The load-bearing mechanism is SynRES's 'Image-Text Aligned Grouping' in Step 2: a pretrained RES model (LISA) produces pseudo-masks on each synthetic image for every candidate expression, pairwise mean IoU across images is computed, and expression pairs above threshold $\tau=0.65$ are grouped into consensus clusters whose averaged masks are re-thresholded into the final synthetic masks. This turns noisy per-expression pseudo-masks into a smaller set of aligned image-expression-mask triplets, which is what makes the dense synthetic supervision usable for training. The two supporting mechanisms are caption-driven synthesis (CoCa captions aggregated into prompts for SANA) and domain-aware augmentation (mosaic composition mixing real and synthetic images, plus probabilistic superclass word replacement in text).

What would settle it

Take a random sample of SynRES synthetic images and have human annotators, or a second independently pretrained segmenter prompted with the same grouped expressions, redraw the masks; if the model trained on the human-corrected masks performs no better on WildRES than the model trained on LISA's pseudo-masks, or if human agreement with the grouped masks is low for attribute-heavy queries, then the pseudo-mask accuracy assumption and the claimed mechanism are falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that the bottleneck in RES is not model capacity but the scarcity of densely paired training signals. Existing datasets pair a single short expression with a mask, or use countable distinctive expressions for multiple targets, leaving models unprepared for long attribute lists and shared-attribute multi-target queries; WildRES quantifies this gap by showing that a current RES model like LISA degrades sharply on such cases. SynRES then closes much of the gap: CoCa captions real image-mask pairs to produce several distinctive expressions per object, these expressions are concatenated into prompts for the SANA text-to-image model, and the resulting synthetic images are pseudo-segmented by a pretrained RES model. Expression pairs whose pseudo-masks agree above a mean-IoU threshold of 0.65 are clustered into consensus groups; the final mask is the averaged and re-thresholded group mask. Mosaic augmentation mixes real and synthetic images for multi-target scenes, and superclass text replacement shifts emphasis from category names to distinguishing attributes. Trained with SynRES, LISA-7B, LISA-13B, and GSVA-7B improve on WildRES and on the classic RefCOCO family, whereas simply adding FreeMask synthetic data hurts performance.

Load-bearing premise

The load-bearing premise is that the pseudo-masks produced by the pretrained LISA model on synthetic SANA images are accurate enough that grouping and averaging them yields correct training supervision; if LISA missegments those images, SynRES's training signal is corrupted and the measured gains may reflect fitting to LISA's prior rather than learning to reason about attributes.

Editorial extensions

If this is right

  • If SynRES is correct, dense synthetic triplets can substitute for manual annotation in RES regimes where human labels are expensive, namely long attribute-heavy queries and multi-target shared-attribute queries.
  • Because gains appear for both LISA and GSVA, the method transfers across model architectures without changing them, so other LMM-based segmentation models should benefit from the same pipeline.
  • The negative result with FreeMask shows that synthetic-data volume alone is not enough; alignment and grouping of masks with expressions is what makes synthetic supervision effective for RES.
  • On classic benchmarks such as RefCOCO, RefCOCO+, RefCOCOg, and gRefCOCO, SynRES maintains or improves gIoU, so the improvements on wild scenarios do not come at the expense of standard RES performance.
  • Ablations show that removing both mosaic and superclass-text augmentation drops performance below the real-data baseline, indicating that these domain-aware augmentations are required for the wild gains, not optional extras.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the near-plateau from 25% to 100% of the synthetic data (98.5% of the full-data score at one quarter of the data) suggests the active ingredient is coverage of attribute combinations and alignment quality rather than dataset size; a targeted test would sample rare attribute combinations and watch whether gains track combinatorial coverage.
  • Editorial inference: WildRES-ID and the SynRES source images both come from the MSCOCO distribution, so part of the in-distribution gain may be domain alignment rather than new reasoning; a stricter test would train on SynRES samples derived from masks and categories held out of the evaluation set.
  • Editorial inference: because the grouping masks are produced by LISA itself, SynRES may inherit LISA's blind spots; generating the pseudo-masks with a second, independently pretrained segmenter and comparing the resulting models would reveal whether the pipeline teaches genuine attribute reasoning or merely reinforces one model's priors.
  • Editorial inference: the benchmark's design choice to avoid count nouns in multi-target expressions mirrors how people actually refer to objects; extending SynRES to video or interactive embodied settings, where manual referring-expression annotation is even costlier, is a natural next step that the paper does not pursue.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces WildRES, a new referring expression segmentation benchmark with long multi-attribute single-target expressions and shared-attribute multi-target expressions, spanning in-distribution (MSCOCO) and domain-shifted (CrowdHuman, Cityscapes, ARMBench) images. The authors also propose SynRES, a pipeline that generates synthetic RES training triplets by: (1) producing distinctive captions from real image-mask pairs and using SANA text-to-image generation to synthesize corresponding images; (2) generating pseudo-masks with a pretrained RES model and grouping expressions by pairwise mean IoU with threshold tau to refine masks; and (3) applying mosaic image augmentation and superclass-based text augmentation. Fine-tuning LISA-7B/13B, GSVA-7B, and GLaMM with SynRES is reported to improve gIoU on WildRES-ID and WildRES-DS, with additional results on RefCOCO/RefCOCO+/RefCOCOg, gRefCOCO, Ref-ZOM, and ReasonSeg.

Significance. If the reported gains are valid under a clean held-out protocol, the paper makes two useful contributions: a new benchmark that targets reasoning dimensions absent from existing RES datasets, and a model-agnostic synthetic-data method that improves several LMM-based RES models. The benchmark design, with explicit many-attribute and shared-attribute splits across multiple domains, is a credible step toward evaluating RES in the wild. The paper also ships code and datasets, and the augmentation ablations are informative. However, the central empirical claim is currently conditional on resolving a protocol ambiguity about validation-set fine-tuning, and the absence of a Pseudo-RIS comparison leaves the claim of superiority over prior synthetic RES data untested. The self-training aspect of the pseudo-mask generation also needs explicit reliability evidence. These issues are load-bearing but addressable, so the paper merits a major revision rather than rejection.

major comments (4)
  1. [§5.2, Tables 1–2 footnotes] The evaluation protocol for WildRES-ID is ambiguous in a way that is load-bearing for the central claim. Section 5.2 states that all models are trained for 5,000 steps with WildRES-ID validation every 100 steps, while the Table 1 footnote says that models trained only with real data use an additional 5,000 fine-tuning steps on the validation set of WildRES-ID, and the Table 2 footnote says that all models are trained with an additional 5,000 fine-tuning steps on the validation set of WildRES-ID. If the validation labels of WildRES-ID participate in fine-tuning, the test numbers in Tables 1 and 2 are not a clean held-out evaluation, and the headline gains of +2.0 gIoU on WildRES-ID and +3.8 gIoU on WildRES-DS cannot be interpreted. Please state explicitly that no WildRES-ID validation labels are used for training or model selection, or rerun the experiments under a protocol that excludes them.
  2. [§2.2, §4, §5, Tables 1–3] Pseudo-RIS is the closest prior method for generating RES-specific pseudo-supervision, and it is discussed in Sections 2.2 and 4 as a motivation for SynRES, but it is never used as a baseline in the experiments. The only synthetic-data baseline in Tables 1 and 2 is FreeMask, and Table 3 has no synthetic-data baseline at all for the classic benchmarks. Without a Pseudo-RIS comparison under the same training recipe, the claim that SynRES improves over existing synthetic RES data generation is not supported. Add Pseudo-RIS as a baseline, or explicitly justify why a direct comparison is not feasible.
  3. [§4.2, Eq. (2)–(4); §5.2] The pseudo-mask supervision used to train LISA is generated by a pretrained RES model, which is the same model family (and in the LISA experiments, the same architecture) that is later fine-tuned with SynRES. This creates a self-training loop: the synthetic training signal is derived from the model's own prior outputs, albeit on newly generated synthetic images. The observed gains on WildRES could therefore reflect adaptation to LISA's prior rather than improved compositional reasoning. Please provide direct evidence of pseudo-mask reliability, for example by measuring agreement with human masks on a held-out subset or on a classic RES dataset, and test whether the gains persist when the masks are generated by a different source such as SAM or a different RES model.
  4. [§5.3, Table 3] The introduction claims that SynRES 'consistently outperforms existing baselines on standard RES benchmarks,' but Table 3 does not fully support this for GSVA-7B: RefCOCO testA drops from 77.8 to 77.5 (-0.3), testB drops from 72.7 to 72.6 (-0.1), and RefCOCO+ testA is unchanged at 68.2. Since the central claim includes classic-benchmark improvements, please either soften the claim to 'maintains or improves' and discuss the small degradations, or provide a statistical or significance analysis showing that these differences are within noise.
minor comments (5)
  1. [§3.2 vs. Appendix A] The main text reports that WildRES contains 724 images and 941 expressions, while Table A in the appendix reports 724 images and 974 expressions; please reconcile the counts.
  2. [§3.2 and Table A] The benchmark is introduced as WildRES-ID and WildRES-DS, but Table A uses the names Wildseg-ID and Wildseg-DS; please unify the notation to avoid confusion.
  3. [§4.2, Eq. (4)] The index k in the refined mask notation is used both for the consensus group G_k and for the final mask index ranging from 1 to l; please clarify the mapping between groups and refined mask indices.
  4. [Figures 2 and 5, Appendix B] The term 'attributions' is used where 'attributes' is meant; this should be corrected throughout for consistency with the rest of the paper.
  5. [Table 1 and Table 2 footnotes] The footnote in Table 1 applies the extra validation-set fine-tuning only to real-data-only models, while the footnote in Table 2 applies it to all models; these statements should be reconciled and made explicit even after the protocol is clarified.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found; the self-training loop is not a derivation-level circularity because final evaluation uses human labels, though the WildRES-ID validation fine-tuning footnote is a protocol concern.

full rationale

The central claim is empirical: SynRES fine-tuning improves gIoU on WildRES-ID, WildRES-DS, and classic RES benchmarks. The derivation chain is: CoCa generates distinctive captions from real image-mask pairs; SANA synthesizes images from concatenated captions; a pretrained RES model (LISA) produces pseudo-masks that are grouped by mIoU threshold and averaged; and the resulting triplets are used to fine-tune LISA or GSVA. No step defines the target metric in terms of the model's own output. Although Step 2 uses LISA to generate supervision and Section 5.2 later fine-tunes LISA on that supervision, the reported evaluations are on human-annotated WildRES and established benchmarks, so the improvement is not an identity or a fitted-value tautology; it is an independent empirical outcome. The mIoU grouping is a filtering and averaging operation, not a fit to the benchmark labels. No load-bearing self-citations appear, no uniqueness theorem is imported from the authors' prior work, and WildRES is a newly introduced human-labeled benchmark rather than a renaming of an existing result. The main concern is protocol hygiene, not circularity: Table 1's footnote says 'Models trained only with real data use an additional 5000 fine-tuning steps on the validation set of WildRES-ID' and Table 2's footnote says 'All models trained with additional 5000 fine-tuning steps on the validation set of WildRES-ID,' which conflicts with Section 3.2's statement that WildRES has no training set. If taken literally, validation labels enter the training signal and weaken the held-out interpretation of the reported gains; this needs clarification or correction, but it is an evaluation-validity issue rather than a derivation that reduces to its own inputs.

Assumptions & free parameters 8 free parameters · 5 assumptions · 0 invented entities

The central claim rests on a set of hand-chosen hyperparameters (tau, p, m, n, grid sizes, data ratio) and on domain assumptions about the reliability of T2I generation and pseudo-mask extraction. No new physical entities or formal objects are introduced; WildRES is a dataset, not an entity. The most consequential dependency is the self-training loop where the model being improved also supplies the pseudo-labels for the synthetic training data.

free parameters (8)
  • mIoU threshold tau = 0.65
    Used in Step 2 to cluster expressions into consensus groups for pseudo-mask refinement. Ablation in Appendix G.1 shows sensitivity, but default chosen by hand.
  • Superclass replacement probability p = 0.7
    Text augmentation probability; ablated in Table F, default chosen by hand.
  • Synthetic images per target m = 6
    Number of T2I images generated per referring target; chosen manually.
  • Max expressions per target n = 5
    Maximum number of synthetic expressions per object; chosen manually.
  • Mosaic grid sizes = 2x2, 3x3
    Composite arrangements mixing 1 real plus 3 or 8 synthetic images; chosen without ablation.
  • Training data ratio = 9:3:3:1:4 (semantic seg : classic RES : VQA : ReasonSeg : SynRES)
    Data mixing ratio for LISA fine-tuning; chosen manually.
  • Fine-tuning steps = 5000
    Fixed step count for all models; chosen manually.
  • Superclass taxonomy = Table C mapping 12 superclasses to original words
    Hand-constructed mapping used for text augmentation; no automatic selection.
assumptions (5)
  • domain assumption T2I model SANA generates synthetic images that faithfully contain the attributes described in the concatenated expressions.
    Step 1 relies on visual-semantic alignment of SANA; if images miss attributes, the pseudo-masks and downstream training are misaligned.
  • domain assumption Pretrained RES model (LISA) produces reliable pseudo-masks on synthetic images.
    Step 2 uses LISA to generate initial masks; incorrect masks would propagate errors into SynRES training data.
  • ad hoc to paper Pairs of expressions with mIoU greater than tau (0.65) refer to the same object.
    The clustering threshold is hand-set; no theoretical justification.
  • domain assumption Superclass replacement does not break the alignment between expression and mask.
    Text augmentation assumes replacing a noun with its superclass still points to the same target; false negatives are acknowledged but said to be mitigated by isolated single objects.
  • standard math Standard IoU and averaging operations are well-defined and correct.
    Used in mask refinement and evaluation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SynRES: Towards Referring Expression Segmentation in the Wild via Synthetic Data." pith.science (2026). https://pith.science/paper/HJUXWN3W

@misc{pith2026250517695,
  author       = {Pith},
  title        = {Pith review of: SynRES: Towards Referring Expression Segmentation in the Wild via Synthetic Data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HJUXWN3W}},
  note         = {Machine review of arXiv:2505.17695}
}
read the original abstract

Despite the advances in Referring Expression Segmentation (RES) benchmarks, their evaluation protocols remain constrained, primarily focusing on either single targets with short queries (containing minimal attributes) or multiple targets from distinctly different queries on a single domain. This limitation significantly hinders the assessment of more complex reasoning capabilities in RES models. We introduce WildRES, a novel benchmark that incorporates long queries with diverse attributes and non-distinctive queries for multiple targets. This benchmark spans diverse application domains, including autonomous driving environments and robotic manipulation scenarios, thus enabling more rigorous evaluation of complex reasoning capabilities in real-world settings. Our analysis reveals that current RES models demonstrate substantial performance deterioration when evaluated on WildRES. To address this challenge, we introduce SynRES, an automated pipeline generating densely paired compositional synthetic training data through three innovations: (1) a dense caption-driven synthesis for attribute-rich image-mask-expression triplets, (2) reliable semantic alignment mechanisms rectifying caption-pseudo mask inconsistencies via Image-Text Aligned Grouping, and (3) domain-aware augmentations incorporating mosaic composition and superclass replacement to emphasize generalization ability and distinguishing attributes over object categories. Experimental results demonstrate that models trained with SynRES achieve state-of-the-art performance, improving gIoU by 2.0% on WildRES-ID and 3.8% on WildRES-DS. Code and datasets are available at https://github.com/UTLLab/SynRES.

Figures

Figures reproduced from arXiv: 2505.17695 by the authors.

Figure 1
Figure 1. (a) Comparison of existing referring expression segmentation (RES) benchmarks with our proposed benchmark, WildRES, which [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Number of attributions in classic RES datasets vs. Wil [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Overview of the step 1 and 2 in SynRES. The process begins by creating distinctive [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Image-text aligned grouping for reliable pseudo mask [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Augmented examples in the step 3. Mosaic augmentation is applied using synthetic images and masks containing one original real [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Qualitative results on WildRES, comparing LISA trained w/o and w/ SynRES. [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

60 extracted references · 52 canonical work pages

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023. 3, 4, 1

    arXiv 2023

  2. [2]

    Vqa: Visual question answering

    Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zitnick, and Devi Parikh. Vqa: Visual question answering. In ICCV, pages 2425–2433,

    work page

  3. [3]

    Coco- stuff: Thing and stuff classes in context

    Holger Caesar, Jasper Uijlings, and Vittorio Ferrari. Coco- stuff: Thing and stuff classes in context. In CVPR, pages 1209–1218, 2018. 1, 2, 3, 4, 7

    work page 2018

  4. [4]

    Detect what you can: De- tecting and representing objects using holistic models and body parts

    Xianjie Chen, Roozbeh Mottaghi, Xiaobai Liu, Sanja Fidler, Raquel Urtasun, and Alan Yuille. Detect what you can: De- tecting and representing objects using holistic models and body parts. In CVPR, pages 1971–1978, 2014. 2, 3, 7

    work page 1971

  5. [5]

    Sam4mllm: Enhance multi- modal large language model for referring expression seg- mentation

    Yi-Chia Chen, Wei-Hua Li, Cheng Sun, Yu-Chiang Frank Wang, and Chu-Song Chen. Sam4mllm: Enhance multi- modal large language model for referring expression seg- mentation. In ECCV, pages 323–340. Springer, 2024. 2, 3

    work page 2024

  6. [6]

    The cityscapes dataset for semantic urban scene understanding

    Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In CVPR, 2016. 2, 4, 7

    work page 2016

  7. [7]

    Divergen: Improving instance segmentation by learning wider data distribution with more diverse generative data

    Chengxiang Fan, Muzhi Zhu, Hao Chen, Yang Liu, Weijia Wu, Huaqi Zhang, and Chunhua Shen. Divergen: Improving instance segmentation by learning wider data distribution with more diverse generative data. In CVPR, pages 3986–3995,

    work page

  8. [8]

    Finding nemo: Negative- mined mosaic augmentation for referring image segmentation

    Seongsu Ha, Chaeyun Kim, Donghwa Kim, Junho Lee, Sangho Lee, and Joonseok Lee. Finding nemo: Negative- mined mosaic augmentation for referring image segmentation. In ECCV, pages 121–137. Springer, 2024. 3, 5

    work page 2024

Show all 60 references
  1. [9]

    Mixgen: A new multi- modal data augmentation

    Xiaoshuai Hao, Yi Zhu, Srikar Appalaraju, Aston Zhang, Wanqian Zhang, Bo Li, and Mu Li. Mixgen: A new multi- modal data augmentation. In WACV, pages 379–389, 2023. 3

    2023

  2. [10]

    Partimagenet: A large, high-quality dataset of parts

    Ju He, Shuo Yang, Shaokang Yang, Adam Kortylewski, Xi- aoding Yuan, Jie-Neng Chen, Shuai Liu, Cheng Yang, Qihang Yu, and Alan Yuille. Partimagenet: A large, high-quality dataset of parts. In ECCV, pages 128–145. Springer, 2022. 2, 3, 7

    2022

  3. [11]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. NeurIPS, 33:6840–6851, 2020. 3

    2020

  4. [12]

    Beyond one-to-one: Rethinking the referring image segmentation

    Yutao Hu, Qixiong Wang, Wenqi Shao, Enze Xie, Zhenguo Li, Jungong Han, and Ping Luo. Beyond one-to-one: Rethinking the referring image segmentation. In ICCV, pages 4067–4077,

  5. [13]

    Gpt-4o system card

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024. 2

    2024 arXiv

  6. [14]

    Armada: Attribute- based multimodal data augmentation

    Xiaomeng Jin, Jeonghwan Kim, Yu Zhou, Kuan-Hao Huang, Te-Lin Wu, Nanyun Peng, and Heng Ji. Armada: Attribute- based multimodal data augmentation. arXiv preprint arXiv:2408.10086, 2024. 3

    2024 arXiv

  7. [15]

    Referitgame: Referring to objects in pho- tographs of natural scenes

    Sahar Kazemzadeh, Vicente Ordonez, Mark Matten, and Tamara Berg. Referitgame: Referring to objects in pho- tographs of natural scenes. In EMNLP, pages 787–798, 2014. 2

    2014

  8. [16]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. In ICCV, pages 4015–4026, 2023. 2, 3

    2023

  9. [17]

    Lisa: Reasoning segmentation via large language model

    Xin Lai, Zhuotao Tian, Yukang Chen, Yanwei Li, Yuhui Yuan, Shu Liu, and Jiaya Jia. Lisa: Reasoning segmentation via large language model. In CVPR, pages 9579–9589, 2024. 1, 2, 3, 5, 6, 7

    2024

  10. [18]

    Bigdatasetgan: Synthe- sizing imagenet with pixel-wise annotations

    Daiqing Li, Huan Ling, Seung Wook Kim, Karsten Kreis, Sanja Fidler, and Antonio Torralba. Bigdatasetgan: Synthe- sizing imagenet with pixel-wise annotations. In CVPR, pages 21330–21340, 2022. 3

    2022

  11. [19]

    Gres: Gener- alized referring expression segmentation

    Chang Liu, Henghui Ding, and Xudong Jiang. Gres: Gener- alized referring expression segmentation. In CVPR, pages 23592–23601, 2023. 1, 2, 3, 4, 7

    2023

  12. [20]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In CVPR, pages 26296–26306, 2024. 2, 3, 7

    2024

  13. [21]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. NeurIPS, 36, 2024. 2, 3, 7

    2024

  14. [22]

    Learning multimodal data augmentation in feature space

    Zichang Liu, Zhiqiang Tang, Xingjian Shi, Aston Zhang, Mu Li, Anshumali Shrivastava, and Andrew Gordon Wilson. Learning multimodal data augmentation in feature space. In ICLR, 2023. 3

    2023

  15. [23]

    Generation and com- prehension of unambiguous object descriptions

    Junhua Mao, Jonathan Huang, Alexander Toshev, Oana Cam- buru, Alan L Yuille, and Kevin Murphy. Generation and com- prehension of unambiguous object descriptions. In CVPR, pages 11–20, 2016. 2, 3, 7

    2016

  16. [24]

    Arm- bench: An object-centric benchmark dataset for robotic ma- nipulation

    Chaitanya Mitash, Fan Wang, Shiyang Lu, Vikedo Terhuja, Tyler Garaas, Felipe Polido, and Manikantan Nambi. Arm- bench: An object-centric benchmark dataset for robotic ma- nipulation. In ICRA, pages 9132–9139. IEEE, 2023. 2, 4, 7

    2023

  17. [25]

    The mapillary vistas dataset for semantic understanding of street scenes

    Gerhard Neuhold, Tobias Ollmann, Samuel Rota Bulo, and Peter Kontschieder. The mapillary vistas dataset for semantic understanding of street scenes. In ICCV, pages 4990–4999,

  18. [26]

    Dataset diffusion: Diffusion-based synthetic data generation for pixel-level semantic segmentation

    Quang Nguyen, Truong Vu, Anh Tran, and Khoi Nguyen. Dataset diffusion: Diffusion-based synthetic data generation for pixel-level semantic segmentation. NeurIPS, 2023. 2, 3

    2023

  19. [27]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In ICML, pages 8748–8763. PMLR, 2021. 3

    2021

  20. [28]

    Paco: Parts and attributes of common objects

    Vignesh Ramanathan, Anmol Kalia, Vladan Petrovic, Yi Wen, Baixue Zheng, Baishan Guo, Rui Wang, Aaron Marquez, Rama Kovvuri, Abhishek Kadian, et al. Paco: Parts and attributes of common objects. In CVPR, pages 7141–7151,

  21. [29]

    Glamm: Pixel grounding large multimodal model

    Hanoona Rasheed, Muhammad Maaz, Sahal Shaji, Abdelrah- man Shaker, Salman Khan, Hisham Cholakkal, Rao M Anwer, Eric Xing, Ming-Hsuan Yang, and Fahad S Khan. Glamm: Pixel grounding large multimodal model. In CVPR, pages 13009–13018, 2024. 2, 3

    2024

  22. [30]

    SAM 2: Segment anything in images and videos

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman R¨adle, Chloe Rolland, Laura Gustafson, Eric Mintun, Junting Pan, Kalyan Vasudev Alwala, Nicolas Carion, Chao-Yuan Wu, Ross Girshick, Piotr Dollar, and Christoph Feichten...

  23. [31]

    Pixellm: Pixel reasoning with large multimodal model

    Zhongwei Ren, Zhicheng Huang, Yunchao Wei, Yao Zhao, Dongmei Fu, Jiashi Feng, and Xiaojie Jin. Pixellm: Pixel reasoning with large multimodal model. In CVPR, pages 26374–26383, 2024. 2, 3

    2024

  24. [32]

    Grounding of textual phrases in images by reconstruction

    Anna Rohrbach, Marcus Rohrbach, Ronghang Hu, Trevor Darrell, and Bernt Schiele. Grounding of textual phrases in images by reconstruction. In ECCV, pages 817–834. Springer,

  25. [33]

    Crowdhuman: A bench- mark for detecting human in a crowd

    Shuai Shao, Zijian Zhao, Boxun Li, Tete Xiao, Gang Yu, Xiangyu Zhang, and Jian Sun. Crowdhuman: A bench- mark for detecting human in a crowd. arXiv preprint arXiv:1805.00123, 2018. 2, 4, 7

    2018 arXiv

  26. [34]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In ICLR, 2021. 3

    2021

  27. [35]

    Dreammask: Boosting open-vocabulary panoptic segmenta- tion with synthetic data

    Yuanpeng Tu, Xi Chen, Ser-Nam Lim, and Hengshuang Zhao. Dreammask: Boosting open-vocabulary panoptic segmenta- tion with synthetic data. arXiv preprint arXiv:2501.02048,

    arXiv

  28. [36]

    Cris: Clip-driven referring image segmentation

    Zhaoqing Wang, Yu Lu, Qiang Li, Xunqiang Tao, Yandong Guo, Mingming Gong, and Tongliang Liu. Cris: Clip-driven referring image segmentation. In CVPR, pages 11686–11695,

  29. [37]

    Towards report- ing bias in visual-language datasets: bimodal augmentation by decoupling object-attribute association

    Qiyu Wu, Mengjie Zhao, Yutong He, Lang Huang, Junya Ono, Hiromi Wakaki, and Yuki Mitsufuji. Towards report- ing bias in visual-language datasets: bimodal augmentation by decoupling object-attribute association. arXiv preprint arXiv:2310.01330, 2023. 3

    arXiv

  30. [38]

    Diffumask: Synthesizing images with pixel-level annotations for semantic segmentation using diffu- sion models

    Weijia Wu, Yuzhong Zhao, Mike Zheng Shou, Hong Zhou, and Chunhua Shen. Diffumask: Synthesizing images with pixel-level annotations for semantic segmentation using diffu- sion models. In ICCV, pages 1206–1217, 2023. 2

    2023

  31. [39]

    Gsva: Generalized segmentation via multimodal large language models

    Zhuofan Xia, Dongchen Han, Yizeng Han, Xuran Pan, Shiji Song, and Gao Huang. Gsva: Generalized segmentation via multimodal large language models. In CVPR, pages 3858– 3869, 2024. 2, 3, 6, 7

    2024

  32. [40]

    Sana: Efficient high-resolution image synthesis with lin- ear diffusion transformers

    Enze Xie, Junsong Chen, Junyu Chen, Han Cai, Haotian Tang, Yujun Lin, Zhekai Zhang, Muyang Li, Ligeng Zhu, Yao Lu, et al. Sana: Efficient high-resolution image synthesis with lin- ear diffusion transformers. arXiv preprint arXiv:2410.10629,

    arXiv

  33. [41]

    Mosaicfusion: Diffusion models as data augmenters for large vocabulary instance segmentation

    Jiahao Xie, Wei Li, Xiangtai Li, Ziwei Liu, Yew Soon Ong, and Chen Change Loy. Mosaicfusion: Diffusion models as data augmenters for large vocabulary instance segmentation. IJCV, pages 1–20, 2024. 3, 5

    2024

  34. [42]

    Bridging vision and language encoders: Parameter-efficient tuning for referring image segmentation

    Zunnan Xu, Zhihong Chen, Yong Zhang, Yibing Song, Xiang Wan, and Guanbin Li. Bridging vision and language encoders: Parameter-efficient tuning for referring image segmentation. In ICCV, pages 17503–17512, 2023. 2

    2023

  35. [43]

    Panoptic scene graph gen- eration

    Jingkang Yang, Yi Zhe Ang, Zujin Guo, Kaiyang Zhou, Wayne Zhang, and Ziwei Liu. Panoptic scene graph gen- eration. In ECCV, pages 178–196. Springer, 2022. 3

    2022

  36. [44]

    Freemask: Synthetic images with dense annotations make stronger segmentation models

    Lihe Yang, Xiaogang Xu, Bingyi Kang, Yinghuan Shi, and Hengshuang Zhao. Freemask: Synthetic images with dense annotations make stronger segmentation models. NeurIPS,

  37. [45]

    Lavt: Language-aware vi- sion transformer for referring image segmentation

    Zhao Yang, Jiaqi Wang, Yansong Tang, Kai Chen, Heng- shuang Zhao, and Philip HS Torr. Lavt: Language-aware vi- sion transformer for referring image segmentation. In CVPR, pages 18155–18165, 2022. 2, 7

    2022

  38. [46]

    Seggen: Supercharging segmentation models with text2mask and mask2img synthesis

    Hanrong Ye, Jason Kuen, Qing Liu, Zhe Lin, Brian Price, and Dan Xu. Seggen: Supercharging segmentation models with text2mask and mask2img synthesis. In ECCV, pages 352–370. Springer, 2024. 2, 3

    2024

  39. [47]

    Pix2cap-coco: Advancing visual comprehension via pixel-level captioning

    Zuyao You, Junke Wang, Lingyu Kong, Bo He, and Zux- uan Wu. Pix2cap-coco: Advancing visual comprehension via pixel-level captioning. arXiv preprint arXiv:2501.13893,

    arXiv

  40. [48]

    From image descriptions to visual denotations: New similarity metrics for semantic inference over event descrip- tions

    Peter Young, Alice Lai, Micah Hodosh, and Julia Hocken- maier. From image descriptions to visual denotations: New similarity metrics for semantic inference over event descrip- tions. Transactions of the Association for Computational Linguistics, 2:67–78, 2014. 3

    2014

  41. [49]

    Coca: Contrastive captioners are image-text foundation models

    Jiahui Yu, Zirui Wang, Vijay Vasudevan, Legg Yeung, Mo- jtaba Seyedhosseini, and Yonghui Wu. Coca: Contrastive captioners are image-text foundation models. TMLR, 2022. 2, 3

    2022

  42. [50]

    Modeling context in referring expressions

    Licheng Yu, Patrick Poirson, Shan Yang, Alexander C Berg, and Tamara L Berg. Modeling context in referring expressions. In ECCV, pages 69–85. Springer, 2016. 1, 2, 3, 7

    2016

  43. [51]

    Pseudo- ris: Distinctive pseudo-supervision generation for referring image segmentation

    Seonghoon Yu, Paul Hongsuck Seo, and Jeany Son. Pseudo- ris: Distinctive pseudo-supervision generation for referring image segmentation. In ECCV, pages 18–36. Springer, 2024. 2, 3, 4

    2024

  44. [52]

    Revisiting counterfactual prob- lems in referring expression comprehension

    Zhihan Yu and Ruifan Li. Revisiting counterfactual prob- lems in referring expression comprehension. In CVPR, pages 13438–13448, 2024. 1

    2024

  45. [53]

    Datasetgan: Efficient labeled data factory with minimal human effort

    Yuxuan Zhang, Huan Ling, Jun Gao, Kangxue Yin, Jean- Francois Lafleche, Adela Barriuso, Antonio Torralba, and Sanja Fidler. Datasetgan: Efficient labeled data factory with minimal human effort. In CVPR, pages 10145–10155, 2021. 3

    2021

  46. [54]

    Evf-sam: Early vision-language fusion for text-prompted seg- ment anything model

    Yuxuan Zhang, Tianheng Cheng, Rui Hu, Lei Liu, Heng Liu, Longjin Ran, Xiaoxin Chen, Wenyu Liu, and Xinggang Wang. Evf-sam: Early vision-language fusion for text-prompted seg- ment anything model. arXiv preprint arXiv:2406.20076, 2024. 2, 3

    2024 arXiv

  47. [55]

    Psalm: Pixelwise segmentation with large multi-modal model

    Zheng Zhang, Yeyao Ma, Enming Zhang, and Xiang Bai. Psalm: Pixelwise segmentation with large multi-modal model. In ECCV, pages 74–91. Springer, 2024. 2, 3

    2024

  48. [56]

    X-paste: Revisiting scalable copy-paste for in- stance segmentation using clip and stablediffusion

    Hanqing Zhao, Dianmo Sheng, Jianmin Bao, Dongdong Chen, Dong Chen, Fang Wen, Lu Yuan, Ce Liu, Wenbo Zhou, Qi Chu, et al. X-paste: Revisiting scalable copy-paste for in- stance segmentation using clip and stablediffusion. In ICML, pages 42098–42109. PMLR, 2023. 3 10

    2023

  49. [57]

    Unleashing text-to-image diffusion models for visual perception

    Wenliang Zhao, Yongming Rao, Zuyan Liu, Benlin Liu, Jie Zhou, and Jiwen Lu. Unleashing text-to-image diffusion models for visual perception. In ICCV, pages 5729–5739,

  50. [58]

    Scene parsing through ade20k dataset

    Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Bar- riuso, and Antonio Torralba. Scene parsing through ade20k dataset. In CVPR, 2017. 2, 3, 7

    2017

  51. [59]

    Generalized decoding for pixel, image, and language

    Xueyan Zou, Zi-Yi Dou, Jianwei Yang, Zhe Gan, Linjie Li, Chunyuan Li, Xiyang Dai, Harkirat Behl, Jianfeng Wang, Lu Yuan, et al. Generalized decoding for pixel, image, and language. In CVPR, pages 15116–15127, 2023. 7

    2023

  52. [60]

    the cat sitting on the bench next to big green wooden boat in the center of the image

    Xueyan Zou, Jianwei Yang, Hao Zhang, Feng Li, Linjie Li, Jianfeng Wang, Lijuan Wang, Jianfeng Gao, and Yong Jae Lee. Segment everything everywhere all at once. NeurIPS, 36:19769–19782, 2023. 2, 7 11 Supplementary Materials A . Details of WildRES 1 B . Referring Expression Attr...

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.