Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

Generalized Decoupled Learning for Enhancing Open-Vocabulary Dense Perception

T0 review · 3 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Pith's one-line read This paper argues that CLIP's dense-perception failures stem from deep-layer attention collapsing onto a few "proxy tokens," and that a decoupled content/context fine-tuning scheme fixes the features without task labels.

desk verdict DeCLIP is a genuine, well-ablated contribution to open-vocabulary dense perception, but the video results are confounded by a backbone swap; the core 2D/3D/pose evidence is controlled and convincing. read the letter →

arxiv 2508.11256 v1 pith:T5BHFWNK submitted 2025-08-15 cs.CV cs.AI

classification cs.CVcs.AI
keywords open-vocabularyperceptionCLIPfine-tuningdecoupledself-attentionknowledgedistillationvisionfoundationmodelsdiffusionmodelattentionsemanticsegmentationrepresentationlearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that CLIP's poor performance on dense perception tasks—object detection, segmentation, pose estimation—has an identifiable cause: in the deep transformer layers, image tokens stop attending to spatially or semantically related regions and concentrate on a few "proxy tokens," producing features that lack local discriminability and spatial consistency. To fix this, DeCLIP decouples CLIP's final self-attention into a context stream and a content stream, and trains each stream against a separate teacher. The context stream imitates affinity maps from a vision foundation model (VFM), with a diffusion model's self-attention maps used to fill in object boundaries; the content stream aligns pooled region features with CLIP's own image-crop [CLS] representations, plus a VFM region-correlation constraint. The paper reports that this decoupled design avoids the trade-off that arises when both kinds of supervision are applied to a single feature space. If the claim holds, CLIP can serve as a drop-in backbone for open-vocabulary perception in 2D, 3D, and video without task-specific labels.

What carries the argument

The key mechanism is the decoupled final attention block. Standard CLIP attention computes an output by attending over values using a softmax of query-key products; DeCLIP rewrites the last block so that a context stream $\mathbf{z}_{\text{context}}$ is produced from the query/key projections, and a content stream $\mathbf{z}_{\text{content}}$ is produced by aggregating values under the context attention. Context distillation uses a KL divergence between CLIP's token-pair similarity matrix and a VFM affinity matrix that has been element-wise multiplied by fused diffusion-model self-attention maps (SD-Guided Semantic Completion). Content distillation uses a softmax-weighted cosine-similarity

What would settle it

Visualize the final-layer attention maps of a DeCLIP-fine-tuned model using background patch tokens as queries. If image tokens still concentrate on a few global vertical lines—the proxy-token artifact—rather than on semantically matching regions, then the proposed mechanism is not what drives the reported gains. A complementary check: fine-tune with the context distillation loss replaced by random or permuted VFM affinity maps; if dense perception performance is unchanged, the affinity teacher is not the active ingredient.

Watch

Extended reading notes

Core claim

DeCLIP's central discovery is that CLIP's dense-feature deficiency is not a general lack of local information but a specific attention pathology: starting around the 9th layer, both the [CLS] token and ordinary image tokens concentrate their attention on a few "proxy tokens," so that features for pixels within the same object are not correlated. The fix is to change what the last attention block computes. The block is split so that one output, the context feature $\mathbf{z}_{\text{context}}$, is generated from queries and keys and trained to match the pairwise cosine-similarity structure of a VFM's dense features; a second output, the content feature $\mathbf{z}_{\text{content}}$, is the va

Load-bearing premise

The load-bearing premise is that CLIP's dense-perception failure is caused by deep-layer attention collapsing onto a few proxy tokens, and that imitating a VFM's affinity maps (completed by diffusion attention) in the context stream is a sufficient teacher without breaking CLIP's vision-language alignment.

Editorial extensions

If this is right

  • Replacing only the CLIP backbone in existing open-vocabulary detectors and segmenters raises novel- and rare-category performance; for example, F-ViT on OV-COCO novel AP50 goes from 37.6 to 43.3 with the ViT-B variant and from 44.3 to 50.2 with ViT-L.
  • Training-free open-vocabulary semantic segmentation—where pixel-text similarity is used directly—improves from 14.1 average mIoU for vanilla CLIP to 44.1 across eight benchmarks, so no task-specific segmentation head is needed.
  • 3D and video instance segmentation improve when DeCLIP replaces CLIP in OpenMask3D/Open3DIS and CLIP-VIS: on ScanNet200, Open3DIS full AP rises from 23.7 to 26.4, and CLIP-VIS with a ViT-L backbone reaches 49.1 AP on YTVIS21.
  • Because DeCLIP is trained only on COCO images without dense labels, the same fine-tuned weights carry over to tasks whose supervision comes from different datasets, including 3D scenes and video sequences.
  • The decoupled features also improve zero-shot region classification accuracy on COCO panoptic at all tested resolutions, indicating that the enhanced dense features retain vision-language alignment.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: the same "proxy-token" attention pathology may occur in other contrastive vision-language transformers, so the content/context decoupling recipe could be applied to models other than CLIP, provided a suitable VFM teacher is chosen.
  • Inference: the diffusion-completion step suggests a general design—use one teacher for semantic relations and a second teacher for boundary integrity—that could improve other affinity-distillation pipelines beyond this exact setup.
  • Inference: downstream methods could deliberately consume only one of the two streams (context for grouping and segmentation, content for recognition), which might yield further gains in training-free pipelines and is directly testable.
  • Inference: the paper's failure-mode account predicts that the vertical-line artifact in deep CLIP attention maps should disappear in DeCLIP's fine-tuned attention; measuring this artifact on held-out images would be a direct check of the mechanism.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes DeCLIP, an unsupervised fine-tuning framework that modifies CLIP's visual encoder to improve dense open-vocabulary perception. The core idea is to decouple the final self-attention block into "content" and "context" streams: context features are distilled to match semantic affinity maps from a VFM (DINOv2), with Stable Diffusion self-attention maps used to complete object boundaries; content features are aligned with CLIP [CLS] representations of image crops, with a region-correlation constraint to preserve dense correlations. The fine-tuned encoder is then dropped into existing open-vocabulary systems for 2D detection/segmentation, training-free segmentation, 3D instance segmentation, video instance segmentation, and 6D pose estimation. The paper reports consistent improvements over baselines, with extensive ablations, sanity checks against the EVA-CLIP default backbone, and sensitivity analyses over VFM choice, dataset source, and hyperparameters.

Significance. If the results hold, DeCLIP would provide a broadly applicable, label-free way to improve CLIP's dense representations, with demonstrated gains across 2D, 3D, video, and pose tasks. The paper is strong in its empirical breadth: code is released, ablations isolate each loss component, sanity checks (Tables 10--12) rule out the EVA-CLIP baseline as the sole source of improvement, and the controlled backbone swaps in detection and segmentation support the central claim. The principal weakness is the uncontrolled comparison in the video experiment, where both the backbone architecture and weights change simultaneously. In addition, the causal 'proxy token' narrative in Section 3.2 is supported only by qualitative attention maps, not by an intervention or quantitative diagnostic; this does not invalidate the method, but the framing is stronger than the evidence. The lack of a same-architecture video baseline is the main load-bearing issue for the paper's broad 'state-of-the-art across a broad spectrum of tasks' claim.

major comments (3)
  1. [§5.2, Table 3] The video instance segmentation comparison is confounded. The CLIP-VIS baseline uses ConvNeXt-B, while CLIP-VIS+DeCLIP uses ViT-B/16 and ViT-L/14. The paper states 'Only the backbone network of CLIP-VIS is replaced with DeCLIP,' but this replacement changes both the architecture (ConvNeXt→ViT) and the weights (original CLIP→DeCLIP). Reported gains (e.g., LV-VIS val AP 32.2→34.8/37.7; YTVIS21 AP 42.1→50.6/54.8) may therefore stem from the ViT backbone being better matched to CLIP-VIS's mask-pooling and memory-bank design, not from the decoupled distillation. Because video is explicitly listed in the abstract as one of the task families supporting the central claim, please add same-backbone baselines: CLIP-VIS with frozen original-CLIP ViT-B/16 and ViT-L/14, using the same feature layers and training settings.
  2. [§3.2, Fig. 2] The 'proxy token' explanation for CLIP's dense-feature deficits is correlational. The paper claims that proxy tokens cause image tokens to lose semantically relevant attention, but no quantitative measure of proxy-token concentration is provided, and no intervention (e.g., masking or suppressing proxy tokens) is performed to show causality. The ablation in Table 8 does demonstrate that decoupled context distillation improves dense tasks, so the method is not invalidated, but the causal framing in the Introduction and Section 3.2 is stronger than the evidence. Please either soften the causal language or add a quantitative diagnostic/intervention to substantiate the mechanism.
  3. [§4.1, Eqs. (4)--(6)] It is unclear whether the decoupled content/context branches are used only during fine-tuning or also at inference. The text says the self-attention module is 'decoupled' and defines content/context features, but downstream experiments 'replace the backbone' with DeCLIP, which suggests the standard CLIP forward pass is used at test time. If the decoupled branches are training-only auxiliaries, this should be stated explicitly, since the gains would then come through gradient updates rather than architectural changes. If the decoupled branches are used at inference, the paper must specify how downstream methods that expect CLIP's standard ViT interface consume the two streams.
minor comments (6)
  1. [§3.1, §4.1] Several equations (Eqs. (1)--(6)) are rendered with missing glyphs and are not interpretable in the provided version. Please ensure all mathematical symbols are typeset correctly.
  2. [Tables 7 and 12] Clarify which VLM baseline is used for the 'DeCLIP (Ours)' numbers: Table 12 reports 64.9 for OpenAI-CLIP and 64.1 for EVA-CLIP, while Table 7 reports 64.1 without specifying. The caption or table notes should state the default VLM consistently.
  3. [§5.2, video experiment] State explicitly whether the vanilla CLIP-VIS baseline uses a frozen backbone, as the DeCLIP version is described as frozen during training. Also specify the exact feature layers used for the ConvNeXt-B baseline.
  4. [Fig. 14(c)] The sensitivity to dataset size/source is reported only for a subset of tasks; state the evaluated tasks and metrics in the figure caption so readers can judge the scope of the robustness claim.
  5. [§5.4, Tables 8--12] No multiple seeds or confidence intervals are reported. Some gains are on the order of 1--3 mIoU/mAP; reporting variance would help assess significance.
  6. [§3.2] The term 'proxy token' is defined only informally. Consider adding a quantitative criterion (e.g., a threshold on attention concentration) or at least a precise operational definition.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claim is supported by external benchmarks and external teacher models; the only self-citation (OV-DQUO) is a non-load-bearing baseline.

full rationale

DeCLIP's training objectives are not equivalent to its evaluation metrics. The context branch (Eqs. 7-10) distills a KL/attention-correlation signal from DINOv2 affinity maps completed by Stable Diffusion self-attention; the content branch (Eqs. 11-13) aligns RoI-pooled region features with CLIP [CLS] crop tokens and VFM region correlations. These teachers are external (DINOv2, SD) or an independently pretrained global signal (CLIP's [CLS]); the reported gains are measured on held-out benchmarks (OV-COCO, OV-LVIS, ScanNet200, LV-VIS, YTVIS, REAL275, etc.) through task-specific heads, not on the training loss. The paper even runs sanity checks (Tables 10-11) to rule out EVA-CLIP as the source of gains. The only overlapping-author citation, OV-DQUO [17], is used as a baseline method in Table 4, not as support for any premise, so it is not load-bearing. A non-circular methodological concern exists in Table 3: the vanilla CLIP-VIS baseline uses ConvNeXt-B while CLIP-VIS+DeCLIP uses ViT-B/16 and ViT-L/14, so the video gains could partly be due to the backbone architecture change; this is a confound, not a self-referential derivation. Footnote 5 notes AM-RADIO is omitted due to lack of OVSS support, a missing-comparison limitation that does not constitute circularity. Overall, no step in the derivation reduces to its own input by construction.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

No new physical or model entities are invented. The method relies on assumed teacher qualities of existing models (DINOv2, Stable Diffusion, CLIP) and on several hyperparameters tuned on validation tasks. The main free parameter is the context loss weight lambda; the rest are standard choices or borrowed settings.

free parameters (6)
  • lambda (context distillation loss weight) = 0.25
    Chosen by sensitivity analysis in Fig. 14(a), balances detection and segmentation.
  • Number of fine-tuned layers = 12 (all)
    Selected via ablation in Fig. 14(b); performance peaks at 12 layers.
  • SD timestep for attention extraction = 45 of 50 steps
    Borrowed from prior work [61,63], not tuned in this paper.
  • CLIP/VFM input resolutions = 560 (CLIP), 490 (DINOv2)
    Set to make token counts equal (1225 tokens) due to different patch sizes.
  • Content distillation region count = grid 1 to 6 blocks per side
    Randomly sampled per image following CLIPSelf [15].
  • RCC and content loss weights = 1.0 (implicit)
    No ablation reported for these weights; set to unit scale.
assumptions (4)
  • domain assumption CLIP's deep-layer 'proxy token' attention is the cause of its weak dense features.
    Motivated by Fig. 2 visualizations; not tested causally.
  • domain assumption DINOv2 semantic affinity maps are a good teacher for spatial consistency.
    Supported by VFM selection ablation Fig. 14(d), but still an empirical choice.
  • domain assumption SD self-attention maps capture object boundaries and complement VFM affinity via multiplication.
    Supported by visual evidence Fig. 6 and ablation Tab. 9; mechanism not separately derived.
  • domain assumption CLIP [CLS] features of image crops provide a valid alignment target for local region features.
    Inherited from CLIPSelf [15].

how reviews work

0 comments
Cite this review

Pith. "Pith review of Generalized Decoupled Learning for Enhancing Open-Vocabulary Dense Perception." pith.science (2026). https://pith.science/paper/T5BHFWNK

@misc{pith2026250811256,
  author       = {Pith},
  title        = {Pith review of: Generalized Decoupled Learning for Enhancing Open-Vocabulary Dense Perception},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/T5BHFWNK}},
  note         = {Machine review of arXiv:2508.11256}
}
read the original abstract

Dense visual perception tasks have been constrained by their reliance on predefined categories, limiting their applicability in real-world scenarios where visual concepts are unbounded. While Vision-Language Models (VLMs) like CLIP have shown promise in open-vocabulary tasks, their direct application to dense perception often leads to suboptimal performance due to limitations in local feature representation. In this work, we present our observation that CLIP's image tokens struggle to effectively aggregate information from spatially or semantically related regions, resulting in features that lack local discriminability and spatial consistency. To address this issue, we propose DeCLIP, a novel framework that enhances CLIP by decoupling the self-attention module to obtain ``content'' and ``context'' features respectively. \revise{The context features are enhanced by jointly distilling semantic correlations from Vision Foundation Models (VFMs) and object integrity cues from diffusion models, thereby enhancing spatial consistency. In parallel, the content features are aligned with image crop representations and constrained by region correlations from VFMs to improve local discriminability. Extensive experiments demonstrate that DeCLIP establishes a solid foundation for open-vocabulary dense perception, consistently achieving state-of-the-art performance across a broad spectrum of tasks, including 2D detection and segmentation, 3D instance segmentation, video instance segmentation, and 6D object pose estimation.} Code is available at https://github.com/xiaomoguhz/DeCLIP

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. DC-Leap: Training-Free Acceleration of dLLMs via Draft-Guided Contiguous Leaping Decoding

    cs.AI 2026-05 conditional novelty 5.0 of 10

    DC-Leap accelerates diffusion LLM decoding by verifying contiguous token spans at a lower confidence threshold and using high-confidence future drafts as look-ahead context, achieving up to 53x speedup with comparable...

Reference graph

Works this paper leans on

116 extracted references · 80 canonical work pages · cited by 1 Pith paper

  1. [1]

    Faster r-cnn: Towards real-time object detection with region proposal networks,

    S. Ren, K. He, R. Girshick, and J. Sun, “Faster r-cnn: Towards real-time object detection with region proposal networks,” �������� �� ������ ����������� ���������� �������, vol. 28, 2015

  2. [2]

    DAB-DETR: Dynamic anchor boxes are better queries for DETR,

    S. Liu, F. Li, H. Zhang, X. Yang, X. Qi, H. Su, J. Zhu, and L. Zhang, “DAB-DETR: Dynamic anchor boxes are better queries for DETR,” in ������������� ���������� �� �������� ���������������, 2022. [Online]. Available: https://openreview.net/forum?id=oMI9PjOb9Jl

  3. [3]

    U-net: Convolutional net- works for biomedical image segmentation,

    O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional net- works for biomedical image segmentation,” in ������� ����� ������� ��� ��� ����������������� ������������������� ����� ���� �������� ������ ����������� ������� �������� ������� ���� ����� ������������ ���� ��� ��. Springer, 2015, pp. 234–241

  4. [4]

    Masked-attention mask transformer for universal image segmenta- tion,

    B. Cheng, I. Misra, A. G. Schwing, A. Kirillov, and R. Girdhar, “Masked-attention mask transformer for universal image segmenta- tion,” in ����������� �� ��� �������� ���������� �� �������� ������ ��� ������� �����������, 2022, pp. 1290–1299

  5. [5]

    Mask dino: Towards a unified transformer-based framework for object detection and segmentation,

    F. Li, H. Zhang, H. Xu, S. Liu, L. Zhang, L. M. Ni, and H.-Y . Shum, “Mask dino: Towards a unified transformer-based framework for object detection and segmentation,” in ����������� �� ��� �������� ���������� �� �������� ������ ��� ������� �����������, 2023, pp. 3041–3050

  6. [6]

    Enhanced training of query-based object detection via selective query recollection,

    F. Chen, H. Zhang, K. Hu, Y .-K. Huang, C. Zhu, and M. Savvides, “Enhanced training of query-based object detection via selective query recollection,” in ����������� �� ��� �������� ���������� �� ���� ����� ������ ��� ������� ����������� ������, June 2023, pp. 23 756– 23 765

  7. [7]

    Deformable detr: Deformable transformers for end-to-end object detection,

    X. Zhu, W. Su, L. Lu, B. Li, X. Wang, and J. Dai, “Deformable detr: Deformable transformers for end-to-end object detection,” ����� �������� ����������������, 2020

  8. [8]

    Open-vocabulary object detection using captions,

    A. Zareian, K. D. Rosa, D. H. Hu, and S.-F. Chang, “Open-vocabulary object detection using captions,” in ����������� �� ��� �������� ���������� �� �������� ������ ��� ������� �����������, 2021, pp. 14 393–14 402

Show all 116 references
  1. [9]

    Aligning bag of regions for open-vocabulary object detection,

    S. Wu, W. Zhang, S. Jin, W. Liu, and C. C. Loy, “Aligning bag of regions for open-vocabulary object detection,” in ����������� �� ��� �������� ���������� �� �������� ������ ��� ������� �����������, 2023, pp. 15 254–15 264

  2. [10]

    Cora: Adapting clip for open- vocabulary detection with region prompting and anchor pre-matching,

    X. Wu, F. Zhu, R. Zhao, and H. Li, “Cora: Adapting clip for open- vocabulary detection with region prompting and anchor pre-matching,” in ����������� �� ��� �������� ���������� �� �������� ������ ��� ������� �����������, 2023, pp. 7031–7040

  3. [11]

    Cat- seg: Cost aggregation for open-vocabulary semantic segmentation,

    S. Cho, H. Shin, S. Hong, A. Arnab, P. H. Seo, and S. Kim, “Cat- seg: Cost aggregation for open-vocabulary semantic segmentation,” in ����������� �� ��� �������� ���������� �� �������� ������ ��� ������� �����������, 2024, pp. 4113–4123

  4. [12]

    Learning transferable visual models from natural language supervi- sion,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever, “Learning transferable visual models from natural language supervi- sion,” in ����������� �� ��� ���� ������������� ���������� �� �������...

  5. [13]

    Scaling language- image pre-training via masking,

    Y . Li, H. Fan, R. Hu, C. Feichtenhofer, and K. He, “Scaling language- image pre-training via masking,” in ����������� �� ��� �������� ���������� �� �������� ������ ��� ������� �����������, 2023, pp. 23 390–23 400

  6. [14]

    Clim: Contrastive language-image mosaic for region representation,

    S. Wu, W. Zhang, L. Xu, S. Jin, W. Liu, and C. C. Loy, “Clim: Contrastive language-image mosaic for region representation,” in ���� �������� �� ��� ���� ���������� �� ��������� ������������, vol. 38, 2024, pp. 6117–6125

  7. [15]

    CLIPSelf: Vision transformer distills itself for open-vocabulary dense prediction,

    S. Wu, W. Zhang, L. Xu, S. Jin, X. Li, W. Liu, and C. C. Loy, “CLIPSelf: Vision transformer distills itself for open-vocabulary dense prediction,” in ��� ������� ������������� ���������� �� �������� ���������������, 2024. [Online]. Available: https://openreview.net/ forum?id=D...

  8. [16]

    Regionclip: Region-based language- image pretraining,

    Y . Zhong, J. Yang, P. Zhang, C. Li, N. Codella, L. H. Li, L. Zhou, X. Dai, L. Yuan, Y . Li �� ���, “Regionclip: Region-based language- image pretraining,” in ����������� �� ��� �������� ���������� �� �������� ������ ��� ������� �����������, 2022, pp. 16 793–16 803

  9. [17]

    Ov-dquo: Open-vocabulary detr with denoising text query train- ing and open-world unknown objects supervision,

    J. Wang, B. Chen, B. Kang, Y . Li, Y . Chen, W. Xian, and H. Chang, “Ov-dquo: Open-vocabulary detr with denoising text query train- ing and open-world unknown objects supervision,” ����� �������� ����������������, 2024

  10. [18]

    Open-vocabulary object de- tection via vision and language knowledge distillation,

    X. Gu, T.-Y . Lin, W. Kuo, and Y . Cui, “Open-vocabulary object de- tection via vision and language knowledge distillation,” ����� �������� ����������������, 2021. 16

  11. [19]

    F-vlm: Open-vocabulary object detection upon frozen vision and language models,

    W. Kuo, Y . Cui, X. Gu, A. Piergiovanni, and A. Angelova, “F-vlm: Open-vocabulary object detection upon frozen vision and language models,” ����� �������� ����������������, 2022

  12. [20]

    Open-vocabulary semantic segmentation with mask-adapted clip,

    F. Liang, B. Wu, X. Dai, K. Li, Y . Zhao, H. Zhang, P. Zhang, P. Vajda, and D. Marculescu, “Open-vocabulary semantic segmentation with mask-adapted clip,” in ����������� �� ��� �������� ���������� �� �������� ������ ��� ������� �����������, 2023, pp. 7061–7070

  13. [21]

    High-resolution image synthesis with latent diffusion models,

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-resolution image synthesis with latent diffusion models,” in ����������� �� ��� �������� ���������� �� �������� ������ ��� ������� �����������, 2022, pp. 10 684–10 695

  14. [22]

    A survey on open-vocabulary detection and seg- mentation: Past, present, and future,

    C. Zhu and L. Chen, “A survey on open-vocabulary detection and seg- mentation: Past, present, and future,” ����� �������� ����������������, 2023

  15. [23]

    Towards open vocabulary learning: A survey,

    J. Wu, X. Li, S. Xu, H. Yuan, H. Ding, Y . Yang, X. Li, J. Zhang, Y . Tong, X. Jiang�� ���, “Towards open vocabulary learning: A survey,” ���� ������������ �� ������� �������� ��� ������� ������������, vol. 46, no. 7, pp. 5092–5113, 2024

  16. [24]

    Object-aware distillation pyramid for open-vocabulary object detection,

    L. Wang, Y . Liu, P. Du, Z. Ding, Y . Liao, Q. Qi, B. Chen, and S. Liu, “Object-aware distillation pyramid for open-vocabulary object detection,” in ����������� �� ��� �������� ���������� �� �������� ������ ��� ������� �����������, 2023, pp. 11 186–11 196

  17. [25]

    Groupvit: Semantic segmentation emerges from text supervision,

    J. Xu, S. De Mello, S. Liu, W. Byeon, T. Breuel, J. Kautz, and X. Wang, “Groupvit: Semantic segmentation emerges from text supervision,” in ����������� �� ��� �������� ���������� �� �������� ������ ��� ������� �����������, 2022, pp. 18 134–18 144

  18. [26]

    Scaling open-vocabulary image segmentation with image-level labels,

    G. Ghiasi, X. Gu, Y . Cui, and T.-Y . Lin, “Scaling open-vocabulary image segmentation with image-level labels,” in �������� ���������� �� �������� ������. Springer, 2022, pp. 540–557

  19. [27]

    Taming self-training for open-vocabulary object detection,

    S. Zhao, S. Schulter, L. Zhao, Z. Zhang, V . K. B. G, Y . Suh, M. Chan- draker, and D. N. Metaxas, “Taming self-training for open-vocabulary object detection,” in ����������� �� ��� �������� ���������� �� �������� ������ ��� ������� ����������� ������, June 2024, pp. 13 938–13 947

  20. [28]

    Detecting twenty-thousand classes using image-level supervision,

    X. Zhou, R. Girdhar, A. Joulin, P. Kr ¨ahenb¨uhl, and I. Misra, “Detecting twenty-thousand classes using image-level supervision,” in �������� ���������� �� �������� ������. Springer, 2022, pp. 350–368

  21. [29]

    Open-vocabulary detr with conditional matching,

    Y . Zang, W. Li, K. Zhou, C. Huang, and C. C. Loy, “Open-vocabulary detr with conditional matching,” in �������� ���������� �� �������� ������. Springer, 2022, pp. 106–122

  22. [30]

    Global knowledge calibration for fast open-vocabulary segmentation,

    K. Han, Y . Liu, J. H. Liew, H. Ding, J. Liu, Y . Wang, Y . Tang, Y . Yang, J. Feng, Y . Zhao �� ���, “Global knowledge calibration for fast open-vocabulary segmentation,” in ����������� �� ��� �������� ������������� ���������� �� �������� ������, 2023, pp. 797–807

  23. [31]

    Densegrounding: Improving dense language- vision semantics for ego-centric 3d visual grounding,

    H. Zheng, H. Shi, Q. Peng, Y . X. Chng, R. Huang, Y . Weng, Z. Shi, and G. Huang, “Densegrounding: Improving dense language- vision semantics for ego-centric 3d visual grounding,” ����� �������� ����������������, 2025

  24. [32]

    Detect anything 3d in the wild,

    H. Zhang, H. Jiang, Q. Yao, Y . Sun, R. Zhang, H. Zhao, H. Li, H. Zhu, and Z. Yang, “Detect anything 3d in the wild,” ����� �������� ����������������, 2025

  25. [33]

    Sam3d: Segment anything in 3d scenes,

    Y . Yang, X. Wu, T. He, H. Zhao, and X. Liu, “Sam3d: Segment anything in 3d scenes,” ����� �������� ����������������, 2023

  26. [34]

    Ovir-3d: Open-vocabulary 3d instance retrieval without training on 3d data,

    S. Lu, H. Chang, E. P. Jing, A. Boularias, and K. Bekris, “Ovir-3d: Open-vocabulary 3d instance retrieval without training on 3d data,” in ���������� �� ����� ��������. PMLR, 2023, pp. 1610–1620

  27. [35]

    Open- vocabulary object 6d pose estimation,

    J. Corsetti, D. Boscaini, C. Oh, A. Cavallaro, and F. Poiesi, “Open- vocabulary object 6d pose estimation,” in����������� �� ��� �������� ���������� �� �������� ������ ��� ������� �����������, 2024, pp. 18 071–18 080

  28. [36]

    Open3dis: Open-vocabulary 3d instance segmentation with 2d mask guidance,

    P. Nguyen, T. D. Ngo, E. Kalogerakis, C. Gan, A. Tran, C. Pham, and K. Nguyen, “Open3dis: Open-vocabulary 3d instance segmentation with 2d mask guidance,” in ����������� �� ��� �������� ���������� �� �������� ������ ��� ������� �����������, 2024, pp. 4018–4028

  29. [37]

    Openmask3d: Open-vocabulary 3d instance segmenta- tion,

    A. Takmaz, E. Fedele, R. W. Sumner, M. Pollefeys, F. Tombari, and F. Engelmann, “Openmask3d: Open-vocabulary 3d instance segmenta- tion,” ����� �������� ����������������, 2023

  30. [38]

    Clip-vis: Adapting clip for open-vocabulary video instance segmentation,

    W. Zhu, J. Cao, J. Xie, S. Yang, and Y . Pang, “Clip-vis: Adapting clip for open-vocabulary video instance segmentation,” ���� ������������ �� �������� ��� ������� ��� ����� ����������, 2024

  31. [39]

    Semantic and sequential alignment for referring video object segmentation,

    F. Pan, H. Fang, F. Li, Y . Xu, Y . Li, L. Benini, and X. Lu, “Semantic and sequential alignment for referring video object segmentation,” in ����������� �� ��� �������� ������ ��� ������� ����������� ������� ����, 2025, pp. 19 067–19 076

  32. [40]

    Unified embedding align- ment for open-vocabulary video instance segmentation,

    H. Fang, P. Wu, Y . Li, X. Zhang, and X. Lu, “Unified embedding align- ment for open-vocabulary video instance segmentation,” in �������� ���������� �� �������� ������. Springer, 2024, pp. 225–241

  33. [41]

    Sigmoid loss for language image pre-training,

    X. Zhai, B. Mustafa, A. Kolesnikov, and L. Beyer, “Sigmoid loss for language image pre-training,” in ����������� �� ��� �������� ������������� ���������� �� �������� ������, 2023, pp. 11 975–11 986

  34. [42]

    Learning mask-aware clip representations for zero-shot segmentation,

    S. Jiao, Y . Wei, Y . Wang, Y . Zhao, and H. Shi, “Learning mask-aware clip representations for zero-shot segmentation,” �������� �� ������ ����������� ���������� �������, vol. 36, pp. 35 631–35 653, 2023

  35. [43]

    Language-driven semantic segmentation,

    B. Li, K. Q. Weinberger, S. Belongie, V . Koltun, and R. Ranftl, “Language-driven semantic segmentation,” ����� �������� ����������������, 2022

  36. [44]

    Open vocabulary semantic segmentation with patch aligned contrastive learning,

    J. Mukhoti, T.-Y . Lin, O. Poursaeed, R. Wang, A. Shah, P. H. Torr, and S.-N. Lim, “Open vocabulary semantic segmentation with patch aligned contrastive learning,” in ����������� �� ��� �������� ���������� �� �������� ������ ��� ������� �����������, 2023, pp. 19 413–19 423

  37. [45]

    Sam-clip: Merging vision foundation models towards semantic and spatial under- standing,

    H. Wang, P. K. A. Vasu, F. Faghri, R. Vemulapalli, M. Farajtabar, S. Mehta, M. Rastegari, O. Tuzel, and H. Pouransari, “Sam-clip: Merging vision foundation models towards semantic and spatial under- standing,” in ����������� �� ��� �������� ���������� �� �������� ������ ��� ��...

  38. [46]

    Open- vocabulary sam: Segment and recognize twenty-thousand classes in- teractively,

    H. Yuan, X. Li, C. Zhou, Y . Li, K. Chen, and C. C. Loy, “Open- vocabulary sam: Segment and recognize twenty-thousand classes in- teractively,” in ���� , 2024

  39. [47]

    Frozenseg: Harmo- nizing frozen foundation models for open-vocabulary segmentation,

    X. Chen, H. Yang, S. Jin, X. Zhu, and H. Yao, “Frozenseg: Harmo- nizing frozen foundation models for open-vocabulary segmentation,” ����� �������� ����������������, 2024

  40. [48]

    Segment anything,

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Lo �� ���, “Segment anything,” in ����������� �� ��� �������� ������������� ���������� �� �������� ������, 2023, pp. 4015–4026

  41. [49]

    Rep- resentation alignment for generation: Training diffusion transformers is easier than you think,

    S. Yu, S. Kwak, H. Jang, J. Jeong, J. Huang, J. Shin, and S. Xie, “Rep- resentation alignment for generation: Training diffusion transformers is easier than you think,” ����� �������� ����������������, 2024

  42. [50]

    A convnet for the 2020s,

    Z. Liu, H. Mao, C.-Y . Wu, C. Feichtenhofer, T. Darrell, and S. Xie, “A convnet for the 2020s,” ����������� �� ��� �������� ���������� �� �������� ������ ��� ������� ����������� ������, 2022

  43. [51]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in ����������� �� ��� ���� ���������� �� �������� ������ ��� ������� �����������, 2016, pp. 770–778

  44. [52]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, “An image is worth 16x16 words: Transformers for image recognition at scale,” ����� �������� ����������������, 2020

  45. [53]

    Attention is all you need,

    A. Vaswani, “Attention is all you need,” �������� �� ������ �������� ���� ���������� �������, 2017

  46. [54]

    Emerging properties in self-supervised vision transformers,

    M. Caron, H. Touvron, I. Misra, H. J ´egou, J. Mairal, P. Bojanowski, and A. Joulin, “Emerging properties in self-supervised vision transformers,” in ����������� �� ��� �������� ������������� ���������� �� �������� ������, 2021, pp. 9650–9660

  47. [55]

    Dinov2: Learning robust visual features without supervision,

    M. Oquab, T. Darcet, T. Moutakanni, H. V o, M. Szafraniec, V . Khali- dov, P. Fernandez, D. Haziza, F. Massa, A. El-Nouby �� ���, “Dinov2: Learning robust visual features without supervision,” ����� �������� ����������������, 2023

  48. [56]

    Sam 2: Segment anything in images and videos,

    N. Ravi, V . Gabeur, Y .-T. Hu, R. Hu, C. Ryali, T. Ma, H. Khedr, R. R ¨adle, C. Rolland, L. Gustafson �� ���, “Sam 2: Segment anything in images and videos,” ����� �������� ����������������, 2024

  49. [57]

    Sclip: Rethinking self-attention for dense vision-language inference,

    F. Wang, J. Mei, and A. Yuille, “Sclip: Rethinking self-attention for dense vision-language inference,” ����� �������� ����������������, 2023

  50. [58]

    Explore the potential of clip for training-free open vocabulary semantic segmentation,

    T. Shao, Z. Tian, H. Zhao, and J. Su, “Explore the potential of clip for training-free open vocabulary semantic segmentation,” in �������� ���������� �� �������� ������. Springer, 2025, pp. 139–156

  51. [59]

    Clearclip: Decomposing clip representations for dense vision-language inference,

    M. Lan, C. Chen, Y . Ke, X. Wang, L. Feng, and W. Zhang, “Clearclip: Decomposing clip representations for dense vision-language inference,” ����� �������� ����������������, 2024

  52. [60]

    Clip-dinoiser: Teaching clip a few dino tricks,

    M. Wysocza ´nska, O. Sim ´eoni, M. Ramamonjisoa, A. Bursuc, T. Trzci ´nski, and P. P ´erez, “Clip-dinoiser: Teaching clip a few dino tricks,” ����� �������� ����������������, 2023

  53. [61]

    Diffusion model is secretly a training-free open vocabulary semantic segmenter,

    J. Wang, X. Li, J. Zhang, Q. Xu, Q. Zhou, Q. Yu, L. Sheng, and D. Xu, “Diffusion model is secretly a training-free open vocabulary semantic segmenter,” ���� ������������ �� ����� ����������, 2025

  54. [62]

    Dynamic prompt learning: Addressing cross-attention leakage for text-based image editing,

    K. Wang, F. Yang, S. Yang, M. A. Butt, and J. van de Weijer, “Dynamic prompt learning: Addressing cross-attention leakage for text-based image editing,” �������� �� ������ ����������� ���������� �������, vol. 36, 2023

  55. [63]

    Cliper: Hierarchically improving spatial representation of clip for open-vocabulary semantic segmentation,

    L. Sun, J. Cao, J. Xie, X. Jiang, and Y . Pang, “Cliper: Hierarchically improving spatial representation of clip for open-vocabulary semantic segmentation,” ����� �������� ����������������, 2024

  56. [64]

    Silc: Improving vision language pretraining with self-distillation,

    M. F. Naeem, Y . Xian, X. Zhai, L. Hoyer, L. Van Gool, and F. Tombari, “Silc: Improving vision language pretraining with self-distillation,” in �������� ���������� �� �������� ������. Springer, 2025, pp. 38–55. 17

  57. [65]

    Exploring open-vocabulary semantic segmentation from clip vision encoder distillation only,

    J. Chen, D. Zhu, G. Qian, B. Ghanem, Z. Yan, C. Zhu, F. Xiao, S. C. Culatana, and M. Elhoseiny, “Exploring open-vocabulary semantic segmentation from clip vision encoder distillation only,” in����������� �� ��� �������� ������������� ���������� �� �������� ������, 2023, pp. 699–710

  58. [66]

    Mask r-cnn,

    K. He, G. Gkioxari, P. Doll ´ar, and R. Girshick, “Mask r-cnn,” in ����������� �� ��� ���� ������������� ���������� �� �������� ������, 2017, pp. 2961–2969

  59. [67]

    Relational knowledge distilla- tion,

    W. Park, D. Kim, Y . Lu, and M. Cho, “Relational knowledge distilla- tion,” in ����������� �� ��� �������� ���������� �� �������� ������ ��� ������� �����������, 2019, pp. 3967–3976

  60. [68]

    Microsoft coco: Common objects in context,

    T.-Y . Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Doll ´ar, and C. L. Zitnick, “Microsoft coco: Common objects in context,” in �������� ����������� ����� ���� �������� ����������� ������� ������������ ��������� ����� ����� ������������ ���� � ��. Springer, ...

  61. [69]

    Decoupled weight decay regularization,

    I. Loshchilov, “Decoupled weight decay regularization,” ����� �������� ����������������, 2017

  62. [70]

    Eva-clip: Improved training techniques for clip at scale,

    Q. Sun, Y . Fang, L. Wu, X. Wang, and Y . Cao, “Eva-clip: Improved training techniques for clip at scale,” ����� �������� ����������������, 2023

  63. [71]

    Vision transform- ers need registers,

    T. Darcet, M. Oquab, J. Mairal, and P. Bojanowski, “Vision transform- ers need registers,” ����� �������� ����������������, 2023

  64. [72]

    Language-grounded indoor 3d semantic segmentation in the wild,

    D. Rozenberszki, O. Litany, and A. Dai, “Language-grounded indoor 3d semantic segmentation in the wild,” in �������� ���������� �� �������� ������. Springer, 2022, pp. 125–141

  65. [73]

    Isbnet: a 3d point cloud instance segmentation network with instance-aware sampling and box-aware dynamic convolution,

    T. D. Ngo, B.-S. Hua, and K. Nguyen, “Isbnet: a 3d point cloud instance segmentation network with instance-aware sampling and box-aware dynamic convolution,” in ����������� �� ��� �������� ���������� �� �������� ������ ��� ������� �����������, 2023, pp. 13 550–13 559

  66. [74]

    Mask3d: Mask transformer for 3d semantic instance segmentation,

    J. Schult, F. Engelmann, A. Hermans, O. Litany, S. Tang, and B. Leibe, “Mask3d: Mask transformer for 3d semantic instance segmentation,” in ���� ���� ������������� ���������� �� �������� ��� ���������� ������. IEEE, 2023, pp. 8216–8223

  67. [75]

    Openscene: 3d scene understanding with open vocabularies,

    S. Peng, K. Genova, C. Jiang, A. Tagliasacchi, M. Pollefeys, T. Funkhouser �� ���, “Openscene: 3d scene understanding with open vocabularies,” in����������� �� ��� �������� ���������� �� �������� ������ ��� ������� �����������, 2023, pp. 815–824

  68. [76]

    A density-based algorithm for discovering clusters in large spatial databases with noise,

    M. Ester, H.-P. Kriegel, J. Sander, X. Xu �� ���, “A density-based algorithm for discovering clusters in large spatial databases with noise,” in ���, vol. 96, no. 34, 1996, pp. 226–231

  69. [77]

    Openins3d: Snap and lookup for 3d open-vocabulary instance seg- mentation,

    Z. Huang, X. Wu, X. Chen, H. Zhao, L. Zhu, and J. Lasenby, “Openins3d: Snap and lookup for 3d open-vocabulary instance seg- mentation,” in �������� ���������� �� �������� ������. Springer, 2024, pp. 169–185

  70. [79]

    In defense of on- line models for video instance segmentation,

    J. Wu, Q. Liu, Y . Jiang, S. Bai, A. Yuille, and X. Bai, “In defense of on- line models for video instance segmentation,” in �������� ���������� �� �������� ������. Springer, 2022, pp. 588–605

  71. [80]

    Simple online and realtime tracking,

    A. Bewley, Z. Ge, L. Ott, F. Ramos, and B. Upcroft, “Simple online and realtime tracking,” in ���� ���� ������������� ���������� �� ����� ���������� ������. Ieee, 2016, pp. 3464–3468

  72. [81]

    Opening up open world tracking,

    Y . Liu, I. E. Zulfikar, J. Luiten, A. Dave, D. Ramanan, B. Leibe, A. O ˇsep, and L. Leal-Taix ´e, “Opening up open world tracking,” in ����������� �� ��� �������� ���������� �� �������� ������ ��� ������� �����������, 2022, pp. 19 045–19 055

  73. [82]

    Xmem: Long-term video object segmentation with an atkinson-shiffrin memory model,

    H. K. Cheng and A. G. Schwing, “Xmem: Long-term video object segmentation with an atkinson-shiffrin memory model,” in �������� ���������� �� �������� ������. Springer, 2022, pp. 640–658

  74. [83]

    Towards open-vocabulary video instance segmentation,

    H. Wang, C. Yan, S. Wang, X. Jiang, X. Tang, Y . Hu, W. Xie, and E. Gavves, “Towards open-vocabulary video instance segmentation,” in ����������� �� ��� �������� ������������� ���������� �� �������� ������, 2023, pp. 4057–4066

  75. [84]

    Video instance segmentation,

    L. Yang, Y . Fan, and N. Xu, “Video instance segmentation,” in ����������� �� ��� �������� ������������� ���������� �� �������� ������, 2019, pp. 5188–5197

  76. [85]

    Lvis: A dataset for large vocabulary instance segmentation,

    A. Gupta, P. Dollar, and R. Girshick, “Lvis: A dataset for large vocabulary instance segmentation,” in ����������� �� ��� �������� ���������� �� �������� ������ ��� ������� �����������, 2019, pp. 5356–5364

  77. [86]

    Occluded video instance segmentation: A benchmark,

    J. Qi, Y . Gao, Y . Hu, X. Wang, X. Liu, X. Bai, S. Belongie, A. Yuille, P. H. Torr, and S. Bai, “Occluded video instance segmentation: A benchmark,” ������������� ������� �� �������� ������, vol. 130, no. 8, pp. 2022–2039, 2022

  78. [87]

    Burst: A benchmark for unifying object recognition, segmentation and tracking in video,

    A. Athar, J. Luiten, P. V oigtlaender, T. Khurana, A. Dave, B. Leibe, and D. Ramanan, “Burst: A benchmark for unifying object recognition, segmentation and tracking in video,” in ����������� �� ��� �������� ������ ���������� �� ������������ �� �������� ������, 2023, pp. 1674– 1683

  79. [88]

    Fs6d: Few-shot 6d pose estimation of novel objects,

    Y . He, Y . Wang, H. Fan, J. Sun, and Q. Chen, “Fs6d: Few-shot 6d pose estimation of novel objects,” in ����������� �� ��� �������� ���������� �� �������� ������ ��� ������� �����������, 2022, pp. 6814–6824

  80. [89]

    Semantically-enriched 3d models for common-sense knowledge,

    M. Savva, A. X. Chang, and P. Hanrahan, “Semantically-enriched 3d models for common-sense knowledge,” in ����������� �� ��� ���� ���������� �� �������� ������ ��� ������� ����������� ���������, 2015, pp. 24–31

  81. [90]

    Normalized object coordinate space for category-level 6d object pose and size estimation,

    H. Wang, S. Sridhar, J. Huang, J. Valentin, S. Song, and L. J. Guibas, “Normalized object coordinate space for category-level 6d object pose and size estimation,” in ����������� �� ��� �������� ���������� �� �������� ������ ��� ������� �����������, 2019, pp. 2642–2651

  82. [91]

    Bop: Benchmark for 6d object pose estimation,

    T. Hodan, F. Michel, E. Brachmann, W. Kehl, A. GlentBuch, D. Kraft, B. Drost, J. Vidal, S. Ihrke, X. Zabulis �� ���, “Bop: Benchmark for 6d object pose estimation,” in ����������� �� ��� �������� ���������� �� �������� ������ ������, 2018, pp. 19–34

  83. [92]

    Bop challenge 2020 on 6d object localization,

    T. Hoda ˇn, M. Sundermeyer, B. Drost, Y . Labb ´e, E. Brachmann, F. Michel, C. Rother, and J. Matas, “Bop challenge 2020 on 6d object localization,” in �������� ����������� ���� ���������� �������� ��� ������ ������ ����� ������������ ���� �� ��. Springer, 2020, pp. 577–594

  84. [93]

    Swin transformer: Hierarchical vision transformer using shifted win- dows,

    Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted win- dows,” in ����������� �� ��� �������� ������������� ���������� �� �������� ������, 2021, pp. 10 012–10 022

  85. [94]

    End-to-end object detection with transformers,

    N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko, “End-to-end object detection with transformers,” in �������� ���������� �� �������� ������. Springer, 2020, pp. 213– 229

  86. [95]

    Region-aware pretraining for open-vocabulary object detection with vision transformers,

    D. Kim, A. Angelova, and W. Kuo, “Region-aware pretraining for open-vocabulary object detection with vision transformers,” in ���� �������� �� ��� �������� ���������� �� �������� ������ ��� ������� �����������, 2023, pp. 11 144–11 154

  87. [96]

    Contrastive feature masking open-vocabulary vision trans- former,

    ——, “Contrastive feature masking open-vocabulary vision trans- former,” in ���� �������� ������������� ���������� �� �������� ������ ������, 2023, pp. 15 556–15 566

  88. [98]

    A simple baseline for open-vocabulary semantic segmentation with pre-trained vision-language model,

    M. Xu, Z. Zhang, F. Wei, Y . Lin, Y . Cao, H. Hu, and X. Bai, “A simple baseline for open-vocabulary semantic segmentation with pre-trained vision-language model,” in �������� ���������� �� �������� ������. Springer, 2022, pp. 736–753

  89. [99]

    Side adapter network for open-vocabulary semantic segmentation,

    M. Xu, Z. Zhang, F. Wei, H. Hu, and X. Bai, “Side adapter network for open-vocabulary semantic segmentation,” in ����������� �� ��� �������� ���������� �� �������� ������ ��� ������� �����������, 2023, pp. 2945–2954

  90. [100]

    Open-vocabulary panoptic segmentation with text-to-image diffusion models,

    J. Xu, S. Liu, A. Vahdat, W. Byeon, X. Wang, and S. De Mello, “Open-vocabulary panoptic segmentation with text-to-image diffusion models,” in ����������� �� ��� �������� ���������� �� �������� ������ ��� ������� �����������, 2023, pp. 2955–2966

  91. [101]

    Convolutions die hard: Open-vocabulary segmentation with single frozen convolutional clip,

    Q. Yu, J. He, X. Deng, X. Shen, and L.-C. Chen, “Convolutions die hard: Open-vocabulary segmentation with single frozen convolutional clip,” �������� �� ������ ����������� ���������� �������, vol. 36, 2024

  92. [102]

    Extract free dense labels from clip,

    C. Zhou, C. C. Loy, and B. Dai, “Extract free dense labels from clip,” in �������� ���������� �� �������� ������. Springer, 2022, pp. 696–712

  93. [103]

    Reco: Retrieve and co-segment for zero-shot transfer,

    G. Shin, W. Xie, and S. Albanie, “Reco: Retrieve and co-segment for zero-shot transfer,” �������� �� ������ ����������� ���������� �������, vol. 35, pp. 33 754–33 767, 2022

  94. [104]

    Learning to generate text-grounded mask for open-world semantic segmentation from only image-text pairs,

    J. Cha, J. Mun, and B. Roh, “Learning to generate text-grounded mask for open-world semantic segmentation from only image-text pairs,” in ����������� �� ��� �������� ���������� �� �������� ������ ��� ������� �����������, 2023, pp. 11 165–11 174

  95. [105]

    Pay attention to your neighbours: Training-free open-vocabulary semantic segmentation,

    S. Hajimiri, I. Ben Ayed, and J. Dolz, “Pay attention to your neighbours: Training-free open-vocabulary semantic segmentation,” in ����������� �� ��� �������� ������ ���������� �� ������������ �� �������� ������, 2025

  96. [106]

    Resclip: Residual attention for training-free dense vision-language inference,

    Y . Yang, J. Deng, W. Li, and L. Duan, “Resclip: Residual attention for training-free dense vision-language inference,” in ����������� �� 18 ��� �������� ������ ��� ������� ����������� ����������, 2025, pp. 29 968–29 978

  97. [107]

    Proxyclip: Proxy attention improves clip for open-vocabulary segmentation,

    M. Lan, C. Chen, Y . Ke, X. Wang, L. Feng, and W. Zhang, “Proxyclip: Proxy attention improves clip for open-vocabulary segmentation,” in �������� ���������� �� �������� ������. Springer, 2024, pp. 70– 88

  98. [108]

    Coco-stuff: Thing and stuff classes in context,

    H. Caesar, J. Uijlings, and V . Ferrari, “Coco-stuff: Thing and stuff classes in context,” in ����������� �� ��� ���� ���������� �� �������� ������ ��� ������� �����������, 2018, pp. 1209–1218

  99. [109]

    Semantic understanding of scenes through the ade20k dataset,

    B. Zhou, H. Zhao, X. Puig, T. Xiao, S. Fidler, A. Barriuso, and A. Torralba, “Semantic understanding of scenes through the ade20k dataset,” ������������� ������� �� �������� ������, vol. 127, pp. 302– 321, 2019

  100. [110]

    The pascal visual object classes (voc) challenge,

    M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zisser- man, “The pascal visual object classes (voc) challenge,” ������������� ������� �� �������� ������, vol. 88, pp. 303–338, 2010

  101. [111]

    The role of context for object detection and semantic segmentation in the wild,

    R. Mottaghi, X. Chen, X. Liu, N.-G. Cho, S.-W. Lee, S. Fidler, R. Urtasun, and A. Yuille, “The role of context for object detection and semantic segmentation in the wild,” in ����������� �� ��� ���� ���������� �� �������� ������ ��� ������� �����������, 2014, pp. 891– 898

  102. [112]

    Decoupling zero-shot semantic segmentation,

    J. Ding, N. Xue, G.-S. Xia, and D. Dai, “Decoupling zero-shot semantic segmentation,” in ����������� �� ��� �������� ���������� �� �������� ������ ��� ������� �����������, 2022, pp. 11 583–11 592

  103. [113]

    The cityscapes dataset for semantic urban scene understanding,

    M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Be- nenson, U. Franke, S. Roth, and B. Schiele, “The cityscapes dataset for semantic urban scene understanding,” in ����������� �� ��� ���� ���������� �� �������� ������ ��� ������� �����������, 2016, pp. 3213–3223

  104. [114]

    Refining clip’s spatial awareness: A visual-centric perspective,

    C. Qiu, Y . Wu, W. Ke, X. Bai, and T. Zhang, “Refining clip’s spatial awareness: A visual-centric perspective,” ����� �������� ����������������, 2025

  105. [115]

    Panoptic seg- mentation,

    A. Kirillov, K. He, R. Girshick, C. Rother, and P. Doll ´ar, “Panoptic seg- mentation,” in ����������� �� ��� �������� ���������� �� �������� ������ ��� ������� �����������, 2019, pp. 9404–9413

  106. [116]

    Am-radio: Agglomerative vision foundation model reduce all domains into one,

    M. Ranzinger, G. Heinrich, J. Kautz, and P. Molchanov, “Am-radio: Agglomerative vision foundation model reduce all domains into one,” in ����������� �� ��� �������� ���������� �� �������� ������ ��� ������� �����������, 2024, pp. 12 490–12 500

  107. [117]

    A tale of two features: Stable diffusion complements dino for zero-shot semantic correspondence,

    J. Zhang, C. Herrmann, J. Hur, L. Polania Cabrera, V . Jampani, D. Sun, and M.-H. Yang, “A tale of two features: Stable diffusion complements dino for zero-shot semantic correspondence,” �������� �� ������ ����������� ���������� �������, vol. 36, pp. 45 533–45 547, 2023

  108. [118]

    Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning,

    P. Sharma, N. Ding, S. Goodman, and R. Soricut, “Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning,” in ����������� �� ��� ���� ������ ������� �� ��� ����� ������� ��� ������������� ����������� ������� �� ���� �������, 2018, pp. ...

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.