REVIEW 1 cited by
VT-CLIP: Enhancing Vision-Language Models with Visual-guided Texts
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Contrastive Language-Image Pre-training (CLIP) has drawn increasing attention recently for its transferable visual representation learning. However, due to the semantic gap within datasets, CLIP's pre-trained image-text alignment becomes sub-optimal on downstream tasks, which severely harms its transferring performance. To better adapt the cross-modality embedding space, we propose to enhance CLIP via Visual-guided Texts, named VT-CLIP. Specifically, we guide textual features of different categories to adaptively explore informative regions on the image and aggregate visual features by attention mechanisms. In this way, the texts become visual-guided, namely, more semantically correlated with downstream images, which greatly benefits the category-wise matching process. In few-shot settings, we evaluate our VT-CLIP on 11 well-known classification datasets to demonstrate its effectiveness.
Forward citations
Cited by 1 Pith paper
-
Text and Image Are Mutually Beneficial: Enhancing Training-Free Few-Shot Classification with CLIP
TIMO improves training-free CLIP few-shot classification by mutually guiding text and image features, and a tuned variant TIMO-S reports state-of-the-art accuracy with roughly 100x less time than training-required methods.
Discussion (0). Continue with ORCID to comment.