Pith. sign in

REVIEW 2 major objections 7 minor 48 references

From Local Details to Global Context: Advancing Vision-Language Models with Attention-Based Selection

T0 review · 2 major / 7 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that attention-guided cropping in image space and in the model's feature space, combined with soft matching of text descriptions, lets a frozen vision-language model outperform prior zero-shot methods and match fine-tuned…

desk verdict Solid training-free CLIP boost with a neat combination, but the paper's signature 'global context' mechanism is under-tested—worth refereeing with a request for a control. read the letter →

arxiv 2505.13233 v1 pith:RPXPLSEU submitted 2025-05-19 cs.CV

Lincan Cai , Jingxuan Kang , Shuang Li , Wenxuan Ma , Binhui Xie , Zhida Qin , Jian Liang This is my paper · ORCID
classification cs.CV
keywords attention-guidedcroppingvision-languagemodelszero-shotclassificationout-of-distributiongeneralizationfeature-spaceselectionsoftmatchingtraining-freeadaptation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a training-free preprocessing step can make frozen vision-language models like CLIP classify images better, both on the original data and on shifted variants. It introduces Attention-Based Selection (ABS), which replaces random cropping with crops centered where a self-supervised attention map signals the main object, and it adds a second crop taken from the model's own feature map before the final transformer layer so the local crop carries global category context. A soft-matching step then weights which LLM-generated class descriptions are allowed to vote for each crop. The reported result is that ABS beats the previous best training-free method on ImageNet variants by up to 6.21 points, outperforms all compared zero-shot baselines on five of six classification datasets, and on several out-of-distribution benchmarks beats methods that fine-tune prompts. The reason to care is that it promises stronger zero-shot classification without any training or per-dataset adaptation.

What carries the argument

The load-bearing mechanism is a pair of attention-guided crops acting as complementary views. Raw-space selection averages DINO's attention maps across heads, sorts the patch attention values, softmaxes the top-k values into a sampling distribution, and draws N crop centers from that distribution with random crop sizes. Feature-space selection applies the same crop to the feature map immediately before CLIP's final transformer layer, resizes the cropped map back to full size with bicubic interpolation, and re-enters it into that final layer so the classification token can combine local crop content with the global semantics already present in the features. Soft matching then produces a per-crop weight vector by taking the softmax of similarities between that crop and every text description, suppressing unrelated descriptions in the final score.

What would settle it

Run ABS on ImageNet with N=60 but replace DINO's attention map with a uniform or spatially shuffled map while keeping crop sizes and soft matching identical. If top-1 accuracy stays near the reported 71.92 on ViT-B/16, then attention-guided selection is not the source of the gain and multi-crop ensembling is doing the work.

Watch

Extended reading notes

Core claim

The paper argues that the failure mode of random-crop visual augmentation is not the cropping itself but where and how the crop is taken. Guided by DINO's attention map, ABS samples crop centers from top-attention patches and random crop sizes around them, producing diverse local views that avoid background objects. To keep global semantics, it takes the original image, extracts the feature map before CLIP's final transformer layer, applies the same crop to that map, bicubic-interpolates it back to full size, and pushes it through the final layer so the classification token interacts with the cropped tokens. This yields features the paper says preserve category-level context that a raw-space crop loses. Finally, for each crop it computes similarities to every LLM-generated description across all classes and softmax-normalizes them, so irrelevant descriptions are down-weighted rather than hard-filtered. The paper claims that this combination accounts for the accuracy gains over the random-crop baseline, with ablations attributing roughly 2.98 percentage points over a description-only baseline on ImageNet, DTD, and ImageNet-V2.

Load-bearing premise

The method assumes that the feature map entering CLIP's last transformer layer already contains enough global category information that cropping it and resizing the crop back to full size still carries that information; if not, the feature-space crop adds nothing beyond an ordinary image crop.

Editorial extensions

If this is right

  • ABS can be dropped onto any frozen CLIP-like encoder without training, so its gains are additive to prompt ensembles and LLM-generated descriptions.
  • Because the method improves out-of-distribution accuracy, attention-guided cropping could reduce sensitivity to background artifacts when the test distribution shifts.
  • The reported gains grow with stronger attention maps, implying further accuracy improvements if the guiding attention map becomes more object-focused.
  • The method stays accurate with as few as 10 crops, so inference cost can be traded down without much loss.
  • On multi-label images such as Food101, the method can be misled by unlabeled objects, meaning label ambiguity rather than crop quality becomes the bottleneck.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not test whether the same crop-then-resize trick could be applied to text-side features; a symmetric text selection step is a natural extension that the reported mechanism does not rule out.
  • Because stronger attention maps give stronger results, replacing DINO with finer-grained segmentation guidance could sharpen crops at object boundaries, though it might over-concentrate on a single object in multi-object scenes.
  • The soft-matching weights could double as a per-crop difficulty signal: high-entropy weight vectors may mark ambiguous crops, which could be used to decide when test-time adaptation is worth applying; the paper does not explore this.
  • A control experiment matching random crops to the same centers and sizes would determine whether the gain comes from the attention guidance itself or from multi-crop ensembling; the paper's ablations do not include that control.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 7 minor

Summary. The paper proposes Attention-Based Selection (ABS), a training-free method for zero-shot classification with vision-language models such as CLIP. ABS first uses a DINO attention map to guide crop selection in the raw image space (raw space selection), obtaining crops centered on salient objects. It then applies the same cropping operation in the feature domain: features are taken from the original image just before the final transformer layer, cropped at the same locations, resized via bicubic interpolation, and fed through the final layer to obtain feature-space crops that supposedly retain global semantic information. A soft matching step then filters LLM-generated text descriptions for each crop by reweighting similarities via a softmax. The final score aggregates similarities between the combined raw/feature crops and the LLM descriptions. The paper reports state-of-the-art results on out-of-distribution generalization and zero-shot classification benchmarks across three CLIP backbones and ten datasets, and compares favorably with few-shot and test-time adaptation methods.

Significance. If the reported results and their interpretation hold, this paper provides a simple, training-free enhancement for zero-shot CLIP classification that requires only a DINO attention map as guidance. The strengths of the work include the breadth of the evaluation (10 datasets, 3 backbones, plus ALIGN, AltCLIP, GroupViT, and BLIP-2), the component-wise ablations showing incremental gains for raw-space selection, feature-space selection, and soft matching, and the release of code. The central conceptual novelty is the feature-space selection mechanism, which is claimed to preserve global semantic information for local crops. However, the current evidence for that mechanism is incomplete: the paper does not isolate the global-context contribution from other factors such as ensembling, interpolation smoothing, or the presence of the original CLS token. The paper itself notes limitations in the final section, but the missing control for the headline mechanism is a load-bearing gap that should be addressed before the claim can be fully accepted.

major comments (2)
  1. [Section 3.3, Eq. (7); Table 7] The central claim that feature-space selection preserves global semantic information from the original image is not isolated from confounds. Table 7 varies the depth of Fmid, which correlates with both global context and the general quality of features, but it never compares the proposed Ffs against a control that removes the original-image global-context channel while keeping the rest of the pipeline identical. A natural control would extract Fmid from the cropped image itself (i.e., f_{l-1}(x_i)) rather than from the original image f_{l-1}(x), and use the corresponding CLS token from that forward pass; alternatively, one could keep the original CLS token but replace the resized patch tokens with a repeated global embedding. Without such a control, the gains attributed to 'global semantic information' could instead stem from ensembling N extra features, from bicubic interpolation smoothing, or from the mere presence of the original CLS token. Please add a control that restricts the global-context channel while keeping the same computational pipeline.
  2. [Table 7 (also Section 4.2)] The experimental setup of Table 7 is not clearly specified and appears inconsistent with the other ablations. The caption and surrounding text do not state which dataset is used, whether raw-space selection and soft matching are included, or the value of N. The reported accuracy at layer 11 (67.74) is lower than the Ffs-only row in Table 4 (69.34 on ImageNet) and much lower than the full ABS result (71.92 for the same backbone), making it impossible for the reader to determine what exactly is being ablated. If Table 7 uses only feature-space selection without the other components, that should be stated; if it uses the full method, the numerical inconsistency must be explained. Without clarification, the depth trend in Table 7 cannot serve as support for the global-context mechanism.
minor comments (7)
  1. [Section 3.1] In the preliminary section, 'we demote f as the image encoder' should read 'we denote f as the image encoder'.
  2. [Figure 4 caption] The caption contains a duplicated word: 'on on ImageNet dataset' should be 'on the ImageNet dataset'.
  3. [Table 6] The row label 'DINO-S/1,6' appears to be a typo for 'DINO-S/16'.
  4. [Implementation details] The method uses M=50 LLM descriptions per class, but the paper does not specify which LLM or prompting procedure is used to obtain these descriptions, nor whether they are taken from an existing repository (e.g., CuPL or WCA). This information is essential for reproducibility and should be included.
  5. [Tables 1 and 2] Standard deviations are reported only for ABS in Table 1 and are absent for the baseline methods and for all results in Tables 2 and 4. Reporting the number of runs and providing standard deviations (or confidence intervals) for all methods would better support the claimed state-of-the-art performance.
  6. [Section 3.4, Eq. (10)] The soft matching weights in Eq. (10) are computed from the same cosine similarities that enter the final score, which can amplify the model's prior in a self-reinforcing manner. While the ablation in Table 4 shows a practical benefit, a brief comment on this property would help the reader interpret the mechanism.
  7. [Table 9] The layout of Table 9 is visually unclear in the manuscript: the column headers for CLIP and the values of N are not well separated, and the caption does not explain the 'Crop+Preprocess' row for CLIP, which does not perform cropping. Please reformat the table and clarify the measured operations.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: ABS is a training-free inference-time scoring pipeline whose components are independently specified and evaluated against external benchmarks.

full rationale

I walked the derivation chain of ABS. Raw-space selection uses DINO attention maps to choose crop centers; feature-space selection crops the pre-final-layer feature map of the original image, resizes it, and passes it through the final transformer layer; soft matching reweights LLM descriptions by a softmax over CLIP cosine similarities. None of these steps fits a parameter to test labels: hyperparameters α, β, K, N, and M are fixed uniformly across datasets, and the final score (Eq. 10) is a deterministic reweighted sum of the same CLIP similarities used at inference. The soft-matching weights are computed from the model's own similarity scores, but the predicted class is still obtained from the resulting score without any label supervision; this is a legitimate scoring design, not a reduction of the prediction to its inputs. The only self-citation in the paper is GDA (Wang et al. 2024), where co-author Jian Liang is an author, but GDA appears solely as a baseline in Table 3 and is not used to justify any premise of the method. The skeptic's concern that Table 7 does not isolate the global-context mechanism is a question about experimental control and confounds, not about circularity of the derivation. The method is self-contained and its reported gains are externally checkable against standard benchmarks.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

No new physical or conceptual entities are introduced. The method's hyperparameters are hand-set but held fixed. The key assumption is that pre-final-layer CLIP features preserve global category semantics under spatial cropping, which is plausible but not proven.

free parameters (5)
  • alpha (crop lower bound) = 0.5
    Hand-set crop size lower bound; held constant across all datasets and backbones (Section 4, Implementation Details).
  • beta (crop upper bound) = 0.9
    Hand-set crop size upper bound; held constant.
  • K (top-k attention patches) = 20
    Number of top attention patches sampled for crop centers; chosen by hand; sensitivity analysis shows small effect (Fig. 4c).
  • N (number of crops) = 60
    Number of raw and feature crops per image; sensitivity analysis shows stable performance from N=10 to N=50 (Fig. 4b).
  • M (number of LLM descriptions per class) = 50
    Number of generated descriptions per class; not varied in main experiments.
assumptions (3)
  • domain assumption DINO's last-layer attention map localizes main objects well enough to guide cropping.
    The entire raw-space selection in Sec. 3.2 depends on this; Fig. 5 suggests it holds on sample images.
  • domain assumption CLIP's features just before the final transformer layer contain global category information that survives spatial cropping and resizing.
    Feature-space selection (Sec. 3.3, Eq. 7) only adds value if this holds; Table 7 shows deeper layers work better, consistent with this.
  • domain assumption LLM-generated fine-grained descriptions improve zero-shot alignment.
    Borrowed from CuPL and WCA; the method uses these descriptions as text targets.

how reviews work

0 comments
Cite this review

Pith. "Pith review of From Local Details to Global Context: Advancing Vision-Language Models with Attention-Based Selection." pith.science (2026). https://pith.science/paper/RPXPLSEU

@misc{pith2026250513233,
  author       = {Pith},
  title        = {Pith review of: From Local Details to Global Context: Advancing Vision-Language Models with Attention-Based Selection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RPXPLSEU}},
  note         = {Machine review of arXiv:2505.13233}
}
read the original abstract

Pretrained vision-language models (VLMs), e.g., CLIP, demonstrate impressive zero-shot capabilities on downstream tasks. Prior research highlights the crucial role of visual augmentation techniques, like random cropping, in alignment with fine-grained class descriptions generated by large language models (LLMs), significantly enhancing zero-shot performance by incorporating multi-view information. However, the inherent randomness of these augmentations can inevitably introduce background artifacts and cause models to overly focus on local details, compromising global semantic understanding. To address these issues, we propose an \textbf{A}ttention-\textbf{B}ased \textbf{S}election (\textbf{ABS}) method from local details to global context, which applies attention-guided cropping in both raw images and feature space, supplement global semantic information through strategic feature selection. Additionally, we introduce a soft matching technique to effectively filter LLM descriptions for better alignment. \textbf{ABS} achieves state-of-the-art performance on out-of-distribution generalization and zero-shot classification tasks. Notably, \textbf{ABS} is training-free and even rivals few-shot and test-time adaptation methods. Our code is available at \href{https://github.com/BIT-DA/ABS}{\textcolor{darkgreen}{https://github.com/BIT-DA/ABS}}.

Figures

Figures reproduced from arXiv: 2505.13233 by the authors.

Figure 1
Figure 1. Random cropping for visual augmentation may [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Similarity between the cropped image obtained in [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Framework overview. Raw space selection: We use DINO’s attention map to guide image cropping, avoiding the inclusion of background objects. Feature selection: The original image is used as input and performs cropping on the feature map corresponding to the fine-grained selection before the final layer, to preserve global semantic information. Soft matching: We calculate a weight matrix to filter out irrelevant text … view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: The sensitivity of three hyperparameter: crop ratio [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: The visualization of the DINO and CLIP attention map and the cropped images guided by the attention maps. [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Examples of Food101 dataset. 13 [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 7
Figure 7. Figure 7: The visualization of the DINO attention map and the cropped images guided by attention maps. [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

48 extracted references · 34 canonical work pages

  1. [1]

    L., Borgeaud, S., Brock, A., Nematzadeh, A., Sharifzadeh, S., Binkowski, M., Barreira, R., Vinyals, O., Zisserman, A., and Simonyan, K

    Alayrac, J., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., Ring, R., Rutherford, E., Cabi, S., Han, T., Gong, Z., Samangooei, S., Monteiro, M., Menick, J. L., Borgeaud, S., Brock, A., Nematzadeh, A., Sharifzadeh, S., Binkowski, M., Barreira, R., Vinyals, O., Zisserman, A., and Simonyan, K. Flaming...

    work page 2022

  2. [2]

    Food-101 -- mining discriminative components with random forests

    Bossard, L., Guillaumin, M., and Van Gool, L. Food-101 -- mining discriminative components with random forests. In ECCV, 2014

    work page 2014

  3. [3]

    D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al

    Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. Language models are few-shot learners. NeurIPS, 33: 0 1877--1901, 2020

    work page 1901

  4. [4]

    Emerging properties in self-supervised vision transformers

    Caron, M., Touvron, H., Misra, I., J \'e gou, H., Mairal, J., Bojanowski, P., and Joulin, A. Emerging properties in self-supervised vision transformers. In ICCV, pp.\ 9650--9660, 2021

    work page 2021

  5. [5]

    Attention-based cropping and erasing learning with coarse-to-fine refinement for fine-grained visual classification

    Chen, J., Li, H., Liang, J., Su, X., Zhai, Z., and Chai, X. Attention-based cropping and erasing learning with coarse-to-fine refinement for fine-grained visual classification. Neurocomputing, 501: 0 359--369, 2022 a

    work page 2022

  6. [6]

    Altclip: Altering the language encoder in clip for extended language capabilities

    Chen, Z., Liu, G., Zhang, B.-W., Ye, F., Yang, Q., and Wu, L. Altclip: Altering the language encoder in clip for extended language capabilities. arXiv preprint arXiv:2211.06679, 2022 b

    arXiv 2022

  7. [7]

    Describing textures in the wild

    Cimpoi, M., Maji, S., Kokkinos, I., Mohamed, S., and Vedaldi, A. Describing textures in the wild. In CVPR, pp.\ 3606--3613, 2014

    work page 2014

  8. [8]

    Imagenet: A large-scale hierarchical image database

    Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In CVPR, pp.\ 248--255, 2009

    2009

Show all 48 references
  1. [9]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR, 2020

    2020

  2. [10]

    Diverse data augmentation with diffusions for effective test-time prompt tuning

    Feng, C.-M., Yu, K., Liu, Y., Khan, S., and Zuo, W. Diverse data augmentation with diffusions for effective test-time prompt tuning. In ICCV, pp.\ 2704--2714, 2023

    2023

  3. [11]

    Clip-adapter: Better vision-language models with feature adapters

    Gao, P., Geng, S., Zhang, R., Ma, T., Fang, R., Zhang, Y., Li, H., and Qiao, Y. Clip-adapter: Better vision-language models with feature adapters. IJCV, 132 0 (2): 0 581--595, 2024

    2024

  4. [12]

    Deep residual learning for image recognition

    He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. In CVPR, pp.\ 770--778, 2016

    2016

  5. [13]

    The many faces of robustness: A critical analysis of out-of-distribution generalization

    Hendrycks, D., Basart, S., Mu, N., Kadavath, S., Wang, F., Dorundo, E., Desai, R., Zhu, T., Parajuli, S., Guo, M., et al. The many faces of robustness: A critical analysis of out-of-distribution generalization. In ICCV, pp.\ 8340--8349, 2021 a

    2021

  6. [14]

    Natural adversarial examples

    Hendrycks, D., Zhao, K., Basart, S., Steinhardt, J., and Song, D. Natural adversarial examples. In CVPR, pp.\ 15262--15271, 2021 b

    2021

  7. [15]

    Scaling up visual and vision-language representation learning with noisy text supervision

    Jia, C., Yang, Y., Xia, Y., Chen, Y.-T., Parekh, Z., Pham, H., Le, Q., Sung, Y.-H., Li, Z., and Duerig, T. Scaling up visual and vision-language representation learning with noisy text supervision. In ICML, pp.\ 4904--4916. PMLR, 2021

    2021

  8. [16]

    Visual prompt tuning

    Jia, M., Tang, L., Chen, B.-C., Cardie, C., Belongie, S., Hariharan, B., and Lim, S.-N. Visual prompt tuning. In ECCV, pp.\ 709--727. Springer, 2022

    2022

  9. [17]

    Efficient test-time adaptation of vision-language models

    Karmanov, A., Guan, D., Lu, S., El Saddik, A., and Xing, E. Efficient test-time adaptation of vision-language models. In CVPR, pp.\ 14162--14171, 2024

    2024

  10. [18]

    U., Rasheed, H., Maaz, M., Khan, S., and Khan, F

    Khattak, M. U., Rasheed, H., Maaz, M., Khan, S., and Khan, F. S. Maple: Multi-modal prompt learning. In CVPR, pp.\ 19113--19122, 2023

    2023

  11. [19]

    Proxyclip: Proxy attention improves clip for open-vocabulary segmentation

    Lan, M., Chen, C., Ke, Y., Wang, X., Feng, L., and Zhang, W. Proxyclip: Proxy attention improves clip for open-vocabulary segmentation. In ECCV, pp.\ 70--88. Springer, 2024

    2024

  12. [20]

    Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation

    Li, J., Li, D., Xiong, C., and Hoi, S. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In ICML, pp.\ 12888--12900. PMLR, 2022

    2022

  13. [21]

    Li, J., Li, D., Savarese, S., and Hoi, S. C. H. BLIP-2: bootstrapping language-image pre-training with frozen image encoders and large language models. In ICML, pp.\ 19730--19742, 2023

    2023

  14. [22]

    Visual-text cross alignment: Refining the similarity score in vision-language models

    Li, J., Li, H., Erfani, S., Feng, L., Bailey, J., and Liu, F. Visual-text cross alignment: Refining the similarity score in vision-language models. CoRR, abs/2406.02915, 2024

    2024 arXiv

  15. [23]

    Liu, H., Li, C., Wu, Q., and Lee, Y. J. Visual instruction tuning. In NeurIPS, 2023

    2023

  16. [24]

    Understanding and mitigating overfitting in prompt tuning for vision-language models

    Ma, C., Liu, Y., Deng, J., Xie, L., Dong, W., and Xu, C. Understanding and mitigating overfitting in prompt tuning for vision-language models. IEEE Transactions on Circuits and Systems for Video Technology, 33 0 (9): 0 4616--4629, 2023

    2023

  17. [25]

    and Vondrick, C

    Menon, S. and Vondrick, C. Visual classification via description from large language models. CoRR, abs/2210.07183, 2022

    2022 arXiv

  18. [26]

    M., Vedaldi, A., Zisserman, A., and Jawahar, C

    Parkhi, O. M., Vedaldi, A., Zisserman, A., and Jawahar, C. Cats and dogs. In CVPR, pp.\ 3498--3505. IEEE, 2012

    2012

  19. [27]

    What does a platypus look like? generating customized prompts for zero-shot image classification

    Pratt, S., Covert, I., Liu, R., and Farhadi, A. What does a platypus look like? generating customized prompts for zero-shot image classification. In ICCV, pp.\ 15691--15701, 2023

    2023

  20. [28]

    W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., and Sutskever, I

    Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., and Sutskever, I. Learning transferable visual models from natural language supervision. In ICML, pp.\ 8748--8763, 2021

    2021

  21. [29]

    Do imagenet classifiers generalize to imagenet? In ICML, pp.\ 5389--5400

    Recht, B., Roelofs, R., Schmidt, L., and Shankar, V. Do imagenet classifiers generalize to imagenet? In ICML, pp.\ 5389--5400. PMLR, 2019

    2019

  22. [30]

    Zero-guidance segmentation using zero segment labels

    Rewatbowornwong, P., Chatthee, N., Chuangsuwanich, E., and Suwajanakorn, S. Zero-guidance segmentation using zero segment labels. In ICCV, pp.\ 1162--1172, 2023

    2023

  23. [31]

    M., Koepke, A., Vinyals, O., Schmid, C., and Akata, Z

    Roth, K., Kim, J. M., Koepke, A., Vinyals, O., Schmid, C., and Akata, Z. Waffling around for performance: Visual classification with random words and broad concepts. In ICCV, pp.\ 15746--15757, 2023

    2023

  24. [32]

    What does clip know about a red circle? visual prompt engineering for vlms

    Shtedritski, A., Rupprecht, C., and Vedaldi, A. What does clip know about a red circle? visual prompt engineering for vlms. In ICCV, pp.\ 11987--11997, 2023

    2023

  25. [33]

    Test-time prompt tuning for zero-shot generalization in vision-language models

    Shu, M., Nie, W., Huang, D.-A., Yu, Z., Goldstein, T., Anandkumar, A., and Xiao, C. Test-time prompt tuning for zero-shot generalization in vision-language models. CoRR, abs/2209.07511, 2022

    2022 arXiv

  26. [34]

    Wang, H., Ge, S., Lipton, Z., and Xing, E. P. Learning robust global representations by penalizing local predictive power. NeurIPS, 32, 2019

    2019

  27. [35]

    A hard-to-beat baseline for training-free clip-based adaptation

    Wang, Z., Liang, J., Sheng, L., He, R., Wang, Z., and Tan, T. A hard-to-beat baseline for training-free clip-based adaptation. CoRR, abs/:2402.04087, 2024

    2024 arXiv

  28. [36]

    Caltech-ucsd birds 200

    Welinder, P., Branson, S., Mita, T., Wah, C., Schroff, F., Belongie, S., and Perona, P. Caltech-ucsd birds 200

  29. [37]

    Groupvit: Semantic segmentation emerges from text supervision

    Xu, J., De Mello, S., Liu, S., Byeon, W., Breuel, T., Kautz, J., and Wang, X. Groupvit: Semantic segmentation emerges from text supervision. In CVPR, pp.\ 18134--18144, 2022

    2022

  30. [38]

    Probing inter-modality: Visual parsing with self-attention for vision-and-language pre-training

    Xue, H., Huang, Y., Liu, B., Peng, H., Fu, J., Li, H., and Luo, J. Probing inter-modality: Visual parsing with self-attention for vision-and-language pre-training. NeurIPS, 34: 0 4514--4528, 2021

    2021

  31. [39]

    Visual-language prompt tuning with knowledge-guided context optimization

    Yao, H., Zhang, R., and Xu, C. Visual-language prompt tuning with knowledge-guided context optimization. In CVPR, pp.\ 6757--6767, 2023

    2023

  32. [40]

    Zang, Y., Li, W., Zhou, K., Huang, C., and Loy, C. C. Unified vision and language prompt learning. CoRR, abs/2210.07225, 2022

    2022 arXiv

  33. [41]

    Tip-adapter: Training-free clip-adapter for better vision-language modeling

    Zhang, R., Fang, R., Zhang, W., Gao, P., Li, K., Dai, J., Qiao, Y., and Li, H. Tip-adapter: Training-free clip-adapter for better vision-language modeling. CoRR, abs/2111.03930, 2021

    2021 arXiv

  34. [42]

    H., Zhou, L., Dai, X., Yuan, L., Li, Y., et al

    Zhong, Y., Yang, J., Zhang, P., Li, C., Codella, N., Li, L. H., Zhou, L., Dai, X., Yuan, L., Li, Y., et al. Regionclip: Region-based language-image pretraining. In CVPR, pp.\ 16793--16803, 2022

    2022

  35. [43]

    Places: A 10 million image database for scene recognition

    Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., and Torralba, A. Places: A 10 million image database for scene recognition. TPAMI, 40 0 (6): 0 1452--1464, 2017

    2017

  36. [44]

    C., and Liu, Z

    Zhou, K., Yang, J., Loy, C. C., and Liu, Z. Conditional prompt learning for vision-language models. In CVPR, pp.\ 16816--16825, 2022 a

    2022

  37. [45]

    C., and Liu, Z

    Zhou, K., Yang, J., Loy, C. C., and Liu, Z. Learning to prompt for vision-language models. IJCV, 130 0 (9): 0 2337--2348, 2022 b

    2022

  38. [46]

    Prompt-aligned gradient for prompt tuning

    Zhu, B., Niu, Y., Han, Y., Wu, Y., and Zhang, H. Prompt-aligned gradient for prompt tuning. In ICCV, pp.\ 15659--15669, 2023

    2023

  39. [47]

    Falip: Visual prompt as foveal attention boosts clip zero-shot performance

    Zhuang, J., Hu, J., Mu, L., Hu, R., Liang, X., Ye, J., and Hu, H. Falip: Visual prompt as foveal attention boosts clip zero-shot performance. In ECCV, pp.\ 236--253. Springer, 2025

    2025

  40. [48]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.