Pith. sign in

REVIEW 3 major objections 6 minor 70 references

ProKeR: A Kernel Perspective on Few-Shot Adaptation of Large Vision-Language Models

T0 review · 3 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Framing Tip-Adapter as a Nadaraya-Watson estimator, ProKeR solves a proximal multi-output kernel ridge regression in closed form and reports state-of-the-art few-shot accuracy across 11 datasets.

desk verdict A useful kernel reframing of Tip-Adapter with strong empirical numbers, but the printed closed-form solution doesn't match the stated objective—a load-bearing error that needs correction before the theory can be trusted. read the letter →

arxiv 2501.11175 v1 pith:555LMP2U submitted 2025-01-19 cs.CV cs.AIcs.LG

Yassir Bendou , Amine Ouasfi , Vincent Gripon , Adnane Boukhayma This is my paper · ORCID
classification cs.CVcs.AIcs.LG MSC 62G0868T45
keywords few-shotadaptationCLIPtraining-freeadapterkernelridgeregressionNadaraya-WatsonestimatorreproducingHilbertspacecachemodelvision-language
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that training-free caching adapters for CLIP, exemplified by Tip-Adapter, are local nonparametric regressors: the cache term is a modified Nadaraya-Watson estimator with an RBF kernel over one-hot labels. Because such local constant fits are biased and only use CLIP pointwise, the paper proposes a global alternative, ProKeR, which learns a correction to CLIP's zero-shot predictor by solving a multi-output kernel ridge regression whose proximal regularizer, the RKHS norm, keeps the solution close to the base predictor. The solution has a closed form, so no fine-tuning is needed, and on the standard 11-dataset few-shot benchmark it reports 69.07% average accuracy in the validation-free setting, ahead of GDA's 67.03%. A sympathetic reader would take the contribution to be a theoretical identity plus a practical adapter that transfers a single set of hyperparameters across datasets.

What carries the argument

The load-bearing object is the proximal multi-output kernel ridge regression objective in Eq. (13), defined on a reproducing kernel Hilbert space with a separable matrix-valued kernel $K_\beta(x,x') = k_\beta(x,x') B$ and an RKHS-norm proximity term $\lambda\|\phi - f_{clip}\|_{\mathcal{H}}^2$ that plays the role of a global regularizer. The representer theorem reduces this objective to a single linear system, Eq. (14), whose closed-form solution makes the method training-free. The analysis path also identifies the Tip-Adapter cache term as a Nadaraya-Watson estimator (Eq. (3)), uses local linear regression and a Mahalanobis metric as intermediate debiasing steps, and finally shows how Mercer/Bochner decompositions with random Fourier features remove the need to store all training samples.

What would settle it

A concrete check is to take a dataset whose CLIP residuals are highly non-smooth under the frozen kernel (for example, fine-grained classes at low shot counts), re-select $\beta$ and $\lambda$ on that dataset's validation split, and compare against the ImageNet-frozen values; if the gap exceeds the margins ProKeR reports over GDA, the claim that a single global kernel transfers across datasets is falsified. A second check is to replace the RBF kernel with a data-dependent kernel and see whether accuracy improves beyond ProKeR's reported numbers, which would show the frozen kernel, not the proximal regularization, is the limiting factor.

Watch

Extended reading notes

Core claim

The central claim is that caching methods are kernel methods. Tip-Adapter's adaptation term coincides, up to normalization, with a Nadaraya-Watson estimator using the RBF kernel, which makes Tip-Adapter a locally constant regression regularized pointwise by CLIP. Starting from this reading, the paper derives ProKeR by minimizing $\sum_i \|\phi(S_i)-L_i\|_2^2 + \lambda\|\phi-f_{clip}\|_{\mathcal{H}}^2$ over a reproducing kernel Hilbert space of vector-valued functions; the representer theorem turns this into $\phi = f_{clip} + \sum_i k_\beta(S_i,\cdot)\gamma_i$ with $\gamma = (I + \lambda^{-1} k_\beta(S,S))^{-1}(L - f_{clip}(S))$. This closed-form proximal kernel ridge regression incorporates global information about the few-shot task while staying near the zero-shot CLIP predictor, and the paper reports consistent gains over prior training-free adapters, including the highest average accuracy on the 11-dataset benchmark in both validation-free and validation-based protocols.

Load-bearing premise

The load-bearing premise is that one RBF kernel with a single scalar bandwidth $\beta$ and one regularizer $\lambda$, chosen on ImageNet and then frozen, can represent the per-class residual corrections $L - f_{clip}(S)$ well enough on all 11 target datasets; if that transferability fails for some dataset, the closed-form ridge regression underfits and the reported margins shrink.

Editorial extensions

If this is right

  • Tip-Adapter-style cache models become analyzable with classical kernel regression tools, giving a shared theory for why caching adapters work and where they lose accuracy.
  • Global task information can be injected into a training-free adapter without gradient optimization, by keeping the solution close to CLIP in the RKHS norm rather than only at test points.
  • Because the solution is closed form, adaptation reduces to solving one linear system; reported runtime is 4.7 seconds on 16-shot ImageNet, on par with Tip-Adapter and GDA.
  • Combining ProKeR with a trained linear base learner (ProKeR + CLAP) outperforms the training-free version and, at 70.73% average, beats the listed training-based methods, suggesting the kernel correction layer also helps when the base predictor is learned.
  • With random Fourier features, the method retains most of its accuracy (70.29% vs 70.46% in the validation-based setting) while its memory requirement no longer grows with the number of shots, addressing a known limitation of cache models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the identity between cache adapters and Nadaraya-Watson regression is right, then bandwidth-selection rules from nonparametric statistics become directly applicable; one testable extension is replacing the single frozen $\beta$ with a per-dataset bandwidth selector, which the paper leaves as future work.
  • The ProKeR objective only requires a pointwise base predictor and a kernel on features, so the same proximal-KRR construction could transfer to other CLIP-based tasks such as retrieval or open-vocabulary detection, not just image classification benchmarks.
  • The 8-shot regime is the one place ProKeR (RBF) trails GDA in the validation-free table, hinting that the fixed ImageNet-selected regularizer may under-regularize or over-regularize on some datasets; a per-dataset $\lambda$ sweep would show how much of the reported margin depends on hyperparameter transfer.
  • The RFF variant points toward a memory-accuracy frontier: using learned or data-dependent Fourier features could close the small gap to the exact kernel while keeping the cache compact.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes ProKeR, a training-free few-shot adaptation method for CLIP. It interprets Tip-Adapter's cache term as a Nadaraya-Watson estimator with an RBF kernel, introduces local linear regression and Mahalanobis-metric variants, and then formulates a proximal kernel ridge regression objective in which the adapted predictor is penalized by its RKHS distance from the zero-shot CLIP predictor. The resulting closed-form adapter is evaluated on the standard 11-dataset few-shot benchmark, with additional experiments on out-of-distribution datasets, different CLIP backbones, kernel ablations, random Fourier feature compression, and a training-based extension. The main claimed result is an average of 69.07% accuracy in the validation-free setting, compared to 67.03% for GDA.

Significance. If the closed-form derivation is corrected, the paper makes a useful contribution: it connects caching adapters to classical kernel regression, proposes a principled proximal RKHS regularizer with a closed-form solution, demonstrates consistent gains over training-free baselines across 11 datasets, and shows that random Fourier features can reduce memory with little accuracy loss. The code release and the use of ImageNet-transferred hyperparameters follow standard practice in this benchmark. I also find that the circularity concern raised in the stress test does not land: the Nadaraya-Watson identity is an algebraic equivalence, and hyperparameters are selected on ImageNet or validation splits rather than on test accuracy. However, the theoretical centerpiece currently contains a factor error in the printed closed-form solution, and the empirical significance cannot be fully assessed without seed counts, error bars, and reported hyperparameter values.

major comments (3)
  1. [§3.5, Eq. (14) and Eq. (19)] The printed closed form is not the minimizer of Eq. (13). Let K = kβ(S,S) and R = L - fclip(S). Writing the residual as h = φ - fclip = Σ_i kβ(S_i,·)γ_i, the objective in Eq. (13) is, up to trace, ||Kγ - R||² + λ γᵀKγ. Its first-order condition is K[(K+λI)γ - R] = 0, and since the RBF kernel matrix is positive definite, γ = (K+λI)^{-1}R. Eq. (14) instead gives (I + K/λ)^{-1}R = λ(K+λI)^{-1}R, which satisfies (K+λI)γ = λR and therefore solves the same problem with the residuals rescaled by λ. The same factor error appears in Eq. (19) for the training-based variant. This is load-bearing because the claim that ProKeR has a closed-form solution to Eq. (13) is the paper's main theoretical contribution. Please correct the formula or explicitly state the objective that the printed formula minimizes, and confirm whether the released code implements Eq. (14) as written or the corrected version.
  2. [§4.1, Tabs. 1-4, App. B] The empirical claim rests on average accuracy differences of 1-4% across 11 datasets, but the paper does not report the actual values of β and λ used for any table, nor the number of random seeds or standard deviations. Appendix B reports sensitivity only as multiples of an unreported base λ. In the few-shot regime, seed selection and hyperparameter values can shift average accuracy by more than the claimed margins. The ImageNet-transferred hyperparameters in the validation-free setting should be listed per shot, and the CoOp-benchmark results should be accompanied by at least several seeds with error bars. Without this, the state-of-the-art claim is not fully verifiable.
  3. [§3.2, Eqs. (4)-(8)] The derivation of Eq. (8) is labelled a Bayes optimal mapping, but it is better described as a regularized empirical-risk solution: the conditional measure dμx is replaced by the kernel-weighted empirical distribution, and the regularizer Rclip is added outside the expectation without a corresponding Bayesian model. Moreover, Eq. (8) is not equivalent to Tip-Adapter in Eq. (2): the cache term in Eq. (8) is normalized by λNK + Z(x), whereas Tip-Adapter uses an unnormalized exponential cache weighted by a fixed α. The paper acknowledges this 'up to a scaling factor', but the discussion should be reframed as showing that Tip-Adapter is an unnormalized, locally constant kernel regression with a fixed mixing weight. This is a useful connection, but it is not a Bayes-optimality argument, and the theoretical-understanding contribution should be stated accordingly.
minor comments (6)
  1. [§3.6, Eq. (15)] Eq. (15) contains a typographical error: it should read kβ(x,x') = ψβ(x)ψβ(x')ᵀ, not ψβ(x)ψβ(xᵀ).
  2. [§3.3] The sentence 'we derive a closed form solution for Eq. (21)' should refer to Eq. (9), since Eq. (21) appears only in the appendix.
  3. [App. A.1, Eq. (20)] The regularization term in Eq. (20) is written as ||q - fclip(x)||² without λ, but the derivative and the resulting Eq. (8) both use λ; Eq. (20) should include λ for consistency.
  4. [Tab. 5] The Epanechnikov kernel expression omits the indicator 1_{||x-y'||≤1}, and the polynomial kernel notation uses y′ inconsistently with the other rows; please clarify whether inputs are normalized before applying these kernels.
  5. [Tab. 7 and §4.5] The text 'our method does need to store additional training samples' for the RFF variant should read 'does not need', and the heading 'Adressing' is misspelled.
  6. [§3.6 and Tab. 2] The number R of random Fourier features used in the experiments is not reported; it should be stated for Tabs. 2 and 7, since the RFF approximation quality depends on R.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: ProKeR is an independent closed-form KRR derivation, and its reported gains come from hyperparameters selected on ImageNet or validation splits, not from test-set feedback.

full rationale

The paper's central derivation chain is self-contained. Tip-Adapter's cache term is algebraically rewritten as an unnormalized Nadaraya-Watson estimator (Eqs. 1-3), and the paper explicitly calls it a 'modified version' of the NW estimator rather than claiming identity; Eq. 8 is then derived from the Bayes optimal mapping objective and acknowledged to be equivalent to Tip-Adapter only up to an input-dependent scaling factor, so no prediction is being relabeled as a new result. ProKeR's closed-form solution in Eq. 14 is a representer-theorem KRR form whose empirical constants (kernel bandwidth β and regularizer λ) are fixed on ImageNet or validation shots and then frozen; the transfer protocol in Section 4.1 does not use target test accuracies to set parameters, so there is no fitted input renamed as a prediction. No load-bearing self-citation or author-imported uniqueness theorem appears: the cited representer-theorem and kernel results are classical external mathematics, and the few author-related references are used only for comparison or as a trainable baseline. One non-circularity concern exists: the printed closed form in Eq. 14, γ=(I+K/λ)^{-1}(L-fclip(S)), is algebraically λ(λI+K)^{-1}(L-fclip(S)) and does not satisfy the first-order stationarity condition of Eq. 13 unless λ=1; this is an internal mathematical-consistency issue that should be corrected, but it is not circularity because it does not make the output equivalent to the input by construction. Accordingly, the circularity score is 0.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

ProKeR's central claim rests on standard kernel-regression tools plus domain assumptions about CLIP features and the representability of residual corrections by an RBF kernel. The two numeric hyperparameters (beta, lambda) and the RFF count are not fully disclosed, which weakens exact reproduction.

free parameters (3)
  • RBF kernel bandwidth beta = not reported in main tables
    Controls locality of the kernel in Eqs. (3) and (14); directly affects predictions.
  • regularization coefficient lambda = not reported in main tables; Tab. 8 shows sensitivity around an optimal value
    Balances fit to few-shot labels versus proximity to fclip in Eqs. (13)-(14).
  • number of random Fourier features R = not stated in paper
    Used in the ProKeR with RFF memory-reduced variant; accuracy and memory trade-off depends on R.
assumptions (5)
  • standard math The Nadaraya-Watson estimator is a consistent local nonparametric regressor with known boundary bias.
    Used in Sec. 3.1-3.3 to motivate the local-constant view of Tip-Adapter and the LLR correction.
  • standard math Representer theorem for separable multi-output RKHS kernels.
    Used to derive the closed-form ProKeR solution in Eq. (14), Sec. 3.5.
  • domain assumption CLIP image and text features are L2-normalized.
    Needed for the equivalence between the dot-product cache term and the Euclidean distance RBF kernel in Eqs. (1)-(2).
  • domain assumption The Bayes-optimal adaptation risk can be replaced by a kernel-weighted empirical cost over one-hot labels.
    Introduced in Eqs. (4)-(7), Sec. 3.2; this is a heuristic approximation, not a proven equivalence.
  • domain assumption The zero-shot CLIP predictor fclip lies in (or can be treated as a fixed additive bias in) the RKHS H.
    Eq. (13) writes the objective with ||phi - fclip||_H, which requires fclip to be in the RKHS; the paper does not prove this.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ProKeR: A Kernel Perspective on Few-Shot Adaptation of Large Vision-Language Models." pith.science (2026). https://pith.science/paper/555LMP2U

@misc{pith2026250111175,
  author       = {Pith},
  title        = {Pith review of: ProKeR: A Kernel Perspective on Few-Shot Adaptation of Large Vision-Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/555LMP2U}},
  note         = {Machine review of arXiv:2501.11175}
}
read the original abstract

The growing popularity of Contrastive Language-Image Pretraining (CLIP) has led to its widespread application in various visual downstream tasks. To enhance CLIP's effectiveness and versatility, efficient few-shot adaptation techniques have been widely adopted. Among these approaches, training-free methods, particularly caching methods exemplified by Tip-Adapter, have gained attention for their lightweight adaptation without the need for additional fine-tuning. In this paper, we revisit Tip-Adapter from a kernel perspective, showing that caching methods function as local adapters and are connected to a well-established kernel literature. Drawing on this insight, we offer a theoretical understanding of how these methods operate and suggest multiple avenues for enhancing the Tip-Adapter baseline. Notably, our analysis shows the importance of incorporating global information in local adapters. Therefore, we subsequently propose a global method that learns a proximal regularizer in a reproducing kernel Hilbert space (RKHS) using CLIP as a base learner. Our method, which we call ProKeR (Proximal Kernel ridge Regression), has a closed form solution and achieves state-of-the-art performances across 11 datasets in the standard few-shot adaptation benchmark.

Figures

Figures reproduced from arXiv: 2501.11175 by the authors.

Figure 1
Figure 1. Fitting comparison between different methods on synthet [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of our training-free method ProKeR. While Tip-Adapter builds a key-value cache model using the few-shot samples, ProKeR incorporates a proximal global regularization based on the zero-shot predictor in a reproducing kernel Hilbert space (RKHS). This allows the use of a richer model without overfitting on the few-shot data. multiple caching methods have been proposed. APE [68] in￾cludes a feature selection s… view at source ↗
Figure 3
Figure 3. Average performance for different methods on 11 image [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Few-shot Performance of Training-free Methods on 11 image classification datasets (CoOp’s benchmark). [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

70 extracted references · 55 canonical work pages

  1. [1]

    Kernels for vector-valued functions: A review

    Mauricio A Alvarez, Lorenzo Rosasco, Neil D Lawrence, et al. Kernels for vector-valued functions: A review. Foundations and Trends® in Machine Learn- ing, 4(3):195–266, 2012. 5

    work page 2012

  2. [2]

    Visual prompting: Modifying pixel space to adapt pre-trained models

    Hyojin Bahng, Ali Jahanian, Swami Sankaranarayanan, and Phillip Isola. Visual prompting: Modifying pixel space to adapt pre-trained models. arXiv preprint arXiv:2203.17274, 2022. 2

    arXiv 2022

  3. [3]

    Improved few-shot visual classification

    Peyman Bateni, Raghav Goyal, Vaden Masrani, Frank Wood, and Leonid Sigal. Improved few-shot visual classification. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 14493–14502, 2020. 5

    work page 2020

  4. [4]

    Food-101–mining discriminative compo- nents with random forests

    Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. Food-101–mining discriminative compo- nents with random forests. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part VI 13 , pp. 446–461. Springer, 2014. 6

    work page 2014

  5. [5]

    Universal multi-task kernels

    Andrea Caponnetto, Charles A Micchelli, Massimil- iano Pontil, and Yiming Ying. Universal multi-task kernels. The Journal of Machine Learning Research, 9:1615–1646, 2008. 5

    work page 2008

  6. [6]

    Plot: Prompt learning with optimal transport for vision-language models

    Guangyi Chen, Weiran Yao, Xiangchen Song, Xinyue Li, Yongming Rao, and Kun Zhang. Plot: Prompt learning with optimal transport for vision-language models. arXiv preprint arXiv:2210.01253, 2022. 2

    arXiv 2022

  7. [7]

    Clip2scene: Towards label-efficient 3d scene understanding by clip

    Runnan Chen, Youquan Liu, Lingdong Kong, Xinge Zhu, Yuexin Ma, Yikang Li, Yuenan Hou, Yu Qiao, and Wenping Wang. Clip2scene: Towards label-efficient 3d scene understanding by clip. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7020–7030, 2023. 2

    work page 2023

  8. [8]

    Describing textures in the wild

    Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describing textures in the wild. In Proceedings of the IEEE con- ference on computer vision and pattern recognition, pp. 3606–3613, 2014. 6

    work page 2014

Show all 70 references
  1. [9]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248–255. Ieee, 2009. 6, 7

    2009

  2. [10]

    Data determines distributional robustness in contrastive language image pre-training (clip)

    Alex Fang, Gabriel Ilharco, Mitchell Wortsman, Yuhao Wan, Vaishaal Shankar, Achal Dave, and Ludwig Schmidt. Data determines distributional robustness in contrastive language image pre-training (clip). In International Conference on Machine Learning , pp. 6216–6234. PMLR, 2022. 1

    2022

  3. [11]

    Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories

    Li Fei-Fei, Rob Fergus, and Pietro Perona. Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. In 2004 conference on computer vision and pattern recognition workshop, pp. 178–178. IEEE,

    2004

  4. [12]

    Clip-adapter: Better vision-language models with feature adapters

    Peng Gao, Shijie Geng, Renrui Zhang, Teli Ma, Rongyao Fang, Yongfeng Zhang, Hongsheng Li, and Yu Qiao. Clip-adapter: Better vision-language models with feature adapters. arXiv preprint arXiv:2110.04544, 2021. 1, 2, 8

    2021 arXiv

  5. [13]

    Reproducing kernel hilbert space, mer- cer’s theorem, eigenfunctions, nystr\" om method, and use of kernels in machine learning: Tutorial and survey

    Benyamin Ghojogh, Ali Ghodsi, Fakhri Karray, and Mark Crowley. Reproducing kernel hilbert space, mer- cer’s theorem, eigenfunctions, nystr\" om method, and use of kernels in machine learning: Tutorial and survey. arXiv preprint arXiv:2106.08443, 2021. 5

    2021 arXiv

  6. [14]

    Open-vocabulary object detection via vision and language knowledge distillation

    Xiuye Gu, Tsung-Yi Lin, Weicheng Kuo, and Yin Cui. Open-vocabulary object detection via vision and language knowledge distillation. arXiv preprint arXiv:2104.13921, 2021. 2

    2021 arXiv

  7. [15]

    Calip: Zero- shot enhancement of clip with parameter-free attention

    Ziyu Guo, Renrui Zhang, Longtian Qiu, Xianzheng Ma, Xupeng Miao, Xuming He, and Bin Cui. Calip: Zero- shot enhancement of clip with parameter-free attention. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 746–754, 2023. 6

    2023

  8. [16]

    The elements of statistical learning: data mining, inference, and prediction, vol- ume 2

    Trevor Hastie, Robert Tibshirani, Jerome H Friedman, and Jerome H Friedman. The elements of statistical learning: data mining, inference, and prediction, vol- ume 2. Springer, 2009. 2, 8

    2009

  9. [17]

    Robust nonparametric regression with metric-space valued output

    Matthias Hein. Robust nonparametric regression with metric-space valued output. Advances in neural infor- mation processing systems, 22, 2009. 4

    2009

  10. [18]

    Eurosat: A novel dataset and deep learning benchmark for land use and land cover classi- fication

    Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classi- fication. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(7):2217– 2226, 2019. 6

    2019

  11. [19]

    The many faces of robustness: A critical analysis of out- of-distribution generalization

    Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, et al. The many faces of robustness: A critical analysis of out- of-distribution generalization. In Proceedings of the IEEE/CVF international ...

    2021

  12. [20]

    Natural adversarial exam- ples

    Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adversarial exam- ples. CVPR, 2021. 7

    2021

  13. [21]

    Scaling up visual and vision- language representation learning with noisy text su- pervision

    Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision- language representation learning with noisy text su- pervision. In International conference on machine learning, pp. 4904–4916...

    2021

  14. [22]

    Operator-valued kernels for learning from func- tional response data

    Hachem Kadri, Emmanuel Duflos, Philippe Preux, Stéphane Canu, Alain Rakotomamonjy, and Julien Au- diffren. Operator-valued kernels for learning from func- tional response data. Journal of Machine Learning Research, 17(20):1–54, 2016. 5

    2016

  15. [23]

    Maple: Multi-modal prompt learning

    Muhammad Uzair Khattak, Hanoona Rasheed, Muham- mad Maaz, Salman Khan, and Fahad Shahbaz Khan. Maple: Multi-modal prompt learning. In The IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023. 2

    2023

  16. [24]

    3d object representations for fine-grained cate- gorization

    Jonathan Krause, Michael Stark, Jia Deng, and Li Fei- Fei. 3d object representations for fine-grained cate- gorization. In Proceedings of the IEEE international conference on computer vision workshops , pp. 554– 561, 2013. 6

    2013

  17. [25]

    Adversarial regres- sion with doubly non-negative weighting matrices

    Tam Le, Truyen Nguyen, Makoto Yamada, Jose Blanchet, and Viet Anh Nguyen. Adversarial regres- sion with doubly non-negative weighting matrices. Ad- vances in Neural Information Processing Systems, 34: 16964–16976, 2021. 2

    2021

  18. [26]

    The power of scale for parameter-efficient prompt tuning

    Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2104.08691, 2021. 2

    2021 arXiv

  19. [27]

    Supervision exists everywhere: A data efficient con- trastive language-image pre-training paradigm

    Yangguang Li, Feng Liang, Lichen Zhao, Yufeng Cui, Wanli Ouyang, Jing Shao, Fengwei Yu, and Junjie Yan. Supervision exists everywhere: A data efficient con- trastive language-image pre-training paradigm. arXiv preprint arXiv:2110.05208, 2021. 2

    arXiv

  20. [28]

    Multimodality helps unimodality: Cross-modal few-shot learning with multimodal mod- els

    Zhiqiu Lin, Samuel Yu, Zhiyi Kuang, Deepak Pathak, and Deva Ramanan. Multimodality helps unimodality: Cross-modal few-shot learning with multimodal mod- els. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19325– 19337, 2023. 1, 2, 3, 7, 8

    2023

  21. [29]

    Frozen clip models are efficient video learners

    Ziyi Lin, Shijie Geng, Renrui Zhang, Peng Gao, Gerard De Melo, Xiaogang Wang, Jifeng Dai, Yu Qiao, and Hongsheng Li. Frozen clip models are efficient video learners. In European Conference on Computer Vision, pp. 388–404. Springer, 2022. 1, 2

    2022

  22. [30]

    Image segmen- tation using text and image prompts

    Timo Lüddecke and Alexander Ecker. Image segmen- tation using text and image prompts. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 7086–7096, 2022. 1, 2

    2022

  23. [31]

    Fine-grained vi- sual classification of aircraft

    Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi. Fine-grained vi- sual classification of aircraft. arXiv preprint arXiv:1306.5151, 2013. 6

    2013 arXiv

  24. [32]

    On learning vector-valued functions

    Charles A Micchelli and Massimiliano Pontil. On learning vector-valued functions. Neural computation, 17(1):177–204, 2005. 5

    2005

  25. [33]

    On estimating regression

    Elizbar A Nadaraya. On estimating regression. Theory of Probability & Its Applications, 9(1):141–142, 1964. 2, 3

    1964

  26. [34]

    Quality not quan- tity: On the interaction between dataset design and robustness of clip

    Thao Nguyen, Gabriel Ilharco, Mitchell Wortsman, Sewoong Oh, and Ludwig Schmidt. Quality not quan- tity: On the interaction between dataset design and robustness of clip. Advances in Neural Information Processing Systems, 35:21455–21469, 2022. 1

    2022

  27. [35]

    Au- tomated flower classification over a large number of classes

    Maria-Elena Nilsback and Andrew Zisserman. Au- tomated flower classification over a large number of classes. In 2008 Sixth Indian conference on computer vision, graphics & image processing , pp. 722–729. IEEE, 2008. 6

    2008

  28. [36]

    Generative local metric learning for kernel regression

    Yung-Kyun Noh, Masashi Sugiyama, Kee-Eung Kim, Frank Park, and Daniel D Lee. Generative local metric learning for kernel regression. Advances in neural information processing systems, 30, 2017. 2, 4, 5

    2017

  29. [37]

    Introduction to radial basis function networks, 1996

    Mark JL Orr et al. Introduction to radial basis function networks, 1996. 3

    1996

  30. [38]

    Black box few-shot adaptation for vision-language models

    Yassine Ouali, Adrian Bulat, Brais Matinez, and Geor- gios Tzimiropoulos. Black box few-shot adaptation for vision-language models. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pp. 15534–15546, 2023. 2

    2023

  31. [39]

    Practical perfor- mance of several data driven bandwidth selectors

    Byeong Park and Berwin Turlach. Practical perfor- mance of several data driven bandwidth selectors. Tech- nical report, Université catholique de Louvain, Center for Operations Research and . . . , 1992. 8

    1992

  32. [40]

    Cats and dogs

    Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar. Cats and dogs. In 2012 IEEE confer- ence on computer vision and pattern recognition, pp. 3498–3505. IEEE, 2012. 6

    2012

  33. [41]

    Learning transferable vi- sual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable vi- sual models from natural language supervision. In Ma- rina Meila ...

    2021

  34. [42]

    Random features for large-scale kernel machines

    Ali Rahimi and Benjamin Recht. Random features for large-scale kernel machines. Advances in neural information processing systems, 20, 2007. 6

    2007

  35. [43]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International conference on machine learning, pp. 8821–8831. Pmlr, 2021. 2

    2021

  36. [44]

    Do imagenet classifiers gen- eralize to imagenet? In International conference on machine learning, pp

    Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do imagenet classifiers gen- eralize to imagenet? In International conference on machine learning, pp. 5389–5400. PMLR, 2019. 6, 7

    2019

  37. [45]

    Consistency-guided prompt learning for vision-language models

    Shuvendu Roy and Ali Etemad. Consistency-guided prompt learning for vision-language models. arXiv preprint arXiv:2306.01195, 2023. 2

    2023 arXiv

  38. [46]

    Multivariate locally weighted least squares regression

    David Ruppert and Matthew P Wand. Multivariate locally weighted least squares regression. The annals of statistics, pp. 1346–1370, 1994. 2

    1994

  39. [47]

    Align your prompts: Test-time prompting with distribution alignment for zero-shot generalization

    Jameel Hassan Abdul Samadh, Hanan Gani, Noor Hazim Hussein, Muhammad Uzair Khattak, Muzammal Naseer, Fahad Khan, and Salman Khan. Align your prompts: Test-time prompting with distribution alignment for zero-shot generalization. In Thirty-seventh Conference on Neural Informatio...

    2023

  40. [48]

    Proposalclip: Unsupervised open-category object proposal generation via exploiting clip cues

    Hengcan Shi, Munawar Hayat, Yicheng Wu, and Jian- fei Cai. Proposalclip: Unsupervised open-category object proposal generation via exploiting clip cues. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pp. 9611–9620,

  41. [49]

    Test-time prompt tuning for zero-shot generalization in vision-language models

    Manli Shu, Weili Nie, De-An Huang, Zhiding Yu, Tom Goldstein, Anima Anandkumar, and Chaowei Xiao. Test-time prompt tuning for zero-shot generalization in vision-language models. Advances in Neural Infor- mation Processing Systems , 35:14274–14289, 2022. 2

    2022

  42. [50]

    A closer look at the few-shot adapta- tion of large vision-language models

    Julio Silva-Rodriguez, Sina Hajimiri, Ismail Ben Ayed, and Jose Dolz. A closer look at the few-shot adapta- tion of large vision-language models. arXiv preprint arXiv:2312.12730, 2023. 2, 5, 6, 7

    2023 arXiv

  43. [51]

    Ucf101: A dataset of 101 human actions classes from videos in the wild

    Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah. Ucf101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0402, 2012. 6

    2012 arXiv

  44. [52]

    Alpha-clip: A clip model focusing on wherever you want

    Zeyi Sun, Ye Fang, Tong Wu, Pan Zhang, Yuhang Zang, Shu Kong, Yuanjun Xiong, Dahua Lin, and Jiaqi Wang. Alpha-clip: A clip model focusing on wherever you want. arXiv preprint arXiv:2312.03818, 2023. 2

    2023 arXiv

  45. [53]

    Sus-x: Training-free name-only transfer of vision-language models

    Vishaal Udandarao, Ankush Gupta, and Samuel Al- banie. Sus-x: Training-free name-only transfer of vision-language models. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pp. 2725–2736, 2023. 2, 3, 6

    2023

  46. [54]

    Ac- tionclip: A new paradigm for video action recognition

    Mengmeng Wang, Jiazheng Xing, and Yong Liu. Ac- tionclip: A new paradigm for video action recognition. arXiv preprint arXiv:2109.08472, 2021. 2

    2021 arXiv

  47. [55]

    A hard-to-beat baseline for training-free clip-based adaptation

    Zhengbo Wang, Jian Liang, Lijun Sheng, Ran He, Zilei Wang, and Tieniu Tan. A hard-to-beat baseline for training-free clip-based adaptation. arXiv preprint arXiv:2402.04087, 2024. 3, 6, 7, 8

    2024 arXiv

  48. [56]

    Metric learn- ing for kernel regression

    Kilian Q Weinberger and Gerald Tesauro. Metric learn- ing for kernel regression. In Artificial intelligence and statistics, pp. 612–619. PMLR, 2007. 5

    2007

  49. [57]

    Sun database: Large- scale scene recognition from abbey to zoo

    Jianxiong Xiao, James Hays, Krista A Ehinger, Aude Oliva, and Antonio Torralba. Sun database: Large- scale scene recognition from abbey to zoo. In 2010 IEEE computer society conference on computer vision and pattern recognition, pp. 3485–3492. IEEE, 2010. 6

    2010

  50. [58]

    Visual- language prompt tuning with knowledge-guided con- text optimization

    Hantao Yao, Rui Zhang, and Changsheng Xu. Visual- language prompt tuning with knowledge-guided con- text optimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion, pp. 6757–6767, 2023. 2

    2023

  51. [59]

    Orthogonal random features

    Felix Xinnan X Yu, Ananda Theertha Suresh, Krzysztof M Choromanski, Daniel N Holtmann-Rice, and Sanjiv Kumar. Orthogonal random features. Ad- vances in neural information processing systems, 29,

  52. [60]

    Task residual for tuning vision-language mod- els

    Tao Yu, Zhihe Lu, Xin Jin, Zhibo Chen, and Xinchao Wang. Task residual for tuning vision-language mod- els. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10899– 10909, 2023. 2, 7

    2023

  53. [61]

    Unified vision and language prompt learning

    Yuhang Zang, Wei Li, Kaiyang Zhou, Chen Huang, and Chen Change Loy. Unified vision and language prompt learning. arXiv preprint arXiv:2210.07225, 2022. 2

    2022 arXiv

  54. [62]

    Tip-adapter: Training-free clip-adapter for better vision-language modeling

    Renrui Zhang, Rongyao Fang, Wei Zhang, Peng Gao, Kunchang Li, Jifeng Dai, Yu Qiao, and Hong- sheng Li. Tip-adapter: Training-free clip-adapter for better vision-language modeling. arXiv preprint arXiv:2111.03930, 2021. 1, 2, 3, 6, 7, 8

    2021 arXiv

  55. [63]

    Pointclip: Point cloud understanding by clip

    Renrui Zhang, Ziyu Guo, Wei Zhang, Kunchang Li, Xupeng Miao, Bin Cui, Yu Qiao, Peng Gao, and Hong- sheng Li. Pointclip: Point cloud understanding by clip. In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition, pp. 8552–8562,

  56. [64]

    Prompt, generate, then cache: Cascade of foundation models makes strong few-shot learners

    Renrui Zhang, Xiangfei Hu, Bohao Li, Siyuan Huang, Hanqiu Deng, Yu Qiao, Peng Gao, and Hongsheng Li. Prompt, generate, then cache: Cascade of foundation models makes strong few-shot learners. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, ...

    2023

  57. [65]

    Dual memory networks: A versatile adaptation approach for vision-language models

    Yabin Zhang, Wenjie Zhu, Hui Tang, Zhiyuan Ma, Kaiyang Zhou, and Lei Zhang. Dual memory networks: A versatile adaptation approach for vision-language models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 28718– 28728, 2024. 3

    2024

  58. [66]

    Conditional prompt learning for vision- language models

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Conditional prompt learning for vision- language models. In IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), 2022. 1, 2

    2022

  59. [67]

    Learning to prompt for vision-language models

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-language models. International Journal of Computer Vision, 130 (9):2337–2348, 2022. 1, 2, 3, 6, 8

    2022

  60. [68]

    Not all features matter: Enhancing few-shot clip with adaptive prior refinement

    Xiangyang Zhu, Renrui Zhang, Bowei He, Aojun Zhou, Dong Wang, Bin Zhao, and Peng Gao. Not all features matter: Enhancing few-shot clip with adaptive prior refinement. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pp. 2605–2615,

  61. [70]

    (20) The derivation of the solution of Eq. 20 is as follows: L = 1 N K N KX i=1 kβ(d(x, Si))∥q − Li∥2 2 + ∥q − fclip(x)∥2 2 ∂L ∂q = 0 ⇒ 1 N K N KX i=1 kβ(d(x, Si)) (q − Li) +λq − λfclip(x) = 0 ⇒ q λN K+ N KX i=1 kβ(d(x, Si)) ! = λN Kfclip(x) + N KX i=1 kβ(d(x, Si))Li ⇒ q = λN ...

  62. [2023]

    Detailed derivations A.1

    2, 3, 6, 7, 8 A. Detailed derivations A.1. Nadaraya-Watson estimator We first derive the solution of the adaptation problem for the Nadaraya-Waston estimator. The adaptation problem writes: ϕ(x) = arg min q 1 N K N KX i=1 kβ(d(x, Si))∥q − Li∥2 2 + ∥q − fclip(x)∥2

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.