Pith. sign in

REVIEW 5 major objections 4 minor 54 references

Cross-Modal Few-Shot Learning with Second-Order Neural Ordinary Differential Equations

T0 review · 5 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read SONO routes CLIP features through a second-order neural ODE and reports consistent few-shot and out-of-distribution gains over prompt- and adapter-based fine-tuning.

desk verdict A competent incremental combination of second-order NODEs, text-initialized classifiers, and text-as-image augmentation for CLIP few-shot adaptation, with solid but incompletely isolated empirical gains. read the letter →

arxiv 2412.15813 v1 pith:JKASZQ7K submitted 2024-12-20 cs.CV

Yi Zhang , Chun-Wun Cheng , Junyi He , Zhihai He , Carola-Bibiane Schönlieb , Yuyan Chen , Angelica I Aviles-Rivero This is my paper · ORCID
classification cs.CV
keywords few-shotlearningcross-modalsecond-orderneuralODEsCLIPfine-tuningdomaingeneralizationtext-basedaugmentationvision-languagemodelsfeaturerefinement
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SONO is a method for fine-tuning CLIP on few-shot image classification: it pushes each visual feature through a second-order neural ordinary differential equation and then through a linear classifier whose weights start as text embeddings. The paper's central claim is that this combination consistently beats existing prompt- and adapter-based CLIP fine-tuning methods, reaching 77.86% average accuracy over 11 datasets at 16 shots and improving out-of-distribution accuracy on ImageNet variants. The motivation is that second-order ODE flows can approximate a wider class of feature transformations than first-order flows, which the authors argue gives smoother, less overfit features when training examples are scarce. The method also augments the N-shot training set with text-prompt features and keeps CLIP frozen, so it adds only 1.54M trainable parameters and trains in minutes.

What carries the argument

The central object is the Second-Order NODE model $S^{(a)}_{\theta}$, defined by the initial-value problem $x''(t) = S^{(a)}_{\theta}(x(t), x'(t), t)$, $x(t_0) = x_0$, $x'(t_0) = g_{\theta}(x(t_0))$. Because standard ODE solvers handle only first-order systems, the paper rewrites the state as $z = [x, x']^{\mathsf{T}}$ and solves $z'(t) = [x', S^{(a)}_{\theta}]^{\mathsf{T}}$ with the fourth-order Runge-Kutta solver, using the first-order adjoint method for backpropagation. This object carries the argument by refining visual features continuously rather than through discrete adapter layers; the paper leans on the universal-approximation property of second-order NODEs to argue that the flow can represent a broader class of transformations. Its two complements are the cross-modal classifier, initialized with text embeddings so the text encoder is not rerun each iteration, and the text-as-image augmentation that adds L class-relevant prompt features to the N-shot training set.

What would settle it

Train the same pipeline with a first-order ODE in place of the second-order one and compare on all 11 datasets; if the first-order version matches SONO within noise, the central second-order claim is falsified.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that a Second-Order Neural ODE module, slotted between CLIP's frozen image encoder and a text-initialized classifier, is a better few-shot adapter than the residual adapters, cache models, and prompt learners it is compared with. The feature update is modeled as $x''(t) = S^{(a)}_{\theta}(x(t), x'(t), t)$ with initial conditions set by the image feature and a learned velocity term; the system is solved with RK4 in the forward pass and trained with the first-order adjoint method. Text-as-image augmentation selects the L prompt features most similar to each class prototype, and the classifier is initialized from text embeddings so the text encoder runs once. The result is 66.37% on 16-shot ImageNet, 77.86% average across 11 datasets, and OOD average gains of 1.58–1.80 points over GraphAdapter, with a 3.95-point gain on ImageNet-A.

Load-bearing premise

The load-bearing premise is that refining features with a second-order ODE flow, rather than a simpler first-order flow, is what produces the accuracy gains; the paper never directly tests that by swapping in a first-order ODE.

Editorial extensions

If this is right

  • If the reported averages hold, SONO at 16 shots (77.86%) becomes the best of the compared CLIP fine-tuning methods by 1.63 points over GraphAdapter and 2.75 points over Tip-Adapter-F.
  • The ablation attributes 3.11 points of ImageNet accuracy to the ODE module and 0.71 points to text-as-image augmentation, so the ODE refinement is the main contributor.
  • At 1.54M parameters and 3.5 minutes on one RTX 3090, the method is much cheaper than CoOp's 15-hour training and uses about a tenth of Tip-Adapter-F's parameters.
  • Domain generalization improves across four ImageNet variants, with the OOD average rising 1.58 points on ResNet-50 and 1.80 points on ViT-B/16 over GraphAdapter, and the largest single gain being 3.95 points on ImageNet-A.
  • Consistent gains over Tip-Adapter-F across ResNet-50, ResNet-101, ViT-B/32, and ViT-B/16 suggest the benefit is not tied to one visual backbone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper never benchmarks its own first-order NODE variant, so the second-order dynamics are not isolated; a first-order ODE with the same residual design might account for much of the gain.
  • The residual ratio $\eta$ is tuned on the same 16-shot ImageNet configuration used for headline results; fixing $\eta$ on a validation split before touching the test sets would tell whether the reported advantage is partly a selection effect.
  • Text-as-image augmentation depends on CLIP's own text-image similarity to pick prompts, which risks circularity on datasets whose class names already determine the easy features; its usefulness should be tested on domains where CLIP's zero-shot alignment is weak.
  • If the universal-approximation argument is what carries the method, a natural test is to vary solver accuracy and integration horizon: the gain should shrink as the solver is coarsened if expressivity is the active ingredient.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper introduces SONO, a few-shot learning method for CLIP that refines visual features with a second-order neural ODE, initializes the classifier from text embeddings, and augments training features with text prompts. The authors evaluate on 11 recognition datasets at 1-16 shots and on four ImageNet distribution-shift benchmarks, reporting that SONO outperforms prompt-based and adapter-based baselines while using modest parameters and training time.

Significance. If the reported gains are real and reproducible, the method is noteworthy for its simplicity and efficiency: it trains in minutes on a single GPU, adds only 1.54M parameters, and the text-as-image augmentation idea is a cheap way to exploit CLIP's shared embedding space. However, the central attribution of the gains to the second-order ODE formulation is not supported by the current experiments, and the paper's theoretical motivation for second-order versus first-order dynamics is not rigorous. The reported margins over strong baselines are small (0.6-3%), making the absence of multiple seeds a material issue.

major comments (5)
  1. [Ablation Studies, Table 2] The ablation labeled SNM '✗' removes the entire ODE module rather than replacing it with a first-order NODE. Since the paper's core claim is that second-order dynamics provide the expressiveness and regularization benefits, the reader cannot tell whether the 3.11% drop is due to the second-order formulation or simply to any residual feature transformation. The first-order predecessor NODE-Adapter (Zhang et al. 2024b) is cited but never benchmarked. Please add a first-order NODE baseline under the identical experimental setup, including the same adapter architecture and text-initialized classifier, to isolate the contribution of the second-order dynamics.
  2. [Residual Ratio η, Table 4] The method section never defines how η enters the feature update; Eq. (2) and Eq. (5) contain no η, so the residual combination described in the text is missing from the formulation. Moreover, Table 4 selects η on the same 16-shot ImageNet configuration used for the headline results and for the domain-generalization source model, which risks selection-on-the-evaluation-set optimism. Please state the exact residual formula (e.g., f_final = η S_θ(f) + (1-η) f) and report η selected on a held-out validation split or show that the reported results are stable across a range of η and multiple seeds.
  3. [Experimental Results, Figure 3 and Table 1] No error bars or multiple-seed results are reported anywhere. In few-shot protocols the specific randomly chosen training examples can change accuracy by more than a percentage point, so the 0.6-3% margins claimed as 'consistently and substantially outperform' may not be statistically meaningful. Please provide mean and standard deviation over at least three seeds for the main comparisons, or explicitly report the seed set used and demonstrate that performance is stable.
  4. [Domain Generalization, Table 1] The claim of consistent improvement is contradicted by Table 1 on the ResNet-50 backbone: on ImageNet-R, SONO achieves 60.75% whereas GraphAdapter achieves 60.86%, a 0.11% deficit. Also, the ViT-B/16 source ImageNet accuracy differs between Table 1 (74.92%) and Table 3 (74.86%) for the same configuration. Please reconcile the numbers and qualify the claim to acknowledge individual dataset exceptions.
  5. [Introduction and Related Work, motivation for second-order NODEs] The motivation states that first-order NODEs cannot approximate a broad class of functions because their flows do not intersect, whereas second-order NODEs are universal approximators. This is not a precise contrast: any second-order ODE system can be rewritten as a first-order system with a doubled state dimension, so the expressiveness argument relies on state augmentation rather than order. The paper should either provide a precise theoretical statement (e.g., the parameter-efficiency results of Massaroli et al. 2020, already cited) or explicitly treat the second-order choice as an empirical design decision validated by the requested first-order baseline.
minor comments (4)
  1. [Ablation Studies, Table 2] The acronym for text-as-image augmentation is written 'TAI' in the text and 'TIA' in the table header; please standardize.
  2. [Method, Eq. (4)] The loss in Eq. (4) uses the symbol 'n' in the summation without defining it; it should be the total number of augmented training features N+L for the class in question.
  3. [Experimental Results, performance comparison] In the text the baseline is sometimes called 'Tip-Adapter' while the tables list 'Tip-Adapter-F'; the distinction should be consistent.
  4. [Implementation Details] The optimization is attributed to AdamW with a citation to Kingma and Ba (2015), which is the Adam paper; please add the correct AdamW reference (Loshchilov and Hutter, 2019) if AdamW is intended.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: SONO's empirical claims are self-contained; the missing first-order NODE ablation and eta selection are experimental gaps, not circular derivation.

full rationale

SONO makes no derived prediction and presents no fitted quantity as an independent result; the paper's claims are empirical comparisons against external baselines on standard benchmarks. The Second-Order NODE component is defined by Eq. (2) as x''(t)=S_theta(x(t),x'(t),t) and is trained with the cross-entropy loss in Eq. (4); there is no equation in which a claimed output is identical to an input by construction, and no parameter fitted to a subset of data is later renamed as a prediction. The only self-citations (Zhang et al. 2024b NODE-Adapter, Cheng et al. 2023, Ordonez et al. 2024) appear in background or related-work descriptions and are not load-bearing for the central few-shot results; the universal-approximation support for second-order NODEs is credited to Kidger (2022), not to the authors' own prior work. The absence of a first-order NODE ablation in Table 2 is a genuine experimental gap that weakens the causal attribution of the gains to second-order dynamics, but it is an evidence/correctness concern rather than circular reasoning. Similarly, selecting the residual ratio eta on 16-shot ImageNet in Table 4 is benchmark hyperparameter selection, not a self-defined prediction. The classifier initialization and text-as-image augmentation both draw on CLIP text embeddings, but the paper never claims these sources are independent; reusing the same embedding space is the method's design, not a derivation of the result from the result. No circular step is exhibited, so the circularity score is 0.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests on several domain assumptions about CLIP text-image geometry and on a handful of hand-set hyperparameters. No genuinely new physical or mathematical entity is introduced. The most load-bearing assumption, that second-order NODEs are better than first-order for few-shot features, is never isolated experimentally.

free parameters (5)
  • Residual ratio eta = 0.6
    Controls the balance between transformed and original visual features; tuned on 16-shot ImageNet (Table 4), the same setting used for headline results.
  • Number of text augmentation features L per class = not reported
    Top-L prompts are selected per class as augmentation features, but L and its sensitivity are not reported in the main text.
  • Prompt codebook size M = 50
    Set to M=50 in Implementation Details with no ablation or sensitivity analysis.
  • ODE integration horizon and solver step = max time step 1000
    RK4 with maximum time step 1000 is used, but the final time t_u and the actual step size are not specified, which affects the scale of feature transformation.
  • Hyperparameter h = 10
    Mentioned as "we empirically set M = 50 and h = 10" but h is never defined or used in the method description.
assumptions (4)
  • domain assumption Second-order NODEs are universal approximators and therefore yield better feature generalization in few-shot learning.
    Invoked in the Introduction and Method using a citation to Kidger 2022; the transfer from approximation theory to few-shot generalization is asserted, not demonstrated.
  • domain assumption Text features from CLIP and CuPL prompts are valid surrogate image samples for training.
    Text-as-Image Augmentation assumes that text embeddings occupy the same classification geometry as image features and that selecting top-L prompts by class-prototype similarity does not bias the classifier.
  • domain assumption The standard CLIP few-shot protocol and baseline set are sufficient for the performance claim.
    Evaluation follows the CoOp and Tip-Adapter protocol, but omits the first-order NODE baseline NODE-Adapter, leaving the marginal benefit of second-order dynamics unmeasured.
  • standard math The adjoint sensitivity method gives correct gradients for the transformed first-order system.
    This is a standard technique from Chen et al. 2018 and is not a source of concern.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Cross-Modal Few-Shot Learning with Second-Order Neural Ordinary Differential Equations." pith.science (2026). https://pith.science/paper/JKASZQ7K

@misc{pith2026241215813,
  author       = {Pith},
  title        = {Pith review of: Cross-Modal Few-Shot Learning with Second-Order Neural Ordinary Differential Equations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JKASZQ7K}},
  note         = {Machine review of arXiv:2412.15813}
}
read the original abstract

We introduce SONO, a novel method leveraging Second-Order Neural Ordinary Differential Equations (Second-Order NODEs) to enhance cross-modal few-shot learning. By employing a simple yet effective architecture consisting of a Second-Order NODEs model paired with a cross-modal classifier, SONO addresses the significant challenge of overfitting, which is common in few-shot scenarios due to limited training examples. Our second-order approach can approximate a broader class of functions, enhancing the model's expressive power and feature generalization capabilities. We initialize our cross-modal classifier with text embeddings derived from class-relevant prompts, streamlining training efficiency by avoiding the need for frequent text encoder processing. Additionally, we utilize text-based image augmentation, exploiting CLIP's robust image-text correlation to enrich training data significantly. Extensive experiments across multiple datasets demonstrate that SONO outperforms existing state-of-the-art methods in few-shot learning performance.

Figures

Figures reproduced from arXiv: 2412.15813 by the authors.

Figure 1
Figure 1. Comparison between (a) Zero-shot CLIP (Radford [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. An overview of our method for K-class N-shot classification. Subfigure (a) illustrates the text-as-image data augmentation process. Subfigure (b) presents the overall architecture of our proposed SONO, consisting of a Second-Order NODEs model S (a) θ and a cross-modal classifier, which is initialized with text embeddings derived from prompts containing class labels. has been applied to Vision-Language Reasoning (Zha… view at source ↗
Figure 3
Figure 3. Classification Performance Comparison on Few-shot Learning, i.e., 1-/2-/4-/8-/16-shot, on 11 benchmark datasets. The top-left is the averaged accuracy over the 11 datasets. Ablation Studies We present an empirical analysis here. Unless specified, our experiments are conducted on the 16-shot ImageNet. Contributions of Major Algorithm Components. In Ta￾ble 2, TAI stands for Text-Image Augmentation, SNM refers to the S… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Ablation results on various ODE solvers: [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

54 extracted references · 44 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Bossard, L.; Guillaumin, M.; and Van Gool, L. 2014. Food-101--mining discriminative components with random forests. In European Conference on Computer Vision. Springer

    work page 2014

  4. [4]

    Chen, G.; Yao, W.; Song, X.; Li, X.; Rao, Y.; and Zhang, K. 2023. Prompt Learning with Optimal Transport for Vision-Language Models. In ICLR

    work page 2023

  5. [5]

    T.; Rubanova, Y.; Bettencourt, J.; and Duvenaud, D

    Chen, R. T.; Rubanova, Y.; Bettencourt, J.; and Duvenaud, D. K. 2018. Neural ordinary differential equations. Advances in neural information processing systems, 31

    2018

  6. [6]

    Continuous U-Net: Faster, Greater and Noiseless

    Cheng, C.-W.; Runkel, C.; Liu, L.; Chan, R. H.; Sch \"o nlieb, C.-B.; and Aviles-Rivero, A. I. 2023. Continuous u-net: Faster, greater and noiseless. arXiv:2302.00626

    work page Pith review arXiv 2023

  7. [7]

    Cimpoi, M.; Maji, S.; Kokkinos, I.; Mohamed, S.; and Vedaldi, A. 2014. Describing textures in the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    work page 2014

  8. [8]

    Deng, J.; Dong, W.; Socher, R.; Li, L.-J.; Li, K.; and Fei-Fei, L. 2009. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition. Ieee

    work page 2009

Show all 54 references
  1. [9]

    Desai, K.; and Johnson, J. 2021. Virtex: Learning visual representations from textual annotations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    2021

  2. [10]

    Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X.; Unterthiner, T.; Dehghani, M.; Minderer, M.; Heigold, G.; Gelly, S.; et al. 2020. An image is worth 16x16 words: transformers for image recognition at scale. In International Conference on Learning Representations

    2020

  3. [11]

    Fei-Fei, L.; Fergus, R.; and Perona, P. 2007. Learning generative visual models from few training examples: An incremental Bayesian approach tested on 101 object categories. Computer Vision and Image Understanding, 106(1)

    2007

  4. [12]

    Gao, P.; Geng, S.; Zhang, R.; Ma, T.; Fang, R.; Zhang, Y.; Li, H.; and Qiao, Y. 2024. Clip-adapter: Better vision-language models with feature adapters. International Journal of Computer Vision, 132

    2024

  5. [13]

    Guo, Z.; Zhang, R.; Qiu, L.; Ma, X.; Miao, X.; He, X.; and Cui, B. 2022. Calip: Zero-shot enhancement of clip with parameter-free attention. arXiv preprint arXiv:2209.14169

    2022 arXiv

  6. [14]

    He, K.; Zhang, X.; Ren, S.; and Sun, J. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    2016

  7. [15]

    Helber, P.; Bischke, B.; Dengel, A.; and Borth, D. 2019. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(7)

    2019

  8. [16]

    Hendrycks, D.; Basart, S.; Mu, N.; Kadavath, S.; Wang, F.; Dorundo, E.; Desai, R.; Zhu, T.; Parajuli, S.; Guo, M.; et al. 2021 a . The many faces of robustness: A critical analysis of out-of-distribution generalization. In Proceedings of the IEEE/CVF International Conference o...

    2021

  9. [17]

    Hendrycks, D.; Zhao, K.; Basart, S.; Steinhardt, J.; and Song, D. 2021 b . Natural adversarial examples. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    2021

  10. [18]

    Jia, C.; Yang, Y.; Xia, Y.; Chen, Y.-T.; Parekh, Z.; Pham, H.; Le, Q.; Sung, Y.-H.; Li, Z.; and Duerig, T. 2021. Scaling up visual and vision-language representation learning with noisy text supervision. In International Conference on Machine Learning

    2021

  11. [19]

    Kidger, P. 2022. On neural differential equations. arXiv:2202.02435

    2022 arXiv

  12. [20]

    P.; and Ba, J

    Kingma, D. P.; and Ba, J. 2015. Adam: A method for stochastic optimization. International Conference on Learning Representations

    2015

  13. [21]

    Krause, J.; Stark, M.; Deng, J.; and Fei-Fei, L. 2013. 3D object representations for fine-grained categorization. In Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops

    2013

  14. [22]

    Li, J.; Li, D.; Xiong, C.; and Hoi, S. 2022 a . Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In International Conference on Machine Learning. PMLR

    2022

  15. [23]

    H.; Zhang, P.; Zhang, H.; Yang, J.; Li, C.; Zhong, Y.; Wang, L.; Yuan, L.; Zhang, L.; Hwang, J.-N.; et al

    Li, L. H.; Zhang, P.; Zhang, H.; Yang, J.; Li, C.; Zhong, Y.; Wang, L.; Yuan, L.; Zhang, L.; Hwang, J.-N.; et al. 2022 b . Grounded language-image pre-training. In CVPR, 10965--10975

    2022

  16. [24]

    Li, X.; Lian, D.; Lu, Z.; Bai, J.; Chen, Z.; and Wang, X. 2024. Graphadapter: Tuning vision-language models with dual knowledge graph. Advances in Neural Information Processing Systems, 36

    2024

  17. [25]

    Maji, S.; Rahtu, E.; Kannala, J.; Blaschko, M.; and Vedaldi, A. 2013. Fine-grained visual classification of aircraft. arXiv preprint arXiv:1306.5151

    2013 arXiv

  18. [26]

    Massaroli, S.; Poli, M.; Park, J.; Yamashita, A.; and Asama, H. 2020. Dissecting neural odes. Advances in Neural Information Processing Systems, 33

    2020

  19. [27]

    Nilsback, M.-E.; and Zisserman, A. 2008. Automated flower classification over a large number of classes. In Indian Conference on Computer Vision, Graphics and Image Processing. IEEE

    2008

  20. [28]

    Norcliffe, A.; Bodnar, C.; Day, B.; Simidjievski, N.; and Li \`o , P. 2020. On second order behaviour in augmented neural odes. Advances in neural information processing systems, 33

    2020

  21. [29]

    C.; Cheng, C.-W.; Huang, J.; Zhang, L.; Yang, G.; Sch \"o nlieb, C.-B.; and Aviles-Rivero, A

    Ordo \ n ez, S. C.; Cheng, C.-W.; Huang, J.; Zhang, L.; Yang, G.; Sch \"o nlieb, C.-B.; and Aviles-Rivero, A. I. 2024. The Missing U for Efficient Diffusion Models. Transactions on Machine Learning Research

    2024

  22. [30]

    M.; Vedaldi, A.; Zisserman, A.; and Jawahar, C

    Parkhi, O. M.; Vedaldi, A.; Zisserman, A.; and Jawahar, C. 2012. Cats and dogs. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    2012

  23. [31]

    Pratt, S.; Covert, I.; Liu, R.; and Farhadi, A. 2023. What does a platypus look like? generating customized prompts for zero-shot image classification. In Proceedings of the IEEE/CVF International Conference on Computer Vision

    2023

  24. [32]

    W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al

    Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning, 8748--8763. PMLR

    2021

  25. [33]

    Rao, Y.; Zhao, W.; Chen, G.; Tang, Y.; Zhu, Z.; Huang, G.; Zhou, J.; and Lu, J. 2022. Denseclip: Language-guided dense prediction with context-aware prompting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 18082--18091

    2022

  26. [34]

    Recht, B.; Roelofs, R.; Schmidt, L.; and Shankar, V. 2019. Do imagenet classifiers generalize to imagenet? In International Conference on Machine Learning. PMLR

    2019

  27. [35]

    Shu, M.; Nie, W.; Huang, D.-A.; Yu, Z.; Goldstein, T.; Anandkumar, A.; and Xiao, C. 2022. Test-time prompt tuning for zero-shot generalization in vision-language models. In Advances in Neural Information Processing Systems, volume 35

    2022

  28. [36]

    Singh, A.; Hu, R.; Goswami, V.; Couairon, G.; Galuba, W.; Rohrbach, M.; and Kiela, D. 2022. Flava: A foundational language and vision alignment model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    2022

  29. [37]

    R.; and Shah, M

    Soomro, K.; Zamir, A. R.; and Shah, M. 2012. UCF101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0402

    2012 arXiv

  30. [38]

    Su, W.; Zhu, X.; Cao, Y.; Li, B.; Lu, L.; Wei, F.; and Dai, J. 2019. Vl-bert: Pre-training of generic visual-linguistic representations. arXiv preprint arXiv:1908.08530

    2019 arXiv

  31. [39]

    N.; Kaiser, .; and Polosukhin, I

    Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, .; and Polosukhin, I. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30

    2017

  32. [40]

    Wang, H.; Ge, S.; Lipton, Z.; and Xing, E. P. 2019. Learning robust global representations by penalizing local predictive power. In Advances in Neural Information Processing Systems, volume 32

    2019

  33. [41]

    Wang, N.; Xie, J.; Luo, H.; Cheng, Q.; Wu, J.; Jia, M.; and Li, L. 2023. Efficient image captioning for edge devices. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37

    2023

  34. [42]

    Wang, Z.; Lu, Y.; Li, Q.; Tao, X.; Guo, Y.; Gong, M.; and Liu, T. 2022. Cris: Clip-driven referring image segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    2022

  35. [43]

    A.; Oliva, A.; and Torralba, A

    Xiao, J.; Hays, J.; Ehinger, K. A.; Oliva, A.; and Torralba, A. 2010. Sun database: Large-scale scene recognition from abbey to zoo. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    2010

  36. [44]

    Yang, J.; Duan, J.; Tran, S.; Xu, Y.; Chanda, S.; Chen, L.; Zeng, B.; Chilimbi, T.; and Huang, J. 2022. Vision-language pre-training with triple contrastive learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    2022

  37. [45]

    Yao, L.; Han, J.; Wen, Y.; Liang, X.; Xu, D.; Zhang, W.; Li, Z.; Xu, C.; and Xu, H. 2022. Detclip: Dictionary-enriched visual-concept paralleled pre-training for open-world detection. Advances in Neural Information Processing Systems, 35

    2022

  38. [46]

    Yu, J.; Wang, Z.; Vasudevan, V.; Yeung, L.; Seyedhosseini, M.; and Wu, Y. 2022. Coca: Contrastive captioners are image-text foundation models. Transactions on Machine Learning Research

    2022

  39. [47]

    Yu, T.; Lu, Z.; Jin, X.; Chen, Z.; and Wang, X. 2023. Task residual for tuning vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    2023

  40. [48]

    Yuan, X.; Lin, Z.; Kuen, J.; Zhang, J.; Wang, Y.; Maire, M.; Kale, A.; and Faieta, B. 2021. Multimodal contrastive training for visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    2021

  41. [49]

    Zhang, J.; Huang, J.; Jin, S.; and Lu, S. 2024 a . Vision-language models for vision tasks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence

    2024

  42. [50]

    Zhang, R.; Zhang, W.; Fang, R.; Gao, P.; Li, K.; Dai, J.; Qiao, Y.; and Li, H. 2022. Tip-adapter: Training-free adaption of clip for few-shot classification. In European Conference on Computer Vision. Springer

    2022

  43. [51]

    Zhang, Y.; Cheng, C.-W.; Yu, K.; He, Z.; Sch \"o nlieb, C.-B.; and Aviles-Rivero, A. I. 2024 b . NODE-Adapter: Neural Ordinary Differential Equations for Better Vision-Language Reasoning. arXiv:2407.08672

    2024 arXiv

  44. [52]

    C.; and Liu, Z

    Zhou, K.; Yang, J.; Loy, C. C.; and Liu, Z. 2022 a . Conditional prompt learning for vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    2022

  45. [53]

    C.; and Liu, Z

    Zhou, K.; Yang, J.; Loy, C. C.; and Liu, Z. 2022 b . Learning to prompt for vision-language models. International Journal of Computer Vision, 130(9)

    2022

  46. [54]

    Zhu, X.; Zhang, R.; He, B.; Zhou, A.; Wang, D.; Zhao, B.; and Gao, P. 2023. Not All Features Matter: Enhancing Few-shot CLIP with Adaptive Prior Refinement. arXiv preprint arXiv:2304.01195

    2023 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.