Enhance Vision-Language Alignment with Noise

Hongyuan Zhang; Sida Huang; Xuelong Li

REVIEW 4 major objections 5 minor 45 references

Enhance Vision-Language Alignment with Noise

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a frozen CLIP can be adapted to downstream classification by learning to inject 'positive-incentive noise' into its visual and text encoders, without adding prompt or adapter modules, and that this outperforms prompt…

desk verdict Empirical gains look real, but the theoretical claim that PiNI works via beneficial 'π-noise' is unsupported: the key independence step is invalid and the training loss allows the noise variance to collapse to zero. read the letter →

arxiv 2412.10817 v2 pith:QCDHBZAG submitted 2024-12-14 cs.CV cs.AI

Sida Huang , Hongyuan Zhang , Xuelong Li This is my paper · ORCID

classification cs.CVcs.AI

keywords vision-languagemodelsCLIPfew-shotclassificationparameter-efficientfine-tuningpositive-incentivenoiseinjectionvariationalinferencepromptdistribution

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that a frozen vision-language model can be adapted to a downstream classification task not by adding learnable prompts or adapter modules, but by learning to inject customized noise into the visual and text encoders. This noise is meant to be 'positive-incentive noise', which should reduce the uncertainty of the prediction task rather than corrupting it. The authors reformulate CLIP's inference as a probabilistic process involving an image, a prompt, and a noise variable, then derive a variational training objective that produces noise from a Gaussian distribution whose mean and variance are predicted by small learned networks. On 11 few-shot classification datasets, the proposed method, PiNI, beats zero-shot CLIP, CoOp, and CLIP-Adapter, with the largest margins at very small training-set sizes. If correct, it shows that the learnable parameters of parameter-efficient fine-tuning can live in the noise distribution rather than in the model architecture.

What carries the argument

The central object is the learned noise distribution p(ε|x, P), modeled as a diagonal Gaussian whose parameters (μ, Σ) come from a small network fθ(x, P), with samples produced via the reparameterization trick ε = Σ·ϵ + μ. The argument is carried by the task-entropy inequality I(T, E) = H(T) − H(T|E) > 0, which states that the noise reduces classification uncertainty. To make this computable, the paper reformulates CLIP inference with an explicit prompt variable, defines H(T|E) as an expectation over images, labels, prompts, and noise, and replaces the intractable posterior p(y|x, ε, P) with a tractable variational approximation q, yielding the training loss in Eq. (13). The noise is factored into visual and text components, injected respectively into the visual features, using a cross-attention generator, and the prompt embeddings, using learnable embeddings.

What would settle it

Estimate the true task entropy H(T|E) on a held-out set by sampling noise from the learned generator and measuring the conditional entropy of labels given noisy embeddings, and compare it with the variational loss L from Eq. (13); if L is not an upper bound, the variational justification fails. A simpler check: train PiNI on a dataset where the label is strongly correlated with low-level image statistics, such as color-biased synthetic data, and test whether the noise generator's output changes when the label is revealed; if it does, the conditional-independence assumption ε ⊥ y | ρ is violated.

Watch

Extended reading notes

Core claim

The central claim is that customizing the noise distribution itself is a viable mechanism for aligning vision and language in a frozen CLIP. The paper shows that if noise ε satisfies I(T, E) > 0, meaning it reduces the entropy of the task, then injecting such π-noise into both the visual encoder's input or features and the text encoder's prompt embeddings can improve classification. They reframe CLIP inference as p(y|x, P) with an explicit prompt set P, define task entropy H(T) = H(y|x, P), and derive a variational upper bound whose minimization produces noise that simplifies the task. The resulting PiNI trains only the small networks that output the noise's mean and variance, leaving all CLIP weights frozen. Empirically, PiNI outperforms the baselines on 11 datasets and shows robustness to distribution shift, especially in low-shot regimes.

Load-bearing premise

The entire theoretical derivation assumes the learned noise contains no information about the true class beyond what the prompt already provides; if the noise distribution actually depends on the label, the proof that the training loss bounds the task entropy breaks, even though the method might still work in practice.

Editorial extensions

If this is right

PiNI fine-tunes a frozen CLIP by learning only the parameters of small noise generators, so the adaptation cost does not scale with the backbone size.
The method's gains are largest at 1, 2, and 4 shots per class, suggesting noise injection is especially sample-efficient when training data are scarce.
Because the noise generator is learned on the source distribution, the same noise-injection procedure transfers to out-of-distribution ImageNet variants, improving average robustness over all three baselines.
The learned prompt distribution is interpretable: sampling from it yields synonyms and grammatical variants such as 'photo', 'photos', 'pic', and 'picture', which shows the method increases semantic diversity rather than only fitting the training classes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper does not test this, but the empirical gains might partly come from stochastic embedding augmentation rather than from true π-noise; a direct check would compare PiNI against injecting random Gaussian noise with the same learned mean and variance.
If the conditional-independence assumption ε ⊥ y | ρ fails in practice, the variational bound in Eq. (6) may still hold approximately, but the theoretical justification would reduce to a heuristic; this could be tested by estimating the true conditional entropy of labels given noisy embeddings on a held-out set.
The same 'learn a noise distribution, keep the base model frozen' recipe could transfer to other dual-encoder architectures and tasks such as retrieval, VQA, or open-vocabulary detection by defining an appropriate task entropy for each.
Injecting noise at inference time, as the paper does, effectively turns a single deterministic prompt into a distribution over prompts; this could serve as a lightweight alternative to test-time ensembling over multiple hand-crafted prompts.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Punchline: the empirical gains look real—PiNI consistently beats CoOp and CLIP-Adapter across 11 datasets, especially in the 1–2 shot regime. But the paper's central claim, that these gains come from learned 'π-noise', is not established. There's a concrete technical flaw in the derivation and a more fundamental issue: nothing in the training objective prevents the noise variance from collapsing to zero.

What's actually new: the specific combination of injecting learned stochastic perturbations into both visual and text streams for CLIP fine-tuning hasn't been done before, as far as I know. The experiments are thorough: 11 datasets, shot counts from 1 to 16, domain generalization, multiple backbones, and a reasonable exploration of noise generator architectures and injection points. Gains over CoOp are modest (1–3 points on average) but consistent, and the low-shot improvement is genuinely useful. Code is public.

The soft spots are real. First, Eq. (9) is wrong: from ε⊥y|ρ you cannot conclude p(ε|x,y,ρ)=p(ε|x,ρ). Conditioning on x matters. Without that step, the variational bound in Eq. (13) isn't an upper bound on H(T|E) as claimed, so the π-noise interpretation doesn't follow. Second, and more damaging, the loss in Eq. (13) has no term that keeps Σθ from shrinking. The reparameterization can trivially set σ to zero and the "noise" becomes a deterministic offset μ(x,P). The paper never reports learned variances, never runs a Σ≡0 ablation, and never checks the defining condition I(T,E)>0. For all we know, the method works because it learns a residual feature transformation, not because noise helps. That would make the title and the theoretical framing misleading.

Minor: the best generator configuration in Table 1 is selected on the same datasets used for final reporting, and there are no error bars, so the 1–3 point improvements may be within run-to-run noise.

Who this is for: people doing parameter-efficient CLIP fine-tuning. They'll find the empirical recipe and the design space useful, but they should ignore the variational derivation.

My recommendation: send to peer review, but with strong pressure for major revision. The empirical contribution deserves scrutiny, and the authors need to add a σ→0 ablation, report variance parameters, and either fix the derivation or drop the π-noise framing. As written, the central claim is not supported.

Referee Report

4 major / 5 minor

Summary. The paper proposes Positive-incentive Noise Injector (PiNI), a method for fine-tuning frozen CLIP by injecting learned noise into the visual and text encoders for few-shot classification. The authors reformulate CLIP inference with a prompt variable, define task entropy, derive a variational upper bound on conditional entropy, and learn Gaussian noise parameters (mu, Sigma) using MLP, CNN, cross-attention, or learnable embeddings. Experiments on 11 datasets report consistent gains over zero-shot CLIP, CoOp, CLIP-Adapter, and Linear Probe, with larger improvements at lower shot counts; additional experiments cover domain generalization and different visual backbones. The paper claims this is the first noise-based fine-tuning scheme for CLIP and that the learned noise is positive-incentive noise in the sense of Eq. (1).

Significance. If the reported gains are robust, PiNI would be a genuinely simple and parameter-efficient alternative to prompt- and adapter-based PEFT, and the code release and broad benchmark are useful assets. The paper also provides a clear exploration of injection locations and generator architectures. However, the theoretical derivation in Section 3.3 contains a load-bearing conditional-independence gap, and the experiments do not currently establish that stochastic noise, rather than a deterministic learnable offset, is responsible for the improvements. Because these points can be addressed with a corrected derivation and targeted ablations, the significance is conditional on that additional evidence.

major comments (4)

[§3.3, Eq. (9)] Equation (9) states that from ε ⊥ y | ρ one can obtain p(ε|x,y,ρ)=p(ε|x,ρ). This inference is invalid: conditional independence of ε and y given ρ does not imply conditional independence given x, and in the graphical model of Figure 2, x may be a collider or a descendant that creates dependence. The equality is needed to pass from Eq. (8) to Eq. (10), so as written the variational upper bound and the claim that the learned noise minimizes H(T|E) do not follow. The correct fix is to define the sampling mechanism so that ε is generated from x and P independently of y, i.e., ε ⊥ y | (x,ρ), and to state that assumption explicitly.
[§3.3–3.4, Eq. (13)] The loss in Eq. (13) is an unregularized expectation of −log q(yi | xi, Gθ(ϵij, xi, P), P) with Gθ(ϵ, x, P) = Σθ(x,P)ϵ + μθ(x,P). Nothing in this objective prevents Σθ from collapsing to zero, in which case the 'noise' reduces to a deterministic input-dependent offset μθ and the method becomes a residual feature adapter. The paper does not report learned variance values, does not compare with a μ-only (Σ≡0) variant, and does not check the defining condition I(T,E)>0 in Eq. (1). The central mechanistic claim that beneficial stochastic noise drives the gains is therefore unsupported; a variance-controlled ablation and a report of Σ statistics are needed.
[§4.2, Table 1] The default combination of injection location and generator architecture is selected from Table 1, which reports 16-shot accuracy on the same datasets used in the main comparisons, but the paper does not state whether this selection was made on a validation split or on the test set. If the latter, the subsequent comparisons to CoOp and CLIP-Adapter are optimistically biased by test-set selection. Please clarify the selection procedure and, if necessary, re-evaluate the chosen configuration on held-out data.
[§A.2, §4.3] The prompts used for PiNI differ from those used for the baselines: Section A.2 states that the template is modified to place all category names at the end and that random words are inserted at the beginning, whereas CLIP, CoOp, and CLIP-Adapter use the hand-crafted template. Since the reported gains over CoOp are small on several datasets, the template change is a plausible confound. A control experiment that evaluates CoOp (or zero-shot CLIP) with the same modified template, or ablates PiNI with the original template, should be reported.

minor comments (5)

[§3.4] The sentence referring to Zhang, Zhu, and Li (2024) and GNNs appears unrelated to the surrounding discussion of noise generators and should be removed or integrated properly.
[§3.4] The phrase 'the noise has the same shape as the input at the injection location' should be made precise for the text side, where the noise shape depends on the number of prompt tokens and the embedding dimension.
[§B.3, Table 8] The nearest-word evidence does not by itself demonstrate richer semantics; if Σ collapses, the sampled prompts are nearly identical, so the table should be accompanied by variance statistics for the learned prompt distribution.
[Throughout] The phrase 'conditional independent' should be 'conditionally independent' (e.g., Section 3.4), and the notation in Figure 2 should be described in a way that makes the assumed generative direction explicit.
[§4.3] The paper should state whether performance numbers are averaged over multiple random seeds and should report standard deviations, since the few-shot regime is known to be noisy and several reported margins are small.

Circularity Check

1 steps flagged · score 4.0 of 10

The π-noise label is assigned by construction: Eq. (13) minimizes the same task entropy used to define π-noise, so the learned perturbation is 'beneficial' by definition; held-out accuracy is the only independent support.

self definitional [Section 3.2, Eqs. (3)-(5) and Section 3.3, Eqs. (6)-(13)]
"The noise E can decrease this uncertainty if it satisfies Eq. (1). ... As H(T) is a constant term for fixed CLIP model, maximizing I(T , E) is equivalent to minimizing H(T |E). ... L ≈ 1/n · m Σ_{i=1}^n Σ_{j=1}^m [− log q(yi|xi, Gθ(ϵij, xi, P), P)]."

Eq. (1) defines π-noise as noise with I(T,E)>0, i.e., H(T)>H(T|E). The training loss in Eq. (13) is a Monte Carlo estimate of exactly H(T|E), with q in place of p. Therefore any perturbation Gθ that lowers the training cross-entropy is, by construction, π-noise on the training distribution. The identification of the learned ε as 'positive-incentive noise' is not an independent result but a restatement of the optimization objective. The held-out accuracy is the only non-circular evidence; the theoretical claim 'we learn beneficial noise' reduces to 'the training cross-entropy decreased.'

full rationale

The paper's empirical evaluation (11 datasets, held-out test accuracy, distribution-shift benchmarks) is independent external evidence, so this is not a 6+ case. However, the theoretical derivation is largely a relabeled cross-entropy objective: Eq. (1) defines π-noise via entropy reduction, Eqs. (3)-(5) define task entropy with and without noise, and Eq. (13) is the empirical cross-entropy of the noisy classifier. Minimizing it guarantees the π-noise condition on training data, making the theoretical conclusion tautological. The self-citations to the authors' prior π-noise papers (Li 2022; Zhang, Huang, and Li 2023) supply the terminology and definition but are not machine-checked or otherwise independent; they are also not the main load-bearing support. The invalid conditional-independence step in Eq. (9) is a correctness flaw rather than a circularity: even if repaired by conditioning on x, the loss remains the training objective. The missing Σ≡0 control is likewise a correctness question about whether stochasticity matters, not a circularity. Overall, partial circularity appears in the theoretical framing, not in the empirical evaluation.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The derived loss is a stochastic classification objective with reparameterized Gaussian noise; the theoretical justification relies on the listed assumptions, several of which are stated briefly or chosen for computational convenience.

free parameters (6)

Noise sample count m = 1
Number of Monte Carlo noise samples per image in Eq. (13); chosen without sensitivity analysis.
Visual noise injection location = visual feature (post-encoder)
Selected from Table 1 as best on ImageNet, Caltech, Pets, Food; no held-out validation.
Text noise injection location = prompt embedding
Selected from Table 1 as best on the same four datasets.
Visual noise generator architecture = Cross-Attention
Selected from Table 1; other architectures (MLP, CNN) were tested and discarded.
Text noise generator = learnable embeddings (constant function)
Reported best in Table 1; effectively a learned continuous prompt with added variance.
Learning rate per dataset = 0.001-0.003
Tuned per dataset (Appendix A.3), not specified exactly.

assumptions (5)

standard math KL divergence is non-negative, so cross-entropy upper-bounds negative log-likelihood.
Used in Eq. (7) to justify the variational bound.
ad hoc to paper The noise and label are conditionally independent given the prompt, and p(ε|x,y,ρ)=p(ε|x,ρ).
Stated in Section 3.3 around Eq. (9); this is load-bearing for the upper bound and is not correctly derived from the graphical model.
domain assumption The noise follows an uncorrelated multivariate Gaussian with diagonal covariance.
Assumed in Section 3.4 to keep the noise generator efficient.
domain assumption The text noise generator can ignore the image and depend only on the prompt set due to computational burden.
Assumed in Section 3.4 'Details of Text Noise Generator'; deviates from Eq. (10) which conditions on x_i.
domain assumption Few-shot training data adequately represents the task distribution X for estimating task entropy.
The task entropy H(T) is estimated from k-shot training samples (Section 4.1); with 1-2 shots this is a very coarse estimate.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Enhance Vision-Language Alignment with Noise." pith.science (2026). https://pith.science/paper/QCDHBZAG

@misc{pith2026241210817,
  author       = {Pith},
  title        = {Pith review of: Enhance Vision-Language Alignment with Noise},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QCDHBZAG}},
  note         = {Machine review of arXiv:2412.10817}
}

abstract

With the advancement of pre-trained vision-language (VL) models, enhancing the alignment between visual and linguistic modalities in downstream tasks has emerged as a critical challenge. Different from existing fine-tuning methods that add extra modules to these two modalities, we investigate whether the frozen model can be fine-tuned by customized noise. Our approach is motivated by the scientific study of beneficial noise, namely Positive-incentive Noise (Pi-noise or $\pi$-noise) , which quantitatively analyzes the impact of noise. It therefore implies a new scheme to learn beneficial noise distribution that can be employed to fine-tune VL models. Focusing on few-shot classification tasks based on CLIP, we reformulate the inference process of CLIP and apply variational inference, demonstrating how to generate $\pi$-noise towards visual and linguistic modalities. Then, we propose Positive-incentive Noise Injector (PiNI), which can fine-tune CLIP via injecting noise into both visual and text encoders. Since the proposed method can learn the distribution of beneficial noise, we can obtain more diverse embeddings of vision and language to better align these two modalities for specific downstream tasks within limited computational resources. We evaluate different noise incorporation approaches and network architectures of PiNI. The evaluation across 11 datasets demonstrates its effectiveness.

Figures

Figures reproduced from arXiv: 2412.10817 by the authors.

**Figure 2.** The probabilistic graphical model of PiNI. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** (a) The inference process of CLIP, whose output features are utilized in PINI. (b) The framework of PiNI, which includes two procedures for adding noise. There are multiple options for the input of noise generators and locations of noise injection, whose potential transmission paths are represented with dashed lines. In the figure, ⊙ denotes the Hadamard product, ⊕ denotes matrix or vector addition, and ⊗ denotes ma… view at source ↗

Figures from the paper (6 more)

**Figure 4.** Figure 4: Visualization of generated noise injected into raw images. The first row shows the raw images. The second row [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]

**Figure 5.** Figure 5: Performance of few-shot learning across 11 datasets. In the top-left subplot, the results are averaged over 11 datasets. [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

**Figure 6.** Figure 6: Performance improvement of PiNI compared to [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

**Figure 7.** Figure 7: Visualization of the noise injected into raw images from Caltech101. [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]

**Figure 8.** Figure 8: Visualization of the noise injected into raw images from Food101. [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]

**Figure 9.** Figure 9: Visualization of the noise injected into raw images from OxfordPets. [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

45 extracted references · 9 canonical work pages

[1]

, " * write output.state after.block = add.period write newline

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

Bishop, C. M. 2006. Pattern recognition and machine learning. Springer google schola, 2: 645--678

2006
[4]

M.; Kucukelbir, A.; and McAuliffe, J

Blei, D. M.; Kucukelbir, A.; and McAuliffe, J. D. 2017. Variational inference: A review for statisticians. Journal of the American statistical Association, 112(518): 859--877

2017
[5]

Bossard, L.; Guillaumin, M.; and Van Gool, L. 2014. Food-101--mining discriminative components with random forests. In Computer Vision--ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part VI 13, 446--461. Springer

2014
[6]

Cimpoi, M.; Maji, S.; Kokkinos, I.; Mohamed, S.; and Vedaldi, A. 2014. Describing textures in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition, 3606--3613

2014
[7]

Cubuk Ekin, D.; Barret, Z.; Dandelion, M.; Vijay, V.; and Le Quoc, V. A. 2019. Learning augmentation strategies from data. In 30th IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

work page 2019
[8]

Deng, J.; Dong, W.; Socher, R.; Li, L.-J.; Li, K.; and Fei-Fei, L. 2009. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, 248--255. Ieee

2009

Show all 45 references

[9]

Fei-Fei, L.; Fergus, R.; and Perona, P. 2004. Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. In 2004 conference on computer vision and pattern recognition workshop, 178--178. IEEE

2004
[10]

Gao, J.; Ruan, J.; Xiang, S.; Yu, Z.; Ji, K.; Xie, M.; Liu, T.; and Fu, Y. 2024 a . Lamm: Label alignment for multi-modal prompt learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 1815--1823

2024
[11]

Gao, P.; Geng, S.; Zhang, R.; Ma, T.; Fang, R.; Zhang, Y.; Li, H.; and Qiao, Y. 2024 b . Clip-adapter: Better vision-language models with feature adapters. International Journal of Computer Vision, 132(2): 581--595

2024
[12]

Guo, Z.; Zhang, R.; Qiu, L.; Ma, X.; Miao, X.; He, X.; and Cui, B. 2023. Calip: Zero-shot enhancement of clip with parameter-free attention. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, 746--754

2023
[13]

Helber, P.; Bischke, B.; Dengel, A.; and Borth, D. 2019. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(7): 2217--2226

2019
[14]

Hendrycks, D.; Basart, S.; Mu, N.; Kadavath, S.; Wang, F.; Dorundo, E.; Desai, R.; Zhu, T.; Parajuli, S.; Guo, M.; et al. 2021 a . The many faces of robustness: A critical analysis of out-of-distribution generalization. In Proceedings of the IEEE/CVF international conference o...

2021
[15]

Hendrycks, D.; Zhao, K.; Basart, S.; Steinhardt, J.; and Song, D. 2021 b . Natural adversarial examples. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 15262--15271

2021
[16]

Houlsby, N.; Giurgiu, A.; Jastrzebski, S.; Morrone, B.; De Laroussilhe, Q.; Gesmundo, A.; Attariyan, M.; and Gelly, S. 2019. adapterParameter-efficient transfer learning for NLP. In International conference on machine learning, 2790--2799. PMLR

2019
[17]

J.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; Chen, W.; et al

Hu, E. J.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; Chen, W.; et al. 2021. LoRA: Low-Rank Adaptation of Large Language Models. In International Conference on Learning Representations

2021
[18]

Jia, M.; Tang, L.; Chen, B.-C.; Cardie, C.; Belongie, S.; Hariharan, B.; and Lim, S.-N. 2022. Visual prompt tuning. In European Conference on Computer Vision, 709--727. Springer

2022
[19]

U.; Rasheed, H.; Maaz, M.; Khan, S.; and Khan, F

Khattak, M. U.; Rasheed, H.; Maaz, M.; Khan, S.; and Khan, F. S. 2023. Maple: Multi-modal prompt learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 19113--19122

2023
[20]

P.; and Welling, M

Kingma, D. P.; and Welling, M. 2014. Auto-Encoding Variational Bayes. stat, 1050: 1

2014
[21]

Krause, J.; Stark, M.; Deng, J.; and Fei-Fei, L. 2013. 3d object representations for fine-grained categorization. In Proceedings of the IEEE international conference on computer vision workshops, 554--561

2013
[22]

Lester, B.; Al-Rfou, R.; and Constant, N. 2021. The Power of Scale for Parameter-Efficient Prompt Tuning. In Moens, M.-F.; Huang, X.; Specia, L.; and Yih, S. W.-t., eds., Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 3045--3059. Online...

2021
[23]

Li, X. 2022. Positive-Incentive Noise. IEEE Transactions on Neural Networks and Learning Systems, 1--7

2022
[24]

Li, X.; Dai, Y.; Ge, Y.; Liu, J.; Shan, Y.; and Duan, L.-Y. 2022. Uncertainty modeling for out-of-distribution generalization. arXiv preprint arXiv:2202.03958

2022 arXiv
[25]

Liu, H.; Li, C.; Li, Y.; and Lee, Y. J. 2024 a . Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 26296--26306

2024
[26]

Liu, H.; Li, C.; Wu, Q.; and Lee, Y. J. 2024 b . Visual instruction tuning. Advances in neural information processing systems, 36

2024
[27]

Liu, Z.; and He, K. 2024. A Decade's Battle on Dataset Bias: Are We There Yet? arXiv preprint arXiv:2403.08632

2024 arXiv
[28]

Maji, S.; Rahtu, E.; Kannala, J.; Blaschko, M.; and Vedaldi, A. 2013. Fine-grained visual classification of aircraft. arXiv preprint arXiv:1306.5151

2013 arXiv
[29]

Nilsback, M.-E.; and Zisserman, A. 2008. Automated flower classification over a large number of classes. In 2008 Sixth Indian conference on computer vision, graphics & image processing, 722--729. IEEE

2008
[30]

M.; Vedaldi, A.; Zisserman, A.; and Jawahar, C

Parkhi, O. M.; Vedaldi, A.; Zisserman, A.; and Jawahar, C. 2012. Cats and dogs. In 2012 IEEE conference on computer vision and pattern recognition, 3498--3505. IEEE

2012
[31]

W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al

Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning, 8748--8763. PMLR

2021
[32]

Recht, B.; Roelofs, R.; Schmidt, L.; and Shankar, V. 2019. Do imagenet classifiers generalize to imagenet? In International conference on machine learning, 5389--5400. PMLR

2019
[33]

R.; and Shah, M

Soomro, K.; Zamir, A. R.; and Shah, M. 2012. UCF101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0402

2012 arXiv
[34]

Sung, Y.-L.; Cho, J.; and Bansal, M. 2022. Vl-adapter: Parameter-efficient transfer learning for vision-and-language tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 5227--5237

2022
[35]

N.; Kaiser, .; and Polosukhin, I

Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, .; and Polosukhin, I. 2017. Attention is all you need. Advances in neural information processing systems, 30

2017
[36]

Wang, H.; Ge, S.; Lipton, Z.; and Xing, E. P. 2019. Learning robust global representations by penalizing local predictive power. Advances in Neural Information Processing Systems, 32

2019
[37]

A.; Oliva, A.; and Torralba, A

Xiao, J.; Hays, J.; Ehinger, K. A.; Oliva, A.; and Torralba, A. 2010. SUN database: Large-scale scene recognition from abbey to zoo. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 3485--3492. IEEE Computer Society

2010
[38]

B.; Goldberg, Y.; and Ravfogel, S

Zaken, E. B.; Goldberg, Y.; and Ravfogel, S. 2022. BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 1--9

2022
[39]

Zhang, H.; Huang, S.; and Li, X. 2023. Variational positive-incentive noise: How noise benefits models. arXiv preprint arXiv:2306.07651

2023 arXiv
[40]

Zhang, H.; Xu, Y.; Huang, S.; and Li, X. 2024. Data Augmentation of Contrastive Learning is Estimating Positive-incentive Noise. arXiv preprint arXiv:2408.09929

2024 arXiv
[41]

Zhang, H.; Zhu, Y.; and Li, X. 2024. Decouple Graph Neural Networks: Train Multiple Simple GNNs Simultaneously Instead of One. IEEE Transactions on Pattern Analysis and Machine Intelligence

2024
[42]

Zhang, Y.; Deng, B.; Li, R.; Jia, K.; and Zhang, L. 2023. Adversarial style augmentation for domain generalization. arXiv preprint arXiv:2301.12643

2023 arXiv
[43]

H.; and Sebe, N

Zhong, Z.; Zhao, Y.; Lee, G. H.; and Sebe, N. 2022. Adversarial style augmentation for domain generalized urban-scene segmentation. Advances in neural information processing systems, 35: 338--350

2022
[44]

C.; and Liu, Z

Zhou, K.; Yang, J.; Loy, C. C.; and Liu, Z. 2022 a . Conditional prompt learning for vision-language models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 16816--16825

2022
[45]

C.; and Liu, Z

Zhou, K.; Yang, J.; Loy, C. C.; and Liu, Z. 2022 b . Learning to prompt for vision-language models. International Journal of Computer Vision, 130(9): 2337--2348

2022

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

, " * write output.state after.block = add.period write newline

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

Bishop, C. M. 2006. Pattern recognition and machine learning. Springer google schola, 2: 645--678

2006

[4] [4]

M.; Kucukelbir, A.; and McAuliffe, J

Blei, D. M.; Kucukelbir, A.; and McAuliffe, J. D. 2017. Variational inference: A review for statisticians. Journal of the American statistical Association, 112(518): 859--877

2017

[5] [5]

Bossard, L.; Guillaumin, M.; and Van Gool, L. 2014. Food-101--mining discriminative components with random forests. In Computer Vision--ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part VI 13, 446--461. Springer

2014

[6] [6]

Cimpoi, M.; Maji, S.; Kokkinos, I.; Mohamed, S.; and Vedaldi, A. 2014. Describing textures in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition, 3606--3613

2014

[7] [7]

Cubuk Ekin, D.; Barret, Z.; Dandelion, M.; Vijay, V.; and Le Quoc, V. A. 2019. Learning augmentation strategies from data. In 30th IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

work page 2019

[8] [8]

Deng, J.; Dong, W.; Socher, R.; Li, L.-J.; Li, K.; and Fei-Fei, L. 2009. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, 248--255. Ieee

2009