Pith. sign in

REVIEW 3 major objections 5 minor 50 references

Token Coordinated Prompt Attention is Needed for Visual Prompting

T0 review · 3 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Per-token prompt matching lifts visual prompting accuracy.

desk verdict A genuinely new token-coordinated prompt mechanism with consistent small gains, but the missing prompt-capacity control leaves the central mechanism attribution under-supported. read the letter →

arxiv 2505.02406 v2 pith:Q4W6P2WZ submitted 2025-05-05 cs.CV

Zichen Liu , Xu Zou , Gang Hua , Jiahuan Zhou This is my paper · ORCID
classification cs.CV
keywords visualpromptingprompttuningvisiontransformerattentionmaskingpooltoken-coordinatedpromptsparameter-efficientfine-tuningfeaturediversity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Visual prompting adapts a frozen pretrained Vision Transformer (ViT) by adding a small number of learnable prompt tokens, but current methods give every token the same prompts, so different tokens attend to similar image regions and the extracted features become redundant. This paper argues that prompts should be token-coordinated: CLS and image tokens should draw from separate prompt pools, and each image token should be matched to a subset of image prompts through a cosine-distance matching rule. The proposed Token Coordinated Prompt Attention (TCPA) module realizes this by building a binary attention mask that lets each token interact only with its assigned prompts, applied to the softmax attention map inside each transformer block. The paper reports that plugging TCPA into several established visual prompting methods improves accuracy on the HTA and VTAB benchmarks by roughly 0.5 to 3 percentage points while adding negligible training time. If this is right, TCPA is a cheap, general upgrade for visual prompting rather than a replacement for it.

What carries the argument

The carrier of the argument is a binary attention mask built from token–prompt matching. A learnable indicator vector is attached to each prompt in each pool; for every image token $h^j_m$ the cosine distance $S(h^j_m,\kappa^k_i)=1-\cos(h^j_m,\kappa^k_i)$ gives an affinity row, and the top-$K_i$ entries are set to one with all others zero (Eqs. 9–11). The CLS token gets the same treatment against the CLS prompt pool. The two masks are concatenated, expanded to the attention-map shape, and multiplied element-wise into the post-softmax attention map $\mathrm{Attn}' = \mathrm{Attn}\odot M$ (Eq. 13), so each query attends only to its matched prompts while all attention weights are still computed in a single pass. This single-pass masking is what keeps the overhead low, and the disentangling of CLS versus image prompts is what the paper credits for preserving the two tokens' distinct roles.

What would settle it

Run TCPA with the same prompt pools and mask density but with random or fixed-permutation token–prompt assignments instead of cosine matching; if accuracy stays roughly the same, the coordination itself is not the source of the gains. As a second check, examine whether post-mask attention row sums collapse or destabilize compared with the unmasked baseline.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central claim is that the failure mode of existing visual prompting is representational collapse: because all tokens use the same prompt tokens in attention, the self-attention matrix stays low-rank after prompting and different tokens extract near-identical, biased features. TCPA breaks this by disentangling prompts into a CLS prompt pool and an image prompt pool, then assigning coordinated prompts per token. For each image token, the cosine distance to learnable prompt indicators produces an affinity matrix, whose top-K entries are binarized into a mask; the image mask and the CLS mask are concatenated and multiplied element-wise into the post-softmax attention map, so queries attend only to their own matched prompts. Training keeps the ViT frozen and optimizes only the prompt pools, the mask indicators, and the classification head, with an auxiliary term that pulls each token's selected indicator close to it. The claimed result is that extracted features become more diverse and discriminative, and that the module reliably improves every method it is attached to.

Load-bearing premise

The load-bearing premises are that the binarized cosine-distance matching assigns stable, meaningful token–prompt pairs and that zeroing entries of the post-softmax attention map without renormalizing leaves the frozen transformer's attention mechanism functionally intact; neither is analyzed in the paper, and if either fails the reported gains could come from the extra parameters rather than from coordination.

Editorial extensions

If this is right

  • Any of the studied visual prompting methods (VP, VPT, DAMVP, AutoVP, VFPT) can be upgraded by inserting TCPA while keeping the ViT frozen, with reported per-dataset gains of roughly 0.5 to 3 percentage points on HTA.
  • VTAB gains appear across all three task groups (Natural, Specialized, Structured), with the largest reported gains in Specialized and Structured.
  • Training-time overhead is small: on CUB the reported per-epoch time rises by about 0.05 to 0.3 seconds depending on the base method.
  • Because attention weights are computed once and masks are applied only before the multiplication with values, enlarging the prompt pool does not multiply feed-forward network cost.
  • The intended payoff is more diverse and discriminative features, which the paper supports with attention-map and t-SNE visualizations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural controlled test the paper does not run is to match the number of extra parameters (prompt pool size and mask sparsity) while ablating the matching rule; without that control, part of the gain could be attributable to capacity rather than coordination.
  • Because the mask thresholding is non-differentiable, the training signal for which prompt a token receives flows only through the auxiliary cosine-distance loss; a differentiable soft assignment might train more smoothly and could be tested as a drop-in variant.
  • The same token-role argument should transfer to other transformer input types, such as video patch tokens or language word tokens, where tokens carry heterogeneous local information, but the paper only demonstrates vision-image tokens.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Token Coordinated Prompt Attention (TCPA), a plug-and-play module for visual prompting with Vision Transformers. TCPA disentangles prompts into a CLS prompt pool and an image prompt pool, and assigns token-specific prompts via cosine-distance matching (Eq. 9), top-K binarization (Eq. 10), and binary masking of the post-softmax attention map (Eq. 13). The authors claim that this token-coordinated assignment makes extracted features more diverse and discriminative, and they report consistent accuracy improvements when TCPA is added to VP, VPT, DAMVP, AutoVP, and VFPT on the HTA and VTAB benchmarks, with negligible training-time overhead. The paper includes ablations of the two main components, hyperparameter sensitivity, attention-map visualizations, t-SNE feature visualizations, and a theoretical motivation based on the low-rankness of self-attention.

Significance. If the central claim is validated, TCPA would be a practically useful, low-overhead plug-in for existing visual prompting methods, with broad applicability. The paper has several strengths: evaluations across two benchmarks and multiple base methods, an ablation showing both components contribute, attention and feature visualizations, an efficiency analysis, and released code. However, the mechanistic attribution of the gains to token-coordinated assignment is not yet established because the experiments do not control for the larger prompt capacity introduced by TCPA, and the reported gains are small and lack uncertainty quantification. The theoretical discussion borrows external results and does not directly prove that prompt coordination is necessary. The central idea is plausible and the empirical evidence is suggestive, but additional controls and analysis are needed before the claim can be accepted.

major comments (3)
  1. [§5.3, Tables 1–3, Figure 4] The experiments do not include an equal-prompt-capacity baseline. TCPA uses a CLS prompt pool of Nc=10 and an image prompt pool of Ni=20 per layer (Section 5.3), i.e., up to 30 prompt tokens per layer, whereas VPT's prompts are replaced rather than augmented and VP/DAMVP/AutoVP have no token-level prompts. Table 3 shows that R-TCPA, which separates CLS and image prompts but does not perform per-token coordination, already accounts for roughly half of the total gain, and Figure 4 shows that prompt-pool size alone moves accuracy by about one point. To support the claim that the improvements come from the token-coordinated matching mechanism (Eqs. 9–13) and not merely from the added prompt capacity, the authors should add a control that uses the same total number of shared prompts without per-token masking, e.g., VPT with 30 prompt tokens per layer or TCPA with all mask entries set to 1. Without such a control, the central mechanistic attribution is confounded.
  2. [§3.3, Eq. 13] Eq. 13 zeroes entries of the post-softmax attention map without renormalizing the rows, so the resulting attention matrix is not row-stochastic. The paper does not analyze how this affects the frozen ViT's attention behavior or training dynamics. The non-differentiable top-K selection in Eq. 10 also lacks any gradient or stability analysis; gradients for the prompt indicators κ appear only through the regularization terms in Eq. 14 and not through the masking path, so the claim that prompts are "automatically assigned" in an end-to-end trained system needs either a straight-through estimator discussion or an empirical stability check. An ablation that renormalizes masked attention rows, or an analysis of matching stability across training, would strengthen the paper considerably.
  3. [§5.4, Tables 1–2] The reported gains are small (0.2–3.1 percentage points) and the paper provides no standard deviations, number of seeds, or significance tests. For VTAB, only group averages are reported, not per-task results, making it impossible to judge whether the "consistent" improvements hold across the 19 individual tasks. Given the headline claim of consistency, the authors should report per-task VTAB numbers and either multi-seed variance or a paired significance test across datasets. Hyperparameters such as top-K, prompt length, and the regularization weights λ_i and λ_c are fixed without sensitivity analysis beyond the pool-size study in Figure 4.
minor comments (5)
  1. [Figure 2 caption] Figure 2's caption contains the LaTeX placeholder "在此处键入公式。" ("type the formula here"); this production artifact must be removed before publication.
  2. [Table 4 and Figure 4] Table 4 contains garbled unicode tokens in its header and Figure 4 has similar encoding artifacts in the axis labels; these need to be regenerated cleanly.
  3. [§5.5.1] The text says "employing all three modules" but only two components (R-TCPA and T-TCPA) are defined and ablated in Table 3; this appears to be a typo and should be corrected.
  4. [Eq. 10] In the definition of the binarized matrix, the text says "elements of the i-th row" but should refer to the m-th row (matching the index of the image token); also the notation for the top-K value should be consistent (K_i with subscript i, not Ki).
  5. [§4, Theorems 4.1–4.2] The theorems are cited from prior work, but the paper's extrapolation that low-rank self-attention implies existing prompts "tend to focus on the same image regions" is not rigorously established; the authors should either provide a direct analysis or soften the claim, since this is the stated motivation for the token-coordination mechanism.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: TCPA's gains are measured on external benchmarks, and its matching mechanism is not derived from the reported results.

full rationale

The paper's central claim is that TCPA improves visual prompting by assigning token-coordinated prompts. This is not definitionally circular: the matching and masking procedure (Eqs. 9-13) is an explicit architectural proposal whose outputs (accuracy on HTA/VTAB, t-SNE plots, attention visualizations) are external to the construction. No parameter is fitted to a subset and then re-reported as a prediction: the prompt pools are trained on the downstream tasks with a standard cross-entropy objective (Eq. 14), and accuracy is evaluated on held-out benchmarks. The theoretical support in Section 4 invokes low-rank self-attention results from Wang et al. (2020) and Kim et al. (2024), both external works, and those theorems are used as motivation rather than as the definition of the method's success. The manuscript contains several references to the authors' own prior work (e.g., Liu et al. 2024a,b,c; Yao et al. 2025; Li & Zhou 2025; Xu et al. 2025), but they appear in related-work context and are not load-bearing for the TCPA derivation. The absence of an equal-prompt-capacity baseline (Section 5.3 uses Nc=10 and Ni=20 prompt pools, while VPT prompts are replaced rather than augmented) is a legitimate experimental confound, but a confound about causal attribution is not circularity: the reported improvements are still measured against external benchmarks and are not consequences of the paper's own definitions.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The paper's result is empirical and introduces no new physical entities. The central claim rests on modelling choices such as frozen pretrained features, the validity of post-softmax attention masking, the stability of top-K prompt selection, and a loose inference from low-rank attention to region overlap. These are recorded as axioms, while pool sizes, regularization weights, and unreported hyperparameters are recorded as free parameters.

free parameters (5)
  • CLS prompt pool size N_c = 10
    Set to 10 for all experiments; sensitivity is shown on CUB in Figure 4, and the chosen value is used for every downstream benchmark.
  • Image prompt pool size N_i = 20
    Set to 20 for all experiments; chosen based on the CUB sensitivity study in Figure 4.
  • Regularization weights lambda_i and lambda_c = 0.5
    Set to 0.5 in Section 5.3 without a sensitivity study; these weights control the prompt-matching regularization in Eq. 14.
  • Top-K selected prompts K_i
    Eq. 10 selects the top K_i largest affinity entries per token row, but K_i is never reported in Section 5.3 or the ablations.
  • Prompt length L_p
    Eq. 4 defines each prompt as a matrix in R^(L_p x D), but the value of L_p used in the experiments is never stated.
assumptions (4)
  • domain assumption Pretrained ViT-B/16 features are a suitable frozen feature extractor for all downstream tasks.
    Invoked throughout Section 5; standard in the prompt tuning literature and shared with the baselines, not specific to TCPA.
  • ad hoc to paper Zeroing entries of the post-softmax attention map in Eq. 13 yields a valid attention operation without renormalization.
    Eq. 13 masks attention weights after softmax and does not renormalize rows; the paper does not analyze the effect on feature magnitudes.
  • ad hoc to paper Low-rank self-attention matrices imply that existing prompts focus on the same image regions, so explicit token-prompt diversity is needed.
    Section 4 draws this inference from Theorems 4.1 and 4.2, but low rank alone does not imply identical attention patterns across prompts or tokens.
  • ad hoc to paper Top-K binarization in Eq. 10 remains stable and trainable enough for end-to-end optimization.
    The indicator function in Eq. 10 is non-differentiable, and the paper provides no gradient approximation or convergence analysis for the matching step.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Token Coordinated Prompt Attention is Needed for Visual Prompting." pith.science (2026). https://pith.science/paper/Q4W6P2WZ

@misc{pith2026250502406,
  author       = {Pith},
  title        = {Pith review of: Token Coordinated Prompt Attention is Needed for Visual Prompting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Q4W6P2WZ}},
  note         = {Machine review of arXiv:2505.02406}
}
read the original abstract

Visual prompting techniques are widely used to efficiently fine-tune pretrained Vision Transformers (ViT) by learning a small set of shared prompts for all tokens. However, existing methods overlook the unique roles of different tokens in conveying discriminative information and interact with all tokens using the same prompts, thereby limiting the representational capacity of ViT. This often leads to indistinguishable and biased prompt-extracted features, hindering performance. To address this issue, we propose a plug-and-play Token Coordinated Prompt Attention (TCPA) module, which assigns specific coordinated prompts to different tokens for attention-based interactions. Firstly, recognizing the distinct functions of CLS and image tokens-global information aggregation and local feature extraction, we disentangle the prompts into CLS Prompts and Image Prompts, which interact exclusively with CLS tokens and image tokens through attention mechanisms. This enhances their respective discriminative abilities. Furthermore, as different image tokens correspond to distinct image patches and contain diverse information, we employ a matching function to automatically assign coordinated prompts to individual tokens. This enables more precise attention interactions, improving the diversity and representational capacity of the extracted features. Extensive experiments across various benchmarks demonstrate that TCPA significantly enhances the diversity and discriminative power of the extracted features. The code is available at https://github.com/zhoujiahuan1991/ICML2025-TCPA.

Figures

Figures reproduced from arXiv: 2505.02406 by the authors.

Figure 1
Figure 1. Above: Visualization of the attention map. The existing visual prompting method VPT (Jia et al., 2022) learns the same prompts for all tokens, resulting in extracted information that lacks distinguishability and comprehensiveness. Our TCPA selects corresponding prompts for different tokens and performs attention interaction, thereby enhancing the diversity and discriminability of the extracted information. Below: Co… view at source ↗
Figure 2
Figure 2. The overall pipeline of our proposed TCPA. For each input sample, embeddings for each image patch are first obtained through the embedding layer. Then, CLS and image tokens adaptively select appropriate prompts from the corresponding CLS and Image Prompt Pools and generate a binary mask. This binary mask is then fed into the attention module to mask certain values in the attention map, enabling attention-based inter… view at source ↗
Figure 3
Figure 3. 3D and 2D attention map of existing visual prompting method VPT (Jia et al., 2022) and Ours. 3.4. Overall Optimization As mentioned above, our TCPA introduces only a few addi￾tional parameters: CLS prompt pool P c and image prompt pool P i . Following (Jia et al., 2022), during training, we maintain the pretrained model’s encoder frozen while al￾lowing only the classification head to be trainable. We denote all lear… view at source ↗
Figures from the paper (4 more)
Figure 5
Figure 5. Figure 5: Feature t-SNE (Van der Maaten & Hinton, 2008) visu￾alization results for our proposed TCPA and comparison method DAMVP (Huang et al., 2023) on GTSRB. can lead to overfitting, which also degrades model perfor￾mance. Optimal performance is achieved when the size of the p…
Figure 4
Figure 4. Figure 4: Influence of hyper-parameters (size of CLS prompt pool Nc, size of image prompt pool Ni) of TCPA on CUB. Despite disentangling the prompts used for the CLS and image tokens, as well as between different image tokens, the increase in computational demand is negligible. …
Figure 6
Figure 6. Figure 6: Feature t-SNE (Van der Maaten & Hinton, 2008) visualization results for our proposed TCPA and comparison method DAMVP on CUB, Cifar100 and SVHN. B. More Attention Visualization Results In [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: The attention map visualization results of CLS and image tokens from the existing visual prompting method VPT (Jia et al., 2022) and our TCPA are presented. 13 [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

50 extracted references · 23 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [3]

    Exploring visual prompts for adapting large-scale models

    Bahng, H., Jahanian, A., Sankaranarayanan, S., and Isola, P. Exploring visual prompts for adapting large-scale models. arXiv preprint arXiv:2203.17274, 2022

    arXiv 2022

  3. [4]

    Food-101--mining discriminative components with random forests

    Bossard, L., Guillaumin, M., and Van Gool, L. Food-101--mining discriminative components with random forests. In ECCV, pp.\ 446--461, Cham, 2014. Springer International Publishing

    work page 2014

  4. [5]

    Z., and Massachusetts Institute of Technology, S

    Cai, H., Gan, C., Massachusetts Institute of Technology, L. Z., and Massachusetts Institute of Technology, S. H. Tinytl: reduce memory, not parameters for efficient on-device learning. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS '20, Red Hook, NY, USA, 2020. Curran Associates Inc. ISBN 9781713829546

    work page 2020

  5. [6]

    Understanding and improving visual prompting: A label-mapping perspective

    Chen, A., Yao, Y., Chen, P.-Y., Zhang, Y., and Liu, S. Understanding and improving visual prompting: A label-mapping perspective. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 19133--19143, 2023. doi:10.1109/CVPR52729.2023.01834

    arXiv 2023

  6. [8]

    Adaptformer: Adapting vision transformers for scalable visual recognition

    Chen, S., Ge, C., Tong, Z., Wang, J., Song, Y., Wang, J., and Luo, P. Adaptformer: Adapting vision transformers for scalable visual recognition. Advances in Neural Information Processing Systems, 35: 0 16664--16678, 2022

    2022

  7. [10]

    Describing textures in the wild

    Cimpoi, M., Maji, S., Kokkinos, I., Mohamed, S., and Vedaldi, A. Describing textures in the wild. In 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp.\ 3606--3613, 2014. doi:10.1109/CVPR.2014.461

    doi:10.1109/cvpr.2014.461 2014

  8. [11]

    Imagenet: A large-scale hierarchical image database

    Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In CVPR, 2009

    2009

Show all 50 references
  1. [12]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020

    2010 arXiv

  2. [13]

    Spottune: Transfer learning through adaptive fine-tuning

    Guo, Y., Shi, H., Kumar, A., Grauman, K., Rosing, T., and Feris, R. Spottune: Transfer learning through adaptive fine-tuning. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 4800--4809, 2019. doi:10.1109/CVPR.2019.00494

    2019

  3. [14]

    E ^ 2 vpt: An effective and efficient approach for visual prompt tuning

    Han, C., Wang, Q., Cui, Y., Cao, Z., Wang, W., Qi, S., and Liu, D. E ^ 2 vpt: An effective and efficient approach for visual prompt tuning. arXiv preprint arXiv:2307.13770, 2023

    2023 arXiv

  4. [15]

    Momentum contrast for unsupervised visual representation learning

    He, K., Fan, H., Wu, Y., Xie, S., and Girshick, R. Momentum contrast for unsupervised visual representation learning. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 9726--9735, 2020. doi:10.1109/CVPR42600.2020.00975

    2020

  5. [16]

    V., Branson, S., Farrell, R., Haber, S., Barry, J., Ipeirotis, P., Perona, P., and Belongie, S

    Horn, G. V., Branson, S., Farrell, R., Haber, S., Barry, J., Ipeirotis, P., Perona, P., and Belongie, S. J. Building a bird recognition app and large scale dataset with citizen scientists: The fine print in fine-grained dataset collection. In CVPR, pp.\ 595--604. IEEE Computer...

    2015

  6. [17]

    Diversity-aware meta visual prompting

    Huang, Q., Dong, X., Chen, D., Zhang, W., Wang, F., Hua, G., and Yu, N. Diversity-aware meta visual prompting. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 10878--10887, 2023

    2023

  7. [18]

    How well do sparse imagenet models transfer? In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 12256--12266, 2022

    Iofinova, E., Peste, A., Kurtz, M., and Alistarh, D. How well do sparse imagenet models transfer? In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 12256--12266, 2022. doi:10.1109/CVPR52688.2022.01195

    2022

  8. [19]

    J., and Shin, J

    Jang, Y., Lee, H., Hwang, S. J., and Shin, J. Learning what and where to transfer. In ICML. PMLR, 05 2019

    2019

  9. [20]

    Visual prompt tuning

    Jia, M., Tang, L., Chen, B.-C., Cardie, C., Belongie, S., Hariharan, B., and Lim, S.-N. Visual prompt tuning. In Computer Vision -- ECCV 2022, pp.\ 709--727, Cham, 2022. Springer Nature Switzerland

    2022

  10. [21]

    Novel dataset for fine-grained image categorization: Stanford dogs

    Khosla, A., Jayadevaprakash, N., Yao, B., and Li, F.-F. Novel dataset for fine-grained image categorization: Stanford dogs. In CVPRW, 2011

    2011

  11. [22]

    Do we really need a large number of visual prompts? Neural Networks, 177: 0 106390, 2024

    Kim, Y., Li, Y., Moitra, A., Yin, R., and Panda, P. Do we really need a large number of visual prompts? Neural Networks, 177: 0 106390, 2024

    2024

  12. [23]

    Learning multiple layers of features from tiny images

    Krizhevsky, A., Hinton, G., et al. Learning multiple layers of features from tiny images. 2009

    2009

  13. [24]

    and Zhou, J

    Li, Q. and Zhou, J. Caprompt: Cyclic prompt aggregation for pre-trained model based class incremental learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp.\ 18421--18429, 2025

    2025

  14. [25]

    Li, X. L. and Liang, P. Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint arXiv:2101.00190, 2021

    2021 arXiv

  15. [26]

    L., Du, Z., Yang, Z., and Tang, J

    Liu, X., Ji, K., Fu, Y., Tam, W. L., Du, Z., Yang, Z., and Tang, J. P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks. arXiv preprint arXiv:2110.07602, 2021 a

    2021 arXiv

  16. [28]

    Compositional prompting for anti-forgetting in domain incremental learning

    Liu, Z., Peng, Y., and Zhou, J. Compositional prompting for anti-forgetting in domain incremental learning. International Journal of Computer Vision, pp.\ 1--18, 2024 a

    2024

  17. [29]

    Ins VP : Efficient instance visual prompting from image itself

    Liu, Z., Peng, Y., and Zhou, J. Ins VP : Efficient instance visual prompting from image itself. In ACM Multimedia 2024, 2024 b . URL https://openreview.net/forum?id=OTjo1q8rWL

    2024

  18. [30]

    Dart: Dual-modal adaptive online prompting and knowledge retention for test-time adaptation

    Liu, Z., Sun, H., Peng, Y., and Zhou, J. Dart: Dual-modal adaptive online prompting and knowledge retention for test-time adaptation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp.\ 14106--14114, 2024 c

    2024

  19. [31]

    and Hutter, F

    Loshchilov, I. and Hutter, F. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017

    2017 arXiv

  20. [32]

    Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., and Ng, A. Y. Reading digits in natural images with unsupervised feature learning. 2011

    2011

  21. [33]

    and Zisserman, A

    Nilsback, M.-E. and Zisserman, A. Automated flower classification over a large number of classes. In 2008 Sixth Indian Conference on Computer Vision, Graphics and Image Processing, pp.\ 722--729, 2008. doi:10.1109/ICVGIP.2008.47

    2008 doi

  22. [34]

    and Favaro, P

    Noroozi, M. and Favaro, P. Unsupervised learning of visual representations by solving jigsaw puzzles. In Computer Vision -- ECCV 2016, pp.\ 69--84, Cham, 2016. Springer International Publishing

    2016

  23. [35]

    Adapterhub: A framework for adapting transformers

    Pfeiffer, J., R \"u ckl \'e , A., Poth, C., Kamath, A., Vuli \'c , I., Ruder, S., Cho, K., and Gurevych, I. Adapterhub: A framework for adapting transformers. arXiv preprint arXiv:2007.07779, pp.\ 46--54, 01 2020. doi:10.18653/v1/2020.emnlp-demos.7

    2007 arXiv

  24. [36]

    Learning multiple visual domains with residual adapters

    Rebuffi, S.-A., Bilen, H., and Vedaldi, A. Learning multiple visual domains with residual adapters. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS'17, pp.\ 506–516, Red Hook, NY, USA, 2017. Curran Associates Inc. ISBN 9781510860964

    2017

  25. [37]

    Stallkamp, J., Schlipsing, M., Salmen, J., and Igel, C. Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition. Neural networks, 32: 0 323--332, 2012. ISSN 0893-6080. doi:https://doi.org/10.1016/j.neunet.2012.02.016

    2012 doi

  26. [38]

    AutoVP: An Automated Visual Prompting Framework and Benchmark

    Tsao, H.-A., Hsiung, L., Chen, P.-Y., Liu, S., and Ho, T.-Y. AutoVP: An Automated Visual Prompting Framework and Benchmark . In The Twelfth International Conference on Learning Representations, 2024

    2024

  27. [39]

    and Hinton, G

    Van der Maaten, L. and Hinton, G. Visualizing data using t-sne. Journal of machine learning research, 2008

    2008

  28. [40]

    The caltech-ucsd birds-200-2011 dataset

    Wah, C., Branson, S., Welinder, P., Perona, P., and Belongie, S. The caltech-ucsd birds-200-2011 dataset. 2011

    2011

  29. [41]

    Lion: Implicit vision prompt tuning

    Wang, H., Chang, J., Luo, X., Sun, J., Lin, Z., and Tian, Q. Lion: Implicit vision prompt tuning. arXiv preprint arXiv:2303.09992, 2023

    2023 arXiv

  30. [42]

    Z., Khabsa, M., Fang, H., and Ma, H

    Wang, S., Li, B. Z., Khabsa, M., Fang, H., and Ma, H. Linformer: Self-attention with linear complexity. arXiv preprint arXiv:2006.04768, 2020

    2006 arXiv

  31. [43]

    Pyramid vision transformer: A versatile backbone for dense prediction without convolutions

    Wang, W., Xie, E., Li, X., Fan, D.-P., Song, K., Liang, D., Lu, T., Luo, P., and Shao, L. Pyramid vision transformer: A versatile backbone for dense prediction without convolutions. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp.\ 548--558, 2021. doi:1...

    2021

  32. [44]

    Transhp: Image classification with hierarchical prompting

    Wang, W., Sun, Y., Li, W., and Yang, Y. Transhp: Image classification with hierarchical prompting. Advances in Neural Information Processing Systems, 36, 2024 a

    2024

  33. [45]

    Revisiting the power of prompt for visual tuning

    Wang, Y., Cheng, L., Fang, C., Zhang, D., Duan, M., and Wang, M. Revisiting the power of prompt for visual tuning. In Forty-first International Conference on Machine Learning, 2024 b . URL https://openreview.net/forum?id=2Y93PtAqCl

    2024

  34. [46]

    Dask: Distribution rehearsing via adaptive style kernel learning for exemplar-free lifelong person re-identification

    Xu, K., Jiang, C., Xiong, P., Peng, Y., and Zhou, J. Dask: Distribution rehearsing via adaptive style kernel learning for exemplar-free lifelong person re-identification. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp.\ 8915--8923, 2025

    2025

  35. [47]

    Selective visual prompting in vision mamba

    Yao, Y., Liu, Z., Cui, Z., Peng, Y., and Zhou, J. Selective visual prompting in vision mamba. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp.\ 22083--22091, 2025

    2025

  36. [48]

    Improving visual prompt tuning for self-supervised vision transformers

    Yoo, S., Kim, E., Jung, D., Lee, J., and Yoon, S. Improving visual prompt tuning for self-supervised vision transformers. arXiv preprint arXiv:2306.05067, 2023

    2023 arXiv

  37. [49]

    Yosinski, J., Clune, J., Bengio, Y., and Lipson, H. How transferable are features in deep neural networks? In Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2, NIPS'14, pp.\ 3320–3328, Cambridge, MA, USA, 2014. MIT Press

    2014

  38. [50]

    B., Ravfogel, S., and Goldberg, Y

    Zaken, E. B., Ravfogel, S., and Goldberg, Y. Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked language-models. arXiv preprint arXiv:2106.10199, 2021

    2021

  39. [51]

    N., and Liu, D

    Zeng, R., Han, C., Wang, Q., Wu, C., Geng, T., Huang, L., Wu, Y. N., and Liu, D. Visual fourier prompt tuning. arXiv preprint arXiv:2411.01327, 2024

    2024 arXiv

  40. [52]

    S., Neumann, M., Dosovitskiy, A., et al

    Zhai, X., Puigcerver, J., Kolesnikov, A., Ruyssen, P., Riquelme, C., Lucic, M., Djolonga, J., Pinto, A. S., Neumann, M., Dosovitskiy, A., et al. A large-scale study of representation learning with the visual task adaptation benchmark. arXiv preprint arXiv:1910.04867, 2019

    1910 arXiv

  41. [53]

    O., Sax, A., Zamir, A., Guibas, L., and Malik, J

    Zhang, J. O., Sax, A., Zamir, A., Guibas, L., and Malik, J. Side-tuning: A baseline for network adaptation via additive side networks. In Computer Vision -- ECCV 2020, pp.\ 698--714, Cham, 2020. Springer International Publishing

    2020

  42. [54]

    Zhang, R., Isola, P., and Efros, A. A. Colorful image colorization. In Computer Vision -- ECCV 2016, pp.\ 649--666, Cham, 2016. Springer International Publishing

    2016

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.