Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

LoR-VP: Low-Rank Visual Prompting for Efficient Vision Model Adaptation

T0 review · 3 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A rank-4 low-rank prompt added to the full resized image adapts frozen vision models better than padding-based visual prompting.

desk verdict Solid, well-run empirical paper on low-rank visual prompting; the rank-4 mechanism story is softer than the efficiency story, but the rank sweep covers much of the gap. read the letter →

arxiv 2502.00896 v3 pith:JMZ6HDOT submitted 2025-02-02 cs.CV

Can Jin , Ying Li , Mingyu Zhao , Shiyu Zhao , Zhenting Wang , Xiaoxiao He , Ligong Han , Tong Che
show 1 more author
Dimitris N. Metaxas
This is my paper · ORCID
classification cs.CV
keywords visualpromptinglow-rankmatrixproductparameter-efficientadaptationtransferlearningvisiontransformerlinearprobingout-of-distributionrobustnessimageclassification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a frozen image classifier can be adapted to new tasks by adding a rank-4 low-rank matrix product directly to the full-resolution resized image, rather than padding tunable frames around a smaller image. The proposed LoR-VP prompt, $P(x)=\operatorname{Resize}_L(x)+B\cdot A$, reaches higher accuracy than padding-based visual prompting across seven architectures and four datasets, while using roughly 5,000 visual-prompt parameters, training up to 6 times faster than AutoVP, and improving accuracy by 3.1% on average. The reason this would matter is practical: if input-level prompting can be both smaller and stronger than existing designs, adapting large vision models to new tasks becomes cheaper and simpler, especially when model weights must stay frozen or the classifier head is very large.

What carries the argument

The load-bearing object is the rank-$r$ factorization $B\cdot A$ added to the resized image. Because $B$ and $A$ are shared across all images and all patches, their product acts as a linear combination of $r$ row-vectors and $r$ column-vectors per color channel, so the same prompt expresses information that is common to entire rows and columns while the individual entries remain free to represent patch-specific adjustments. This replaces the $cL^2$ free parameters of a full additive image prompt with $2crL$ parameters, and replaces the resize-pad-mask pipeline with a single resize-plus-add operation. The paper couples this input design with linear probing on the classifier head as the output transformation, which avoids the costly label-mapping sequences used by padding baselines.

What would settle it

Train a full-rank additive visual prompt of the same form, $P(x)=\operatorname{Resize}_L(x)+V$ with $V\in\mathbb{R}^{3\times 224\times 224}$, using the same optimizer, epochs, and linear-probing head on at least one pair where LoR-VP is reported to win, such as ViT-B/16-P on Tiny-ImageNet, and inspect the singular-value spectrum of the learned $V$; if full-rank $V$ matches or exceeds LoR-VP accuracy, or if the learned spectrum does not decay rapidly, the paper's claim that low rank supplies the useful inductive bias would be weakened, though the efficiency advantage would remain.

Watch

Extended reading notes

Core claim

The paper's central claim is that the bottleneck of existing visual prompting is not prompt size but where and how the prompt touches the image. Padding-based prompts modify only border patches and treat each patch independently; LoR-VP instead resizes the input to $L\times L$ and adds $B\cdot A$ with $B\in\mathbb{R}^{c\times L\times r}$ and $A\in\mathbb{R}^{c\times r\times L}$ for $r=4$, a per-channel low-rank matrix that reaches every pixel and encodes shared row and column structure while allowing patch-specific coefficients. With zero-initialization of $B$ and Gaussian initialization of $A$, training starts from an unchanged image. Across ResNet, ViT, Swin, and CLIP models on ImageNet, Tiny-ImageNet, and CIFAR benchmarks, the paper reports consistent gains over AutoVP, ILM-VP, CLIP-VP, and linear probing, including a 3.1% average improvement over AutoVP, 5 times fewer epochs and 6 times less training time, and 18 times fewer visual-prompt parameters, with the margin growing to 5.06 points on ImageNet-1K with a 21K-class Swin-B head. The paper also reports stronger out-of-distribution generalization on four ImageNet variants and extensions to object detection and semantic segmentation.

Load-bearing premise

The load-bearing assumption is that a rank-4 additive prompt is at least as capable, per unit of data and training, as an unrestricted full-image prompt; this is tested only indirectly, because the paper's main experiments do not include a full-rank additive prompt as a same-cost baseline.

Editorial extensions

If this is right

  • Visual prompting no longer needs to search over resize size and padding width: a fixed resize to $L$ plus a rank-4 additive prompt is reported to be sufficient across models and datasets.
  • Because the prompt touches every pixel, the method works on models with very large classifier heads, where pad-based methods with full mapping struggle; the reported ImageNet-1K gain over AutoVP with Swin-B is 5.06 points.
  • The reported out-of-distribution robustness on ImageNet-R, ImageNet-Sketch, ImageNet-A, and ImageNet-V2 suggests the adapted model generalizes beyond the source distribution.
  • The parameter and time reductions, roughly 5K prompt parameters, 5 times fewer epochs than AutoVP, and 10 times fewer than ILM-VP, make the method attractive for resource-constrained deployment without changing inference latency materially.
  • The paper reports that the same visual prompt transfers beyond classification, improving detection AP50 by about 4 points over AutoVP and segmentation mIoU by 1.1 points on Pascal VOC.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: since the paper's rank sweep shows accuracy saturating at $r=4$ for linear probing, an experiment the paper leaves implicit is to learn a full-rank additive prompt on the same task and inspect its singular-value spectrum; a rapidly decaying spectrum would validate low rank as a learned prior, while a flat spectrum would suggest the gains come from the additive full-image form
  • Editorial inference: the prompt is a single global matrix added to every image, so it can be read as a dataset-level input shift; conditioning $B$ or $A$ on the input or on the task domain is a natural next step the paper does not explore.
  • Editorial inference: the large parameter and time reductions suggest LoR-VP could combine naturally with other parameter-efficient adaptations, such as lightweight adapters or per-class prompt banks, though the paper does not test such compositions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes LoR-VP, a visual prompting method that adapts frozen image classifiers by resizing inputs to 224x224 and adding a low-rank additive prompt P(x) = Resize_L(x) + B·A with rank r = 4, using linear probing as the output transformation. The authors report that this simple input-level prompt outperforms prior pad-based visual prompting methods (AutoVP, ILM-VP, CLIP-VP) and the LP baseline across seven network architectures and several datasets, while reducing training time and visual prompt parameters. A preliminary study compares four prompt designs and motivates the low-rank design; ablation studies address output transformation, rank sensitivity, and component contributions. Code is released.

Significance. If the reported results hold, LoR-VP is a genuinely simple and parameter-efficient input-level adaptation method with practical value: it uses only 5K visual prompt parameters, trains substantially faster than AutoVP and ILM-VP, and improves accuracy on several benchmarks including large-scale ImageNet-21K-pretrained models. The paper's strengths include the controlled comparisons of output transformations in Tables 3 and 12, the component ablation in Table 5, efficiency measurements in Table 2, OOD evaluation in Table 1, and a public code release. However, the central mechanistic claim that the rank-4 low-rank structure is the source of the gains is not fully isolated in the main experiments, and the absence of error bars leaves several small accuracy differences without statistical support.

major comments (3)
  1. [Section 3.2 / Figure 2 vs. Section 5.2 / Figures 4-5] The controlled baselines that isolate the rank-4 prompt design from full-resolution resizing and the LP head appear only in the preliminary study and are absent from the main results. Patch-Free (a full-rank additive prompt with cL^2 parameters) and Patch-Same (a shared-patch prompt) are compared with LoR-VP only on two model/dataset pairs and with FM as the output transformation. The main tables and figures never include these baselines with the matched LP output transformation. Since Patch-Free is exactly the comparison that tests whether low rank, rather than full-resolution resizing plus an additive prompt, drives the improvement, omitting it from the main benchmarks leaves the paper's central inductive-bias claim under-supported. Please add Patch-Free and Patch-Same to the main comparisons with matched output heads and report whether rank-4 still outperforms them.
  2. [Section 5.1 / Figure 6] The rank r=4 and other hyperparameters appear to be selected on the same datasets used for the headline results. Section 5.1 states that optimal hyperparameters are determined through grid search, and Figure 6 selects the optimal rank using Tiny-ImageNet and CIFAR-100, which are also main evaluation datasets. There is no described held-out validation split. This selection-on-evaluation procedure risks inflating the reported gains and makes the 3.1% average improvement over AutoVP difficult to interpret. Please specify the validation procedure, use held-out splits for hyperparameter selection, or report the sensitivity of the final numbers to the chosen rank in a way that accounts for selection.
  3. [Figures 2, 4, 5 and Table 1] The paper reports results averaged over three runs but provides no error bars, standard deviations, or significance tests. Several of the headline differences are small: for example, in Table 1 the ImageNet-A gap between LoR-VP and LP is 0.35 (27.89 vs. 27.54), and in Figure 4 the ResNet-50 Tiny-ImageNet gap over the best baseline is about 1.03. Without variance information, the reader cannot tell whether the claimed improvements are meaningful or within run-to-run noise. Please add error bars or standard deviations for all three-run results and, where appropriate, a paired significance test for the main comparisons.
minor comments (5)
  1. [Section 5.1, Networks paragraph] The text says 'We employ six architectures' but then lists seven architectures: ResNet-18, ResNet-50, ViT-B/32, ResNet-50-P, ViT-B/16-P, Swin-B, and CLIP. Please correct this to seven.
  2. [Section 4.1, Eq. (1)] Equation (1) writes B · A with B in R^{c x L x r} and A in R^{c x r x L}, but it is not immediately obvious that the product is a per-channel matrix product. Please clarify the indexing, e.g., by writing P_c(x) = Resize_L(x)_c + B_c A_c or by noting that the multiplication is performed independently for each channel c.
  3. [Section 3.2] The term 'fully connected label (FM)' in Section 3.2 is inconsistent with the term 'full mapping (FM)' used in Section 2 and elsewhere. Please unify the terminology.
  4. [References] The references 'et al. Yeh' and 'et al. Zhang' are incomplete and should list the actual author names; this is a formatting issue that affects reproducibility of the bibliography.
  5. [Captions of Figures 2, 4, 5] The captions refer to markers ⋆ and • but do not define what each marker denotes. Please define them in each caption or in a common legend.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: LoR-VP's reported gains are measured empirically; the low-rank design is motivated by a preliminary study and validated by ablations, not derived from its own outputs.

full rationale

LoR-VP is an empirical method paper. Its central claim is backed by measured test accuracies, and the only defining equation, P(x)=Resize_L(x)+B·A, is the method itself; the reported improvements are obtained by optimizing B and A on training data and evaluating on test sets, not by fitting a parameter to the target result. The low-rank hypothesis is motivated by the preliminary comparisons in Section 3.2/Figure 2 and then separately probed by rank sweeps and ablations in Section 5.5. Even though Figure 2 also shows LoR-VP's own accuracy, that is ordinary empirical validation, not a definitional or fitted-input circularity. The self-citations in the related-work section (e.g., Jin et al. 2023, Han et al. 2023, Han et al. 2024) are contextual and not load-bearing; no uniqueness theorem from the authors is invoked to force the low-rank form. The absence of a full-rank additive prompt in the main tables is a legitimate strength-of-evidence concern about the inductive-bias claim, but it is a comparison/ablation gap, not a circular reduction. Under the hard rules, flagging circularity requires exhibiting a specific equation or fitted quantity that reproduces the result by construction, and no such step exists here.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The method relies on the standard visual prompting premise and on the paper's own claim that a rank-4 factorization of an additive image prompt is a useful inductive bias. The only fitted numeric inputs are the rank, learning rate, and training schedule, all chosen by the authors on the evaluation benchmarks. No new physical entities are introduced.

free parameters (3)
  • rank r of the low-rank prompt = 4
    Selected by ablation in Figure 6 on Tiny-ImageNet and CIFAR-100; the paper reports that rank 4 is optimal for linear probing and full mapping. This value is tuned on datasets that also appear in the main evaluation.
  • number of training epochs = 20 (10 for Swin-B on ImageNet-1K)
    The efficiency claim rests on reaching high accuracy in 20 epochs; this schedule was set by the authors and not compared to longer schedules for LoR-VP.
  • learning rate = 0.02 (SGD), 40 for CLIP
    Chosen by grid search per Table 8. It affects final accuracy and convergence, but is a standard hyperparameter rather than a scientific constant.
assumptions (3)
  • domain assumption A frozen pre-trained vision model can be adapted to a new task by adding tunable parameters to the input image, with weights fixed.
    This is the founding assumption of visual prompting, adopted from Bahng et al. (2022), Chen et al. (2023), and Tsao et al. (2024). It is invoked throughout Sections 3.1 and 4.
  • domain assumption Linear probing of the frozen backbone features is an adequate output transformation for the downstream tasks.
    Section 4.2 and Table 8 use a linear classifier head instead of label mapping; the method's reported superiority is evaluated with this output head.
  • ad hoc to paper Rank-4 multiplicative structure of the prompt induces a useful inductive bias for sharing information across rows and columns while retaining patch-specific detail.
    Proposed in Section 4.1 based on the preliminary study in Section 3.2. No theoretical justification is given, and the main experiments do not compare against a full-rank additive prompt.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LoR-VP: Low-Rank Visual Prompting for Efficient Vision Model Adaptation." pith.science (2026). https://pith.science/paper/JMZ6HDOT

@misc{pith2026250200896,
  author       = {Pith},
  title        = {Pith review of: LoR-VP: Low-Rank Visual Prompting for Efficient Vision Model Adaptation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JMZ6HDOT}},
  note         = {Machine review of arXiv:2502.00896}
}
read the original abstract

Visual prompting has gained popularity as a method for adapting pre-trained models to specific tasks, particularly in the realm of parameter-efficient tuning. However, existing visual prompting techniques often pad the prompt parameters around the image, limiting the interaction between the visual prompts and the original image to a small set of patches while neglecting the inductive bias present in shared information across different patches. In this study, we conduct a thorough preliminary investigation to identify and address these limitations. We propose a novel visual prompt design, introducing Low-Rank matrix multiplication for Visual Prompting (LoR-VP), which enables shared and patch-specific information across rows and columns of image pixels. Extensive experiments across seven network architectures and four datasets demonstrate significant improvements in both performance and efficiency compared to state-of-the-art visual prompting methods, achieving up to 6 times faster training times, utilizing 18 times fewer visual prompt parameters, and delivering a 3.1% improvement in performance. The code is available as https://github.com/jincan333/LoR-VP.

Figures

Figures reproduced from arXiv: 2502.00896 by the authors.

Figure 1
Figure 1. Illustration of various visual prompting methods applied to target domain data: [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Preliminary Investigation Results. Performance comparison of various VP designs. Our VP method demonstrates competitive or superior performance in several configurations. The final performance of each method is marked by ⋆ or •, with all results averaged over three runs. 4 [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Our VP Design. We resize the image to a resolution of L × L and initialize two low-rank matrices B and A as tunable parameters. B · A serves as the visual prompt and is directly added to the resized images. This design allows shared information in rows and columns and also allows patch-specific information in different patches. various visual domains. Unlike previous works such as ILM-VP (Chen et al., 2023) and Auto… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: , we can observe that: ❶ LOR-VP consistently outperforms all baselines across all network and dataset combinations, achieving an average improvement of 3.2% and 2.1% over AutoVP and LP, respectively. ❷ LOR-VP converges significantly faster than the baselines, reaching …
Figure 5
Figure 5. Figure 5: Performance of ImageNet-21K Pre-trained Models on ImageNet-1K and Tiny￾ImageNet. Performance comparison of LOR-VP and four baseline methods. The models are pre￾trained on either ImageNet-21K-P or ImageNet-21K and then tuned on the respective downstream datasets. The fi…
Figure 6
Figure 6. Figure 6: The Impact of Rank in LOR-VP [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. HeteroPROPMT: A Real-time and Privacy-Preserving Heterogeneous Collaborative Perception Framework

    cs.CV 2026-07 conditional novelty 5.0 of 10

    A prompt-based, privacy-preserving heterogeneity-alignment framework that uses low-rank FiLM to adapt BEV features and an autoencoder-based classifier for metadata-free modality routing.

Reference graph

Works this paper leans on

55 extracted references · 28 canonical work pages · cited by 1 Pith paper

  1. [1]

    Can SGD learn recurrent neural networks with provable generalization? In Advances in Neural Information Processing Systems (NeurIPS), pp.\ 10310--10320, 2019

    Zeyuan Allen-Zhu and Yuanzhi Li. Can SGD learn recurrent neural networks with provable generalization? In Advances in Neural Information Processing Systems (NeurIPS), pp.\ 10310--10320, 2019

    work page 2019

  2. [2]

    On the backward stability of SGD and its use for asymptotic model selection in neural networks

    Zeyuan Allen-Zhu and Yuanzhi Li. On the backward stability of SGD and its use for asymptotic model selection in neural networks. Journal of Machine Learning Research (JMLR), 21 0 (190): 0 1--50, 2020

    work page 2020

  3. [3]

    On the convergence rate of training recurrent neural networks

    Zeyuan Allen-Zhu, Yuanzhi Li, Zhao Song, and Yingyu Wang. On the convergence rate of training recurrent neural networks. In Advances in Neural Information Processing Systems (NeurIPS), pp.\ 10031--10041, 2019

    work page 2019

  4. [4]

    Backward feature correction: How deep learning performs deep learning

    Zeyuan Allen-Zhu, Yuanzhi Li, and Zhao Song. Backward feature correction: How deep learning performs deep learning. In Proceedings of the 33rd International Conference on Neural Information Processing Systems, pp.\ 13540--13550, 2020

    work page 2020

  5. [5]

    Exploring visual prompts for adapting large-scale models

    Hyojin Bahng, Ali Jahanian, Swami Sankaranarayanan, and Phillip Isola. Exploring visual prompts for adapting large-scale models. arXiv preprint arXiv:2203.17274, 2022

    arXiv 2022

  6. [6]

    Yolov4: Optimal speed and accuracy of object detection

    Alexey Bochkovskiy, Chien-Yao Wang, and Hong-Yuan Mark Liao. Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934, 2020

    arXiv 2004

  7. [7]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gr...

    1901

  8. [8]

    Sample-specific masks for visual reprogramming-based prompting

    Chengyi Cai, Zesheng Ye, Lei Feng, Jianzhong Qi, and Feng Liu. Sample-specific masks for visual reprogramming-based prompting. In Forty-first International Conference on Machine Learning, 2024

    work page 2024

Show all 55 references
  1. [9]

    Singular value thresholding algorithm for matrix completion

    Jian-Feng Cai et al. Singular value thresholding algorithm for matrix completion. SIAM Journal on Optimization, 2010

    2010

  2. [10]

    Understanding and improving visual prompting: A label-mapping perspective

    Aochuan Chen, Yuguang Yao, Pin-Yu Chen, Yihua Zhang, and Sijia Liu. Understanding and improving visual prompting: A label-mapping perspective. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 19133--19143, 2023

    2023

  3. [11]

    Encoder-decoder with atrous separable convolution for semantic image segmentation

    Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European conference on computer vision (ECCV), pp.\ 801--818, 2018

    2018

  4. [12]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp.\ 248--255. Ieee, 2009

    2009

  5. [13]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In International Con...

    2020

  6. [14]

    Neural networks are more expressive than kernel methods: A representational perspective

    Amirata Ghorbani, Song Mei, Theodor Misiakiewicz, and Andrea Montanari. Neural networks are more expressive than kernel methods: A representational perspective. In International Conference on Learning Representations (ICLR), 2020

    2020

  7. [15]

    Low-rank tensor approximation techniques

    Lars Grasedyck et al. Low-rank tensor approximation techniques. SIAM Journal on Matrix Analysis and Applications, 2013

    2013

  8. [16]

    Svdiff: Compact parameter space for diffusion fine-tuning

    Ligong Han, Yinxiao Li, Han Zhang, Peyman Milanfar, Dimitris Metaxas, and Feng Yang. Svdiff: Compact parameter space for diffusion fine-tuning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.\ 7323--7334, 2023

    2023

  9. [17]

    Proxedit: Improving tuning-free real image editing with proximal guidance

    Ligong Han, Song Wen, Qi Chen, Zhixing Zhang, Kunpeng Song, Mengwei Ren, Ruijiang Gao, Anastasis Stathopoulos, Xiaoxiao He, Yuxiao Chen, et al. Proxedit: Improving tuning-free real image editing with proximal guidance. In Proceedings of the IEEE/CVF Winter Conference on Applic...

    2024

  10. [18]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 770--778, 2016

    2016

  11. [19]

    Momentum contrast for unsupervised visual representation learning

    Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 9729--9738, 2020

    2020

  12. [20]

    The many faces of robustness: A critical analysis of out-of-distribution generalization

    Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, et al. The many faces of robustness: A critical analysis of out-of-distribution generalization. In Proceedings of the IEEE/CVF international c...

    2021

  13. [21]

    Natural adversarial examples

    Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adversarial examples. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 15262--15271, 2021 b

    2021

  14. [22]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021

    2021 arXiv

  15. [23]

    Visual prompt tuning

    Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim. Visual prompt tuning. In European Conference on Computer Vision (ECCV), 2022. URL https://arxiv.org/abs/2203.12119

    2022 arXiv

  16. [24]

    Visual prompting upgrades neural network sparsification: A data-model perspective

    Can Jin, Tianjin Huang, Yihua Zhang, Mykola Pechenizkiy, Sijia Liu, Shiwei Liu, and Tianlong Chen. Visual prompting upgrades neural network sparsification: A data-model perspective. arXiv preprint arXiv:2312.01397, 2023

    2023 arXiv

  17. [25]

    Initialization and regularization of factorized neural layers

    Mikhail Khodak, David Macko, and Christopher De Sa. Initialization and regularization of factorized neural layers. In Advances in Neural Information Processing Systems (NeurIPS), 2021

    2021

  18. [26]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky et al. Learning multiple layers of features from tiny images. 2009

    2009

  19. [27]

    Tiny imagenet visual recognition challenge

    Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. 2015

    2015

  20. [28]

    Low-rank adaptation of large neural networks

    Xiang Li et al. Low-rank adaptation of large neural networks. In International Conference on Learning Representations (ICLR), 2016

    2016

  21. [29]

    Low-rank matrix recovery

    Xiang Li et al. Low-rank matrix recovery. Foundations and Trends® in Machine Learning, 2018

    2018

  22. [30]

    Prefix-tuning: Optimizing continuous prompts for generation

    Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Pap...

    2021

  23. [31]

    Learning overparameterized neural networks via stochastic gradient descent on structured data

    Yuanzhi Li and Yingyu Liang. Learning overparameterized neural networks via stochastic gradient descent on structured data. In Advances in Neural Information Processing Systems (NeurIPS), pp.\ 8157--8166, 2018

    2018

  24. [32]

    Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning

    Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin A Raffel. Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning. Advances in Neural Information Processing Systems, 35: 0 1950--1965, 2022

    1950

  25. [33]

    Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing

    Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 55 0 (9): 0 1--35, 2023

    2023

  26. [34]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pp.\ 10012--10022, 2021

    2021

  27. [35]

    Blackvip: Black-box visual prompting for robust transfer learning

    Changdae Oh, Hyeji Hwang, Hee-young Lee, YongTaek Lim, Geunyoung Jung, Jiyoung Jung, Hosik Choi, and Kyungwoo Song. Blackvip: Black-box visual prompting for robust transfer learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p...

    2023

  28. [36]

    Generalization guarantees for neural networks via harnessing the low-rank structure of the jacobian

    Samet Oymak, Zalan Fabian, Mingchen Li, and Mahdi Soltanolkotabi. Generalization guarantees for neural networks via harnessing the low-rank structure of the jacobian. arXiv preprint arXiv:1906.05392, 2019

    1906 arXiv

  29. [37]

    Maxk-gnn: Extremely fast gpu kernel design for accelerating graph neural networks training

    Hongwu Peng, Xi Xie, Kaustubh Shivdikar, Md Amit Hasan, Jiahui Zhao, Shaoyi Huang, Omer Khan, David Kaeli, and Caiwen Ding. Maxk-gnn: Extremely fast gpu kernel design for accelerating graph neural networks training. In Proceedings of the 29th ACM International Conference on Ar...

    2024

  30. [38]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp...

    2021

  31. [39]

    Do imagenet classifiers generalize to imagenet? In International conference on machine learning, pp.\ 5389--5400

    Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do imagenet classifiers generalize to imagenet? In International conference on machine learning, pp.\ 5389--5400. PMLR, 2019

    2019

  32. [40]

    Imagenet-21k pretraining for the masses

    Tal Ridnik, Emanuel Ben-Baruch, Asaf Noy, and Lihi Zelnik-Manor. Imagenet-21k pretraining for the masses. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1), 2021

    2021

  33. [41]

    Low-rank matrix factorization for deep neural network training with high-dimensional output targets

    Tara N Sainath, Brian Kingsbury, George Saon, Hagen Soltau, Abdel-rahman Mohamed, George E Dahl, and Bhuvana Ramabhadran. Low-rank matrix factorization for deep neural network training with high-dimensional output targets. In IEEE International Conference on Acoustics, Speech ...

    2013

  34. [42]

    Logan IV, Eric Wallace, and Sameer Singh

    Taylor Shin, Yasaman Razeghi, Robert L. Logan IV, Eric Wallace, and Sameer Singh. A uto P rompt: E liciting K nowledge from L anguage M odels with A utomatically G enerated P rompts. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (eds.), Proceedings of the 2020 Conferen...

    2020 doi

  35. [43]

    Transfer learning without knowing: Reprogramming black-box machine learning models with scarce data and limited resources

    Yun-Yun Tsai, Pin-Yu Chen, and Tsung-Yi Ho. Transfer learning without knowing: Reprogramming black-box machine learning models with scarce data and limited resources. In International Conference on Machine Learning (ICML). PMLR, 2020. URL https://proceedings.mlr.press/v119/tsa...

    2020

  36. [44]

    AutoVP: An Automated Visual Prompting Framework and Benchmark

    Hsi-Ai Tsao, Lei Hsiung, Pin-Yu Chen, Sijia Liu, and Tsung-Yi Ho. AutoVP: An Automated Visual Prompting Framework and Benchmark . In The Twelfth International Conference on Learning Representations, 2024

    2024

  37. [45]

    Learning robust global representations by penalizing local predictive power

    Haohan Wang, Songwei Ge, Zachary Lipton, and Eric P Xing. Learning robust global representations by penalizing local predictive power. Advances in Neural Information Processing Systems, 32, 2019

    2019

  38. [46]

    Lirong Wu, Cheng Tan, Zicheng Liu, Zhangyang Gao, Haitao Lin, and Stan Z. Li. Learning to augment graph structure for both homophily and heterophily graphs. In Proceedings of the European Conference on Machine Learning (ECML), 2022. URL https://link.springer.com/article/10.100...

    2022 doi

  39. [47]

    Prompt tuning for unified multimodal pretrained models

    Hao Yang, Junyang Lin, An Yang, Peng Wang, and Chang Zhou. Prompt tuning for unified multimodal pretrained models. In Findings of the Association for Computational Linguistics: ACL 2023, pp.\ 402--416, Toronto, Canada, July 2023. Association for Computational Linguistics. doi:...

    2023 doi

  40. [48]

    et al. Yeh. Lora: A unified low-rank adaptation framework for stable diffusion. arXiv preprint arXiv:2409.14983, 2023

    2023 arXiv

  41. [49]

    et al. Zhang. Svd-llm: Truncation-aware singular value decomposition for large language model compression. arXiv preprint arXiv:2403.07378, 2023

    2023 arXiv

  42. [50]

    Facial landmark detection by deep multi-task learning

    Zhifei Zhang, Ping Luo, Chen Change Loy, and Xiaoou Tang. Facial landmark detection by deep multi-task learning. In European conference on computer vision, pp.\ 94--108. Springer, 2014

    2014

  43. [51]

    Energy-efficient image classification on low-power iot devices

    Menglong Zhao, Wanli Ouyang, Xiaogang Li, and Xiaowei Wang. Energy-efficient image classification on low-power iot devices. In IEEE Transactions on Circuits and Systems for Video Technology, pp.\ 205--215. IEEE, 2016

    2016

  44. [52]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  45. [53]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  46. [54]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  47. [55]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.