Pith. sign in

REVIEW 3 major objections 4 minor 53 references

Split convolution adaptation into channels and space, keep priors intact

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · glm-5.2

2026-07-09 22:55 UTC pith:WU4X5QG5

load-bearing objection Solid convolutional PEFT method with a real ablation gap the 3 major comments →

arxiv 2607.06918 v1 pith:WU4X5QG5 submitted 2026-07-08 cs.CV cs.AIcs.LG

LoCA: Spatially-Aware Low-Rank Convolutional Adaptation of Vision Foundation Models

Sojung An , Junha Lee , Sujeong You , Nam Ik Cho , Donghyun Kim This is my paper
classification cs.CV cs.AIcs.LG
keywords parameter-efficient fine-tuninglow-rank adaptationconvolutional neural networkssingular value decompositionspatial inductive biasvision foundation models
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Low-Rank Adaptation (LoRA) has become the standard method for efficiently fine-tuning large pre-trained models, but it was designed for the two-dimensional weight matrices found in transformer attention layers. When applied to convolutional layers—which store weights as four-dimensional tensors coupling spatial kernel shape with input-output channels—LoRA flattens the kernel into a 2D matrix, destroying the spatial topology that gives convolutions their inductive bias. This paper proposes Low-Rank Convolutional Adaptation (LoCA), which separates the adaptation update into two independent paths: a low-rank channel mixing path that handles dense cross-channel dependencies without touching spatial structure, and a spatial basis refinement path that extracts principal spatial patterns from the pre-trained kernel via singular value decomposition of the spatial covariance matrix and learns to adjust them. The two paths are composed additively, with the spatial update placed only on the depthwise diagonal to keep it decoupled from cross-channel mixing. The method also introduces hierarchical rank scheduling that allocates larger adaptation ranks to deeper, wider network stages. Experiments span fine-grained classification, domain-generalized semantic segmentation, and text-to-image generation, with LoCA matching or exceeding prior PEFT methods while using fewer trainable parameters than convolutional LoRA.

Core claim

The central claim is that decoupling a convolutional weight update into separate channel-mixing and spatial-basis-refinement components preserves the spatial inductive bias of pre-trained convolutional kernels, and that this preservation is what enables better downstream adaptation than naive flattened-kernel low-rank methods. The mechanism carrying this claim is the SVD-based spatial basis: by computing the spatial covariance of the pre-trained kernel, extracting its principal spatial components as a learnable basis, and restricting spatial adaptation to depthwise diagonal coefficients, the method maintains the kernel's inherent spatial structure while still allowing both channel and space-

What carries the argument

The core mechanism is a two-path reparameterization of the convolutional weight update. Path one is a standard low-rank channel update (B_c times A_c) operating in flattened kernel space but reshaped back to 4D, handling cross-channel mixing. Path two is a spatial basis expansion: the pre-trained kernel slices are standardized and their spatial covariance matrix is decomposed via SVD to produce a set of principal spatial basis patterns S, which become learnable parameters; depthwise diagonal coefficients U_dw control how each channel combines these spatial patterns. The final update adds the channel mixing tensor to a depthwise-diagonal spatial tensor. A hierarchical rank scheduler scales r^

Load-bearing premise

The method assumes that the principal spatial patterns extracted from pre-trained convolutional kernels—via SVD of the spatial covariance matrix—constitute a useful, compact basis for downstream spatial adaptation. If the spatial structure most relevant to a new task is not well-captured by the pre-trained kernel's dominant spatial components, or if the fixed spatial rank (kernel height times width) is too small or too large for the task, the spatial basis refinement path may

What would settle it

A direct test would compare LoCA against a variant that uses the same channel mixing path and hierarchical rank scheduling but replaces the SVD-based spatial basis with either (a) random spatial bases, (b) no spatial path at all, or (c) a joint low-rank update in flattened space with matching parameter count. If such ablations match LoCA's performance, the spatial basis refinement is not the load-bearing component. The paper's own ablation (Table E) shows the spatial basis adds 0.87 mIoU over channel mixing alone in DGSS, but this margin is modest and task-specific.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If spatial-channel decoupling is the key, then any PEFT method applied to convolutional or mixed architectures should explicitly preserve kernel geometry rather than flattening it, potentially shifting how adapter modules are designed across vision and multimodal models.
  • The SVD-based spatial basis approach could extend to other structured weight tensors beyond convolutions—such as grouped linear layers, mixture-of-experts routing weights, or multi-axis attention—where flattening similarly destroys structural priors.
  • Hierarchical rank scheduling tied to stage width suggests that uniform rank allocation is suboptimal for any hierarchical backbone, which may influence PEFT design for transformer hierarchies as well as convolutional ones.
  • The finding that singular value spectra evolve more smoothly under decoupled adaptation (Figure G) provides a diagnostic tool: one could monitor SV dynamics during training as an early indicator of whether a PEFT method is preserving or collapsing representational diversity.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper acknowledges that the fixed spatial basis (set to kh*kw) constrains representation capacity for 1x1 convolutions, which dominate architectures like ConvNeXt. If 1x1 kernels are the majority of adapted layers, the spatial path may contribute little for those layers, and the gains may come primarily from the channel mixing path plus the hierarchical rank scheduler rather than from spatial
  • The covariance-SVD initialization is evaluated against alternatives (Table F) with modest margins (0.1-0.3 points). This raises the question of whether the specific SVD initialization matters or whether any structured initialization preserving spatial topology would yield similar benefits.
  • If the spatial basis is frozen (learning rate multiplier 0.0 as reported in the DGSS setup), then the spatial path reduces to learning diagonal coefficients on fixed bases—essentially a depthwise scaling—which resembles existing methods more than the fully learnable basis described in the general formulation.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes Low-Rank Convolutional Adaptation (LoCA), a Parameter-Efficient Fine-Tuning (PEFT) method designed specifically for convolutional layers in Vision Foundation Models (VFMs). Standard LoRA flattens 4D convolutional kernels into 2D matrices, which entangles spatial and channel information. To resolve this, LoCA decouples the adaptation into two paths: (1) a low-rank channel adaptation for dense cross-channel mixing, and (2) an SVD-based spatial basis refinement that operates on the depthwise diagonal. The method also introduces a hierarchical rank scheduling strategy that allocates adaptation capacity based on the stage-specific channel width. The authors evaluate LoCA on fine-grained classification (VTAB-1k, FGVC), domain-generalized semantic segmentation (DGSS), and subject-driven text-to-image generation (DreamBooth), demonstrating competitive or state-of-the-art performance across various convolutional backbones such as ConvNeXt, ResNet, and MambaVision.

Significance. Adapting convolutional layers efficiently is a relevant problem, as convolutions remain fundamental in modern hybrid VFM backbones. The paper provides a clear mathematical formulation for decoupling channel and spatial adaptation and provides reproducible code. The empirical evaluation is broad, spanning classification, segmentation, and generation. The mechanistic analysis of the Effective Receptive Field (ERF) and singular value evolution provides useful insights into how structural preservation aids adaptation. However, the central mechanistic claim regarding the benefits of spatial-channel decoupling is somewhat undermined by confounding factors in the ablation studies, which prevents the significance from being fully realized without further clarification.

major comments (3)
  1. The ablation in Table E does not cleanly isolate the contribution of the decoupled design from module placement and initialization. The 'Channel Mixing' variant (Eq. 4) is mathematically identical to naive flattened convolutional LoRA (Eq. 2), as both use B ∈ R^{C_out×r} and A ∈ R^{r×(C_in·k_h·k_w)}. However, Table E shows 'Channel Mixing' achieving +1.21 over 'LoRA Convolution' (64.95 → 66.16). Since the formulations are the same, this gain must stem from confounded factors, likely module placement (as the paper mentions following Conv-Adapter [4] placement strategies) or initialization. The authors must clarify exactly what differs between the 'LoRA Convolution' and 'Channel Mixing' rows in Table E to validate the claim that the decoupled design itself drives the improvement.
  2. The paper claims that the SVD-based spatial basis is crucial for preserving pre-trained spatial priors. However, Table F shows that the 'Covariance SVD' initialization yields only marginal gains over simpler initializations (e.g., 66.5 vs 66.2-66.3 on DGSS). Furthermore, the spatial basis term (Eq. 5-7) essentially acts as a structured depthwise adaptation. The paper does not directly compare the SVD-based spatial basis against a simpler depthwise LoRA baseline (without the SVD initialization). To support the claim that the SVD-based structure is load-bearing, the authors should provide a direct comparison against a depthwise LoRA variant with standard initialization.
  3. In Section 4.2, the spatial rank is fixed to r_sp = k_h * k_w. The paper notes in Section H (Limitations) that this fixed spatial basis design constrains the representation capacity for 1x1 kernels, which are prevalent in modern architectures like ConvNeXt. Given that the primary experiments heavily feature ConvNeXt, this limitation seems significant. The authors should discuss how this constraint impacts the reported results and whether the gains are primarily driven by the channel path rather than the spatial path for 1x1 convolutions.
minor comments (4)
  1. In Table A, the formatting of the ResNet-50 LoRA row appears to have spacing issues (e.g., '65.06 82.5356.2165.01'). This should be corrected for readability.
  2. Figure labels (e.g., Fig. A, Fig. B, Fig. C) are used in the main text but appear to be placeholders. Ensure that figures are numbered sequentially as per standard journal formatting (Fig. 1, Fig. 2, etc.).
  3. In Section 5.1, the paper mentions using 'sequential CA insertion for ConvNeXt and parallel k×k CA adaptation for ResNet'. It would be helpful to briefly define what 'CA insertion' refers to in this context, as it is not a standard term and relies heavily on the Conv-Adapter [4] reference.
  4. The reference to 'DINOv3-ConvNeXt [38]' in Section 5.3 appears to cite a 2025 arXiv preprint. Ensure that all citations are complete and up-to-date.

Circularity Check

0 steps flagged

No circularity found; derivation is self-contained

full rationale

The paper's derivation chain is self-contained and does not reduce to its inputs by construction. The SVD-based spatial basis (Sec. 4.2) is derived from the pre-trained kernel's spatial covariance matrix C_sp = W_norm^T W_norm, not from downstream task data, so the 'preservation of spatial priors' claim is not circularly forced by fitting to target benchmarks. The channel adaptation path (Eq. 4) is indeed mathematically identical in form to naive flattened LoRA (Eq. 2), as the skeptic notes, but the paper does not claim Eq. 4 alone is the novelty — the contribution is the composition (Eq. 7-8) that adds the depthwise spatial basis term. The ablation confound (Table E: 'Channel Mixing' outperforming 'LoRA Convolution' despite identical math) is a legitimate experimental design concern, but it belongs to correctness risk, not circularity: the paper is not defining a quantity in terms of the thing it claims to predict. No self-citation chain is load-bearing: SoMA [47] (Yun et al.), FSF [5] (Chen et al.), and Conv-Adapter [4] (Chen et al.) have no author overlap with the present paper. The hierarchical rank scheduling (Sec. 4.4) is a heuristic allocation strategy, not a fitted-then-predicted result. Initialization strategy comparison (Table F) evaluates alternatives against external benchmarks. No step in the derivation chain reduces to its own inputs by definition, fit, or self-citation.

Axiom & Free-Parameter Ledger

4 free parameters · 3 axioms · 0 invented entities

The method relies on standard linear algebra (SVD) and established PEFT assumptions. No new physical entities or forces are introduced. The free parameters are standard hyperparameters for low-rank adaptation methods.

free parameters (4)
  • Channel rank (r_ch) = 16 (default)
    The rank of the low-rank channel adaptation matrices B_c and A_c. Selected empirically.
  • Scaling factor (alpha) = 16.0 (default)
    Constant scaling factor for the channel adaptation term.
  • Global rank budget (R)
    Used in hierarchical rank scheduling to determine stage-specific channel ranks.
  • Learning rate multipliers = [0.5, 0.1, 0.0]
    Applied to backbone, u_dw, and s_basis respectively (Appendix D.1).
axioms (3)
  • domain assumption Weight changes during model adaptation possess a low intrinsic rank.
    Inherited from LoRA [21], foundational to the PEFT approach.
  • domain assumption Pre-trained convolutional kernels contain reusable spatial structure that can be captured by SVD of their spatial covariance.
    Justifies the SVD-based spatial basis refinement in Sec. 4.2.
  • domain assumption Convolutional architectures encode coarse-to-fine information across stages, requiring stage-dependent adaptation capacity.
    Motivates the hierarchical rank scheduling in Sec. 4.4.

pith-pipeline@v1.1.0-glm · 28769 in / 2174 out tokens · 233872 ms · 2026-07-09T22:55:39.162255+00:00 · methodology

0 comments
read the original abstract

Pre-trained Vision Foundation Models (VFMs) provide strong visual representations for diverse downstream tasks. The key challenge of VFM adaptation stems from the prohibitive costs of full fine-tuning and catastrophic forgetting. To address this, Low-Rank Adaptation (LoRA) has emerged as the prevailing paradigm for Parameter-Efficient Fine-Tuning (PEFT). However, LoRA is typically designed for transformer self-attention layers parameterized by 2D matrices. Since convolutional kernels inherently couple spatial and channel information within a 4D tensor, forcing them into a monolithic 2D matrix disrupts the inherent spatial topology. In this paper, we propose Low-Rank Convolutional Adaptation (LoCA), a convolution-aware PEFT framework that addresses spatial-channel entanglement by decoupling channel and spatial adaptation. LoCA introduces a low-rank channel adaptation for dense cross-channel mixing and refines spatial bases extracted from pre-trained kernels via Singular Value Decomposition (SVD). Experimental results show that LoCA preserves pre-trained spatial priors and achieves competitive or state-of-the-art performance across fine-grained classification, domain-generalized semantic segmentation, and generative benchmarks.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

53 extracted references · 53 canonical work pages · 4 internal anchors

  1. [1]

    Transactions on Machine Learning Research (2024),https://openreview.net/forum?id=aloEru2qCG, featured Certification

    Biderman, D., Portes, J., Ortiz, J.J.G., Paul, M., Greengard, P., Jennings, C., King, D., Havens, S., Chiley, V., Frankle, J., Blakeney, C., Cunningham, J.P.: LoRA learns less and forgets less. Transactions on Machine Learning Research (2024),https://openreview.net/forum?id=aloEru2qCG, featured Certification

    work page 2024

  2. [2]

    In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., Lin, H

    Cai, H., Gan, C., Zhu, L., Han, S.: Tinytl: Reduce memory, not parameters for efficient on-device learning. In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., Lin, H. (eds.) Advances in Neural Information Processing Systems. vol. 33, pp. 11285–11297. Curran Associates, Inc. (2020),https://proceedings.neurips.cc/ paper_files/paper/2020/file/81f7acab...

    work page 2020

  3. [3]

    In: Proceedings of the IEEE/CVF international conference on computer vision

    Caron, M., Touvron, H., Misra, I., Jégou, H., Mairal, J., Bojanowski, P., Joulin, A.: Emerging properties in self-supervised vision transformers. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 9650–9660 (2021)

    work page 2021

  4. [4]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops

    Chen, H., Tao, R., Zhang, H., Wang, Y., Li, X., Ye, W., Wang, J., Hu, G., Savvides, M.: Conv-adapter: Exploring parameter efficient transfer learning for convnets. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops. pp. 1551–1561 (June 2024)

    work page 2024

  5. [5]

    In: The Thirteenth International Conference on Learning Representations (2025), https://openreview.net/forum?id=E5YmIBvOqV

    Chen, W., Miao, Z., Qiu, Q.: Large convolutional model tuning via filter subspace. In: The Thirteenth International Conference on Learning Representations (2025), https://openreview.net/forum?id=E5YmIBvOqV

    work page 2025

  6. [6]

    In: The Eleventh International Conference on Learn- ing Representations (2023),https://openreview.net/forum?id=plKu2GByCNW

    Chen, Z., Duan, Y., Wang, W., He, J., Lu, T., Dai, J., Qiao, Y.: Vision transformer adapter for dense predictions. In: The Eleventh International Conference on Learn- ing Representations (2023),https://openreview.net/forum?id=plKu2GByCNW

    work page 2023

  7. [7]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Cheng, B., Misra, I., Schwing, A.G., Kirillov, A., Girdhar, R.: Masked-attention mask transformer for universal image segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 1290–1299 (June 2022)

    work page 2022

  8. [8]

    Contributors, M.: MMSegmentation: Openmmlab semantic segmentation toolbox and benchmark.https://github.com/open-mmlab/mmsegmentation(2020)

    work page 2020

  9. [9]

    In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (June 2016)

    Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., Schiele, B.: The cityscapes dataset for semantic urban scene understanding. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (June 2016)

    work page 2016

  10. [10]

    LoRA-C: Parameter-Efficient Fine-Tuning of Robust CNN for IoT Devices

    Ding, C., Cao, X., Xie, J., Fan, L., Wang, S., Lu, Z.: Lora-c: Parameter-efficient fine-tuning of robust cnn for iot devices. arXiv preprint arXiv:2410.16954 (2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  11. [11]

    In: Bouamor, H., Pino, J., Bali, K

    Ding, N., Lv, X., Wang, Q., Chen, Y., Zhou, B., Liu, Z., Sun, M.: Sparse low- rank adaptation of pre-trained language models. In: Bouamor, H., Pino, J., Bali, K. (eds.) Proceedings of the 2023 Conference on Empirical Methods in Natu- ral Language Processing. pp. 4133–4145. Association for Computational Linguis- tics, Singapore (Dec 2023).https://doi.org/1...

    work page doi:10.18653/v1/2023.emnlp-main.252 2023

  12. [12]

    In: In- ternational Conference on Learning Representations (2021),https://openreview

    Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., Houlsby, N.: An image is worth 16x16 words: Transformers for image recognition at scale. In: In- ternational Conference on Learning Representations (2021),https://openreview. net/forum?id=YicbFdNTTy

    work page 2021

  13. [13]

    Gal, R., Alaluf, Y., Atzmon, Y., Patashnik, O., Bermano, A.H., Chechik, G., Cohen-Or, D.: An image is worth one word: Personalizing text-to-image genera- tion using textual inversion (2022).https://doi.org/10.48550/ARXIV.2208.01618, https://arxiv.org/abs/2208.01618 LoCA 17

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2208.01618 2022

  14. [14]

    In: First Conference on Language Modeling (2024),https://openreview

    Gu, A., Dao, T.: Mamba: Linear-time sequence modeling with selective state spaces. In: First Conference on Language Modeling (2024),https://openreview. net/forum?id=tEYskw1VY2

    work page 2024

  15. [15]

    Guo, D., Rush, A.M., Kim, Y.: Parameter-efficient transfer learning with diff prun- ing. In: Proceedings of the 59th annual meeting of the association for computa- tional linguistics and the 11th international joint conference on natural language processing (volume 1: Long papers). pp. 4884–4896 (2021)

    work page 2021

  16. [16]

    Transactions on Machine Learning Research (2024),https://openreview.net/forum?id=lIsCS8b6zj

    Han, Z., Gao, C., Liu, J., Zhang, J., Zhang, S.Q.: Parameter-efficient fine-tuning for large models: A comprehensive survey. Transactions on Machine Learning Research (2024),https://openreview.net/forum?id=lIsCS8b6zj

    work page 2024

  17. [17]

    In: Proceedings of the Computer Vision and Pattern Recognition Con- ference

    Hatamizadeh, A., Kautz, J.: Mambavision: A hybrid mamba-transformer vision backbone. In: Proceedings of the Computer Vision and Pattern Recognition Con- ference. pp. 25261–25270 (2025)

    work page 2025

  18. [18]

    In: Proceedings of the computer vision and pattern recognition conference

    He, H., Zhang, J., Cai, Y., Chen, H., Hu, X., Gan, Z., Wang, Y., Wang, C., Wu, Y., Xie, L.: Mobilemamba: Lightweight multi-receptive visual mamba network. In: Proceedings of the computer vision and pattern recognition conference. pp. 4497– 4507 (2025)

    work page 2025

  19. [19]

    He,K.,Zhang,X.,Ren,S.,Sun,J.:Deepresiduallearningforimagerecognition.In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (June 2016)

    work page 2016

  20. [20]

    In: Proceedings of the 36th International Conference on Machine Learning (2019)

    Houlsby, N., Giurgiu, A., Jastrzebski, S., Morrone, B., De Laroussilhe, Q., Ges- mundo, A., Attariyan, M., Gelly, S.: Parameter-efficient transfer learning for NLP. In: Proceedings of the 36th International Conference on Machine Learning (2019)

    work page 2019

  21. [21]

    In: International Con- ference on Learning Representations (2022),https://openreview.net/forum?id= nZeVKeeFYf9

    Hu, E.J., yelong shen, Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.: LoRA: Low-rank adaptation of large language models. In: International Con- ference on Learning Representations (2022),https://openreview.net/forum?id= nZeVKeeFYf9

    work page 2022

  22. [22]

    In: Computer Vision – ECCV 2022

    Jia, M., Tang, L., Chen, B.C., Cardie, C., Belongie, S., Hariharan, B., Lim, S.N.: Visual prompt tuning. In: Computer Vision – ECCV 2022. pp. 709–727. Springer, Springer Nature Switzerland, Cham (2022)

    work page 2022

  23. [23]

    In: First Workshop on Fine-Grained Visual Categorization, IEEE Conference on Computer Vision and Pattern Recognition

    Khosla, A., Jayadevaprakash, N., Yao, B., Fei-Fei, L.: Novel dataset for fine-grained image categorization. In: First Workshop on Fine-Grained Visual Categorization, IEEE Conference on Computer Vision and Pattern Recognition. Colorado Springs, CO (June 2011)

    work page 2011

  24. [24]

    In: Computer Vision – ECCV 2022

    Kim, D., Wang, K., Sclaroff, S., Saenko, K.: A broad study of pre-training for domain generalization and adaptation. In: Computer Vision – ECCV 2022. pp. 621–638. Springer, Springer Nature Switzerland, Cham (2022)

    work page 2022

  25. [25]

    In: Proceedings of the IEEE International Conference on Computer Vision (ICCV) Workshops (June 2013)

    Krause, J., Stark, M., Deng, J., Fei-Fei, L.: 3d object representations for fine- grained categorization. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV) Workshops (June 2013)

    work page 2013

  26. [26]

    In: Zong, C., Xia, F., Li, W., Navigli, R

    Li, X.L., Liang, P.: Prefix-tuning: Optimizing continuous prompts for generation. In: Zong, C., Xia, F., Li, W., Navigli, R. (eds.) Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th Interna- tional Joint Conference on Natural Language Processing (Volume 1: Long Papers). pp. 4582–4597. Association for Comp...

    work page doi:10.18653/v1/2021.acl-long.353 2021

  27. [27]

    In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T

    Li, Y., Mao, H., Girshick, R., He, K.: Exploring plain vision transformer back- bones for object detection. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds.) Computer Vision – ECCV 2022. pp. 280–296. Springer Nature Switzerland, Cham (2022) 18 S. An et al

    work page 2022

  28. [28]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Liu, Z., Mao, H., Wu, C.Y., Feichtenhofer, C., Darrell, T., Xie, S.: A convnet for the 2020s. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 11976–11986 (June 2022)

    work page 2022

  29. [29]

    In: International Conference on Learning Representations (2019),https://openreview.net/forum? id=Bkg6RiCqY7

    Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. In: International Conference on Learning Representations (2019),https://openreview.net/forum? id=Bkg6RiCqY7

    work page 2019

  30. [30]

    In: Lee, D., Sugiyama, M., Luxburg, U., Guyon, I., Garnett, R

    Luo, W., Li, Y., Urtasun, R., Zemel, R.: Understanding the effective receptive field in deep convolutional neural networks. In: Lee, D., Sugiyama, M., Luxburg, U., Guyon, I., Garnett, R. (eds.) Advances in Neural Information Processing Systems. vol. 29. Curran Associates, Inc. (2016),https://proceedings.neurips.cc/paper_ files/paper/2016/file/c8067ad1937f...

    work page 2016

  31. [31]

    In: Globerson, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J., Zhang, C

    Meng, F., Wang, Z., Zhang, M.: Pissa: Principal singular values and singular vec- tors adaptation of large language models. In: Globerson, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J., Zhang, C. (eds.) Advances in Neural In- formation Processing Systems. vol. 37, pp. 121038–121072. Curran Associates, Inc. (2024).https://doi.org/10.52202/...

    work page doi:10.52202/079017- 2024

  32. [32]

    In: Proceedings of the IEEE International Conference on Computer Vision (ICCV) (Oct 2017)

    Neuhold, G., Ollmann, T., Rota Bulo, S., Kontschieder, P.: The mapillary vistas dataset for semantic understanding of street scenes. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV) (Oct 2017)

    work page 2017

  33. [33]

    PEFT A2Z: Parameter-Efficient Fine-Tuning Survey for Large Language and Vision Models

    Prottasha, N.J., Chowdhury, U.R., Mohanto, S., Nuzhat, T., Sami, A.A., Ali, M.S., Sobuj, M.S.I., Raman, H., Kowsher, M., Garibay, O.O.: Peft a2z: parameter- efficient fine-tuning survey for large language and vision models. arXiv preprint arXiv:2504.14117 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  34. [34]

    In: Meila, M., Zhang, T

    Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., Sutskever, I.: Learning transferable visual models from natural language supervision. In: Meila, M., Zhang, T. (eds.) Proceedings of the 38th International Conference on Machine Learning. Proceed- ings of Machine Learning Res...

    work page 2021

  35. [35]

    In: The Thirty-ninth Annual Conference on Neural Information Processing Systems (2026),https:// openreview.net/forum?id=3pF7rt9fQM

    Ran, W., Zhang, W., Pang, S., Zhu, Q., Liu, J., Liu, J., Cao, X., Li, Q., Yan, Y., Ma, C.: Correlated low-rank adaptation for convnets. In: The Thirty-ninth Annual Conference on Neural Information Processing Systems (2026),https:// openreview.net/forum?id=3pF7rt9fQM

    work page 2026

  36. [36]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 10684– 10695 (June 2022)

    work page 2022

  37. [37]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Ruiz, N., Li, Y., Jampani, V., Pritch, Y., Rubinstein, M., Aberman, K.: Dream- booth: Fine tuning text-to-image diffusion models for subject-driven generation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 22500–22510 (June 2023)

    work page 2023

  38. [38]

    Siméoni, O., Vo, H.V., Seitzer, M., Baldassarre, F., Oquab, M., Jose, C., Khalidov, V., Szafraniec, M., Yi, S., Ramamonjisoa, M., Massa, F., Haziza, D., Wehrstedt, L., Wang, J., Darcet, T., Moutakanni, T., Sentana, L., Roberts, C., Vedaldi, A., Tolan, J., Brandt, J., Couprie, C., Mairal, J., Jégou, H., Labatut, P., Bojanowski, P.: DINOv3 (2025),https://ar...

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  39. [39]

    In: Proceedings LoCA 19 of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (June 2015)

    Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Proceedings LoCA 19 of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (June 2015)

    work page 2015

  40. [40]

    In: Proceedings of the IEEE conference on computer vision and pattern recognition

    Van Horn, G., Branson, S., Farrell, R., Haber, S., Barry, J., Ipeirotis, P., Perona, P., Belongie, S.: Building a bird recognition app and large scale dataset with citizen scientists: The fine print in fine-grained dataset collection. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 595–604 (2015)

    work page 2015

  41. [41]

    Wah, C., Branson, S., Welinder, P., Perona, P., Belongie, S.: The caltech-ucsd birds-200-2011 dataset. Tech. rep., California Institute of Technology (2011)

    work page 2011

  42. [42]

    In: CVPR (2024)

    Wei, Z., Chen, L., Jin, Y., Ma, X., Liu, T., Ling, P., Wang, B., Chen, H., Zheng, J.: Stronger fewer & superior: Harnessing vision foundation models for domain generalized semantic segmentation. In: CVPR (2024)

    work page 2024

  43. [43]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Woo, S., Debnath, S., Hu, R., Chen, X., Liu, Z., Kweon, I.S., Xie, S.: Convnext v2: Co-designing and scaling convnets with masked autoencoders. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 16133–16142 (June 2023)

    work page 2023

  44. [44]

    IEEE Transactions on Pattern Analysis and Machine Intelligence pp

    Xu, L., Xie, H., Qin, S.J., Tao, X., Wang, F.L.: Parameter-efficient fine-tuning methods for pretrained language models: A critical review and assessment. IEEE Transactions on Pattern Analysis and Machine Intelligence pp. 1–20 (2026).https: //doi.org/10.1109/TPAMI.2026.3657354

    work page doi:10.1109/tpami.2026.3657354 2026

  45. [45]

    (eds.) Advances in Neural Information Processing Systems

    Yosinski, J., Clune, J., Bengio, Y., Lipson, H.: How transferable are features in deep neural networks? In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N., Weinberger, K. (eds.) Advances in Neural Information Processing Systems. vol. 27. Curran Associates, Inc. (2014),https://proceedings.neurips.cc/paper_ files/paper/2014/file/532a2f85b6977104bc93f...

    work page 2014

  46. [46]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (June 2020)

    Yu, F., Chen, H., Wang, X., Xian, W., Chen, Y., Liu, F., Madhavan, V., Dar- rell, T.: Bdd100k: A diverse driving dataset for heterogeneous multitask learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (June 2020)

    work page 2020

  47. [47]

    In: Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Yun, S., Chae, S., Lee, D., Ro, Y.: Soma: Singular value decomposed minor com- ponents adaptation for domain generalizable representation learning. In: Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 25602–25612 (June 2025)

    work page 2025

  48. [48]

    In: High-dimensional Learning Dynamics 2024: The Emergence of Structure and Reasoning (2024),https://openreview.net/forum? id=u3sssLLu4y

    Yunis, D., Patel, K.K., Wheeler, S., Savarese, P.H.P., Vardi, G., Frankle, J., Livescu, K., Maire, M., Walter, M.: Rank minimization, alignment and weight decay in neural networks. In: High-dimensional Learning Dynamics 2024: The Emergence of Structure and Reasoning (2024),https://openreview.net/forum? id=u3sssLLu4y

    work page 2024

  49. [49]

    In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T

    Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) Computer Vision – ECCV

    work page

  50. [50]

    pp. 818–833. Springer International Publishing, Cham (2014)

    work page 2014

  51. [51]

    In: The Eleventh Interna- tional Conference on Learning Representations (2023),https://openreview.net/ forum?id=lq62uWRJjiY

    Zhang, Q., Chen, M., Bukharin, A., He, P., Cheng, Y., Chen, W., Zhao, T.: Adap- tive budget allocation for parameter-efficient fine-tuning. In: The Eleventh Interna- tional Conference on Learning Representations (2023),https://openreview.net/ forum?id=lq62uWRJjiY

    work page 2023

  52. [52]

    IEEE Transactions on Pattern Analysis and Machine Intelligence47(7), 5268–5280 (2025).https://doi.org/10

    Zhang, Y., Zhou, K., Liu, Z.: Neural prompt search. IEEE Transactions on Pattern Analysis and Machine Intelligence47(7), 5268–5280 (2025).https://doi.org/10. 1109/TPAMI.2024.3435939

    work page arXiv 2025

  53. [53]

    a photo of [V] [C]

    Zhong, Z., Tang, Z., He, T., Fang, H., Yuan, C.: Convolution meets loRA: Param- eter efficient finetuning for segment anything model. In: The Twelfth International Conference on Learning Representations (2024),https://openreview.net/forum? id=ezscMer8L0 Supplementary Material LoCA: Spatially-Aware Low-Rank Convolutional Adaptation of Vision Foundation Mod...

    work page arXiv 2024