TR-PTS: Task-Relevant Parameter and Token Selection for Efficient Tuning

Guangtao Zhai; Guangyang Wu; Haoran Yang; Mingyang Yi; Siqi Luo; Xiaohong Liu; Yi Xin

REVIEW 4 major objections 7 minor 39 references

TR-PTS: Task-Relevant Parameter and Token Selection for Efficient Tuning

T0 review · 4 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A unified task-aware scheme that selects which parameters to unfreeze and which image tokens to keep can beat full fine-tuning of a Vision Transformer while updating less than one percent of its weights.

desk verdict A competent, incremental PEFT paper whose joint parameter-and-token selection is plausible but whose central sparse-insertion claim rests on thin evidence; referee it, but ask for statistical rigor and full hyperparameter disclosure. read the letter →

arxiv 2507.22872 v1 pith:W7LSUOHM submitted 2025-07-30 cs.CV

Siqi Luo , Haoran Yang , Yi Xin , Mingyang Yi , Guangyang Wu , Guangtao Zhai , Xiaohong Liu This is my paper

classification cs.CV

keywords TR-PTSparameter-efficientfine-tuningFisherinformationmatrixtokenselectionmergingVisionTransformerVTAB-1kFGVC

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

TR-PTS is a method for adapting a pre-trained Vision Transformer to a new visual task that decides, before training, which parameters to unfreeze and, during training, which image tokens to keep. The parameter side ranks weights by their Fisher information, approximated as the squared gradient of the loss, and allocates trainable connections per layer in proportion to task relevance. The token side uses the [CLS] token's attention scores to keep the most informative patches and merges the rest into one weighted token. The two selections are coordinated: token reduction is applied mostly in layers that were given fewer trainable parameters, based on the paper's observation that such layers also carry less informative tokens. On the FGVC and VTAB-1k benchmarks, the method reports average accuracies that exceed full fine-tuning by 3.40 and 10.35 percentage points while updating only 0.60% and 0.34% of the pre-trained model's parameters.

What carries the argument

The central machinery is a Fisher-information-guided binary mask on the weights, paired with a [CLS]-attention token selector and merger. The Fisher Information Matrix is approximated by the squared gradient of the cross-entropy loss; its diagonal gives each parameter a task-relevance score. Layer importance is computed as the share of the top-M% parameters that fall in each layer, and each neuron in a layer is assigned $C_l$ trainable connections via $C_l = \max(1, w_l/\min(w) \cdot C_{\min})$. Token selection uses the attention score $a_i = \frac{\exp(q_{\mathrm{CLS}}\cdot k_i)}{\sum_j \exp(q_{\mathrm{CLS}}\cdot k_j)}$ for each patch token, keeping the top $\lfloor \rho N \rfloor$ tokens and fusing the rest into $x_{\mathrm{merged}} = \sum_{i\in I} a_i x_i / \sum_{i\in I} a_i$. The two are joined by a "sparse insertion" rule: token reduction is applied at layers with few trainable connections, which the paper's Table 5 reports is more accurate than applying it to dense or randomly chosen layers.

What would settle it

A concrete test would be to run TR-PTS with token reduction placed in dense, sparse, and random layers across many tasks and look for a task where sparse placement underperforms dense or random; the paper's Table 5 shows only four datasets. A second test would measure the sparsity-redundancy correlation directly by computing layer-wise Fisher scores and token-importance distributions for a new task and checking whether they are actually negatively correlated before applying the method.

Watch

Extended reading notes

Core claim

The paper claims that parameter and token selection for fine-tuning should be task-driven and coupled, not fixed and independent. It introduces a layer-wise Fisher Information Matrix score to find the connections most sensitive to the target task, guarantees every layer keeps at least one active connection per neuron, and scales the number of trainable connections per layer by its task importance. In parallel, it scores image tokens by their attention weight from the [CLS] token, keeps the top fraction, and merges the discarded tokens into a single weighted token so no information is wholly lost. The coordinated rule is that token reduction happens preferentially in parameter-sparse layers, where the paper argues tokens are less informative. With both mechanisms active, TR-PTS reports average Top-1 accuracy of 91.94% on FGVC and 75.92% on VTAB-1k, beating full fine-tuning, GPS, SSF, LoRA, and other baselines while adding no extra parameters at inference time.

Load-bearing premise

The joint strategy assumes that layers whose parameters look task-irrelevant under the Fisher score also carry tokens that are safe to drop, so applying token reduction to sparse layers preserves accuracy; if that correlation fails for a given task, informative tokens could be discarded and the accuracy gains would shrink.

Editorial extensions

If this is right

Fine-tuning a ViT-B/16 with 0.34% of its parameters and a shortened token sequence can exceed full fine-tuning by 10.35 mean Top-1 points on the 19 tasks of VTAB-1k.
On the five FGVC tasks, the same recipe exceeds full fine-tuning by 3.40 points while updating 0.60% of parameters, with the largest per-task gains on Stanford Dogs and NABirds.
Because TR-PTS adds no modules and merges discarded tokens into a single weighted token, the deployed model carries no extra parameters and performs fewer FLOPs and less per-sample memory than the compared PEFT baselines.
Token reduction should be applied in parameter-sparse layers rather than dense ones; the paper's comparisons show that "dense" token selection can hurt accuracy while "sparse" selection improves it.
Task-driven parameter sets overlap little across tasks, so the Fisher-based allocation selects genuinely task-specific connections rather than a fixed sparse mask.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the layerwise sparsity-redundancy correlation holds beyond the four datasets shown in Table 5, the same coordination rule could be ported to other transformer families and modalities by setting each layer's token-reduction rate as a decreasing function of its parameter-importance score.
The Fisher mask is computed once at the start; a testable variant would recompute it after token selection starts changing the input distribution, since the most task-relevant parameters may shift as the token stream is refined.
The reported VTAB gain over full fine-tuning is not yet isolated from the token-selection effect: comparing full fine-tuning with the same token selection and merging would tell how much of the 10.35-point gain comes from parameter selection alone.
Cheaper importance signals such as gradient norms or attention entropy could replace the Fisher scores; the paper argues for Fisher over gradient magnitude but does not include those alternatives in its comparisons.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The headline is accurate: this is a solid, incremental PEFT paper. The new thing is the joint formulation – FIM-based layer-wise parameter selection (close to GPS) combined with attention-based token selection and merging (close to ToMe) – plus the sparse-insertion rule that applies token reduction preferentially to parameter-sparse layers. The writing is clear, the method is sensible, and the experiments cover 24 datasets with a sensible baseline set. There is a code link, which is good practice, and the ablations (Table 4) show both components help.

That said, the soft spots are real and need to be addressed before I'd trust the headline numbers. First, the claimed SOTA margins over GPS are thin: +0.74% on VTAB-1k and +0.16% on FGVC. There are no error bars, no multiple seeds, and no hyperparameter settings reported (M, C_min, rho, Fisher sample size, which layers get token selection). That makes it impossible to distinguish a real improvement from noise. Second, the sparse-insertion rule is load-bearing: the paper's central efficiency-accuracy win depends on the claim that layers with fewer task-relevant parameters also encode less informative tokens. The support for this is just Table 5, which covers 4 of the 24 datasets, reports differences of 0.1–1.0 accuracy points, and shows no variance or correlation coefficient. I think the stress-test note is correct to flag this as a weakly evidenced correlation. If it fails on other tasks, applying token selection at sparse layers could discard informative tokens, and the method's advantage could erode.

The reader's concern about circularity is minor, and I agree it doesn't force the central result. The placement rule was tuned on the benchmark, but accuracy is still measured on held-out test sets. That's a small burden, not a fatal one.

Overall: this is a competent contribution that deserves a serious referee, but the revision needs multi-seed results with variance, full hyperparameter reporting, and a more thorough validation of the sparse-insertion heuristic, ideally across a broader set of tasks or with a quantitative measure of the sparsity–redundancy correlation. I would not desk-reject it, but I would not accept it in this form either.

Referee Report

4 major / 7 minor

Summary. The paper proposes TR-PTS, a fine-tuning method for Vision Transformers that combines Fisher-Information-based task-relevant parameter selection (with layer-wise allocation) and attention-based task-relevant token selection/merging. The two mechanisms are coordinated by applying token reduction preferentially to layers with sparse task-relevant parameters, based on an empirical correlation claimed in Section 3.4. The method is evaluated on FGVC (5 datasets) and VTAB-1k (19 datasets) using ViT-B/16 pretrained on ImageNet-21k, reporting state-of-the-art mean accuracy (75.92 on VTAB-1k, 91.94 on FGVC) while tuning only 0.34% and 0.60% of parameters, respectively, and reducing FLOPs, memory, and inference time.

Significance. If the reported results hold, TR-PTS is a practically useful contribution: it shows that task-aware parameter and token selection can be combined to reduce both training and inference cost while matching or exceeding full fine-tuning accuracy on 24 datasets. The paper is clearly written, the method is simple and well motivated, and the code is promised to be released. The two mechanisms are individually reasonable, and the benchmark coverage is broad. However, the claimed state-of-the-art advantage over GPS rests on small average margins (0.74 and 0.16 percentage points) without error bars or multiple seeds, and the core design rule for joint selection (sparse-insertion) is supported by only a four-dataset comparison with no quantitative correlation measure. The efficiency numbers also depend on unreported hyperparameters. These issues currently prevent the central simultaneous efficiency-accuracy claim from being fully verified, though the methodology is sound in principle.

major comments (4)

[Section 4.2, Tables 2 and 3] The state-of-the-art claim over GPS rests on average margins of 0.74 percentage points on VTAB-1k and 0.16 percentage points on FGVC, with no error bars or multiple seeds. The FGVC margin in particular is smaller than typical seed-to-seed variation for this benchmark suite, so the improvement is not statistically supported as reported. Please report mean and standard deviation over at least three runs, or provide paired seed-matched comparisons against GPS, for the main tables.
[Sections 3.2-3.4 and 4.1] Key hyperparameters are not reported: M (top-Fisher percentage in Eq. (3)), C_min in Eq. (4), the token select rate ρ in Eq. (9), the number of samples used for the Fisher approximation in Eq. (2), and the specific layers at which token selection is applied. The main results in Tables 2 and 3 therefore cannot be reproduced or independently evaluated, and the accuracy-efficiency trade-off is not fully specified. Please add a hyperparameter table or state these values per benchmark.
[Section 3.4, Table 5] The sparse-insertion rule is the load-bearing design choice that unifies parameter and token selection, but its empirical support is thin. Table 5 covers only four datasets that come from the same evaluation benchmarks, reports no error bars, and shows no consistent advantage for Sparse placement: at ρ=0.8, Sparse is equal to Dense on Flower102 (99.1 vs 99.1) and worse than Dense on Patch Camelyon (87.2 vs 87.4). The differences are typically 0.1-1.0 points, within the expected noise. To support the claimed correlation between parameter sparsity and token redundancy, please provide a quantitative per-layer analysis (e.g., correlation between FIM-based parameter sparsity and attention entropy or token redundancy) over a wider set of tasks, or validate the sparse-insertion rule on held-out tasks not used to motivate it.
[Section 4.2, Figure 4] The computational cost claims (shortest inference time, lowest memory, lowest FLOPs) are difficult to verify because the paper does not state the token schedule used in the efficiency measurements: which layers perform token selection, what ρ is used, and what the resulting average token counts are. Inference time is also reported without variance despite being averaged over only 500 forward passes. Please specify the configuration behind Figure 4 so the reported efficiency numbers can be reproduced.

minor comments (7)

[Abstract] The abstract contains grammatical errors: 'benchmark' should be 'benchmarks', and 'The code are available' should be 'The code is available'.
[Section 1] In the FGVC results description, 'slightly surpassing GPS (91.78%) and and outperforming full fine-tuning' contains a duplicated 'and'.
[Section 3.3] The heading 'Task-Relevant T oken Selection' contains a stray space in 'Token'.
[Figure 5 caption] The caption reads 'Differernt Datasets Top 1% FIM Parameter Distribution'; 'Differernt' should be 'Different'.
[Section 3.2.2, Eq. (4)] The formula Cl = max(1, wl/min(w) * Cmin) is undefined if the least important layer has wl = 0, which would make min(w) = 0. Please clarify how zero-importance layers are handled, since the text states that every layer retains at least one active connection.
[Table 5] The terms 'Dense', 'Sparse', and 'Random' are not quantitatively defined. Please state the criterion used to classify layers as dense or sparse (e.g., top/bottom fraction by selected-connection density) and how 'Random' placement is sampled.
[Section 4.1] The sentence 'pre-train the model with both supervised method' should be rephrased, e.g., 'pre-trained with supervised methods' or 'supervised pretraining'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reported gains are empirical benchmark results, and the method's selection rules are stated approximations or ablation-validated design choices rather than definitions that force the measured outcomes.

full rationale

TR-PTS's parameter selection is defined by FIM scores (Eq. 1-2) and layer-wise allocation (Eq. 3-6); token selection is defined by CLS attention scores (Eq. 7-9) and weighted merging (Eq. 10-11). Neither quantity is defined in terms of the final accuracy or efficiency numbers it is said to achieve. The sparse-insertion rule in Section 3.4 is motivated by an empirical correlation reported in Table 5 and validated by ablations; this is a design choice supported by measurements, not a fitted parameter renamed as a prediction. The headline improvements (3.40% on FGVC, 10.35% on VTAB-1k) are test-set measurements, not outputs derived from the method's equations. Self-citations appear in related work and benchmark references ([8], [18], [19], [20], [24], [31]-[35]), but none carries a load-bearing uniqueness proof or derivation step; the FIM and attention-score foundations are external citations. No step in the paper reduces by construction to its own inputs, so the circularity score is 0.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The method rests on standard domain assumptions about Fisher information and attention scores, plus one ad hoc correlation between parameter sparsity and token redundancy that is validated on four tasks from the same benchmark. All key hyperparameters (M, C_min, rho, Fisher sample size) are free parameters not specified in the paper.

free parameters (5)

M (top-Fisher percentage) = not reported
Determines the pool of candidate parameters for layer allocation; value not given for main experiments.
C_min (minimum connections per neuron) = not reported
Controls per-layer allocation in Eq. 4; not specified.
rho (token select rate) = not reported for main experiments; 0.95/0.8 in ablation
Selects fraction of tokens kept (Eq. 9); main experiments' value not stated.
Fisher estimation sample size = not reported
Number of examples or batches used to estimate Eq. 2 is not given.
Sparse-insertion layer selection = not formalized
The rule for which layers receive token reduction is described qualitatively in Section 3.4, with no precise threshold.

assumptions (3)

domain assumption The diagonal Fisher information matrix, approximated by squared cross-entropy gradients, is a valid estimator of each parameter's task relevance.
Invoked in Section 3.2.1 (Eq. 2); this is a standard PEFT heuristic but remains an unproved modeling assumption.
domain assumption Attention scores of the [CLS] token reflect the task-relevance of image tokens.
Invoked in Section 3.3.1 (Eq. 8); used by prior token pruning work, but not proven.
ad hoc to paper Layers with sparse task-relevant parameters tend to encode less informative tokens.
Introduced in Section 3.4 and supported only by Table 5 on four datasets from the same benchmark used for evaluation.

how reviews work

0 comments

Cite this review

Pith. "Pith review of TR-PTS: Task-Relevant Parameter and Token Selection for Efficient Tuning." pith.science (2026). https://pith.science/paper/W7LSUOHM

@misc{pith2026250722872,
  author       = {Pith},
  title        = {Pith review of: TR-PTS: Task-Relevant Parameter and Token Selection for Efficient Tuning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/W7LSUOHM}},
  note         = {Machine review of arXiv:2507.22872}
}

read the original abstract

Large pre-trained models achieve remarkable performance in vision tasks but are impractical for fine-tuning due to high computational and storage costs. Parameter-Efficient Fine-Tuning (PEFT) methods mitigate this issue by updating only a subset of parameters; however, most existing approaches are task-agnostic, failing to fully exploit task-specific adaptations, which leads to suboptimal efficiency and performance. To address this limitation, we propose Task-Relevant Parameter and Token Selection (TR-PTS), a task-driven framework that enhances both computational efficiency and accuracy. Specifically, we introduce Task-Relevant Parameter Selection, which utilizes the Fisher Information Matrix (FIM) to identify and fine-tune only the most informative parameters in a layer-wise manner, while keeping the remaining parameters frozen. Simultaneously, Task-Relevant Token Selection dynamically preserves the most informative tokens and merges redundant ones, reducing computational overhead. By jointly optimizing parameters and tokens, TR-PTS enables the model to concentrate on task-discriminative information. We evaluate TR-PTS on benchmark, including FGVC and VTAB-1k, where it achieves state-of-the-art performance, surpassing full fine-tuning by 3.40% and 10.35%, respectively. The code are available at https://github.com/synbol/TR-PTS.

Figures

Figures reproduced from arXiv: 2507.22872 by the authors.

**Figure 1.** Overview of Our TR-PTS. TR-PTS selectively retains task-relevant tokens and parameters for efficient fine-tuning. However, with state-of-the-art models now comprising billions or even trillions of parameters [22, 27], this conventional approach has become increasingly impractical due to its exorbitant computational and storage requirements. Mitigating the inefficiencies of full fine-tuning has led researchers in … view at source ↗

**Figure 2.** Visualizing the Impact of Each Token on the Final Prediction. The label ”wrong” indicates an incorrect classification. These results demonstrate that the final prediction relies primarily on a subset of the most task-relevant tokens. approaches optimize parameter selection and token processing separately, although the informativeness of tokens depends heavily on the task, particularly in ViT-based models. As shown… view at source ↗

**Figure 3.** Architecture of Our Proposed TR-PTS Framework. Top: For Task-Relevant Parameter Selection, we use the Fisher Information Matrix to determine each layer’s importance and dynamically set the number of trainable connections per neuron (Nl); critical layers receive more parameters, while every layer updates at least one connection. Bottom: For Task-Relevant Token Selection, attention scores from the [CLS] token identify… view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: Comparison of Different Methods in terms of Test Time (ms), Test Memory Usage (GB), and FLOPs (G). The results show that the TR-PTS method achieves the lowest FLOPs and memory consumption while maintaining competitive test time. Method Dataset CUB-200 -2011 NABirds Oxf…

**Figure 5.** Figure 5: Differernt Datasets Top 1% FIM Parameter Distribution. results demonstrate that TR-PTS not only delivers competitive accuracy in a demanding fine-grained classification setting but also significantly enhances computational efficiency. Computational Cost. In [PITH_FU…

**Figure 6.** Figure 6: Visualization of Task-Relevant Tokens Selected by TRPTS Using ViT-B/16 with 12 Layers. As the number of layers increases, our method increasingly focuses on task-relevant tokens [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

**Figure 7.** Figure 7: Visualization of the Overlapping Rate Among TaskDriven Parameter Sets. This heatmap shows the overlap between parameter sets across tasks. Darker shades indicate higher similarity, while lighter shades highlight task-specific differences. parameters tend to encode le…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

39 extracted references · 31 canonical work pages

[1]

Token merging: Your vit but faster, 2023

Daniel Bolya, Cheng-Yang Fu, Xiaoliang Dai, Peizhao Zhang, Christoph Feichtenhofer, and Judy Hoffman. Token merging: Your vit but faster, 2023. 3

work page 2023
[2]

Language models are few-shot learners

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Sub- biah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. In Advances in Neural Informa- tion Processing Systems (NeurIPS), 2020. 1

work page 2020
[3]

Adaptformer: Adapting vision transformers for scalable visual recognition

Shoufa Chen, Chongjian Ge, Zhan Tong, Jiangliu Wang, Yib- ing Song, Jue Wang, and Ping Luo. Adaptformer: Adapting vision transformers for scalable visual recognition. Advances in Neural Information Processing Systems (NeurIPS) , 2022. 2, 5, 6

work page 2022
[4]

Novel datasets for fine-grained image categoriza- tion

E Dataset. Novel datasets for fine-grained image categoriza- tion. In Proceedings of the IEEE Conference on Computer Vi- sion and Pattern Recognition Workshops (CVPR Workshops),

work page
[5]

Bert: Pre-training of deep bidirectional transform- ers for language understanding

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transform- ers for language understanding. Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), 2018. 1

work page 2018
[6]

An image is worth 16x16 words: Transform- ers for image recognition at scale

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, et al. An image is worth 16x16 words: Transform- ers for image recognition at scale. In Proceedings of the In- ternational Conference on Learning Representations (ICLR) ,

work page
[7]

An image is worth 16x16 words: Transformers for image recognition at scale

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. Proceedings of the International Conference on Learn- ing Representati...

work page 2021
[8]

Multi-source fully test-time adaptation

Yuntao Du, Siqi Luo, Yi Xin, Mingcai Chen, Shuai Feng, Mu- jie Zhang, and Chonngjun Wang. Multi-source fully test-time adaptation. Neural Networks, 181:106661, 2025. 1

work page 2025

Show all 39 references

[9]

Adaptive token sampling for efficient vision transformers

Mohsen Fayyaz, Soroush Abbasi Koohpayegani, Farnoush Rezaei Jafari, Sunando Sengupta, Hamid Reza Vaezi Joze, Eric Sommerlade, Hamed Pirsiavash, and J¨urgen Gall. Adaptive token sampling for efficient vision transformers. In European Conference on Computer Vision , pages 396–41...

2022
[10]

Fine-grained car detection for visual census estimation

Timnit Gebru, Jonathan Krause, Yilun Wang, Duyun Chen, Jia Deng, and Li Fei-Fei. Fine-grained car detection for visual census estimation. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), 2017. 5

2017
[11]

Parameter-efficient transfer learning for nlp

Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for nlp. In Proceedings of the International Con- ference on Machine Learning (ICML), 2019. 1, 2, 5, 6

2019
[12]

Lora: Low-rank adaptation of large language models

Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. Proceedings of the International Conference on Learning Representations (ICLR), 2022. 2, 5, 6

2022
[13]

Lora: Low-rank adaptation of large language models

Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3,
[14]

Visual prompt tuning

Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim. Visual prompt tuning. In Proceedings of the European Conference on Computer Vision (ECCV), 2022. 1, 2, 5, 6

2022
[15]

Adam: A method for stochastic optimization

Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 ,

arXiv
[16]

Overcoming catastrophic forgetting in neural networks

James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of scien...

2017
[17]

Scaling & shifting your features: A new baseline for efficient model tuning

Dongze Lian, Daquan Zhou, Jiashi Feng, and Xinchao Wang. Scaling & shifting your features: A new baseline for efficient model tuning. In Advances in Neural Information Processing Systems (NeurIPS), 2022. 2, 5, 6

2022
[18]

Sparse-tuning: Adapting vision transformers with efficient fine-tuning and in- ference

Ting Liu, Xuyang Liu, Siteng Huang, Liangtao Shi, Zunnan Xu, Yi Xin, Quanjun Yin, and Xiaohong Liu. Sparse-tuning: Adapting vision transformers with efficient fine-tuning and in- ference. arXiv preprint arXiv:2405.14700, 2024. 1

2024
[19]

M2ist: Multi-modal interactive side-tuning for efficient refer- ring expression comprehension

Xuyang Liu, Ting Liu, Siteng Huang, Yi Xin, Yue Hu, Long Qin, Donglin Wang, Yuanyuan Wu, and Honggang Chen. M2ist: Multi-modal interactive side-tuning for efficient refer- ring expression comprehension. IEEE Transactions on Cir- cuits and Systems for Video Technology (TCSVT), 2025. 1

2025
[20]

Enhancing test time adaptation with few-shot guidance

Siqi Luo, Yi Xin, Yuntao Du, Zhongwei Wan, Tao Tan, Guangtao Zhai, and Xiaohong Liu. Enhancing test time adaptation with few-shot guidance. arXiv preprint arXiv:2409.01341, 2024. 1

2024
[21]

Automated flower classification over a large number of classes

Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In 2008 Sixth Indian conference on computer vision, graphics & image processing, 2008. 5

2008
[22]

Gpt-4 technical report

OpenAI. Gpt-4 technical report. arXiv preprint arXiv:2304.10592, 2023. 1

2023 arXiv
[23]

Pytorch: An imperative style, high-performance deep learning library

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Ad- vances in neural information processing systems, 3...

2019
[24]

Lumina- image 2.0: A unified and efficient image generative frame- work

Qi Qin, Le Zhuo, Yi Xin, Ruoyi Du, Zhen Li, Bin Fu, Yiting Lu, Jiakang Yuan, Xinyue Li, Dongyang Liu, et al. Lumina- image 2.0: A unified and efficient image generative frame- work. arXiv preprint arXiv:2503.21758, 2025. 1

2025 arXiv
[25]

Dynamicvit: Efficient vision trans- formers with dynamic token sparsification

Yongming Rao, Wenliang Zhao, Benlin Liu, Jiwen Lu, Jie Zhou, and Cho-Jui Hsieh. Dynamicvit: Efficient vision trans- formers with dynamic token sparsification. Advances in Neu- ral Information Processing Systems (NeurIPS), 2021. 3

2021
[26]

Imagenet-21k pretraining for the masses

Tal Ridnik, Emanuel Ben-Baruch, Asaf Noy, and Lihi Zelnik- Manor. Imagenet-21k pretraining for the masses. In Thirty- fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1), 2021. 5

2021
[27]

Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971, 2023

Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth ´ee Lacroix, Baptiste Rozi`ere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971, 2023. 1

2023 arXiv
[28]

Building a bird recognition app and large scale dataset with citizen scientists: The fine print in fine-grained dataset collection

Grant Van Horn, Steve Branson, Ryan Farrell, Scott Haber, Jessie Barry, Panos Ipeirotis, Pietro Perona, and Serge Be- longie. Building a bird recognition app and large scale dataset with citizen scientists: The fine print in fine-grained dataset collection. In Proceedings of t...

2015
[29]

The caltech-ucsd birds-200-2011 dataset

Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-ucsd birds-200-2011 dataset. California Institute of Technology, 2011. 5

2011
[30]

Zero- tprune: Zero-shot token pruning through leveraging of the at- tention graph in pre-trained transformers

Hongjie Wang, Bhishma Dedhia, and Niraj K Jha. Zero- tprune: Zero-shot token pruning through leveraging of the at- tention graph in pre-trained transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16070–16079, 2024. 3

2024
[31]

Vmt-adapter: Parameter-efficient transfer learning for multi- task dense scene understanding

Yi Xin, Junlong Du, Qiang Wang, Zhiwen Lin, and Ke Yan. Vmt-adapter: Parameter-efficient transfer learning for multi- task dense scene understanding. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), 2024. 1

2024
[32]

Mmap : Multi-modal alignment prompt for cross- domain multi-task learning

Yi Xin, Junlong Du, Qiang Wang, Ke Yan, and Shouhong Ding. Mmap : Multi-modal alignment prompt for cross- domain multi-task learning. In Proceedings of the AAAI Con- ference on Artificial Intelligence (AAAI), 2024. 1

2024
[33]

V-petl bench: A unified visual parameter- efficient transfer learning benchmark

Yi Xin, Siqi Luo, Xuyang Liu, Haodi Zhou, Xinyu Cheng, Christina E Lee, Junlong Du, Haozhe Wang, MingCai Chen, Ting Liu, et al. V-petl bench: A unified visual parameter- efficient transfer learning benchmark. Advances in neural in- formation processing systems, 37:80522–80535, 2024. 1

2024
[34]

Parameter-efficient fine- tuning for pre-trained vision models: A survey.arXiv preprint arXiv:2402.02242, 2024

Yi Xin, Siqi Luo, Haodi Zhou, Junlong Du, Xiaohong Liu, Yue Fan, Qing Li, and Yuntao Du. Parameter-efficient fine- tuning for pre-trained vision models: A survey.arXiv preprint arXiv:2402.02242, 2024. 2, 5

2024
[35]

Lumina-mgpt 2.0: Stand-alone autore- gressive image modeling

Yi Xin, Juncheng Yan, Qi Qin, Zhen Li, Dongyang Liu, Shicheng Li, Victor Shea-Jay Huang, Yupeng Zhou, Renrui Zhang, Le Zhuo, et al. Lumina-mgpt 2.0: Stand-alone autore- gressive image modeling. arXiv preprint arXiv:2507.17801,

arXiv
[36]

Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked language-models

Elad Ben Zaken, Shauli Ravfogel, and Yoav Goldberg. Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked language-models. Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), 2022. 1, 2, 5, 6

2022
[37]

A large-scale study of representation learning with the visual task adaptation benchmark

Xiaohua Zhai, Joan Puigcerver, Alexander Kolesnikov, Pierre Ruyssen, Carlos Riquelme, Mario Lucic, Josip Djolonga, Andre Susano Pinto, Maxim Neumann, Alexey Dosovit- skiy, et al. A large-scale study of representation learning with the visual task adaptation benchmark. arXiv pr...

1910 arXiv
[38]

Neural prompt search

Yuanhan Zhang, Kaiyang Zhou, and Ziwei Liu. Neural prompt search. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2024. 2

2024
[39]

Gradient-based parameter selection for efficient fine-tuning

Zhi Zhang, Qizhe Zhang, Zijun Gao, Renrui Zhang, Ekaterina Shutova, Shiji Zhou, and Shanghang Zhang. Gradient-based parameter selection for efficient fine-tuning. InProceedings of the IEEE Conference on Computer Vision and Pattern Recog- nition (CVPR), 2024. 1, 2, 3, 5, 6

2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

Token merging: Your vit but faster, 2023

Daniel Bolya, Cheng-Yang Fu, Xiaoliang Dai, Peizhao Zhang, Christoph Feichtenhofer, and Judy Hoffman. Token merging: Your vit but faster, 2023. 3

work page 2023

[2] [2]

Language models are few-shot learners

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Sub- biah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. In Advances in Neural Informa- tion Processing Systems (NeurIPS), 2020. 1

work page 2020

[3] [3]

Adaptformer: Adapting vision transformers for scalable visual recognition

Shoufa Chen, Chongjian Ge, Zhan Tong, Jiangliu Wang, Yib- ing Song, Jue Wang, and Ping Luo. Adaptformer: Adapting vision transformers for scalable visual recognition. Advances in Neural Information Processing Systems (NeurIPS) , 2022. 2, 5, 6

work page 2022

[4] [4]

Novel datasets for fine-grained image categoriza- tion

E Dataset. Novel datasets for fine-grained image categoriza- tion. In Proceedings of the IEEE Conference on Computer Vi- sion and Pattern Recognition Workshops (CVPR Workshops),

work page

[5] [5]

Bert: Pre-training of deep bidirectional transform- ers for language understanding

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transform- ers for language understanding. Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), 2018. 1

work page 2018

[6] [6]

An image is worth 16x16 words: Transform- ers for image recognition at scale

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, et al. An image is worth 16x16 words: Transform- ers for image recognition at scale. In Proceedings of the In- ternational Conference on Learning Representations (ICLR) ,

work page

[7] [7]

An image is worth 16x16 words: Transformers for image recognition at scale

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. Proceedings of the International Conference on Learn- ing Representati...

work page 2021

[8] [8]

Multi-source fully test-time adaptation

Yuntao Du, Siqi Luo, Yi Xin, Mingcai Chen, Shuai Feng, Mu- jie Zhang, and Chonngjun Wang. Multi-source fully test-time adaptation. Neural Networks, 181:106661, 2025. 1

work page 2025