A Survey on Prompt Tuning

Nigel Collier; Yixuan Su; Zongqian Li

REVIEW 2 major objections 5 minor 1 cited by

A Survey on Prompt Tuning

T0 review · 2 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This survey organizes prompt-tuning methods into two branches—direct prompt learning and transfer learning—and analyzes each method's design, insights, and trade-offs.

desk verdict A useful but overclaimed survey taxonomy: the direct-versus-transfer split works well, but the paper forgets to include PPT in its own taxonomy, and the specific citation swap the reader flagged doesn't hold up. read the letter →

arxiv 2507.06085 v2 pith:OAPJMIBU submitted 2025-07-08 cs.CL

Zongqian Li , Yixuan Su , Nigel Collier This is my paper · ORCID

classification cs.CL

keywords prompttuningparameter-efficientfine-tuningsoftpromptstransferlearningdirectmixture-of-expertslow-rankdecompositionlargelanguagemodels

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Prompt tuning adapts a frozen language model by prepending trainable continuous vectors to its input. This survey's central claim is that every prompt-tuning method belongs to one of two families: direct prompt learning, which trains soft prompts in a single stage on the target task, and transfer learning, which draws on prompts or knowledge from source tasks. Within those families the survey distinguishes general optimization, encoder-based, decomposition, and mixture-of-experts designs, and it catalogs each method's innovations, advantages, and limitations. The value of the claim, if true, is that a fast-growing and scattered literature acquires a stable map that also highlights open problems in computational efficiency and training stability.

What carries the argument

The machinery is the two-branch taxonomy built on the soft prompt, a trainable matrix $P \in \mathbb{R}^{m \times d}$ prepended to the embedded input $[P; E(x)]$ while the language model stays frozen. The split criterion is training regime: a single training stage on the target task versus reliance on source-task prompts or knowledge. The taxonomy does the work of grouping the surveyed methods into coherent families so that design choices—encoders, low-rank decompositions, mixture-of-experts routing, and attention-based prompt mixing—can be compared across the literature.

What would settle it

Compare each method entry against the abstract and content of the cited paper; a finding that a substantial share of entries describe a different method than the cited source would show the taxonomy is not a reliable map of the prompt-tuning literature.

Watch

Extended reading notes

Core claim

The core claim is the taxonomy itself: existing prompt-tuning approaches can be classified into direct prompt learning and transfer learning. Direct prompt learning covers methods that optimize prompt embeddings, prefix key-value pairs, or mixtures of short prompts directly on the target task, subcategorized into general, encoder-based, decomposition, and mixture-of-experts branches. Transfer learning covers methods that initialize, mix, or decompose prompts using knowledge from source tasks, subcategorized into general, encoder-based, and decomposition branches. The survey reads each method through this lens and reports the recurring insights that the taxonomy is meant to surface: soft prompts exhibit redundant low-rank structure, performance depends on model scale, and training is sensitive to initialization and learning rate.

Load-bearing premise

The survey's usefulness assumes that each method description and its citation accurately reflect the cited paper, so that a reader can use the taxonomy as a reliable guide to the literature.

Editorial extensions

If this is right

If the taxonomy holds, a new prompt-tuning paper can be located quickly by asking whether it trains directly on the target task or transfers from source tasks.
The survey's challenges section identifies computational efficiency and training stability as the main bottlenecks, giving future prompt-tuning research two concrete targets.
The low-rank decomposition and mixture-of-experts branches point to a cross-method trend: soft prompts have redundancy and can be compressed or routed without sacrificing task performance.
The transfer-learning branches show that prompt reuse can help low-resource and few-shot settings, which is where the survey says the gains matter most.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: the same two-branch taxonomy could classify prompt-tuning variants outside this survey's list, such as visual or multimodal soft prompts, because the criteria are architectural and data-agnostic.
Editorial inference: the convergence of several methods on low-rank decomposition suggests a testable hypothesis that learned soft prompts are near low-rank; measuring the effective rank of trained prompts across tasks would settle this.
Editorial inference: the taxonomy does not define a precedence rule for hybrids that both transfer from source prompts and train directly on the target, so a reader must classify those by the dominant mechanism.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

First off: this is a serviceable survey. The direct-prompt-learning vs transfer-learning split is a reasonable organizing principle, and the per-method write-ups (Prompt Tuning, XPrompt, P-Tuning v2, P-Tuning, RPT, Prefix Tuning, DPT, DePT, SMoP, SPoT, ATTEMPT, TransPrompt, CTPT, MPT) are generally accurate, with useful insights and limitations for each. The visualizations comparing frameworks are a nice touch and will help newcomers. None of this is new research, but as a survey it's a legitimate contribution to the prompt-tuning/PEFT community.

Now the soft spots. The reader's strongest concrete evidence—that P-Tuning v2 and P-Tuning citations are swapped—doesn't survive inspection. In the references, Liu et al. 2022b is the ACL 2022 'P-tuning: Prompt tuning can be comparable...' paper (which is the P-Tuning v2 paper), and Liu et al. 2023 is 'GPT understands, too' (P-Tuning v1). That matches the descriptions in Section 3, so the citation attributions are actually fine.

The real problem is coverage, not citation mislabeling. The paper cites Gu et al. (2022), PPT—pre-trained prompt tuning—in Section 2 as an example of transfer learning in prompt tuning, but PPT never appears in Section 4's taxonomy or in the appendix list of papers. That's a direct counterexample to the abstract's claim of categorizing existing approaches and to the 'comprehensive categorization' contribution. If PPT is a transfer method, leaving it out undercuts the central organizational claim. There may be other omissions too, but PPT is the one the paper itself invokes and then forgets.

Two smaller issues. The paper includes the authors' own PT-MoE method in the taxonomy and describes it as state-of-the-art without external benchmarking; it's fine to include your own work, but the SOTA claim should be toned down or clearly labeled as the authors' own evaluation. Also, the introduction promises 'three primary contributions' but lists four bullets, and there are a few typos ('fiengrained', 'promptsk'). Minor, but they add to the impression of haste.

Overall: the taxonomy is plausible and the method summaries are reliable enough to be useful to a reader who wants a map of the field. The omission of PPT from the taxonomy is a real gap but it's fixable, not fatal. The clearer it is about selection criteria, the more the 'comprehensive' claim will hold.

Who is this for: newcomers and practitioners looking for an entry point to prompt tuning; it's less useful for experts who already know the landscape. It deserves a serious referee rather than a desk reject, but I would want the coverage gap and the self-promotion issue addressed before publication. Send it to review with those conditions.

Referee Report

2 major / 5 minor

Summary. The manuscript is a survey of prompt tuning, a parameter-efficient adaptation technique in which a frozen language model is steered by trainable continuous vectors prepended to the input. The paper's central contribution is a two-branch taxonomy: direct prompt learning and transfer learning. Direct learning is subdivided into general, encoder-based, decomposition, and mixture-of-experts methods; transfer learning is subdivided into general, encoder-based, and decomposition methods. For each method the survey gives a paragraph-level description of the design, insights, advantages, and limitations, and Figure 2 provides schematic comparisons. The paper closes with a discussion of current challenges (computational efficiency, training instability, initialization, model-scale dependence, explainability) and future directions.

Significance. The paper addresses a useful niche: existing surveys tend to cover either all of PEFT or all of prompting, whereas a focused survey of prompt tuning can serve as an entry point for practitioners. The high-level direct-versus-transfer partition is a plausible organizing principle, and the visualizations in Figure 2 help readers see the architectural differences among methods. The paper is clearly written and cites most of the canonical works in the area. That said, the value of a survey depends on its coverage and on the consistency of its taxonomy; the omission of PPT from the transfer-learning branch undercuts the paper's own 'comprehensive categorization' claim. I also checked the P-Tuning references: the citations in Section 3 are internally consistent, so the specific citation-swap concern in the reader's report does not survive inspection.

major comments (2)

[Section 4 and Appendix A] The transfer-learning branch is incomplete in a way that directly undercuts the paper's central claim. Section 2 defines transfer learning in prompt tuning by citing PPT (Gu et al., 2022), and Section 5.1 cites it again, but Section 4's transfer-learning branch lists only SPoT, ATTEMPT, TransPrompt, CTPT, and MPT, and Appendix A likewise omits PPT from all transfer-learning subsections. Since Section 1 promises a 'comprehensive categorization' and the abstract says existing approaches are classified into two categories, the omission of a canonical transfer method from the taxonomy is an internal inconsistency. Please classify PPT (for example, under general transfer) or explicitly narrow the survey's scope and soften the comprehensiveness claim.
[Section 3] The 'encoder-based methods' subcategory is applied without a stated criterion. P-Tuning genuinely uses a prompt encoder (LSTM/MLP), but RPT uses a residual MLP reparameterization and Prefix-Tuning uses an MLP-reparameterized prefix; grouping these three together as 'encoder-based' while treating DPT and DePT as 'decomposition-based' is not self-explanatory. Please define the boundary of 'encoder-based' (for example, any learned reparameterization network) or rename the subcategory, since the usefulness of the taxonomy depends on such definitions.

minor comments (5)

[Section 1] The introduction lists four bullets after saying there are 'three primary contributions'; please renumber or expand the count to four.
[Section 3] The claim that Prompt Tuning is 'the first method that finetunes the input tokens' is historically fragile given Prefix-Tuning, which also tunes continuous prompts albeit at every layer; please soften or qualify this statement.
[Section 3 (PT-MoE paragraph)] PT-MoE is described as 'achieving state-of-the-art performance' and 'outperforming other prompt tuning methods' without any numbers or an external comparison being reported in the survey; since PT-MoE is one of the authors' own methods, please add the original paper's evaluation details or qualify the claim.
[Section 3 and Appendix A] Please proofread: 'fiengrained' in the P-Tuning paragraph, 'promptsk' in the SMoP paragraph, and 'Decomposirion' in Appendix A are typos.
[Section 5.1] The initialization challenge cites Liu et al. (2022a), which is a few-shot PEFT paper; if the intended reference is specifically about prompt initialization, please choose a more direct citation or clarify the connection.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation; survey taxonomy rests on external methods, with minor non-load-bearing self-citations.

full rationale

The paper is a survey, not a derivation. Its central claim is a two-branch taxonomy of prompt tuning methods (direct vs transfer learning), and each leaf of the taxonomy is populated by external works (Lester et al., 2021; Vu et al., 2022; Asai et al., 2022; Wang et al., 2021/2023; etc.) whose descriptions are cited rather than derived from the authors' own results. No equation or fitted parameter is renamed as a prediction. The only in-house method included, PT-MoE (Li et al., 2025c), is asserted to be 'state-of-the-art' and to 'outperform other prompt tuning methods,' but this claim is peripheral to the organizing taxonomy and does not support any derived quantity; removing it would not alter the classification. The other self-citations (2024; 2025a; 2025b; Li & Cole, 2025) are contextual. The skeptic's proposed citation-swap evidence does not survive inspection: Liu et al. (2022b) is indeed the ACL 2022 P-Tuning v2 paper and Liu et al. (2023) is 'GPT understands, too,' matching the descriptions in Section 3. The real weakness is coverage—Gu et al. (2022) PPT is invoked to define transfer learning in Section 2 yet never assigned to the Section 4 taxonomy—but an omission is a completeness/correctness flaw, not circularity. Therefore the central claim has independent content and no step is equivalent by construction to its input; a score of 1 reflects the minor, non-load-bearing self-citation concern rather than actual circularity.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The survey rests on assumptions about representation, taxonomy validity, and representativeness; no free parameters or invented entities apply to a review.

assumptions (3)

domain assumption The cited papers are accurately represented in the survey's descriptions.
The survey's value depends on faithful summaries of each method; the P-Tuning v2 mis-citation in Section 3 indicates this assumption is false.
ad hoc to paper The direct-versus-transfer taxonomy is a valid partition of prompt tuning methods.
The taxonomy is introduced by the authors and not derived from a formal analysis; it could exclude methods that do not fit the branches.
domain assumption The selected 15 methods are representative of the prompt tuning field.
No selection criteria are stated, so representativeness is asserted rather than demonstrated.

how reviews work

0 comments

Cite this review

Pith. "Pith review of A Survey on Prompt Tuning." pith.science (2026). https://pith.science/paper/OAPJMIBU

@misc{pith2026250706085,
  author       = {Pith},
  title        = {Pith review of: A Survey on Prompt Tuning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OAPJMIBU}},
  note         = {Machine review of arXiv:2507.06085}
}

read the original abstract

This survey reviews prompt tuning, a parameter-efficient approach for adapting language models by prepending trainable continuous vectors while keeping the model frozen. We classify existing approaches into two categories: direct prompt learning and transfer learning. Direct prompt learning methods include: general optimization approaches, encoder-based methods, decomposition strategies, and mixture-of-experts frameworks. Transfer learning methods consist of: general transfer approaches, encoder-based methods, and decomposition strategies. For each method, we analyze method designs, innovations, insights, advantages, and disadvantages, with illustrative visualizations comparing different frameworks. We identify challenges in computational efficiency and training stability, and discuss future directions in improving training robustness and broadening application scope.

Figures

Figures reproduced from arXiv: 2507.06085 by the authors.

**Figure 2.** Illustration of different prompt tuning (PT) methods. Prompt Tuning directly prepends soft prompts to input. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

AutoFed: Personalized Federated Traffic Prediction via Adaptive Prompt
cs.LG 2025-12 unverdicted novelty 6.0 of 10

AutoFed introduces a prompt-based personalized federated learning method that achieves superior traffic prediction performance on real-world datasets by sharing compact prompt matrices while preserving local specificity.

Reference graph

Works this paper leans on

12 extracted references · 2 canonical work pages · cited by 1 Pith paper

[4]

Zhao, W., Gupta, A., Chung, T., and Huang, J

URL https://www.sciencedirect.com/ science/article/pii/S2666651024000056. Zhao, W., Gupta, A., Chung, T., and Huang, J. SPC: Soft prompt construction for cross domain generaliza- tion. In Can, B., Mozes, M., Cahyawijaya, S., Saphra, N., Kassner, N., Ravfogel, S., Ravichander, A., Zhao, C., Augenstein, I., Rogers, A., Cho, K., Grefenstette, E., and V oita,...

arXiv 2023
[9]

doi: 10.18653/v1/2021.emnlp-main.221

Association for Computational Linguistics. doi: 10.18653/v1/2021.emnlp-main.221. URL https:// aclanthology.org/2021.emnlp-main.221. Wang, L., Zhang, X., Su, H., and Zhu, J. A Comprehensive Survey of Continual Learning: Theory, Method and Application . IEEE Transactions on Pattern Analysis & Machine Intelligence, 46(08):5362–5383, August 2024. ISSN 1939-35...

arXiv 2021
[353]

acl-long.353

URL https://aclanthology.org/2021. acl-long.353. Li, Z. and Cole, J. M. Auto-generating question-answering datasets with domain-specific knowledge for language models in scientific tasks. Digital Discovery, 2025. doi: 10.1039/D4DD00307A. URL http://dx.doi.org/ 10.1039/D4DD00307A. Li, Z., Su, Y ., and Collier, N. 500xcompressor: Generalized prompt compress...

arXiv 2021
[368]

ReasonGraph: Visualisation of Reasoning Paths

URL https://aclanthology.org/2025. naacl-long.368/. Li, Z., Shareghi, E., and Collier, N. Reasongraph: Vi- sualisation of reasoning paths, 2025b. URL https: //arxiv.org/abs/2503.03979. Li, Z., Su, Y ., and Collier, N. Pt-moe: An efficient fine- tuning framework for integrating mixture-of-experts into prompt tuning, 2025c. URL https://arxiv.org/ abs/2505.0...

work page Pith review arXiv 2025
[421]

findings-acl.421

URL https://aclanthology.org/2023. findings-acl.421. Schulhoff, S., Ilie, M., Balepur, N., Kahadze, K., Liu, A., Si, C., Li, Y ., Gupta, A., Han, H., Schulhoff, S., Dulepet, P. S., Vidyadhara, S., Ki, D., Agrawal, S., Pham, C., Kroiz, G., Li, F., Tao, H., Srivastava, A., Costa, H. D., Gupta, S., Rogers, M. L., Goncearenco, I., Sarli, G., Galynker, I., Pes...

arXiv 2023
[446]

emnlp-main.446

URL https://aclanthology.org/2022. emnlp-main.446. Ben Zaken, E., Goldberg, Y ., and Ravfogel, S. BitFit: Sim- ple parameter-efficient fine-tuning for transformer-based masked language-models. In Muresan, S., Nakov, P., and Villavicencio, A. (eds.), Proceedings of the 60th An- nual Meeting of the Association for Computational Lin- guistics (Volume 2: Shor...

doi:10.18653/v1/2022.acl-short.1 2022
[576]

acl-long.576

URL https://aclanthology.org/2022. acl-long.576. Han, Z., Gao, C., Liu, J., Zhang, J., and Zhang, S. Q. Parameter-efficient fine-tuning for large models: A com- prehensive survey. Transactions on Machine Learn- ing Research, 2024. ISSN 2835-8856. URL https: //openreview.net/forum?id=lIsCS8b6zj. Hinton, G., Vinyals, O., and Dean, J. Distilling the knowl- e...

arXiv 2022
[780]

findings-emnlp.780

URL https://aclanthology.org/2023. findings-emnlp.780. Yao, Y ., Zhang, A., Zhang, Z., Liu, Z., Chua, T.-S., and Sun, M. Cpt: Colorful prompt tuning for pre-trained vision-language models. AI Open, 5:30–38, 2024. ISSN 2666-6510. doi: https://doi.org/10.1016/j.aiopen.2024.01

work page doi:10.1016/j.aiopen.2024.01 2023

Show all 12 references

[890]

findings-emnlp.890

URL https://aclanthology.org/2023. findings-emnlp.890. Xu, Y ., Zeng, Z., and Shen, Z. Efficient cross-task prompt tuning for few-shot conversational emotion recognition. In Bouamor, H., Pino, J., and Bali, K. (eds.), Findings of the Association for Computational Linguistics: ...

2023 doi
[2021]

doi: 10.18653/v1/2021.emnlp-main.243

Association for Computational Linguistics. doi: 10.18653/v1/2021.emnlp-main.243. URL https:// aclanthology.org/2021.emnlp-main.243. Li, X. L. and Liang, P. Prefix-tuning: Optimizing continu- ous prompts for generation. In Zong, C., Xia, F., Li, W., and Navigli, R. (eds.), Proc...

2021 doi
[2023]

doi: 10.18653/v1/2023.emnlp-main.884

Association for Computational Linguistics. doi: 10.18653/v1/2023.emnlp-main.884. URL https:// aclanthology.org/2023.emnlp-main.884. Gu, Y ., Han, X., Liu, Z., and Huang, M. PPT: Pre-trained prompt tuning for few-shot learning. In Muresan, S., Nakov, P., and Villavicencio, A. (...

2023 doi
[2024]

Tay, Y ., Dehghani, M., Bahri, D., and Metzler, D

URL https://openreview.net/forum? id=KjegfPGRde. Tay, Y ., Dehghani, M., Bahri, D., and Metzler, D. Efficient transformers: A survey. ACM Comput. Surv., 55(6), De- cember 2022. ISSN 0360-0300. doi: 10.1145/3530811. URL https://doi.org/10.1145/3530811. Vu, T., Lester, B., Const...

2022 doi

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [4]

Zhao, W., Gupta, A., Chung, T., and Huang, J

URL https://www.sciencedirect.com/ science/article/pii/S2666651024000056. Zhao, W., Gupta, A., Chung, T., and Huang, J. SPC: Soft prompt construction for cross domain generaliza- tion. In Can, B., Mozes, M., Cahyawijaya, S., Saphra, N., Kassner, N., Ravfogel, S., Ravichander, A., Zhao, C., Augenstein, I., Rogers, A., Cho, K., Grefenstette, E., and V oita,...

arXiv 2023

[2] [9]

doi: 10.18653/v1/2021.emnlp-main.221

Association for Computational Linguistics. doi: 10.18653/v1/2021.emnlp-main.221. URL https:// aclanthology.org/2021.emnlp-main.221. Wang, L., Zhang, X., Su, H., and Zhu, J. A Comprehensive Survey of Continual Learning: Theory, Method and Application . IEEE Transactions on Pattern Analysis & Machine Intelligence, 46(08):5362–5383, August 2024. ISSN 1939-35...

arXiv 2021

[3] [353]

acl-long.353

URL https://aclanthology.org/2021. acl-long.353. Li, Z. and Cole, J. M. Auto-generating question-answering datasets with domain-specific knowledge for language models in scientific tasks. Digital Discovery, 2025. doi: 10.1039/D4DD00307A. URL http://dx.doi.org/ 10.1039/D4DD00307A. Li, Z., Su, Y ., and Collier, N. 500xcompressor: Generalized prompt compress...

arXiv 2021

[4] [368]

ReasonGraph: Visualisation of Reasoning Paths

URL https://aclanthology.org/2025. naacl-long.368/. Li, Z., Shareghi, E., and Collier, N. Reasongraph: Vi- sualisation of reasoning paths, 2025b. URL https: //arxiv.org/abs/2503.03979. Li, Z., Su, Y ., and Collier, N. Pt-moe: An efficient fine- tuning framework for integrating mixture-of-experts into prompt tuning, 2025c. URL https://arxiv.org/ abs/2505.0...

work page Pith review arXiv 2025

[5] [421]

findings-acl.421

URL https://aclanthology.org/2023. findings-acl.421. Schulhoff, S., Ilie, M., Balepur, N., Kahadze, K., Liu, A., Si, C., Li, Y ., Gupta, A., Han, H., Schulhoff, S., Dulepet, P. S., Vidyadhara, S., Ki, D., Agrawal, S., Pham, C., Kroiz, G., Li, F., Tao, H., Srivastava, A., Costa, H. D., Gupta, S., Rogers, M. L., Goncearenco, I., Sarli, G., Galynker, I., Pes...

arXiv 2023

[6] [446]

emnlp-main.446

URL https://aclanthology.org/2022. emnlp-main.446. Ben Zaken, E., Goldberg, Y ., and Ravfogel, S. BitFit: Sim- ple parameter-efficient fine-tuning for transformer-based masked language-models. In Muresan, S., Nakov, P., and Villavicencio, A. (eds.), Proceedings of the 60th An- nual Meeting of the Association for Computational Lin- guistics (Volume 2: Shor...

doi:10.18653/v1/2022.acl-short.1 2022

[7] [576]

acl-long.576

URL https://aclanthology.org/2022. acl-long.576. Han, Z., Gao, C., Liu, J., Zhang, J., and Zhang, S. Q. Parameter-efficient fine-tuning for large models: A com- prehensive survey. Transactions on Machine Learn- ing Research, 2024. ISSN 2835-8856. URL https: //openreview.net/forum?id=lIsCS8b6zj. Hinton, G., Vinyals, O., and Dean, J. Distilling the knowl- e...

arXiv 2022

[8] [780]

findings-emnlp.780

URL https://aclanthology.org/2023. findings-emnlp.780. Yao, Y ., Zhang, A., Zhang, Z., Liu, Z., Chua, T.-S., and Sun, M. Cpt: Colorful prompt tuning for pre-trained vision-language models. AI Open, 5:30–38, 2024. ISSN 2666-6510. doi: https://doi.org/10.1016/j.aiopen.2024.01

work page doi:10.1016/j.aiopen.2024.01 2023