Pith. sign in

REVIEW 3 major objections 6 minor 38 references

Efficient and Privacy-Preserving Soft Prompt Transfer for LLMs

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read POST shows that soft prompts tuned privately on a small distilled model can be transferred to a larger LLM using only public data, recovering most of the accuracy of full prompt tuning.

desk verdict Solid empirical work on private soft prompt transfer, but Eq 5's KL divergence over output differences is mathematically undefined as printed, and the alpha selection leaks private test information. read the letter →

arxiv 2506.16196 v1 pith:CH7M3SBM submitted 2025-06-19 cs.LG

Xun Wang , Jing Xu , Franziska Boenisch , Michael Backes , Christopher A. Choquette-Choo , Adam Dziedzic This is my paper · ORCID
classification cs.LG
keywords softprompttransfertuningknowledgedistillationdifferentialprivacylargelanguagemodelsprivacy-preservingadaptationparameter-efficientfine-tuningmembershipinference
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Soft prompts that adapt large language models are normally tuned directly on the big model, which forces users either to pay for expensive full-model backpropagation or to send private data to the model provider. This paper proposes POST, a framework that instead tunes the soft prompt locally on a small model distilled from the large one, then transfers that prompt back to the large LLM using only a small public dataset. The authors set out to show that this pipeline keeps the private data confidential, can add differential privacy guarantees, and still reaches accuracy close to full prompt tuning on the large model. On classification and open-ended generation tasks across three model families, POST beats direct prompt transfer and prior transfer baselines by large margins, reduces compute, and loses little utility when differential privacy is turned on.

What carries the argument

The central machinery is knowledge-distilled feature alignment plus a two-term public-data transfer loss. The provider distills the target LLM into a small student (for example, Llama2-7b to a 2-layer model) using a combination of logit KL, language-model, and embedding-cosine losses, so the student matches the teacher's predictive behavior enough to make prompts portable. The transfer step then optimizes the target prompt $p_t$ to minimize $L=(1-\alpha)L_1+\alpha L_2$, with $L_1$ the KL divergence between the prompted outputs of the two models and $L_2$ the KL divergence between the prompt-induced output changes, both evaluated on public data. Local tuning can run under PromptDPSGD for $(\epsilon,\delta)$-differential privacy, and DP post-processing carries the guarantee through the transfer.

What would settle it

Run POST with a same-size student that has equal task accuracy but was produced by ordinary fine-tuning on the distillation corpus instead of knowledge distillation. The paper's Table 6 predicts a clear drop in transferred accuracy (up to roughly 15 points on tweet); finding no drop would falsify the feature-alignment premise.

Watch

Extended reading notes

Core claim

The paper's central claim is that a soft prompt's task knowledge is not permanently locked to the model it was tuned on. If the small source model is produced by knowledge distillation from the target LLM, the tuned prompt can be re-encoded for the target by optimizing the target prompt on public data to match two quantities: the source model's prompted output logits, and the direction of the output change that the prompt induces relative to the un-prompted model. The claim is supported experimentally: transferred prompts on Llama2-7b, GPT2-XL, and Roberta-base approach the Full PT upper bound, improve over direct transfer by a large amount, and outperform previous zero-shot transfer and discrete-prompt baselines, while differentially private tuning reduces membership-inference leakage.

Load-bearing premise

The transfer step assumes the distilled student's feature space is aligned with the large model's, so matching the student's prompted logits and output-change direction on public data is enough to re-encode the prompt's task knowledge in the large model.

Editorial extensions

If this is right

  • Users can adapt a hosted LLM to private tasks without ever sending training data to the provider, since local tuning and public-data transfer are separate from the private data.
  • Differentially private local tuning transfers to the large model with little utility loss, and membership-inference AUC drops relative to non-private prompts.
  • Small public datasets of under roughly 100 samples and a few thousand transfer steps are enough to reach convergence, so the transfer step is cheap.
  • Prompt tuning on a small student model cuts runtime substantially, with a sixfold speedup on the largest tested dataset before accounting for the one-time distillation cost.
  • Soft prompt transfer now works across model families and for open-ended generation, not only classification, giving providers a reason to offer soft-prompt adaptation without collecting user data.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication the authors leave implicit is that the same two-term transfer loss could be applied to other parameter-efficient adapters, such as prefix or LoRA-style modules, whenever a distilled student is available.
  • The observed regularizing effect of DP noise suggests a testable extension: adding small noise during local tuning might improve transfer generally, even for users who do not require formal privacy.
  • Because public-dataset choice matters for transfer quality, a systematic or synthetic public-data selection method would remove the remaining heuristic; the paper's task-similarity finding is a starting point for such a rule.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes POST, a three-stage framework for privately adapting a hosted LLM: the provider distills the large model into a small student; the user tunes a soft prompt on the student with private data, optionally under differential privacy; and the provider transfers the prompt to the large model using only a small public dataset. The transfer step optimizes a weighted combination of two KL-based losses, one matching prompted outputs and one matching prompt-induced output changes. Experiments on Roberta-base, GPT2-XL, and Llama2-7b across five classification and two generation tasks report accuracies close to full prompt tuning, large gains over direct transfer and zero-shot, DP via PromptDPSGD, LiRA leakage reduction, runtime savings, and comparisons to DP-OPT and zero-shot transfer baselines.

Significance. If the headline results hold, POST is a useful contribution: it gives a concrete protocol for avoiding private-data exposure to LLM providers while retaining most of the utility of full prompt tuning, with substantial compute savings, and it is one of the first demonstrations that soft prompts can be transferred effectively using public data only. The evaluation is unusually broad: three model families, multiple private/public dataset pairs, DP and non-DP settings, generation tasks, membership-inference analysis, runtime accounting, and a large ablation suite. The authors also release code, which materially supports reproducibility. However, the two concerns below affect the exact formulation of the transfer objective and the validity of the reported selection protocol, so the contribution needs revision before the claims can be accepted as stated.

major comments (3)
  1. [Section 4.3, Eq. (5)] The loss L2 as printed is mathematically undefined: the arguments of KLDiv are differences of model outputs. If Phi returns probability vectors, each difference can contain negative entries and does not sum to one; if Phi returns logits, KL divergence requires probabilities. No normalization or alternative divergence is specified. Because L2 is the term that drives transfer when the target model has strong zero-shot performance or the student is weak, the reported transfer results cannot be reproduced from the paper alone. Please state the actual operation used in the code (e.g., softmax followed by KL, MSE, or cosine distance) and correct Eq. (5) accordingly.
  2. [Section 5.2, Tables 1/12/20, Eq. (6)] The reported headline accuracies appear to be selected using private test labels. Section 5.1 defines Full ZS as the teacher's zero-shot accuracy on the private test sets, and Eq. (6) uses this ZS and the compressed model's accuracy C to set alpha. Table 20 then reports the 'best-performing alpha' by transferred accuracy on the same private test sets, and the alpha values in Table 12 do not always match the heuristic row of Table 20 (e.g., Llama2-7b sst2: Table 12 uses 0.9 while the heuristic gives 1.0). Additionally, Table 1 reports two public datasets per private task and the summary/Table 5 use the better of the two. As a result, the numbers in the main tables do not demonstrate a deployable privacy-preserving protocol, because the transfer step in practice would not know private test labels. Please specify how alpha and the public dataset are chosen from information available to the parties (e.g., a private validation split held out by the user, or a purely public criterion) and re-run the main comparisons under that protocol.
  3. [Section 4.3 and Section 5.6, Eq. (4)] Even after fixing Eq. (5), Eq. (4) needs the same clarification: KLDiv is only defined if both arguments are probability distributions, so the paper should state whether Phi_t and Phi_s outputs are softmax distributions or logits, and whether a temperature or smoothing is applied. The current notation leaves this ambiguous, which is a reproducibility blocker for the central transfer objective.
minor comments (6)
  1. [Table 12] The title contains a typo: 'Trasnfer' should be 'Transfer'.
  2. [Section D.7] The phrase 'pertaining data leakage' should be 'pretraining data leakage'.
  3. [Table 18] The word 'analze' should be 'analyze' in the caption.
  4. [Table 5] The entry '59,54' uses a decimal comma instead of a decimal point; please standardize numeric formatting.
  5. [Table 9] The tweet ground-truth label list for label 1 has a stray extra closing bracket: '[ " moderate", " neutral", " balanced"]]'.
  6. [Equation (6)] The heuristic in Eq. (6) can produce negative values when ZS < RG; please state whether values are clipped to a valid range and what the user should do in that case.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation: the transfer objective is optimized on public data and scored on private held-out data; the only notable issues are test-informed hyperparameter selection and a formal ambiguity in Eq (5), neither of which reduces the claimed result to its inputs.

full rationale

The paper's central transfer claim is empirical rather than definitional. In Section 4.3, the target prompt pt is optimized on a public dataset Dpub through the loss L=(1-alpha)L1+alpha L2, with L1 and L2 in Eqs (4)-(5) matching the small model's prompted outputs and output differences. The reported private test accuracy is a held-out evaluation metric and does not appear as an optimization input in Eq (3). The distilled model Phi_s is obtained by knowledge distillation from Phi_t (Section 4.1), but this is an architectural/enabling step, not a mathematical forcing of the transfer result; the ablation in Table 6 gives independent evidence that distillation helps. The alpha heuristic in Eq (6) uses ZS and C, which in the simulated evaluation are private test accuracies, and Table 5 reports the best public dataset; this is test-set-informed hyperparameter selection that can optimistically bias the reported numbers, but it is not a constructional equivalence because the transfer optimization itself never sees the private labels. The paper's self-citations, notably PromptDPSGD from Duan et al. (2023a), provide an existing differentially private tuning mechanism with a peer-reviewed privacy proof; they are not used to justify the novel transfer step and are therefore not load-bearing circularity. One formal concern remains: Eq (5) writes a KL divergence between differences of model outputs, which are not probability distributions as printed, so the loss is not well-defined from the paper alone; the released code may implement a valid surrogate. This is a reproducibility defect, not a circularity, since it does not make the reported result equal to its input by construction.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The main free parameter is alpha, which is effectively tuned using private test accuracies in the evaluation. The two load-bearing domain assumptions are KD feature alignment and public-data transfer sufficiency. No new physical or architectural entities are postulated.

free parameters (1)
  • alpha (balance between L1 and L2 in transfer loss) = 0.0 to 1.0 per model/dataset; see Table 12, e.g., Roberta sst2=0.8, Llama2 sst2=0.9
    Eq 3 defines L = (1-alpha) L1 + alpha L2. Eq 6 sets alpha from ZS (large model zero-shot accuracy) and C (compressed model accuracy) on the private task's test set. These accuracies are not available to the provider in the real setup, so the reported transfer results are conditioned on test-set-informed tuning. Table 20 shows a sweep and the heuristic row.
assumptions (3)
  • domain assumption Knowledge distillation on BookCorpus (Eq 1) produces a small model whose feature space is aligned with the large model enough to enable prompt transfer.
    Section 4.1 uses KD to compress the model; Table 6 shows transfer performance degrades sharply without this alignment, so the claim depends on it, but the paper does not prove it for arbitrary models.
  • domain assumption Matching output distributions and prompt-induced output changes on public data (Eqs 4-5) is sufficient to transfer the private task to the large model.
    Section 4.3; no formal guarantee; supported only by the reported experiments (Tables 1 and 13-14).
  • standard math PromptDPSGD provides (epsilon, delta)-DP for the prompt and the transfer step is DP post-processing.
    Section 4.2 relies on Duan et al. (2023a) for the DP proof and on the standard post-processing property; the paper does not re-derive these.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Efficient and Privacy-Preserving Soft Prompt Transfer for LLMs." pith.science (2026). https://pith.science/paper/CH7M3SBM

@misc{pith2026250616196,
  author       = {Pith},
  title        = {Pith review of: Efficient and Privacy-Preserving Soft Prompt Transfer for LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CH7M3SBM}},
  note         = {Machine review of arXiv:2506.16196}
}
read the original abstract

Prompting has become a dominant paradigm for adapting large language models (LLMs). While discrete (textual) prompts are widely used for their interpretability, soft (parameter) prompts have recently gained traction in APIs. This is because they can encode information from more training samples while minimizing the user's token usage, leaving more space in the context window for task-specific input. However, soft prompts are tightly coupled to the LLM they are tuned on, limiting their generalization to other LLMs. This constraint is particularly problematic for efficiency and privacy: (1) tuning prompts on each LLM incurs high computational costs, especially as LLMs continue to grow in size. Additionally, (2) when the LLM is hosted externally, soft prompt tuning often requires sharing private data with the LLM provider. For instance, this is the case with the NVIDIA NeMo API. To address these issues, we propose POST (Privacy Of Soft prompt Transfer), a framework that enables private tuning of soft prompts on a small model and subsequently transfers these prompts to a larger LLM. POST uses knowledge distillation to derive a small model directly from the large LLM to improve prompt transferability, tunes the soft prompt locally, optionally with differential privacy guarantees, and transfers it back to the larger LLM using a small public dataset. Our experiments show that POST reduces computational costs, preserves privacy, and effectively transfers high-utility soft prompts.

Figures

Figures reproduced from arXiv: 2506.16196 by the authors.

Figure 1
Figure 1. POSTB Framework. 1 An LLM provider compresses their LLM Φt into a smaller model Φs through knowledge distillation. 2 The private data owner learns a specific soft prompt ps on Φs using their private dataset (optionally with differential privacy guarantees). 3 The LLM provider obtains the soft prompt pt for solving the user’s task by transferring ps to the target LLM Φt—solely relying on a small public dataset and no… view at source ↗
Figure 3
Figure 3. Effect of Number of Transfer Steps (arisetv) [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 2
Figure 2. Effect of Number of Public Samples (arisetv). Effect of Number of Public Samples used for Transfer. We first investigate the influence of the size of the public dataset required to complete the prompt transfer. Our results in [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Checkpoint Loss on Training Steps for Distilled RoBERTa Models with Varying Numbers of Layers. The plot illustrates the convergence behavior of 1-layer, 2-layer, 3-layer, and 4-layer distilled models. The dashed vertical line represents the point at which a specific ch…
Figure 5
Figure 5. Figure 5: Analysis of Transferred Accuracy versus Compressed PT Accuracy and Checkpoint Loss at 50,000, 100,000, 200,000 and 500,000 Distillation Steps. (a) compares transferred accuracy to compressed PT accuracy for different distilled Roberta models. (b) compares transferred a…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 22 canonical work pages

  1. [1]

    B., Mironov, I., Talwar, K., and Zhang, L

    Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K., and Zhang, L. Deep learning with differential privacy. InProceedings of the 2016 ACM SIGSAC conference on computer and communications security, pp. 308–318,

    work page 2016

  2. [5]

    Clark, C., Lee, K., Chang, M.-W., Kwiatkowski, T., Collins, M., and Toutanova, K

    URL https://lmsys.org/blog/ 2023-03-30-vicuna/. Clark, C., Lee, K., Chang, M.-W., Kwiatkowski, T., Collins, M., and Toutanova, K. BoolQ: Exploring the surpris- ing difficulty of natural yes/no questions. In Burstein, J., Doran, C., and Solorio, T. (eds.),Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational ...

    work page 2023

  3. [7]

    Cal- ibrating noise to sensitivity in private data analysis

    Dwork, C., McSherry, F., Nissim, K., and Smith, A. Cal- ibrating noise to sensitivity in private data analysis. In Theory of Cryptography: Third Theory of Cryptography Conference, TCC 2006, New York, NY, USA, March 4-7,

    2006

  4. [8]

    αce αlm αcos Learning Rate Batch Size 5.0 2.0 1.0 0.00025 5 B.2

    Table 8:Hyperparameters in Knowledge Distillation. αce αlm αcos Learning Rate Batch Size 5.0 2.0 1.0 0.00025 5 B.2. Datasets and Tasks We use the following datasets as private sets or transfer sets in our experiments: • sst2: The Stanford Sentiment Treebank (sst2) is a binary sentiment classification dataset containing sentences from movie reviews, labele...

    work page 2022

  5. [11]

    Distilling the knowledge in a neural network.arXiv preprint arXiv:1503.02531, 2(7),

    Hinton, G., Vinyals, O., Dean, J., et al. Distilling the knowledge in a neural network.arXiv preprint arXiv:1503.02531, 2(7),

    arXiv

  6. [12]

    Table 11:Hyperparameters used during Prompt Transfer. Model Batch Size Optimizer Learning Rate Roberta-base 32 Adam 0.001 GPT2-XL 8 Adam 0.001 Llama2-7b 4 Adam 0.0005 Table 12:Setting ofαfor Different Datasets and Models during Prompt Trasnfer. Dataset Model sst2 imdb tweet arisetv Roberta-base 0.8 0.8 0.2 0.4 GPT2-XL 0.7 0.4 0.0 0.6 Llama2-7b 0.9 1.0 0.6...

    work page arXiv

  7. [14]

    Dataset posted on Hugging Face

    URL https: //huggingface.co/datasets/vinaykudari/ acled-information-extraction. Dataset posted on Hugging Face. Lester, B., Al-Rfou, R., and Constant, N. The power of scale for parameter-efficient prompt tuning.arXiv preprint arXiv:2104.08691,

    arXiv

  8. [16]

    Liu, H., Tam, D., Muqeeth, M., Mohta, J., Huang, T., Bansal, M., and Raffel, C. A. Few-shot parameter-efficient fine- tuning is better and cheaper than in-context learning.Ad- vances in Neural Information Processing Systems, 35: 1950–1965,

    work page 1950

Show all 38 references
  1. [17]

    A conversational movie search system based on con- ditional random fields.13th Annual Conference of the International Speech Communication Association 2012, INTERSPEECH 2012, 3, 01

    Liu, J., Cyphers, S., Pasupat, P., Mcgraw, I., and Glass, J. A conversational movie search system based on con- ditional random fields.13th Annual Conference of the International Speech Communication Association 2012, INTERSPEECH 2012, 3, 01

    2012

  2. [19]

    URL http://arxiv.org/ abs/1907.11692. Maas, A. L., Daly, R. E., Pham, P. T., Huang, D., Ng, A. Y ., and Potts, C. Learning word vectors for sentiment analysis. InProceedings of the 49th Annual Meeting of 11 Efficient and Privacy-Preserving Soft Prompt Transfer for LLMs the Ass...

    1907 arXiv

  3. [20]

    Bissyand ´e, T

    Philippy, F., Guo, S., Haddadan, S., Lothritz, C., Klein, J., and F. Bissyand ´e, T. Soft prompt tuning for cross- lingual transfer: When less is more. In V ´azquez, R., Mickus, T., Tiedemann, J., Vuli´c, I., and ¨Ust¨un, A. (eds.), Proceedings of the 1st Workshop on Modular a...

    2024

  4. [21]

    URL https://aclanthology.org/ 2024.moomin-1.2/

    Association for Computational Linguistics. URL https://aclanthology.org/ 2024.moomin-1.2/. Qin, G. and Eisner, J. Learning how to ask: Querying lms with mixtures of soft prompts. InProceedings of the 2021 Conference of the North American Chapter of the Association for Computat...

    2024

  5. [22]

    Proceedings of the 11th International Conference on Learning Representations (ICLR); 2023 Mai 1-5; Kigali, Rwanda.International Conference on Learning Represen- tations (ICLR),

    2023

  6. [23]

    Semeval-2017 task 4: Sentiment analysis in twitter

    Rosenthal, S., Farra, N., and Nakov, P. Semeval-2017 task 4: Sentiment analysis in twitter. InProceedings of the 11th international workshop on semantic evaluation (SemEval- 2017), pp. 502–518,

    2017

  7. [24]

    Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter.ArXiv, abs/1910.01108,

    Sanh, V ., Debut, L., Chaumond, J., and Wolf, T. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter.ArXiv, abs/1910.01108,

    1910 arXiv

  8. [25]

    and Sch ¨utze, H

    Schick, T. and Sch ¨utze, H. Few-shot text genera- tion with pattern-exploiting training.arXiv preprint arXiv:2012.11926,

    2012 arXiv

  9. [27]

    L., Wallace, E., and Singh, S

    Shin, T., Razeghi, Y ., Logan IV , R. L., Wallace, E., and Singh, S. Autoprompt: Eliciting knowledge from lan- guage models with automatically generated prompts. In Proceedings of the 2020 Conference on Empirical Meth- ods in Natural Language Processing (EMNLP). Associa- tion ...

    2020

  10. [28]

    Su, Y ., Wang, X., Qin, Y ., Chan, C.-M., Lin, Y ., Wang, H., Wen, K., Liu, Z., Li, P., Li, J., et al

    URL https: //arxiv.org/abs/2408.11796. Su, Y ., Wang, X., Qin, Y ., Chan, C.-M., Lin, Y ., Wang, H., Wen, K., Liu, Z., Li, P., Li, J., et al. On transferability of prompt tuning for natural language processing. InPro- ceedings of the 2022 Conference of the North American Chapt...

    2022 arXiv

  11. [29]

    Touvron, H., Martin, L., Stone, K

    URL https: //openreview.net/forum?id=oZtt0pRnOl. Touvron, H., Martin, L., Stone, K. R., Albert, P., Almahairi, A., Babaei, Y ., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., Bikel, D. M., Blecher, L., Ferrer, C. C., Chen, M., Cucurull, G., Esiobu, D., Fernandes, J., Fu,...

    arXiv

  12. [30]

    Wang, Y ., Wang, J., and Zhang, X

    URL https: //arxiv.org/abs/2403.18955. Wang, Y ., Wang, J., and Zhang, X. Parameter-efficient on- line knowledge distillation for pretrained language mod- els.Expert Systems with Applications, 265:126040,

    arXiv

  13. [32]

    Offsite-tuning: Transfer learn- ing without full model.arXiv preprint arXiv:2302.04870,

    Xiao, G., Lin, J., and Han, S. Offsite-tuning: Transfer learn- ing without full model.arXiv preprint arXiv:2302.04870,

    arXiv

  14. [33]

    Factual probing is [mask]: Learning vs

    Zhong, Z., Friedman, D., and Chen, D. Factual probing is [mask]: Learning vs. learning to recall. InProceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 5017–5033,

    2021

  15. [36]

    The results show that using the same dataset for both private and public sets consistently achieves the best performance

    The datasets fall into three categories: sentiment classification (sst2, imdb, tweet, mpqa), disaster detection (disaster), and topic classification (arisetv, agnews). The results show that using the same dataset for both private and public sets consistently achieves the best ...

    2016

  16. [37]

    reduces the precision of model parameters to lower bit-widths, effectively decreasing model size. However, since our framework involves transmitting the proxy model to the user, quantization primarily reduces storage requirements without adequately protecting the intellectual ...

    2015

  17. [38]

    Knowledge Distillation Setup.We also investigated the best way of performing KD to improve prompt transferability

    The results demonstrate that combining all three loss terms consistently yields the best transfer performance, validating our choice of loss function for KD. Knowledge Distillation Setup.We also investigated the best way of performing KD to improve prompt transferability. In p...

  18. [39]

    Both too few or too many tokens in the soft prompt lead to sub-optimal performance

    We find that a prompt length of 50-100 tokens is a good range, and our method is not very sensitive to the token length. Both too few or too many tokens in the soft prompt lead to sub-optimal performance. Additionally, too many tokens need more computing resources. Table 21:Ab...

    2023

  19. [2006]

    Membership inference attacks from first principles

    Carlini, N., Chien, S., Nasr, M., Song, S., Terzis, A., and Tram `er, F. Membership inference attacks from first principles. In2022 IEEE Symposium on Secu- rity and Privacy (SP), pp. 1897–1914,

    1914

  20. [2015]

    T., Zhang, C., Li, Z., Li, B., and Wang, Z

    Hong, J., Wang, J. T., Zhang, C., Li, Z., Li, B., and Wang, Z. Dp-opt: Make large language model your privacy-preserving prompt engineer. ArXiv, abs/2312.03724,

    arXiv

  21. [2016]

    Han, X., Zhao, W., Ding, N., Liu, Z., and Sun, M

    URL https://arxiv.org/abs/1510.00149. Han, X., Zhao, W., Ding, N., Liu, Z., and Sun, M. Ptr: Prompt tuning with rules for text classification.AI Open, 3:182–192,

    arXiv

  22. [2017]

    URL https://openreview.net/ forum?id=rJqFGTslg. Li, X. and Roth, D. Learning question classifiers. In COLING 2002: The 19th International Conference on Computational Linguistics,

    2002

  23. [2018]

    Kudari, V

    URL https://arxiv.org/abs/1712.05877. Kudari, V . Acled information extraction dataset. Hugging Face,

    arXiv

  24. [2019]

    Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K

    URL https://www.kaggle.com/c/nlp-getting- started. Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. Bert: Pre-training of deep bidirectional transformers for lan- guage understanding.arXiv preprint arXiv:1810.04805,

    arXiv

  25. [2020]

    and Sch¨utze, H

    Schick, T. and Sch¨utze, H. Exploiting cloze-questions for few-shot text classification and natural language infer- ence. InProceedings of the 16th Conference of the Eu- ropean Chapter of the Association for Computational Linguistics: Main Volume, pp. 255–269, 2021a. Schick, T...

    2021

  26. [2021]

    M., Hauth, A., et al

    10 Efficient and Privacy-Preserving Soft Prompt Transfer for LLMs Gemini-Team, Anil, R., Borgeaud, S., Wu, Y ., Alayrac, J.- B., Yu, J., Soricut, R., Schalkwyk, J., Dai, A. M., Hauth, A., et al. Gemini: a family of highly capable multimodal models.arXiv preprint arXiv:2312.11805,

    arXiv

  27. [2022]

    Chiang, W.-L., Li, Z., Lin, Z., Sheng, Y ., Wu, Z., Zhang, H., Zheng, L., Zhuang, S., Zhuang, Y ., Gonzalez, J

    doi: 10.1109/SP46214.2022.9833649. Chiang, W.-L., Li, Z., Lin, Z., Sheng, Y ., Wu, Z., Zhang, H., Zheng, L., Zhuang, S., Zhuang, Y ., Gonzalez, J. E., Stoica, I., and Xing, E. P. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, March

    2022

  28. [2023]

    D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al

    Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. Language models are few-shot learners. Advances in neural information processing systems, 33: 1877–1901,

    1901

  29. [2024]

    Hambardzumyan, K., Khachatrian, H., and May, J

    URL https://openreview.net/ forum?id=5h0qf7IBZZ. Hambardzumyan, K., Khachatrian, H., and May, J. Warp: Word-level adversarial reprogramming. InACL-IJCNLP 2021-59th Annual Meeting of the Association for Com- putational Linguistics and the 11th International Joint Conference on ...

    2021

  30. [2025]

    Hard prompts made easy: Gradient-based discrete optimization for prompt tuning and discovery.ArXiv, abs/2302.03668,

    Wen, Y ., Jain, N., Kirchenbauer, J., Goldblum, M., Geiping, J., and Goldstein, T. Hard prompts made easy: Gradient-based discrete optimization for prompt tuning and discovery.ArXiv, abs/2302.03668,

    arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.