Impact of Fine-Tuning Methods on Memorization in Large Language Models

Chuxiong Wu; Jie Hou; Lannan Luo; Qiang Zeng

REVIEW 3 major objections 5 minor 1 cited by

Impact of Fine-Tuning Methods on Memorization in Large Language Models

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that prompt-based fine-tuning methods memorize LLM fine-tuning data far less than parameter-based methods while achieving competitive performance, and that this privacy gap grows with model scale.

desk verdict Useful new MIA comparison of parameter- vs prompt-based tuning, but the privacy conclusion runs ahead of the evidence: near-random MIA AUC is not yet shown to mean low memorization. read the letter →

arxiv 2507.00258 v1 pith:OUWFPAW7 submitted 2025-06-30 cs.CL cs.AI

Jie Hou , Chuxiong Wu , Lannan Luo , Qiang Zeng This is my paper

classification cs.CLcs.AI

keywords membershipinferenceattacksmemorizationfine-tuningprompttuningparameter-efficientLoRALLMprivacyprefix

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether the way an LLM is fine-tuned determines how much of the fine-tuning data the model memorizes. It claims that prompt-based methods (prefix tuning, prompt tuning, and P-tuning), which keep weights frozen and learn soft prompts, memorize far less than parameter-based methods (full fine-tuning, head tuning, and LoRA), while matching task performance on validation perplexity. Across model scales from 117M to 7B parameters, membership-inference AUC for prompt-based methods stays near the random-guess range of 0.48 to 0.50, whereas parameter-based methods climb to 0.8 to 0.99. If true, choosing prompt-based fine-tuning is a privacy-preserving alternative when fine-tuning data is sensitive, at the cost of only modest perplexity increases on some tasks.

What carries the argument

The measuring instrument is a set of membership inference attacks (MIAs), which attempt to determine whether a given sample was part of the fine-tuning set; the primary score is the LOSS attack (the sample's loss on the fine-tuned model), calibrated by the reference-based attack that subtracts the loss on the frozen pre-trained model. The explanatory mechanism is the separation of loss distributions: parameter-based tuning lowers the model's loss on training samples relative to non-training samples, while prompt-based tuning mostly re-weights attention through prepended soft tokens and leaves the internal attention distribution nearly unchanged, so membership and non-membership losses overlap. The paper quotes the identity $t^{\text{pt}}_i = A^{\text{pt}}_{i0} W_V S_1 + (1 - A^{\text{pt}}_{i0}) t_i$ from Petrov et al. (2024) to argue that a prefix only shifts attention to different content rather than altering the relative distribution of the input, which is why prompt-based fine-tuning does not induce the representational shift that MIAs detect.

What would settle it

Fine-tune the same base model, such as LLaMA2-7B, on the same data with prefix tuning and with LoRA, then run a verbatim data-extraction attack that prompts each model to continue or complete held-out fine-tuning samples; if the prompt-tuned model reproduces training text at rates close to the LoRA model, the claim that prompt-based methods are privacy-preserving is falsified.

Watch

Extended reading notes

Core claim

The paper establishes that parameter-based fine-tuning methods—full fine-tuning, head tuning, and LoRA—exhibit substantially higher vulnerability to membership inference attacks than prompt-based methods—prefix tuning, prompt tuning, and P-tuning—across three datasets and several open-source models. On LLaMA2-7B, parameter-based methods reach LOSS-attack AUC scores above 0.8 on WebNLG and Wikitext, while prompt-based methods stay between 0.48 and 0.65, close to random guessing. The gap widens with model scale: full fine-tuning's AUC rises from roughly 0.60 on the smallest GPT-2 model to 0.99 on GPT-2 XL and near 1.0 on LLaMA3-1B and LLaMA2-7B, whereas prompt-based methods remain near 0.48 to 0.50 at every scale. The paper also finds that prompt-based methods show no rise in memorization over training epochs and that structured data-to-text tasks like WebNLG produce somewhat higher memorization for prompt-based methods than language modeling or summarization tasks.

Load-bearing premise

The paper's conclusion rests on treating membership-inference AUC as the operational definition of memorization; if a prompt-tuned model can still be induced to reproduce its fine-tuning data even when an MIA cannot distinguish those samples, the privacy benefit would not hold.

Editorial extensions

If this is right

Adopting prompt-based fine-tuning over parameter-based fine-tuning would reduce membership-inference AUC from roughly 0.8-0.99 to 0.48-0.63 on the datasets and models tested, at the price of only slight increases in validation perplexity on some tasks.
Model scale, which sharply amplifies memorization under full fine-tuning and LoRA (AUC reaching 0.99-1.0 at 1B-7B parameters), would no longer be a primary driver of memorization risk if prompt-based methods are used.
Prompt-based fine-tuning is not uniformly safe: on the structured data-to-text task WebNLG, its AUC rises above 0.58, so task type should be considered in privacy decisions.
For LoRA, memorization depends on placement: adapting projection layers, or both attention and projection layers, produces higher MIA scores than adapting attention layers alone.
During training, parameter-based fine-tuning's MIA AUC rises with epochs while prompt-based methods stay flat, so longer training schedules amplify the privacy gap.
The results imply that privacy comparisons of fine-tuning are incomplete unless they hold the tuning paradigm fixed; simply choosing a 'lightweight' parameter method like LoRA is not equivalent to choosing a prompt-based method.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because the reference-based attack uses the frozen pre-trained model as its baseline, part of the reported gap may reflect that prompt-based fine-tuning changes sample losses less than parameter-based fine-tuning; a reference model fine-tuned on a disjoint dataset might reveal a different gap, so the AUC numbers should not be read as an absolute measure of extractable memorization.
The paper's privacy conclusion is only as strong as the MIA proxy; a direct data-extraction test on prompt-tuned models would settle whether 'low memorization' means low extractability, and that test is a natural next experiment not reported here.
The prefix-attention identity suggests a testable prediction: for tasks that require genuinely new attention patterns, prompt-based methods will underperform parameter-based ones, and any privacy advantage may come with a capability ceiling tied to the pre-trained model's existing attention behavior.
In deployment, prompt-based fine-tuning could become the default for privacy-sensitive adaptation when only the prompt is exposed, but the structured-task finding implies that sensitive structured mappings, such as personal data-to-text generation, still need targeted auditing.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is a genuinely new empirical comparison, and the main pattern—parameter-based tuning (FFT, head, LoRA) gives high MIA AUC while prompt-based tuning sits near random—is consistent across the tables. That is worth something. There was no clean prompt-versus-parameter comparison under one MIA suite in the cited literature. The datasets span task structure, they include three model families, and they track AUC over training epochs. I believe the central measurement is plausible.

The soft spots are real but not fatal to the measurement. First, no error bars and no released code; every AUC is a single run. Table 3 conflicts with the Section 6 text on the GPT-2 FFT values (table: 0.6012 to 0.9897; text: 0.8892 to 0.9837), which makes the scale claim harder to trust. Second, "competitive performance" is overstated on Xsum and WebNLG, where prompt methods' PPL is notably worse. Third—and most important—the paper equates MIA AUC with memorization. The Ref attack subtracts the frozen pre-trained baseline; a prompt that reduces loss uniformly on the task distribution can keep member/non-member losses close even if generation would leak specific strings. The paper defers extraction attacks to future work, which is honest, but then the abstract's "more privacy-preserving" conclusion is one step ahead of the evidence. The scale claim in Table 3 uses only the LOSS attack, so it inherits the same gap.

I don't think the stress test torpedoes the paper. Low AUC under four attacks is a legitimate observation, just not the full memorization picture. Before publication the paper needs error bars, corrected table/text numbers, honest performance framing, and either an extraction experiment or a clearly scoped claim that they measure MIA vulnerability, not memorization per se.

This deserves a serious referee—the question is important, the comparison is new, and the flaws are fixable. I would send it to review with a major-revision request, and I would bring it to a privacy reading group as a discussion piece, quoting it as "MIA resistance" rather than "no memorization."

Referee Report

3 major / 5 minor

Summary. This manuscript studies how the choice of fine-tuning method affects memorization of fine-tuning data, measured through four membership-inference attacks (LOSS, Ref, Zlib, Min-K%) and validation perplexity. Experiments on Wikitext, WebNLG, and Xsum with GPT-2 variants, LLaMA2-7B, and LLaMA3-1B report that parameter-based methods (FFT, FT head, LoRA) achieve high attack AUC that increases with model size and training epochs, while prompt-based methods (prefix tuning, prompt tuning, P-tuning) stay near random AUC while achieving competitive validation perplexity. The paper concludes that parameter-based fine-tuning is more prone to leaking private information and that prompt-based fine-tuning is a more privacy-preserving option.

Significance. If the result holds, the paper provides a useful privacy-utility comparison for practitioners choosing among fine-tuning strategies, and the breadth of methods, datasets, and models is a real strength. The consistent direction of the empirical pattern across datasets and attack families is credible, and the limitations paragraph is candid. However, the headline privacy conclusion goes beyond what the chosen metrics can establish: all four MIA scores are loss-based, and the paper explicitly defers data-extraction attacks to future work. The significance therefore depends on an unverified equivalence between resistance to these MIAs and actual non-memorization.

major comments (3)

[§3, §4.3, §7, Observations #1 and #3] The central privacy claim equates low MIA AUC with low memorization. All four attacks are functions of token log-losses, and the Ref score subtracts the frozen pre-trained model's loss. A learned prompt can lower loss uniformly on the task distribution, including for non-members, without changing the relative member/non-member ordering; near-random AUC is therefore compatible with the model retaining extractable training strings in prompt vectors or in generation behavior. Since Section 7 explicitly defers data-extraction attacks to future work, the evidence for 'more privacy-preserving' is incomplete. Please add a generation-based extraction evaluation (e.g., prefix-completion verbatim recall or inserted canary memorization) or restrict the conclusion to 'lower vulnerability to these MIAs'.
[§6, Table 3] The text in Section 6 states that 'the AUC under FFT increases from 0.8892 with the smallest GPT-2 model to 0.9837 with GPT-2 XL,' but Table 3 reports 0.6012 and 0.9897 for the same stated setting (Wikitext, 5 epochs). Since this sentence is used to support Observation 3's scale claim, the discrepancy is load-bearing. Please correct the numbers and ensure that every scale-related claim is read from the same table and checkpoint.
[§4.4, Tables 2 and 3] All AUC values come from a single run, with no error bars, confidence intervals, or significance tests. The main parameter-vs-prompt differences are large, but the paper also makes finer claims, such as 'close to random guessing' for an AUC of 0.6260 and distinctions among prompt methods differing by 0.01-0.03. Please report multiple seeds, bootstrap confidence intervals, or a significance test for the central tables so that the reader can assess the stability of the observations.

minor comments (5)

[§5.2] The equation for prefix tuning, t^pt_i = A^pt_i0 W_V S_1 + (1 - A^pt_i0) t_i, is not self-contained: S_1 and the status of t_i are undefined, and the paraphrase of Petrov et al. (2024) is too loose. Please define all symbols and state the theorem precisely, or remove the equation.
[§6] The text says the results are presented 'across both datasets,' but Table 3 reports only the Wikitext dataset. Please correct this wording or add the second dataset.
[§4.4 and Table 2] Section 4.4 says models are fine-tuned for 15 epochs, while Table 2 reports results after 5 epochs. Please clarify that all AUC tables use the epoch-5 checkpoint and that Figures 1 and 3 show trajectories over epochs.
[§1 and §2] The claim of being 'the first study to systematically investigate' memorization across fine-tuning methods should be softened given that Mireshghallah et al. (2022b), cited elsewhere in the paper, already studies memorization in fine-tuned autoregressive language models; please distinguish the present contribution more carefully.
[§6.1, Observation #4] Observation 4 asserts that prompt-based tuning leads to stronger memorization in structured tasks, but this rests on a single structured dataset (WebNLG). Please either add a second structured task or frame the claim as a hypothesis.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: empirical MIA measurements against externally defined attacks drive the conclusions.

full rationale

The paper's central claims are empirical measurements of membership-inference attack (MIA) AUC on models fine-tuned by different methods. The attacks (LOSS, Ref, Zlib, Min-K%) are defined in prior external work and are not fitted, renamed, or re-derived by the authors. The Ref attack uses a frozen pre-trained model as baseline, but this is the definition from Mireshghallah et al. (2022a), not a construction of the present paper, and the same qualitative pattern appears under the other three attacks. The scale claim in Table 3 uses only the LOSS attack, but LOSS is still an externally defined metric requiring no fitted parameters. The Petrov et al. (2024) citation is invoked only as a post hoc interpretation of observed attention behavior; it is not used to derive the measured AUCs or to select fine-tuning methods. The paper also explicitly lists data extraction attacks as future work in Section 7, which is a limitation in scope rather than a circular step. No self-citations are load-bearing, no uniqueness theorem is imported from the authors, and no prediction reduces by construction to an input or fitted parameter. The conclusion therefore stands as an independent empirical finding, even if one might debate whether MIA resistance fully captures memorization for privacy purposes.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central claim is empirical and depends on no fitted constants. The main unstated costs are the MIA proxy assumption, the comparability of models in the scale study, and the explanatory link to Petrov et al. No new physical or architectural entities are introduced.

free parameters (4)

Epoch count and evaluation checkpoint = 15 epochs trained; results reported at epoch 5
The choice of epoch 5 affects all AUC and PPL comparisons; no sensitivity analysis across epochs is shown despite AUC rising with epochs for parameter-based methods.
LoRA rank and alpha = r=16, alpha=32
Taken from the LoRA default and applied to query and key projections; the paper itself attributes LoRA's scale-dependent memorization to parameter budget, so this choice is load-bearing for Observation 3.
Virtual token counts for prompt methods = Prefix 5, P-tuning 20, Prompt tuning 8
Fixed per the original papers; prompt capacity could plausibly change memorization and was not varied.
Learning rate and optimizer schedule = AdamW, lr 5e-5, linear scheduler
Identical across all methods; no tuning or per-method adjustment, so the comparison is tied to this single schedule.

assumptions (3)

domain assumption MIA AUC on a balanced train/test sample is a valid proxy for memorization of fine-tuning data.
Sections 3 and 4.3 define memorization through MIA AUC. MIAs measure distinguishability, which can arise from overfitting or task adaptation and does not directly measure extractable verbatim memorization.
domain assumption The models in the scale study are comparable because they share architecture and pre-training data.
Section 6 states this to isolate model scale, but GPT-2 and LLaMA families differ in architecture and pre-training corpora, so the scale trend is partly confounded.
ad hoc to paper Petrov et al. (2024) theorem that prefixes redistribute attention without changing relative input distribution implies low memorization for prompt-based tuning.
Section 5.2 uses this cited theorem as the formal foundation of its explanation; the step from attention redistribution to MIA vulnerability is asserted rather than proven in this paper.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Impact of Fine-Tuning Methods on Memorization in Large Language Models." pith.science (2026). https://pith.science/paper/OUWFPAW7

@misc{pith2026250700258,
  author       = {Pith},
  title        = {Pith review of: Impact of Fine-Tuning Methods on Memorization in Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OUWFPAW7}},
  note         = {Machine review of arXiv:2507.00258}
}

read the original abstract

As the capabilities of pre-trained large language models (LLMs) continue to advance, the "pre-train and fine-tune" paradigm has become increasingly mainstream, leading to the development of various fine-tuning methods. However, the privacy risks arising from memorization during fine-tuning have received relatively little attention. To address this gap, we categorize popular fine-tuning approaches and assess their impact on memorization through the lens of membership inference attacks (MIAs). Our results show that, compared to parameter-based fine-tuning, prompt-based fine-tuning achieves competitive performance while exhibiting lower vulnerability to MIAs. Furthermore, prompt-based methods maintain low memorization regardless of model scale. These findings suggest that parameter-based fine-tuning is more prone to leaking private information, whereas prompt-based fine-tuning serves as a more privacy-preserving option.

Figures

Figures reproduced from arXiv: 2507.00258 by the authors.

**Figure 1.** Performance of the LOSS attack against LLaMA2-7B on WebNLG, Wikitext, and Xsum. than that of FT head, yet its AUC remains between 0.55 and 0.65, close to random guessing. Observation ♯2 Parameter-based fine-tuning exhibits increasing memorization over training epochs, while prompt-based fine-tuning maintains consistently low memorization throughout training. We further analyze how memorization evolves during traini… view at source ↗

**Figure 2.** Loss distributions of membership and non [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** Validation PPL against LLaMA2-7B on WebNLG, Wikitext and Xsum. does model size influence memorization under different fine-tuning strategies? Observation ♯3 Model size significantly enhances memorization in parameter-based fine-tuning methods, while prompt-based methods show minimal sensitivity and maintain consistently low memorization. To systematically investigate the impact of different pre-trained model siz… view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

What do Reward Models Memorize?
cs.LG 2026-07 conditional novelty 7.0 of 10

Counterfactual memorization maps show RMs misallocate capacity to easy pairs, memorize dataset artifacts, and overgeneralize length/compliance on unseen pairs.

Reference graph

Works this paper leans on

26 extracted references · 8 canonical work pages · cited by 1 Pith paper

[1]

Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramèr. 2022. https://doi.org/10.1109/SP46214.2022.9833649 Membership inference attacks from first principles . In 2022 IEEE Symposium on Security and Privacy (SP), pages 1897--1914

arXiv 2022
[2]

Nicholas Carlini, Chang Liu, \'U lfar Erlingsson, Jernej Kos, and Dawn Song. 2019. https://www.usenix.org/conference/usenixsecurity19/presentation/carlini The secret sharer: Evaluating and testing unintended memorization in neural networks . In 28th USENIX Security Symposium (USENIX Security 19), pages 267--284, Santa Clara, CA. USENIX Association

work page 2019
[3]

Nicholas Carlini, Florian Tram \`e r, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, \'U lfar Erlingsson, Alina Oprea, and Colin Raffel. 2021. https://www.usenix.org/conference/usenixsecurity21/presentation/carlini-extracting Extracting training data from large language models . In 30th USENIX Secur...

work page 2021
[4]

Li Dong, Nan Yang, Wenhui Wang, Furu Wei, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou, and Hsiao-Wuen Hon. 2019. https://proceedings.neurips.cc/paper/2019/hash/c20bb2d9a50d5ac1f713f8b34d9aac5a-Abstract.html Unified language model pre-training for natural language understanding and generation . In NeurIPS, pages 13042--13054

work page 2019
[5]

Claire Gardent, Anastasia Shimorina, Shashi Narayan, and Laura Perez-Beltrachini. 2017. https://doi.org/10.18653/v1/P17-1017 Creating training corpora for NLG micro-planners . In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 179--188, Vancouver, Canada. Association for Computational ...

doi:10.18653/v1/p17-1017 2017
[6]

Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen - Zhu, Yuanzhi Li, Shean Wang, and Weizhu Chen

Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen - Zhu, Yuanzhi Li, Shean Wang, and Weizhu Chen. 2021. https://arxiv.org/abs/2106.09685 Lora: Low-rank adaptation of large language models . CoRR, abs/2106.09685

arXiv 2021
[7]

Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. https://doi.org/10.18653/v1/2021.emnlp-main.243 The power of scale for parameter-efficient prompt tuning . In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3045--3059, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics

doi:10.18653/v1/2021.emnlp-main.243 2021
[8]

Xiang Lisa Li and Percy Liang. 2021. https://doi.org/10.18653/v1/2021.acl-long.353 Prefix-tuning: Optimizing continuous prompts for generation . In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 4582--4597, Onl...

doi:10.18653/v1/2021.acl-long.353 2021

Show all 26 references

[9]

Xiao Liu, Yanan Zheng, Zhengxiao Du, Ming Ding, Yujie Qian, Zhilin Yang, and Jie Tang. 2021. https://arxiv.org/abs/2103.10385 Gpt understands, too . Preprint, arXiv:2103.10385

2021 arXiv
[10]

Justus Mattern, Fatemehsadat Mireshghallah, Zhijing Jin, Bernhard Schoelkopf, Mrinmaya Sachan, and Taylor Berg-Kirkpatrick. 2023. https://doi.org/10.18653/v1/2023.findings-acl.719 Membership inference attacks against language models via neighbourhood comparison . In Findings o...

2023 doi
[11]

Matthieu Meeus, Shubham Jain, Marek Rei, and Yves-Alexandre de Montjoye. 2024. https://www.usenix.org/conference/usenixsecurity24/presentation/meeus Did the neurons read your book? document-level membership inference for large language models . In 33rd USENIX Security Symposiu...

2024
[12]

Kevin Meng, David Bau, Alex J Andonian, and Yonatan Belinkov. 2022. https://openreview.net/forum?id=-h6WAS6eE4 Locating and editing factual associations in GPT . In Advances in Neural Information Processing Systems

2022
[13]

Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2017. https://openreview.net/forum?id=Byj72udxe Pointer sentinel mixture models . In International Conference on Learning Representations

2017
[14]

Fatemehsadat Mireshghallah, Kartik Goyal, Archit Uniyal, Taylor Berg-Kirkpatrick, and Reza Shokri. 2022 a . https://doi.org/10.18653/v1/2022.emnlp-main.570 Quantifying privacy risks of masked language models using membership inference attacks . In Proceedings of the 2022 Confe...

2022 doi
[15]

Fatemehsadat Mireshghallah, Archit Uniyal, Tianhao Wang, David Evans, and Taylor Berg-Kirkpatrick. 2022 b . https://doi.org/10.18653/v1/2022.emnlp-main.119 An empirical analysis of memorization in fine-tuned autoregressive language models . In Proceedings of the 2022 Conferenc...

2022 doi
[16]

Cohen, and Mirella Lapata

Shashi Narayan, Shay B. Cohen, and Mirella Lapata. 2018. https://doi.org/10.18653/v1/D18-1206 Don`t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization . In Proceedings of the 2018 Conference on Empirical Methods in Natura...

2018 doi
[17]

Aleksandar Petrov, Philip Torr, and Adel Bibi. 2024. https://openreview.net/forum?id=JewzobRhay When do prompting and prefix-tuning work? a theory of capabilities and limitations . In The Twelfth International Conference on Learning Representations

2024
[18]

Alec Radford and Karthik Narasimhan. 2018. https://api.semanticscholar.org/CorpusID:49313245 Improving language understanding by generative pre-training

2018
[19]

Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. https://api.semanticscholar.org/CorpusID:160025533 Language models are unsupervised multitask learners

2019
[20]

Weijia Shi, Anirudh Ajith, Mengzhou Xia, Yangsibo Huang, Daogao Liu, Terra Blevins, Danqi Chen, and Luke Zettlemoyer. 2024. https://openreview.net/forum?id=zWqr3MQuNs Detecting pretraining data from large language models . In The Twelfth International Conference on Learning Re...

2024
[21]

Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. 2017. https://arxiv.org/abs/1610.05820 Membership inference attacks against machine learning models . In 2017 IEEE symposium on security and privacy (SP), pages 3--18. IEEE

2017 arXiv
[22]

Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton - Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu,...

2023 doi arXiv
[23]

YUTONG WU, Han Qiu, Shangwei Guo, Jiwei Li, and Tianwei Zhang. 2024. https://openreview.net/forum?id=7WsivwyHrS You only query once: An efficient label-only membership inference attack . In The Twelfth International Conference on Learning Representations

2024
[24]

Samuel Yeom, Irene Giacomelli, Matt Fredrikson, and Somesh Jha. 2018. https://doi.org/10.1109/CSF.2018.00027 Privacy Risk in Machine Learning: Analyzing the Connection to Overfitting . In 2018 IEEE 31st Computer Security Foundations Symposium (CSF), pages 268--282, Los Alamito...

2018
[25]

online" 'onlinestring :=

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...
[26]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramèr. 2022. https://doi.org/10.1109/SP46214.2022.9833649 Membership inference attacks from first principles . In 2022 IEEE Symposium on Security and Privacy (SP), pages 1897--1914

arXiv 2022

[2] [2]

Nicholas Carlini, Chang Liu, \'U lfar Erlingsson, Jernej Kos, and Dawn Song. 2019. https://www.usenix.org/conference/usenixsecurity19/presentation/carlini The secret sharer: Evaluating and testing unintended memorization in neural networks . In 28th USENIX Security Symposium (USENIX Security 19), pages 267--284, Santa Clara, CA. USENIX Association

work page 2019

[3] [3]

Nicholas Carlini, Florian Tram \`e r, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, \'U lfar Erlingsson, Alina Oprea, and Colin Raffel. 2021. https://www.usenix.org/conference/usenixsecurity21/presentation/carlini-extracting Extracting training data from large language models . In 30th USENIX Secur...

work page 2021

[4] [4]

Li Dong, Nan Yang, Wenhui Wang, Furu Wei, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou, and Hsiao-Wuen Hon. 2019. https://proceedings.neurips.cc/paper/2019/hash/c20bb2d9a50d5ac1f713f8b34d9aac5a-Abstract.html Unified language model pre-training for natural language understanding and generation . In NeurIPS, pages 13042--13054

work page 2019

[5] [5]

Claire Gardent, Anastasia Shimorina, Shashi Narayan, and Laura Perez-Beltrachini. 2017. https://doi.org/10.18653/v1/P17-1017 Creating training corpora for NLG micro-planners . In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 179--188, Vancouver, Canada. Association for Computational ...

doi:10.18653/v1/p17-1017 2017

[6] [6]

Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen - Zhu, Yuanzhi Li, Shean Wang, and Weizhu Chen

Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen - Zhu, Yuanzhi Li, Shean Wang, and Weizhu Chen. 2021. https://arxiv.org/abs/2106.09685 Lora: Low-rank adaptation of large language models . CoRR, abs/2106.09685

arXiv 2021

[7] [7]

Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. https://doi.org/10.18653/v1/2021.emnlp-main.243 The power of scale for parameter-efficient prompt tuning . In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3045--3059, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics

doi:10.18653/v1/2021.emnlp-main.243 2021

[8] [8]

Xiang Lisa Li and Percy Liang. 2021. https://doi.org/10.18653/v1/2021.acl-long.353 Prefix-tuning: Optimizing continuous prompts for generation . In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 4582--4597, Onl...

doi:10.18653/v1/2021.acl-long.353 2021