Memorization in NLP Fine-tuning Methods
read the original abstract
Large language models are shown to present privacy risks through memorization of training data, and several recent works have studied such risks for the pre-training phase. Little attention, however, has been given to the fine-tuning phase and it is not well understood how different fine-tuning methods (such as fine-tuning the full model, the model head, and adapter) compare in terms of memorization risk. This presents increasing concern as the "pre-train and fine-tune" paradigm proliferates. In this paper, we empirically study memorization of fine-tuning methods using membership inference and extraction attacks, and show that their susceptibility to attacks is very different. We observe that fine-tuning the head of the model has the highest susceptibility to attacks, whereas fine-tuning smaller adapters appears to be less vulnerable to known extraction attacks.
This paper has not been read by Pith yet.
Forward citations
Cited by 5 Pith papers
-
Clinically Grounded Privacy Evaluation of Medical LMs
Presents a clinically grounded privacy evaluation framework for medical LMs that measures verbatim memorization and semantic leakage of diagnoses across tiers of adversarial access, finding high leakage from routine m...
-
Unveiling Privacy Risks in Multi-modal Large Language Models: Task-specific Vulnerabilities and Mitigation Challenges
Introduces MM-Privacy dataset and evaluations showing MLLMs leak sensitive data from images in various tasks, highlighting task inconsistency effects.
-
Private Seeds, Public LLMs: Realistic and Privacy-Preserving Synthetic Data Generation
RPSG generates realistic synthetic replicas of private text by combining private seeds with public LLMs and a formal differential privacy mechanism in candidate selection.
-
Memory-Efficient Differentially Private Training with Gradient Random Projection
DP-GRAPE reduces memory in differentially private neural network training by using random Gaussian projections on gradients instead of SVD, achieving comparable privacy-utility tradeoffs to DP-SGD and scaling to 6.7B ...
-
Vision Language Model Helps Private Information De-Identification in Vision Data
VisShield with OPTIC dataset enables VLMs to localize and mask private text in vision data via instruction tuning for privacy preservation.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.