Pith. sign in

REVIEW 5 major objections 6 minor 2 cited by

Leaner Training, Lower Leakage: Revisiting Memorization in LLM Fine-Tuning with LoRA

T0 review · 5 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read LoRA fine-tuning cuts data leakage to near zero while keeping task quality.

desk verdict First systematic LoRA memorization study with a plausible qualitative finding, but the 'near-zero' absolute claim is hostage to a weak extraction attack and missing error bars. read the letter →

arxiv 2506.20856 v1 pith:LSPRX63H submitted 2025-06-25 cs.LG cs.CLcs.CR

Fei Wang , Baochun Li This is my paper
classification cs.LGcs.CLcs.CR
keywords memorizationLoRAlow-rankadaptationfine-tuningdataextractionprivacyplagiarismdetectionsentencesimilarity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that LoRA, a parameter-efficient fine-tuning method, is not only cheaper than full fine-tuning but also memorizes far less of the training data. Using a plagiarism detector and a relaxed sentence-similarity metric, the authors find that full fine-tuning shows rising extractable memorization as model size and data duplication increase, while LoRA and head-only tuning stay near zero. LoRA's generated text peaks at cosine similarities of about 0.4-0.6, below the 0.8 threshold treated as extractable, and its ROUGE and BLEU scores remain close to those of full fine-tuning. If the finding holds, LoRA becomes a privacy-preserving default for fine-tuning on sensitive data.

What carries the argument

The load-bearing object is LoRA (low-rank adaptation), which represents the weight update as $W = W_0 + \alpha BA$, with $B \in \mathbb{R}^{d \times r}$, $A \in \mathbb{R}^{r \times k}$, and $r \ll \min(d,k)$; freezing $W_0$ and training only the small matrices $A$ and $B$ is the mechanism the paper credits for low memorization. The measurement side has two detectors: an automated plagiarism detector that retrieves candidate documents with Elasticsearch and aligns text with the PAN 2014 algorithm to flag verbatim, paraphrase, and idea plagiarism, and a sentence-similarity screen that compares generated samples to the fine-tuning corpus with cosine similarity over 768-dimensional embeddings, treating 0.8 as the extractability threshold. Extraction itself follows the standard black-box attack: seed generation with a start-of-sentence token and sample autoregressively with top-$k$ and top-$p$ truncation.

What would settle it

Run a stronger extraction attack on a LoRA fine-tuned model, such as beam search or corpus-guided prompting with the fine-tuning data as candidate references, and count how many generated sequences exceed 0.8 similarity or match training spans verbatim; if a substantial fraction does, the near-zero claim fails.

Watch

Extended reading notes

Core claim

The central discovery is that fine-tuning strategy changes the memorization trend. Under plagiarism detection, full fine-tuning produces verbatim and paraphrase plagiarism that grows with model scale and with data duplication, reaching paraphrase plagiarism above 50% at duplication level 20 on GPT-2 Small; LoRA and head-only fine-tuning stay at near-zero plagiarism rates across all scales and duplication levels. Under a more permissive sentence-similarity metric, full fine-tuning shifts the distribution of extracted samples toward the 0.8 threshold and beyond, while LoRA keeps the distribution peaked around 0.4-0.5 and, across all tested ranks, scaling factors, dropout rates, and model scales up to Llama 3.1 8B, keeps maximum similarity below 0.8. The paper reads this as evidence that LoRA's low-rank weight update restricts memorization of exact training sequences while still supporting task adaptation. This contradicts the earlier membership-inference result that head-only tuning memorizes most; here head-only behaves like LoRA under the plagiarism and similarity metrics.

Load-bearing premise

The load-bearing premise is that the measurement chain faithfully detects memorization: the plagiarism detector and the 0.8 cosine-similarity threshold are proxies for extractable leakage, and the start-of-sentence sampling attack is a sufficient stand-in for a real adversary.

Editorial extensions

If this is right

  • Practitioners fine-tuning on private data can adopt LoRA as a default that keeps extractable memorization near zero while retaining most task performance.
  • The memorization amplifiers from pre-training, model scale and data duplication, do not carry over to LoRA fine-tuning; duplicated fine-tuning data does not push LoRA past the extraction threshold.
  • LoRA hyperparameter choice is a privacy lever: increasing rank or scaling factor raises similarity slightly, while a moderate dropout rate lowers it without sacrificing utility.
  • The earlier head-only finding is metric-dependent: under MIA-based measures head-only looked riskiest, but under plagiarism and similarity measures it behaves like LoRA, so conclusions about which fine-tuning method leaks depend on the evaluator.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the near-zero result is measured against plagiarism and cosine similarity, so a stronger adversary that uses the fine-tuning corpus as a candidate list, or that optimizes a prompt to maximize likelihood of a target span, could still extract content the current detector misses.
  • Beyond the paper: if the mechanism is the small parameter budget of the adapter, then even lighter adapters such as prompt or prefix tuning should show similar or stronger memorization resistance, a testable comparison the paper does not run.
  • Beyond the paper: planting repeated 'canary' sequences in the fine-tuning set and measuring whether LoRA reproduces them after many epochs would directly test whether the effect is absolute or just below the 0.8 threshold.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper empirically compares post-fine-tuning memorization under full, head-only, and LoRA fine-tuning across GPT-2 model sizes, data duplication levels, and a dialogue task, using a plagiarism-detection metric and a cosine-similarity metric. It reports that full fine-tuning exhibits rising plagiarism and similarity scores with model scale and duplication, while LoRA and head-only fine-tuning remain near zero, and that LoRA hyperparameters such as rank, scaling factor, and dropout modulate a modest similarity shift. The authors conclude that LoRA reduces memorization risk while retaining task utility and position LoRA as a privacy-preserving fine-tuning alternative.

Significance. If the central result holds, this is a practically useful contribution: it is the first systematic study of memorization in LoRA fine-tuning, it uses two complementary metrics, and it varies model scale, duplication, and LoRA hyperparameters while also reporting task utility. The direction of the LoRA-versus-full-FT gap is consistent across GPT-2 model sizes and duplication levels, and the inclusion of ROUGE, BLEU, and readability metrics is a strength. The significance is, however, conditional: the extraction protocol is a single weak adversary, all results are single runs without variance estimates, and the Llama experiments do not include a full-fine-tuning baseline. I agree with the stress-test concern that the SOT-only attack is a genuine blind-spot risk, and I note that the authors themselves concede in Section 7 that stronger attacks could enhance extractability.

major comments (5)
  1. [Sec. 2.1 / 4.2] The extraction attack conditions on only a start-of-sentence token, whereas the memorization definition invoked from Carlini et al. (2021) is prefix-based and requires a prefix that elicits the target sequence. The paper's own Section 7 concedes that stronger attacks, such as those in Zhang et al. (2023) and Kassem et al. (2024), could enhance extractability. Because the 'near-zero plagiarism-based memorization' claim is produced entirely by this SOT-prompted protocol, the conclusion currently conflates 'not extracted by this weak attack' with 'not memorized.' I request a prefix-conditioned extraction test using prompts derived from the fine-tuning corpus or a loss-based targeted extraction method, and a report of whether LoRA's near-zero rates survive such an adversary.
  2. [Figs. 1-5; Tables 1-2] All results appear to come from a single run per configuration; no seeds, error bars, or confidence intervals are reported. The qualitative gap between full fine-tuning and LoRA is large, but the abstract's wording 'significantly reduces' and the conclusion's 'consistently results in' need a measure of run-to-run variability. I ask for at least 3-5 seeds for the main model-size and duplication comparisons, with means and standard deviations, because the duplication resampling itself can introduce variance.
  3. [Sec. 5.3; Figs. 6-8] The Llama experiments contain no full fine-tuning condition; Figs. 6-8 compare only LoRA configurations against the pre-trained model. The abstract's claim that the comparison covers 'the family of GPT-2 and Llama 3 models' is therefore unsupported: the LoRA-versus-full-FT claim is established only on GPT-2. Either add a full-fine-tuning baseline on at least one Llama model, or restrict the central claim to the GPT-2 family.
  4. [Sec. 2.2 vs. Sec. 4.2] The definition of head-only fine-tuning is internally inconsistent and differs from the prior work cited for the 'surprising divergence.' Section 2.2 defines head-only as freezing all layers except the final projection layer, while Section 4.2 states that 'last two transformer layers along with the final language modeling head' are updated. Mireshghallah et al. (2022b), the prior study whose head-only result is contrasted, freeze all transformer layers. The claimed divergence from prior findings is therefore not a like-for-like comparison; please either use the same head-only configuration as the prior work or reframe the claim.
  5. [Sec. 5.1] The 0.8 cosine-similarity threshold is asserted as 'typically used' with no citation or calibration analysis. Since 'extractable memorization' is operationalized as exceeding this threshold, the threshold choice is load-bearing for the near-zero conclusion for LoRA and head-only tuning. If the threshold is miscalibrated, the qualitative claim could be a metric artifact. Please provide a citation for the threshold or run a sensitivity analysis across several thresholds (e.g., 0.6, 0.7, 0.8, 0.9) and show that the LoRA-versus-full-FT ordering is unchanged.
minor comments (6)
  1. [Sec. 2.1 vs. Sec. 4.2] The extraction protocol is described with top-k=40 in Section 2.1 but with top-k=50 in Section 4.2; please clarify which setting produced the reported results and whether the k=40 setting was also tested.
  2. [Sec. 5.3] The duplication factor is defined as a positive integer in Section 4.2, but Section 5.3 uses 'ρ=0'; please state explicitly that ρ=0 means no additional duplication (each unique sample appears once).
  3. [Table 1] The table header is difficult to parse, especially the repeated 'FT' column labels; please reformat so each condition (Before FT, Full FT, Head-only FT, LoRA FT) has one clearly labeled column group.
  4. [Figs. 2 and 3] In the preprint version, legends and labels overlap the plotting area in several panels; adjusting placement would improve readability.
  5. [Sec. 5.1] The term 'extractable memorization' is used in Section 6 as if it were directly measured, but the cosine-similarity metric only scores generated samples against the corpus; please define the term at first use in Section 5.1 to avoid implying that a direct extraction attack was run for the similarity metric.
  6. [General] The paper does not state whether the fine-tuning and evaluation code will be released; providing a repository would substantially aid reproducibility, especially because the plagiarism-detection pipeline has several external components.

Circularity Check

0 steps flagged · score 0.0 of 10

No material circularity: the LoRA-versus-full-FT memorization ordering is an empirical measurement under symmetrically applied metrics; the Section 7 attack-strength caveat is a validity concern, not a circular reduction.

full rationale

No circular step is present in the paper's derivation chain. The central claim — that LoRA fine-tuning exhibits near-zero plagiarism-based memorization and lower training-data similarity than full fine-tuning — is an empirical comparative measurement: the authors fine-tune GPT-2 and Llama models, generate 1,000 samples per model under a fixed extraction protocol (Sections 2.1 and 4.2), and score those samples with the PAN-2014 plagiarism detector (Lee et al. 2023) and with cosine similarity against the fine-tuning corpus at a 0.8 threshold (Section 5.1). Both metrics are applied identically to the pre-trained baseline, full fine-tuning, head-only fine-tuning, and LoRA, so the relative ordering is an outcome of the experiment, not a consequence of any fitted parameter; no parameter is fitted to a subset of the data and then re-reported as a 'prediction.' The LoRA weight update W = W0 + α·BA is a definition quoted from Hu et al. (2022), and it is not used to derive the memorization result. The paper contains no load-bearing self-citations at all (the authors cite no prior work of their own). The genuine weakness in the paper, flagged in Section 7, is the concession that 'it remains unclear whether more advanced data extraction attacks, such as those proposed in Zhang et al. (2023); Kassem et al. (2024), could enhance the extractability of fine-tuning data.' That is a measurement-validity limitation: the start-of-sentence-prompted, top-k/top-p attack may under-detect memorization that a stronger adversary would extract. A weak attack or a miscalibrated threshold is a correctness risk, not a circularity, because the LoRA-versus-full-FT ordering is not guaranteed by the protocol; full fine-tuning in fact produces escalating plagiarism rates and right-shifted similarity distributions under the very same protocol. Hence the derivation is self-contained against the benchmarks it compares, and the finding is a clean non-finding on circularity.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The central comparison relies on the chosen similarity threshold, the extraction decoding settings, and the assumption that the detection tools capture memorization. The paper itself acknowledges the lack of theoretical analysis and the possibility that stronger attacks would change results (Section 7). No new entities are introduced.

free parameters (4)
  • sentence similarity threshold = 0.8
    Chosen by hand as the cutoff for calling a generated sample memorized; the paper does not cite a source or calibrate it, and the percentage of LoRA samples 'exceeding threshold' depends directly on it.
  • decoding hyperparameters = top-k=50, top-p=0.9, temperature=0.8
    Chosen for the extraction attack; no sensitivity analysis shows the results are robust to decoding choices.
  • LoRA default hyperparameters = rank=16, alpha=16, dropout=0.05
    Standard configuration used for the headline LoRA results; later varied in hyperparameter studies, but the central claim is established under this single configuration.
  • fine-tuning schedule = 10 epochs, learning rate 2e-4, batch size 4 per device, sequence length 512
    Single training schedule used for all GPT-2 comparisons; no evidence that the conclusions hold across schedules.
assumptions (6)
  • domain assumption The plagiarism detection framework of Lee et al. (2023) correctly classifies memorized generated text.
    Used in Sections 4.1-4.2 as the primary evidence that LoRA and head-only fine-tuning produce near-zero memorization; if the detector has blind spots, the conclusion overstates safety.
  • domain assumption Sentence embeddings from All-MPNet-Base-V2 and cosine similarity capture semantic memorization, with 0.8 as a meaningful extraction boundary.
    Introduced in Section 5.1; the paper's relaxed-metric conclusions rest on this embedding and threshold.
  • domain assumption Generating from a start-of-sentence token with top-k and top-p sampling is a sufficient proxy for a real data extraction adversary.
    Sections 2.1 and 4.2; the paper itself notes in Limitations that advanced attacks might extract more, so this is a load-bearing simplification.
  • domain assumption The Arxiver dataset (2023 arXiv papers) was not part of GPT-2's pre-training corpus.
    Section 4.2 justifies this by publication date; if any fine-tuning text actually appeared in pre-training, the similarity baselines and post-FT comparisons would be confounded.
  • domain assumption Single-seed measurements are representative enough to compare fine-tuning methods.
    No seeds or repeated runs are reported; all figures show one trajectory per setting.
  • standard math The LoRA update W = W0 + alpha * B * A from Hu et al. (2022) is the correct background model.
    Section 2.2 presents this as standard background for the method.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Leaner Training, Lower Leakage: Revisiting Memorization in LLM Fine-Tuning with LoRA." pith.science (2026). https://pith.science/paper/LSPRX63H

@misc{pith2026250620856,
  author       = {Pith},
  title        = {Pith review of: Leaner Training, Lower Leakage: Revisiting Memorization in LLM Fine-Tuning with LoRA},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LSPRX63H}},
  note         = {Machine review of arXiv:2506.20856}
}
read the original abstract

Memorization in large language models (LLMs) makes them vulnerable to data extraction attacks. While pre-training memorization has been extensively studied, fewer works have explored its impact in fine-tuning, particularly for LoRA fine-tuning, a widely adopted parameter-efficient method. In this work, we re-examine memorization in fine-tuning and uncover a surprising divergence from prior findings across different fine-tuning strategies. Factors such as model scale and data duplication, which strongly influence memorization in pre-training and full fine-tuning, do not follow the same trend in LoRA fine-tuning. Using a more relaxed similarity-based memorization metric, we demonstrate that LoRA significantly reduces memorization risks compared to full fine-tuning, while still maintaining strong task performance.

Figures

Figures reproduced from arXiv: 2506.20856 by the authors.

Figure 1
Figure 1. Verbatim, idea, and paraphrase memorization [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. ROUGE-L scores across various fine-tuning [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The histogram distribution of the top sentence [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: The distribution of the top sentence similarity scores for [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Violin plots depicting the top sentence simi [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 7
Figure 7. Figure 7: A comparison of BLEU, ROUGE-1, ROUGE￾2, and ROUGE-L scores of Llama 3.2 1B Instruct mod￾els and Llama 3.1 8B Instruct models before and after LoRA fine-tuning. 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 Sentence Similarity 0 20 40 60 80 100 Frequency Distribution of Sentence Simi…
Figure 6
Figure 6. Figure 6: BLEU, ROUGE-L, and sentence similarity distribution of fine-tuned Llama 3.2 1B Instruct mod￾els under different LoRA hyperparameter settings on the ChatDoctor dataset. The label “16-16-0.05”, for example, represents the rank r = 16, the scaling factor α = 16, and the d…

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. How Many Bits Can an Adapter Write? Measuring the Capacity and Memorization of Parameter-Efficient Fine-Tuning

    cs.LG 2026-07 conditional novelty 7.0 of 10

    LoRA adapters store roughly 1.7–2.8 bits per trainable parameter, with capacity set by placement and the frozen base's structure rather than parameter count; RL with verifiable rewards writes essentially no memorized ...

  2. LLM Harms: A Taxonomy and Discussion

    cs.CY 2025-12 unverdicted novelty 3.0 of 10

    This paper proposes a taxonomy of LLM harms in five categories and suggests mitigation strategies plus a dynamic auditing system for responsible development.

Reference graph

Works this paper leans on

23 extracted references · 15 canonical work pages · cited by 2 Pith papers

  1. [1]

    Muhammet Hatipoglu Alican Acar, Alara Dirik. 2024. Arxiver. https://huggingface.co/datasets/neuralwork/arxiver

    work page 2024

  2. [2]

    Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. 2023. https://openreview.net/forum?id=TatRHT_1cK Quantifying memorization across neural language models . In The Eleventh International Conference on Learning Representations

    2023

  3. [3]

    Nicholas Carlini, Chang Liu, \' U lfar Erlingsson, Jernej Kos, and Dawn Song. 2019. The secret sharer: Evaluating and testing unintended memorization in neural networks. In Proceedings of the 28th USENIX Conference on Security Symposium, pages 267--284, USA. USENIX Association

    work page 2019

  4. [4]

    Nicholas Carlini, Florian Tram \`e r, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, \'U lfar Erlingsson, Alina Oprea, and Colin Raffel. 2021. https://www.usenix.org/conference/usenixsecurity21/presentation/carlini-extracting Extracting training data from large language models . In 30th USENIX Secur...

    2021

  5. [5]

    Rudolf Franz Flesch. 1948. https://api.semanticscholar.org/CorpusID:39344661 A new readability yardstick. The Journal of Applied Psychology, 32 3:221--33

    work page 1948

  6. [6]

    Clinton Gormley and Zachary Tong. 2015. Elasticsearch: The Definitive Guide, 1st edition. O'Reilly Media, Inc

    work page 2015

  7. [7]

    Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for NLP . In International conference on machine learning, pages 2790--2799. PMLR

    work page 2019

  8. [8]

    Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. https://openreview.net/forum?id=nZeVKeeFYf9 Lo RA : Low-rank adaptation of large language models . In International Conference on Learning Representations

    2022

Show all 23 references
  1. [9]

    Nikhil Kandpal, Eric Wallace, and Colin Raffel. 2022. https://proceedings.mlr.press/v162/kandpal22a.html Deduplicating training data mitigates privacy risks in language models . In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings ...

    2022

  2. [10]

    Aly M Kassem, Omar Mahmoud, Niloofar Mireshghallah, Hyunwoo Kim, Yulia Tsvetkov, Yejin Choi, Sherif Saad, and Santu Rana. 2024. Alpaca against vicuna: Using llms to uncover memorization of llms. arXiv preprint arXiv:2403.04801

    2024 arXiv

  3. [11]

    Jooyoung Lee, Thai Le, Jinghui Chen, and Dongwon Lee. 2023. https://doi.org/10.1145/3543507.3583199 Do language models plagiarize? In Proceedings of the ACM Web Conference 2023, pages 3637--3647. Association for Computing Machinery

    2023

  4. [12]

    Leybzon and Corentin Kervadec

    Danny D. Leybzon and Corentin Kervadec. 2024. https://aclanthology.org/2024.blackboxnlp-1.4/ Learning, forgetting, remembering: Insights from tracking LLM memorization during training . In Proceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks ...

    2024

  5. [13]

    Yunxiang Li, Zihan Li, Kai Zhang, Ruilong Dan, Steve Jiang, and You Zhang. 2023. Chatdoctor: A medical chat model fine-tuned on a large language model meta-ai (llama) using medical domain knowledge. Cureus, 15(6)

    2023

  6. [14]

    Chin-Yew Lin. 2004. https://aclanthology.org/W04-1013/ ROUGE : A package for automatic evaluation of summaries . In Text Summarization Branches Out, pages 74--81. Association for Computational Linguistics

    2004

  7. [15]

    G Harry Mc Laughlin. 1969. SMOG grading-a new readability formula. Journal of reading, 12(8):639--646

    1969

  8. [16]

    Fatemehsadat Mireshghallah, Kartik Goyal, Archit Uniyal, Taylor Berg-Kirkpatrick, and Reza Shokri. 2022 a . https://aclanthology.org/2022.emnlp-main.570/ Quantifying privacy risks of masked language models using membership inference attacks . In Proceedings of the 2022 Confere...

    2022

  9. [17]

    Fatemehsadat Mireshghallah, Archit Uniyal, Tianhao Wang, David Evans, and Taylor Berg-Kirkpatrick. 2022 b . https://aclanthology.org/2022.emnlp-main.119/ An empirical analysis of memorization in fine-tuned autoregressive language models . In Proceedings of the 2022 Conference ...

    2022

  10. [18]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. https://doi.org/10.3115/1073083.1073135 BLEU : A method for automatic evaluation of machine translation . In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, page 311–318. As...

    2002

  11. [19]

    Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language Models are Unsupervised Multitask Learners . OpenAI Blog, 1(8):9

    2019

  12. [20]

    Avi Schwarzschild, Zhili Feng, Pratyush Maini, Zachary Chase Lipton, and J Zico Kolter. 2024. https://openreview.net/forum?id=KFmRMvzAZy Rethinking LLM memorization through the lens of adversarial compression . In The Thirty-eighth Annual Conference on Neural Information Proce...

    2024

  13. [21]

    Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. 15(1):1929–1958

    2014

  14. [22]

    Shenglai Zeng, Yaxin Li, Jie Ren, Yiding Liu, Han Xu, Pengfei He, Yue Xing, Shuaiqiang Wang, Jiliang Tang, and Dawei Yin. 2024. https://doi.org/10.18653/v1/2024.acl-long.216 Exploring memorization in fine-tuned language models . pages 3917--3948, Bangkok, Thailand

    2024 doi

  15. [23]

    Zhexin Zhang, Jiaxin Wen, and Minlie Huang. 2023. https://aclanthology.org/2023.acl-long.709/ ETHICIST : Targeted training data extraction through loss smoothed soft prompting and calibrated confidence estimation . In Proceedings of the 61st Annual Meeting of the Association f...

    2023

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.