Pith. sign in

REVIEW 4 major objections 6 minor 160 references

SoK: The Privacy Paradox of Large Language Models: Advancements, Privacy Risks, and Mitigation

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper argues that a complete account of LLM privacy must cover four interacting risk surfaces—training data, user prompts, generated outputs, and LLM agents—and that the last three are comparatively underexplored and under-mitigated.

desk verdict A useful four-part taxonomy of LLM privacy risks, but the paper's own literature-search accounting is too inconsistent to back the 'comprehensive' claim as written. read the letter →

arxiv 2506.12699 v2 pith:JFR2LVDF submitted 2025-06-15 cs.CR cs.HC

Yashothara Shanmugarasa , Ming Ding , M.A.P Chamikara , Thierry Rakotoarivelo This is my paper
classification cs.CRcs.HC
keywords largelanguagemodelsprivacysystematizationofknowledgepromptLLMagentsmembershipinferenceattacksdifferentialin-contextlearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This systematization-of-knowledge paper tries to establish that LLM privacy cannot be reduced to training-data leakage. It proposes four categories—training data, user prompts, generated outputs, and LLM agents—and argues that privacy risks from user interaction, such as attribute inference from seemingly harmless prompts, regurgitation of prompt data in outputs, and data propagation by agents, are distinct and comparatively underexplored. If the taxonomy holds, privacy research and regulation should widen their focus from model training to the full interaction pipeline. The paper also reviews current mitigations and finds them uneven, with output and agent privacy having the fewest mature defenses.

What carries the argument

The organizing device is a four-part taxonomy of privacy challenges (training data, prompts, outputs, agents) paired with a cause–attack–consequence structure for training data and a mitigation mapping that assigns techniques to specific challenges. The taxonomy does the work: it separates memorization-based risks from inference-based and propagation-based risks, and it reveals where mitigation is thin, namely in outputs and agents.

What would settle it

Run the stated search protocol (post-2022 papers, the listed databases, and the stated exclusion criteria) and compare the recovered corpus with the paper's counts; if the recovered set differs materially, or if a comprehensive search surfaces a privacy risk that does not fit any of the four categories, the taxonomy's completeness claim fails. A simpler check is to resolve the discrepancy among 116 (text), 128 (figure), and 149 (sum of category counts).

Watch

Extended reading notes

Core claim

The paper's central claim is that a complete account of LLM privacy must cover four categories: privacy in training data, privacy in user prompts, privacy in LLM-generated outputs, and privacy involving LLM agents. Existing surveys concentrate on the first; the paper argues that interaction-time risks—attribute inference from innocuous prompts, regurgitation of prompt data in outputs, and data propagation by agents—are distinct and understudied. It then reviews mitigation mechanisms for each category and concludes that current defenses are piecemeal, often relying on trusting the provider or on heuristics that fail against inference.

Load-bearing premise

The survey's map of the field is only as good as its literature search, and that search is reported inconsistently: the text says 116 studies were included, the search figure says 128, and the category counts sum to 149; if the corpus is not representative, the taxonomy and the claim that prompts, outputs, and agents are underexplored may be incomplete.

Editorial extensions

If this is right

  • Privacy risk assessment for LLM systems should treat prompts, outputs, and agents as first-class risk surfaces alongside training data.
  • Existing mitigation techniques are largely siloed; future work should aim for unified and adaptive privacy frameworks rather than isolated fixes.
  • Named-entity recognition and rule-based redaction are insufficient against inference-based privacy attacks, so LLM-based anonymization and local small models become more important.
  • LLM agents introduce data-persistence and third-party propagation risks that require separate safeguards such as access restriction and sandboxed execution.
  • Fine-tuning, in-context learning, and retrieval-augmented generation with small private datasets are highlighted as underexplored privacy-critical settings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The four-category taxonomy could be extended to multimodal models, where the boundaries between prompts, outputs, and agents blur because inputs and outputs include images, audio, and sensor data.
  • The paper's title suggests a capability–privacy tension: more helpful LLMs may require more personal context, a trade-off that could be formalized as a utility–privacy frontier.
  • A testable extension would be to build benchmark suites, similar to agent risk benchmarks, that measure prompt-level and output-level privacy leakage in addition to agent safety.
  • If LLM-based anonymization becomes a standard defense, it may trigger an arms race with stronger inference models, suggesting that formal guarantees like differential privacy may eventually be needed rather than heuristic obfuscation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This SoK paper maps privacy risks in large language models into four categories: privacy in training data, privacy in user prompts, privacy in LLM-generated outputs, and privacy in LLM agents. For each category, it surveys attack mechanisms and mitigation techniques, and it closes with research directions and an appendix describing a PRISMA-based literature selection. The paper claims to fill a gap left by prior surveys that focus mainly on training-data privacy, arguing that prompt-, output-, and agent-level privacy are underexplored.

Significance. If the underlying corpus is representative, the four-category taxonomy is a useful organizing device for a fast-moving field, and the synthesis of mitigation work on prompts, outputs, and agents is a genuine contribution that goes beyond earlier training-data-centric surveys. The paper is generally faithful to the cited literature in its qualitative descriptions of attacks and defenses, and the explicit PRISMA protocol is a strength in principle. However, the paper's central 'comprehensive map' claim is only as strong as the literature-selection accounting, and that accounting is currently internally inconsistent and not reproducible from the text.

major comments (4)
  1. [Appendix 6.1 / Figure 5] The PRISMA accounting is internally inconsistent and must be reconciled. The text reports 101 records after screening and then states that the study comprised 116 papers; Figure 5 reports 113 records after screening plus 15 snowballed records, yielding 128 included studies; and the category counts in the same text (51+24+8+24+9+14+12+7) sum to 149. These numbers cannot all be correct. If categories overlap, the overlap must be disclosed; if they are exclusive, the sum must equal the total. As written, a reader cannot determine the corpus size, and the 'comprehensive' and 'underexplored' claims are not checkable.
  2. [Appendix 6.1] The exclusion criteria are not operationalized and appear inconsistent with the reference list. The paper states that it excluded papers with no citations or published in non-A/A*/B venues, yet the reference list is dominated by arXiv preprints. Either the venue criterion is applied more loosely than stated, or the corpus is biased toward a particular subcommunity. The paper also excludes 'technical papers focused on root techniques with modifications,' a criterion that is too vague to reproduce. The authors should provide the complete list of included studies and justify how each exclusion criterion was applied.
  3. [Section 1.4] The gap claim is self-contradictory. The text says 'we could not identify any previous surveys or SoKs related to the privacy challenges during LLM deployment and user interaction,' but in the same section it cites [31] as 'a timely and highly relevant overview' of privacy-preserving prompt engineering, which is a survey of prompt privacy. The authors need to clarify how [31] differs from their contribution or revise the novelty claim.
  4. [Sections 2.2-2.4, 3.2-3.4, and Appendix 6.1] The assignment of papers to the four categories is not governed by an explicit coding protocol, and the boundaries are unclear. For example, leakage of user data through in-context learning is discussed both as a prompt-privacy issue (Section 3.2.3) and as an output-privacy issue (Section 3.3.1), while agent-related exposure to third-party tools is separated from output privacy only by the agent's autonomy (Section 3.4.3). Without a coding rule or a statement of whether a paper can appear in multiple categories, the category counts and the taxonomy itself are not independently verifiable.
minor comments (6)
  1. [Title / Section 1] The phrase 'Privacy Paradox' in the title is never defined or discussed in the body; consider adding a brief definition or removing the term to avoid implying a specific behavioral or economic phenomenon.
  2. [Section 3.2.1 / Reference [68]] The paper refers to 'EmojiCrypt' in the text, but the reference list entry [68] is titled 'PromptCrypt'; please align the name used in the text with the actual cited work.
  3. [Table 2 caption] The table legend refers to icons for open/closed-source systems and threat models, but the symbols are not visible in the extracted text; please verify that the legend renders correctly in the final PDF.
  4. [Section 1.6] The scope statement says the paper deliberately does not delve extensively into training-data privacy, yet Section 3.1 is a substantial treatment of training-data attacks and mitigations; the scope description should be adjusted to reflect the actual coverage.
  5. [Section 2.2, footnote 1] The claim about The New York Times reporting exposure of chatbot conversations and login credentials is attributed to a footnote about the Italy ban on ChatGPT; please verify that the cited article supports the specific claim, or replace the citation.
  6. [Reference list] Several references are incomplete or informal, e.g., [32] cites a tweet as evidence for real-world agent data collection and [7] includes 'et al' in the author list; these should be cleaned up before publication.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the taxonomy is an inductive literature summary with no self-citations, fitted parameters, or derivation returning to its own input.

full rationale

This paper is a SoK survey, not a derivation chain. Its central contribution is the four-category taxonomy (training data, prompts, outputs, and agents) presented in Section 2 as 'Our comprehensive analysis of the literature reveals four main categories of privacy challenges,' supported by external references such as [36, 80], [68, 146], [86, 124], and [39, 109]. The categories are inductive summaries of cited work, not quantities defined in terms of one another, so no self-definitional loop is present. No parameter is fitted, no quantity is predicted from a calibrated input, and no prior result by the same authors is invoked: the author list of this paper does not overlap with any cited work in the reference list, so the self-citation patterns that would make a result forced by citation are absent. The Appendix 6.1 PRISMA accounting is internally inconsistent (the text reports 116 papers, Figure 5 reports 128, and the listed category counts sum to 149), and the exclusion criteria are not fully justified; however, this is a reproducibility and validity flaw, not circularity. An inconsistent or non-representative corpus can make the 'underexplored' claim unsupported, but it does not make the taxonomy definitionally equivalent to its inputs. Similarly, Section 1.4 cites [31] as a 'timely and highly relevant overview' of prompt privacy while later claiming that no previous surveys cover user interaction and deployment; this is a gap-claim inconsistency worth flagging for correctness, but it is not a circular reduction. Because the paper is self-contained against external benchmarks and makes no prediction that reduces by construction to its own inputs, the appropriate finding is no significant circularity.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

As a SoK, the paper adds no fitted parameters and no invented entities. Its load-bearing inputs are empirical claims from cited papers plus a methodological assumption that the PRISMA selection yielded a representative corpus. The taxonomy's exhaustiveness is assumed, not demonstrated.

assumptions (4)
  • domain assumption LLMs can memorize and reproduce verbatim sensitive data from training corpora.
    Invoked in Section 3.1.1 and cited to [20, 80, 128]; if false, the training-data privacy risk category collapses.
  • domain assumption LLMs' contextual inference capabilities can deduce private attributes from innocuous prompts.
    Invoked in Section 3.2.2 and based on [107, 108]; underpins the prompt-inference risk category.
  • ad hoc to paper Privacy risks in LLM systems are exhaustively partitionable into training data, prompts, outputs, and agents.
    The four-category taxonomy is the paper's organizing claim; no proof of exhaustiveness is provided.
  • ad hoc to paper The PRISMA search and exclusion criteria yielded a representative sample of the relevant literature.
    The paper's conclusions about research gaps depend on this; Appendix 6.1 describes the selection but does not provide search strings or the inclusion/exclusion list.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SoK: The Privacy Paradox of Large Language Models: Advancements, Privacy Risks, and Mitigation." pith.science (2026). https://pith.science/paper/JFR2LVDF

@misc{pith2026250612699,
  author       = {Pith},
  title        = {Pith review of: SoK: The Privacy Paradox of Large Language Models: Advancements, Privacy Risks, and Mitigation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JFR2LVDF}},
  note         = {Machine review of arXiv:2506.12699}
}
read the original abstract

Large language models (LLMs) are sophisticated artificial intelligence systems that enable machines to generate human-like text with remarkable precision. While LLMs offer significant technological progress, their development using vast amounts of user data scraped from the web and collected from extensive user interactions poses risks of sensitive information leakage. Most existing surveys focus on the privacy implications of the training data but tend to overlook privacy risks from user interactions and advanced LLM capabilities. This paper aims to fill that gap by providing a comprehensive analysis of privacy in LLMs, categorizing the challenges into four main areas: (i) privacy issues in LLM training data, (ii) privacy challenges associated with user prompts, (iii) privacy vulnerabilities in LLM-generated outputs, and (iv) privacy challenges involving LLM agents. We evaluate the effectiveness and limitations of existing mitigation mechanisms targeting these proposed privacy challenges and identify areas for further research.

Figures

Figures reproduced from arXiv: 2506.12699 by the authors.

Figure 1
Figure 1. A Multi-Faceted View of the Four Identified Privacy Challenges in LLMs [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. The Workflow of LLM Agents The transition from text-based interactions to those involving sensitive data with agents operating in the real world raises signifi￾cant privacy concerns [99]. For example, physical interaction-based LLM agents heighten privacy risks by collecting real-world data such as facial photos (cameras) and speech audio (microphones) [32]. A significant challenge in such an agent-based system is t… view at source ↗
Figure 3
Figure 3. The Privacy Challenges in LLM Training Data [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Summary of Privacy Challenges in LLM Systems and Mitigation Techniques in the Literature [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 5
Figure 5. Figure 5: PRISMA and Snowballing Approach for Paper [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

160 extracted references · 13 canonical work pages

  1. [31]

    Kennedy Edemacu and Xintao Wu. 2024. Privacy Preserving Prompt Engineer- ing: A Survey. arXiv preprint arXiv:2404.06001 (2024)

    arXiv 2024

  2. [1]

    Accountability Act. 1996. Health insurance portability and accountability act of 1996. Public law 104 (1996), 191

    1996

  3. [2]

    UK AI Safety Institute. 2024. Inspect AI: Framework for Large Language Model Evaluations. UK AI Safety Institute. https://github.com/UKGovernmentBEIS/ inspect_ai

    2024

  4. [3]

    Kanwal, Tegan Maharaj, Asja Fischer, Aaron Courville, Yoshua Bengio, and Simon Lacoste-Julien

    Devansh Arpit, Stanisław Jastrzundefinedbski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder S. Kanwal, Tegan Maharaj, Asja Fischer, Aaron Courville, Yoshua Bengio, and Simon Lacoste-Julien. 2017. A closer look at memorization in deep networks. In Proceedings of the 34th International Con- ference on Machine Learning - Volume 70 (Sydney, NSW, Au...

    2017

  5. [4]

    Shubhi Asthana, Ruchi Mahindru, Bing Zhang, and Jorge Sanz. 2025. Adap- tive PII Mitigation Framework for Large Language Models. arXiv preprint arXiv:2501.12465 (2025)

    arXiv 2025

  6. [5]

    Eugene Bagdasaryan, Ren Yi, Sahra Ghalebikesabi, Peter Kairouz, Marco Gruteser, Sewoong Oh, Borja Balle, and Daniel Ramage. 2024. Air Gap: Protect- ing Privacy-Conscious Conversational Agents. arXiv preprint arXiv:2405.05175 (2024)

    arXiv 2024

  7. [7]

    Yuntao Bai, Saurav Kadavath, Sandipan Kundu, and Amanda Askell et al. 2022. Constitutional AI: Harmlessness from AI Feedback. arXiv:2212.08073 [cs.CL] https://arxiv.org/abs/2212.08073

    arXiv 2022

  8. [8]

    Borja Balle, Giovanni Cherubin, and Jamie Hayes. 2022. Reconstructing training data with informed adversaries. In2022 IEEE Symposium on Security and Privacy (SP). IEEE, 1138–1156

    2022

Show all 160 references
  1. [9]

    Mislav Balunovic, Dimitar Dimitrov, Nikola Jovanović, and Martin Vechev. 2022. Lamp: Extracting text from gradients with language model priors. Advances in Neural Information Processing Systems 35 (2022), 7641–7654

    2022

  2. [10]

    Dipto Barman, Ziyi Guo, and Owen Conlan. 2024. The dark side of language models: Exploring the potential of llms in multimedia disinformation generation and dissemination. Machine Learning with Applications (2024), 100545

    2024

  3. [11]

    Kristian González Barman, Nathan Wood, and Pawel Pawlowski. 2024. Beyond transparency and explainability: on the need for adequate and contextualized user guidelines for LLM use. Ethics and Information Technology 26, 3 (2024), 47

    2024

  4. [12]

    Rouzbeh Behnia, Mohammadreza Reza Ebrahimi, Jason Pacheco, and Balaji Padmanabhan. 2022. Ew-tune: A framework for privately fine-tuning large language models with differential privacy. In2022 IEEE International Conference on Data Mining Workshops (ICDMW) . IEEE, 560–566

    2022

  5. [13]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901

    2020

  6. [14]

    Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert- Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. 2021. Extracting training data from large language models. In30th USENIX security symposium (USENIX Security 21) ...

    2021

  7. [15]

    Ting-Yun Chang, Jesse Thomason, and Robin Jia. 2023. Do Localization Methods Actually Localize Memorized Data in LLMs? arXiv preprint arXiv:2311.09060 (2023)

    2023 arXiv

  8. [16]

    Tianyu Chen, Hangbo Bao, Shaohan Huang, Li Dong, Binxing Jiao, Daxin Jiang, Haoyi Zhou, Jianxin Li, and Furu Wei. 2022. The-x: Privacy-preserving trans- former inference with homomorphic encryption.arXiv preprint arXiv:2206.00216 (2022)

    2022 arXiv

  9. [17]

    Yu Chen, Tingxin Li, Huiming Liu, and Yang Yu. 2023. Hide and Seek (HaS): A Lightweight Framework for Prompt Privacy Protection. arXiv:2309.03057 [cs.CR]

    2023 arXiv

  10. [18]

    Steffi Chern, Zhen Fan, and Andy Liu. 2024. Combating Adversarial Attacks with Multi-Agent Debate. arXiv preprint arXiv:2401.05998 (2024)

    2024 arXiv

  11. [19]

    Junjie Chu, Zeyang Sha, Michael Backes, and Yang Zhang. 2024. Conversation Reconstruction Attack Against GPT Models. arXiv preprint arXiv:2402.02987 (2024)

    2024 arXiv

  12. [20]

    Badhan Chandra Das, M Hadi Amini, and Yanzhao Wu. 2024. Security and privacy challenges of large language models: A survey. arXiv preprint arXiv:2402.00888 (2024)

    2024 arXiv

  13. [21]

    Leo de Castro, Antigoni Polychroniadou, and Daniel Escudero. 2024. Privacy- Preserving Large Language Model Inference via GPU-Accelerated Fully Homo- morphic Encryption. In Neurips Safe Generative AI Workshop 2024

    2024

  14. [22]

    Jieren Deng, Yijue Wang, Ji Li, Chao Shang, Hang Liu, Sanguthevar Rajasekaran, and Caiwen Ding. 2021. Tag: Gradient attack on transformer-based language models. arXiv preprint arXiv:2103.06819 (2021)

    2021 arXiv

  15. [23]

    Sentry Developers. [n. d.]. PII and Data Scrubbing — develop.sentry.dev. https: //develop.sentry.dev/pii. [Accessed 24-04-2024]

    2024

  16. [24]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv:1810.04805 [cs.CL] https://arxiv.org/abs/1810.04805

    2019 arXiv

  17. [25]

    Peng Ding, Jun Kuang, Dan Ma, Xuezhi Cao, Yunsen Xian, Jiajun Chen, and Shujian Huang. 2023. A Wolf in Sheep’s Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily.arXiv preprint arXiv:2311.08268 (2023)

    2023 arXiv

  18. [26]

    Xinshuai Dong, Anh Tuan Luu, Min Lin, Shuicheng Yan, and Hanwang Zhang

  19. [27]

    Wenyu Du, Tongxu Luo, Zihan Qiu, Zeyu Huang, Yikang Shen, Reynold Cheng, Yike Guo, and Jie Fu. 2024. Stacking Your Transformers: A Closer Look at Model Growth for Efficient LLM Pre-Training. arXiv preprint arXiv:2405.15319 (2024)

    2024 arXiv

  20. [28]

    Haonan Duan, Adam Dziedzic, Mohammad Yaghini, Nicolas Papernot, and Franziska Boenisch. 2023. On the privacy risk of in-context learning. In The 61st Annual Meeting Of The Association For Computational Linguistics

    2023

  21. [29]

    Michael Duan, Anshuman Suri, Niloofar Mireshghallah, Sewon Min, Weijia Shi, Luke Zettlemoyer, Yulia Tsvetkov, Yejin Choi, David Evans, and Hannaneh Hajishirzi. 2024. Do Membership Inference Attacks Work on Large Language Models? arXiv preprint arXiv:2402.07841 (2024)

    2024 arXiv

  22. [30]

    Travis Dyde. 2023. Documentation on the emergence, current iterations, and possible future of Artificial Intelligence with a focus on Large Language Models. (2023)

    2023

  23. [32]

    Ekenstam

    L. Ekenstam. 2023. Linus on x: "if you think this is creepy. . . you should know what all major stores in the world knows about you as a shopper they use in-store, online, cell-tower, and more to keep track of everything. . . everything. cambridge analytics is a joke in compar...

    2023

  24. [33]

    Ronen Eldan and Mark Russinovich. 2023. Who’s Harry Potter? Approximate Unlearning in LLMs. arXiv:2310.02238 [cs.CL]

    2023 arXiv

  25. [34]

    Tao Fan, Yan Kang, Guoqiang Ma, Weijing Chen, Wenbin Wei, Lixin Fan, and Qiang Yang. 2023. Fate-llm: A industrial grade federated learning framework for large language models. arXiv preprint arXiv:2310.10049 (2023)

    2023 arXiv

  26. [35]

    Richard Fang, Rohan Bindu, Akul Gupta, and Daniel Kang. 2024. LLM Agents can Autonomously Exploit One-day Vulnerabilities. arXiv preprint arXiv:2404.08144 (2024)

    2024 arXiv

  27. [36]

    Vitaly Feldman. 2020. Does learning require memorization? a short tale about a long tail. In Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing. 954–959. Yashothara Shanmugarasa, Ming Ding, M.A.P. Chamikara, and Thierry Rakotoarivelo

    2020

  28. [37]

    Wenjie Fu, Huandong Wang, Chen Gao, Guanghua Liu, Yong Li, and Tao Jiang

  29. [38]

    Neil Zhenqiang Gong and Bin Liu. 2018. Attribute inference attacks in online social networks. ACM Transactions on Privacy and Security (TOPS) 21, 1 (2018), 1–30

    2018

  30. [39]

    Xiangming Gu, Xiaosen Zheng, Tianyu Pang, Chao Du, Qian Liu, Ye Wang, Jing Jiang, and Min Lin. 2024. Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fast. arXiv preprint arXiv:2402.08567 (2024)

    2024 arXiv

  31. [40]

    Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A Smith. 2020. Don’t stop pretraining: Adapt language models to domains and tasks. arXiv preprint arXiv:2004.10964 (2020)

    2020 arXiv

  32. [41]

    David Haber. 2024. Introducing Lakera Guard – Bringing Enterprise-Grade Security to LLMs with One Line of Code | Lakera – Protecting AI teams that dis- rupt the world. — lakera.ai. https://www.lakera.ai/blog/lakera-guard-overview. [Accessed 15-05-2024]

    2024

  33. [42]

    Meng Hao, Hongwei Li, Hanxiao Chen, Pengzhi Xing, Guowen Xu, and Tianwei Zhang. 2022. Iron: Private inference on transformers. Advances in neural information processing systems 35 (2022), 15718–15731

    2022

  34. [43]

    Florian Hartmann, Duc-Hieu Tran, Peter Kairouz, Victor Cărbune, et al. 2024. Can LLMs get help from other LLMs without revealing private information? arXiv preprint arXiv:2404.01041 (2024)

    2024 arXiv

  35. [44]

    Wenyue Hua, Xianjun Yang, Zelong Li, Cheng Wei, and Yongfeng Zhang. 2024. TrustAgent: Towards Safe and Trustworthy LLM-based Agents through Agent Constitution. arXiv preprint arXiv:2402.01586 (2024)

    2024 arXiv

  36. [45]

    Daphne Ippolito, Florian Tramèr, Milad Nasr, Chiyuan Zhang, Matthew Jagielski, Katherine Lee, Christopher A Choquette-Choo, and Nicholas Carlini. 2022. Preventing verbatim memorization in language models gives a false sense of privacy. arXiv preprint arXiv:2210.17546 (2022)

    2022 arXiv

  37. [46]

    Abhyuday Jagannatha, Bhanu Pratap Singh Rawat, and Hong Yu. 2021. Member- ship inference attack susceptibility of clinical language models. arXiv preprint arXiv:2104.08305 (2021)

    2021 arXiv

  38. [47]

    Matthew Jagielski, Om Thakkar, Florian Tramer, Daphne Ippolito, Katherine Lee, Nicholas Carlini, Eric Wallace, Shuang Song, Abhradeep Thakurta, Nicolas Papernot, et al. 2022. Measuring forgetting of memorized training examples. arXiv preprint arXiv:2207.00099 (2022)

    2022 arXiv

  39. [48]

    Tanqiu Jiang, Zian Wang, Jiacheng Liang, Changjiang Li, Yuhui Wang, and Ting Wang. 2025. RobustKV: Defending Large Language Models against Jailbreak Attacks via KV Eviction. In The Thirteenth International Conference on Learning Representations. https://openreview.net/forum?id...

    2025

  40. [49]

    Junfeng Jiao, Saleh Afroogh, Yiming Xu, and Connor Phillips. 2024. Navigating llm ethics: Advancements, challenges, and future directions. arXiv preprint arXiv:2406.18841 (2024)

    2024 arXiv

  41. [50]

    Nikhil Kandpal, Krishna Pillutla, Alina Oprea, Peter Kairouz, Christopher A Choquette-Choo, and Zheng Xu. 2023. User inference attacks on large language models. arXiv preprint arXiv:2310.09266 (2023)

    2023 arXiv

  42. [51]

    Nikhil Kandpal, Eric Wallace, and Colin Raffel. 2022. Deduplicating training data mitigates privacy risks in language models. In International Conference on Machine Learning. PMLR, 10697–10707

    2022

  43. [52]

    Antonia Karamolegkou, Jiaang Li, Li Zhou, and Anders Søgaard. 2023. Copyright violations and large language models. arXiv preprint arXiv:2310.13771 (2023)

    2023 arXiv

  44. [53]

    Aly Kassem, Omar Mahmoud, and Sherif Saad. 2023. Preserving Privacy Through Dememorization: An Unlearning Technique For Mitigating Mem- orization Risks In Language Models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , Houda Bouamor...

    2023 doi

  45. [54]

    Yigitcan Kaya, Sanghyun Hong, and Tudor Dumitras. 2020. On the Effectiveness of Regularization Against Membership Inference Attacks. arXiv:2006.05336 [cs.LG] https://arxiv.org/abs/2006.05336

    2020 arXiv

  46. [55]

    Sunder Ali Khowaja, Parus Khuwaja, and Kapal Dev. 2023. Chatgpt needs spade (sustainability, privacy, digital divide, and ethics) evaluation: A review. arXiv preprint arXiv:2305.03123 (2023)

    2023 arXiv

  47. [56]

    Siwon Kim, Sangdoo Yun, Hwaran Lee, Martin Gubri, Sungroh Yoon, and Seong Joon Oh. 2024. Propile: Probing privacy leakage in large language models. Advances in Neural Information Processing Systems 36 (2024)

    2024

  48. [57]

    Jacob Leon Kröger, Leon Gellrich, Sebastian Pape, Saba Rebecca Brause, and Stefan Ullrich. 2022. Personal information inference from voice recordings: User awareness and privacy concerns. Proc. Priv. Enhancing Technol. 2022, 1 (2022), 6–27

    2022

  49. [58]

    Weirui Kuang, Bingchen Qian, Zitao Li, Daoyuan Chen, Dawei Gao, Xuchen Pan, Yuexiang Xie, Yaliang Li, Bolin Ding, and Jingren Zhou. 2023. Federatedscope- llm: A comprehensive package for fine-tuning large language models in feder- ated learning. arXiv preprint arXiv:2309.00363 (2023)

    2023 arXiv

  50. [59]

    Harrison Lee, Samrat Phatale, Hassan Mansoor, Kellie Lu, Thomas Mesnard, Colton Bishop, Victor Carbune, and Abhinav Rastogi. 2023. Rlaif: Scaling reinforcement learning from human feedback with ai feedback. arXiv preprint arXiv:2309.00267 (2023)

    2023 arXiv

  51. [60]

    Katherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison-Burch, and Nicholas Carlini. 2021. Deduplicating training data makes language models better. arXiv preprint arXiv:2107.06499 (2021)

    2021 arXiv

  52. [61]

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rock- täschel, et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing...

    2020

  53. [62]

    Haoran Li, Dadi Guo, Wei Fan, Mingshi Xu, Jie Huang, Fanpu Meng, and Yangqiu Song. 2023. Multi-step jailbreaking privacy attacks on chatgpt. arXiv preprint arXiv:2304.05197 (2023)

    2023 arXiv

  54. [63]

    Kunhong Li, Ruwei Huang, and Bo Yang. 2025. Privacy-Preserving Text Classi- fication on Deep Neural Network. Neural Processing Letters 57, 2 (2025), 29

    2025

  55. [64]

    Linyang Li, Demin Song, and Xipeng Qiu. 2022. Text adversarial purification as defense against adversarial attacks. arXiv preprint arXiv:2203.14207 (2022)

    2022 arXiv

  56. [65]

    Tianshi Li, Sauvik Das, Hao-Ping Lee, Dakuo Wang, Bingsheng Yao, and Zhiping Zhang. 2024. Human-Centered Privacy Research in the Age of Large Language Models. arXiv preprint arXiv:2402.01994 (2024)

    2024 arXiv

  57. [66]

    Xuechen Li, Florian Tramer, Percy Liang, and Tatsunori Hashimoto. 2021. Large language models can be strong differentially private learners. arXiv preprint arXiv:2110.05679 (2021)

    2021 arXiv

  58. [67]

    Yuanchun Li, Hao Wen, Weijun Wang, Xiangyu Li, Yizhen Yuan, Guohong Liu, Jiacheng Liu, Wenxing Xu, Xiang Wang, Yi Sun, et al. 2024. Personal llm agents: Insights and survey about the capability, efficiency and security. arXiv preprint arXiv:2401.05459 (2024)

    2024 arXiv

  59. [68]

    Guo Lin, Wenyue Hua, and Yongfeng Zhang. 2024. PromptCrypt: Prompt Encryption for Secure Communication with Large Language Models. arXiv preprint arXiv:2402.05868 (2024)

    2024 arXiv

  60. [69]

    Tong Liu, Yingjie Zhang, Zhe Zhao, Yinpeng Dong, Guozhu Meng, and Kai Chen

  61. [70]

    Yi Liu, Gelei Deng, Zhengzi Xu, Yuekang Li, Yaowen Zheng, Ying Zhang, Lida Zhao, Tianwei Zhang, and Yang Liu. 2023. Jailbreaking chatgpt via prompt engineering: An empirical study. arXiv preprint arXiv:2305.13860 (2023)

    2023 arXiv

  62. [71]

    Zechun Liu, Changsheng Zhao, Forrest Iandola, Chen Lai, Yuandong Tian, Igor Fedorov, Yunyang Xiong, Ernie Chang, Yangyang Shi, Raghuraman Kr- ishnamoorthi, Liangzhen Lai, and Vikas Chandra. 2024. MobileLLM: Op- timizing Sub-billion Parameter Language Models for On-Device Use C...

    2024 arXiv

  63. [72]

    Scott M Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. Advances in neural information processing systems 30 (2017)

    2017

  64. [73]

    Aatish Mandelecha. 2024. How to Secure Sensitive Data in LLM Prompts? — strac.io. https://www.strac.io/blog/secure-sensitive-data-in-llm-prompts. [Accessed 14-05-2024]

    2024

  65. [74]

    E McGowan. 2024. Is ChatGPT’s use of people’s data even legal? — blog.avast.com. https://blog.avast.com/chatgpt-data-use-legal. [Accessed 04-04-2024]

    2024

  66. [75]

    Fatemehsadat Mireshghallah, Archit Uniyal, Tianhao Wang, David K Evans, and Taylor Berg-Kirkpatrick. 2022. An empirical analysis of memorization in fine-tuned autoregressive language models. InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processi...

    2022

  67. [76]

    Sumeet Ramesh Motwani, Mikhail Baranchuk, Martin Strohmeier, Vijay Bolina, Philip HS Torr, Lewis Hammond, and Christian Schroeder de Witt. 2024. Secret Collusion Among Generative AI Agents. arXiv preprint arXiv:2402.07510 (2024)

    2024 arXiv

  68. [77]

    Silen Naihin, David Atkinson, Marc Green, Merwane Hamadi, Craig Swift, Dou- glas Schonholtz, Adam Tauman Kalai, and David Bau. 2023. Testing language model agents safely in the wild. arXiv preprint arXiv:2311.10538 (2023)

    2023 arXiv

  69. [78]

    Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, et al. 2021. Webgpt: Browser-assisted question-answering with human feedback. arXiv preprint arXiv:2112.09332 (2021)

    2021 arXiv

  70. [79]

    Milad Nasr, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski, A Feder Cooper, Daphne Ippolito, Christopher A Choquette-Choo, Eric Wallace, Florian Tramèr, and Katherine Lee. 2023. Scalable extraction of training data from (production) language models. arXiv preprint arXiv:...

    2023 arXiv

  71. [80]

    Seth Neel and Peter Chang. 2023. Privacy issues in large language models: A survey. arXiv preprint arXiv:2312.06717 (2023)

    2023 arXiv

  72. [81]

    Helen Nissenbaum. 2004. Privacy as contextual integrity. Wash. L. Rev. 79 (2004), 119

    2004

  73. [82]

    Chris Olah, Arvind Satyanarayan, Ian Johnson, Shan Carter, Ludwig Schu- bert, Katherine Ye, and Alexander Mordvintsev. 2018. The building blocks of interpretability. Distill 3, 3 (2018), e10

    2018

  74. [83]

    Matthew J Page, Joanne E McKenzie, Patrick M Bossuyt, Isabelle Boutron, Tammy C Hoffmann, Cynthia D Mulrow, Larissa Shamseer, Jennifer M Tetzlaff, Elie A Akl, Sue E Brennan, et al. 2021. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. bmj 372 ...

    2021

  75. [84]

    Xianghe Pang, Shuo Tang, Rui Ye, Yuxin Xiong, Bolun Zhang, Yanfeng Wang, and Siheng Chen. 2024. Self-Alignment of Large Language Models via Monopolylogue-based Social Scene Simulation. arXiv preprint arXiv:2402.05699 (2024)

    2024 arXiv

  76. [85]

    Ildikó Pilán, Pierre Lison, Lilja Øvrelid, Anthi Papadopoulou, David Sánchez, and Montserrat Batet. 2022. The text anonymization benchmark (tab): A dedicated corpus and evaluation framework for text anonymization. Computational Linguistics 48, 4 (2022), 1053–1101

    2022

  77. [86]

    Aman Priyanshu, Supriti Vijay, Ayush Kumar, Rakshit Naidu, and Fatemehsadat Mireshghallah. 2023. Are chatbots ready for privacy-sensitive applications? an investigation into input regurgitation and prompt-induced sanitization. arXiv preprint arXiv:2305.15008 (2023)

    2023 arXiv

  78. [87]

    Md Abdur Rahman, Lamyaa Alqahtani, Amna Albooq, and Alaa Ainousah. 2024. A Survey on Security and Privacy of Large Multimodal Deep Learning Mod- els: Teaching and Learning Perspective. In 2024 21st Learning and Technology Conference (L&T). IEEE, 13–18

    2024

  79. [88]

    Abhinav Rao, Sachin Vashistha, Atharva Naik, Somak Aditya, and Monojit Choudhury. 2023. Tricking llms into disobedience: Formalizing, analyzing, and detecting jailbreaks. arXiv preprint arXiv:2305.14965 (2023)

    2023 arXiv

  80. [89]

    Protection Regulation. 2016. Regulation (EU) 2016/679 of the European Parlia- ment and of the Council. Regulation (eu) 679 (2016), 2016

    2016

  81. [90]

    Protection Regulation. 2024. AI Act, Regulation (EU) 2024/1689 of the European Parliament and of the Council. Regulation (eu) 1689 (2024)

    2024

  82. [91]

    Jingjing Ren, Ashwin Rao, Martina Lindorfer, Arnaud Legout, and David Choffnes. 2016. Recon: Revealing and controlling pii leaks in mobile network traffic. In Proceedings of the 14th Annual International Conference on Mobile Systems, Applications, and Services. 361–374

    2016

  83. [92]

    Donghwan Rho, Taeseong Kim, Minje Park, Jung Woo Kim, Hyunsik Chae, Ernest K Ryu, and Jung Hee Cheon. 2024. Encryption-friendly LLM architecture. arXiv preprint arXiv:2410.02486 (2024)

    2024 arXiv

  84. [93]

    Why should i trust you?

    Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. " Why should i trust you?" Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. 1135–1144

    2016

  85. [94]

    Toran Bruce Richards. 2023. Auto-gpt: Autonomous artificial intelligence soft- ware agent. https://github.com/Significant-Gravitas/AutoGPT. [Accessed 08-04-2024]

    2023

  86. [95]

    Jae Hun Ro, Srinadh Bhojanapalli, Zheng Xu, Yanxiang Zhang, and Ananda Theertha Suresh. 2024. Efficient Language Model Architectures for Dif- ferentially Private Federated Learning. arXiv preprint arXiv:2403.08100 (2024)

    2024 arXiv

  87. [96]

    Alexander Robey, Eric Wong, Hamed Hassani, and George J Pappas. 2023. Smoothllm: Defending large language models against jailbreaking attacks.arXiv preprint arXiv:2310.03684 (2023)

    2023 arXiv

  88. [97]

    Oscar J Romero, John Zimmerman, Aaron Steinfeld, and Anthony Tomasic. 2023. Synergistic integration of large language models and cognitive architectures for robust ai: An exploratory analysis. In Proceedings of the AAAI Symposium Series, Vol. 2. 396–405

    2023

  89. [98]

    Lorenzo Rovida and Alberto Leporati. 2024. Transformer-based language models and homomorphic encryption: An intersection with bert-tiny. In Proceedings of the 10th ACM International Workshop on Security and Privacy Analytics . 3–13

    2024

  90. [99]

    Yangjun Ruan, Honghua Dong, Andrew Wang, Silviu Pitis, Yongchao Zhou, Jimmy Ba, Yann Dubois, Chris J Maddison, and Tatsunori Hashimoto. 2023. Identifying the risks of lm agents with an lm-emulated sandbox. arXiv preprint arXiv:2309.15817 (2023)

    2023 arXiv

  91. [100]

    Zhang Ruoyan, Zheng Zhongxiang, and Bao Wankang. 2025. Practical Secure Inference Algorithm for Fine-tuned Large Language Model Based on Fully Homomorphic Encryption. arXiv preprint arXiv:2501.01672 (2025)

    2025 arXiv

  92. [101]

    Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. Distil- BERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108 (2019)

    2019 arXiv

  93. [102]

    Weiyan Shi, Aiqi Cui, Evan Li, Ruoxi Jia, and Zhou Yu. 2021. Selective differential privacy for language modeling. arXiv preprint arXiv:2108.12944 (2021)

    2021 arXiv

  94. [103]

    Taylor Shin, Yasaman Razeghi, Robert L Logan IV, Eric Wallace, and Sameer Singh. 2020. Autoprompt: Eliciting knowledge from language models with automatically generated prompts. arXiv preprint arXiv:2010.15980 (2020)

    2020 arXiv

  95. [104]

    Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. 2017. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP) . IEEE, 3–18

    2017

  96. [105]

    Tanmay Singh, Harshvardhan Aditya, Vijay K Madisetti, and Arshdeep Bahga

  97. [106]

    Victoria Smith, Ali Shahin Shamsabadi, Carolyn Ashurst, and Adrian Weller

  98. [107]

    Robin Staab, Mark Vero, Mislav Balunovic, and Martin Vechev. 2024. Beyond Memorization: Violating Privacy via Inference with Large Language Models. In The Twelfth International Conference on Learning Representations . https: //openreview.net/forum?id=kmn0BhQk7p

    2024

  99. [108]

    Robin Staab, Mark Vero, Mislav Balunović, and Martin Vechev. 2024. Large Language Models are Advanced Anonymizers. arXiv preprint arXiv:2402.13846 (2024)

    2024 arXiv

  100. [109]

    Journal of Software Engineering and Applications 17, 1 (2024), 1–22

    Whispered Tuning: Data Privacy Preservation in Fine-Tuning LLMs through Differential Privacy. Journal of Software Engineering and Applications 17, 1 (2024), 1–22

    2024

  101. [110]

    Nishant Subramani, Sasha Luccioni, Jesse Dodge, and Margaret Mitchell. 2023. Detecting personal information in training corpora: an analysis. In Proceedings of the 3rd Workshop on Trustworthy Natural Language Processing (TrustNLP 2023). 208–220

    2023

  102. [111]

    arXiv preprint arXiv:2310.01424 (2023)

    Identifying and mitigating privacy risks stemming from language models: A survey. arXiv preprint arXiv:2310.01424 (2023)

    2023 arXiv

  103. [112]

    Baraniuk

    Jasper Tan, Blake Mason, Hamid Javadi, and Richard G. Baraniuk. 2022. Param- eters or Privacy: A Provable Tradeoff Between Overparameterization and Mem- bership Inference. arXiv:2202.01243 [stat.ML] https://arxiv.org/abs/2202.01243

    2022 arXiv

  104. [113]

    Xinyu Tang, Richard Shin, Huseyin A Inan, Andre Manoel, Fatemehsadat Mireshghallah, Zinan Lin, Sivakanth Gopi, Janardhan Kulkarni, and Robert Sim. 2023. Privacy-preserving in-context learning with differentially private few-shot generation. arXiv preprint arXiv:2309.11765 (2023)

    2023 arXiv

  105. [114]

    Lukas Struppek, Minh Hieu Le, Dominik Hintersdorf, and Kristian Kersting

  106. [115]

    arXiv preprint arXiv:2402.09132 (2024)

    Exploring the Adversarial Capabilities of Large Language Models. arXiv preprint arXiv:2402.09132 (2024)

    2024 arXiv

  107. [116]

    Jeffrey G Wang, Jason Wang, Marvin Li, and Seth Neel. 2024. Pandora’s White-Box: Increased Training Data Leakage in Open LLMs. arXiv preprint arXiv:2402.17012 (2024)

    2024 arXiv

  108. [117]

    Zhiqing Sun, Yikang Shen, Qinhong Zhou, Hongxin Zhang, and et al Chen

  109. [118]

    Advances in Neural Information Processing Systems 36 (2024)

    Principle-driven self-alignment of language models from scratch with minimal human supervision. Advances in Neural Information Processing Systems 36 (2024)

    2024

  110. [119]

    Tong Wu, Ashwinee Panda, Jiachen T Wang, and Prateek Mittal. 2023. Privacy- preserving in-context learning for large language models. In The Twelfth Inter- national Conference on Learning Representations

    2023

  111. [120]

    Xiaodong Wu, Ran Duan, and Jianbing Ni. 2023. Unveiling security, privacy, and ethical concerns of chatgpt. Journal of Information and Intelligence (2023)

    2023

  112. [121]

    Yu Tian, Xiao Yang, Jingyuan Zhang, Yinpeng Dong, and Hang Su. 2023. Evil geniuses: Delving into the safety of llm-based agents. arXiv preprint arXiv:2311.11855 (2023)

    2023 arXiv

  113. [122]

    Minh N Vu, Truc Nguyen, Tre’R Jeter, and My T Thai. 2024. Analysis of Privacy Leakage in Federated Large Language Models. arXiv preprint arXiv:2403.04784 (2024)

    2024 arXiv

  114. [123]

    Fuzhao Xue, Yao Fu, Wangchunshu Zhou, Zangwei Zheng, and Yang You. 2024. To repeat or not to repeat: Insights from scaling llm under token-crisis.Advances in Neural Information Processing Systems 36 (2024)

    2024

  115. [124]

    Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. 2024. Jailbroken: How does llm safety training fail? Advances in Neural Information Processing Systems 36 (2024)

    2024

  116. [125]

    Jules White, Quchen Fu, Sam Hays, Michael Sandborn, Carlos Olea, Henry Gilbert, Ashraf Elnashar, Jesse Spencer-Smith, and Douglas C Schmidt. 2023. A prompt pattern catalog to enhance prompt engineering with chatgpt. arXiv preprint arXiv:2302.11382 (2023)

    2023 arXiv

  117. [126]

    Linyao Yang, Hongyang Chen, Zhao Li, Xiao Ding, and Xindong Wu. 2024. Give us the facts: Enhancing large language models with knowledge graphs for fact-aware language modeling. IEEE Transactions on Knowledge and Data Engineering (2024)

    2024

  118. [127]

    Zhou Yang, Zhipeng Zhao, Chenyu Wang, Jieke Shi, Dongsun Kim, DongGyun Han, and David Lo. 2023. What do code models memorize? an empirical study on large language models of code. arXiv preprint arXiv:2308.09932 (2023)

    2023 arXiv

  119. [128]

    Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, et al. 2023. The rise and potential of large language model based agents: A survey. arXiv preprint arXiv:2309.07864 (2023)

    2023 arXiv

  120. [129]

    Daliang Xu, Wangsong Yin, Xin Jin, Ying Zhang, Shiyun Wei, Mengwei Xu, and Xuanzhe Liu. 2023. Llmcad: Fast and scalable on-device large language model inference. arXiv preprint arXiv:2309.04255 (2023)

    2023 arXiv

  121. [130]

    Yuanshun Yao, Xiaojun Xu, and Yang Liu. 2023. Large language model unlearn- ing. arXiv preprint arXiv:2310.10683 (2023)

    2023 arXiv

  122. [131]

    Biwei Yan, Kun Li, Minghui Xu, Yueyan Dong, Yue Zhang, Zhaochun Ren, and Xiuzheng Cheng. 2024. On Protecting the Data Privacy of Large Language Models (LLMs): A Survey. arXiv preprint arXiv:2403.05156 (2024)

    2024 arXiv

  123. [132]

    Haomiao Yang, Kunlan Xiang, Mengyu Ge, Hongwei Li, Rongxing Lu, and Shui Yu. 2024. A comprehensive overview of backdoor attacks in large language models within communication networks. IEEE Network (2024)

    2024

  124. [133]

    Keun Soo Yim. 2023. Privacy-friendly Personalization of LLM Responses Using Hashed Entity Injection. Technical Disclosure Commons (2023)

    2023

  125. [134]

    Jin Yong Yoo and Yanjun Qi. 2021. Towards improving adversarial training of NLP models. arXiv preprint arXiv:2109.00544 (2021)

    2021 arXiv

  126. [135]

    Yifan Yao, Jinhao Duan, Kaidi Xu, Yuanfang Cai, Zhibo Sun, and Yue Zhang

  127. [136]

    High-Confidence Computing (2024), 100211

    A survey on large language model (llm) security and privacy: The good, the bad, and the ugly. High-Confidence Computing (2024), 100211

    2024

  128. [137]

    Yixiang Yao, Fei Wang, Srivatsan Ravi, and Muhao Chen. 2024. Privacy- Preserving Language Model Inference with Instance Obfuscation.arXiv preprint arXiv:2402.08227 (2024)

    2024 arXiv

  129. [138]

    Yifan Zeng, Yiran Wu, Xiao Zhang, Huazheng Wang, and Qingyun Wu. 2024. Au- toDefense: Multi-Agent LLM Defense against Jailbreak Attacks. arXiv preprint arXiv:2403.04783 (2024)

    2024 arXiv

  130. [139]

    Rui Ye, Wenhao Wang, Jingyi Chai, Dihan Li, Zexi Li, Yinda Xu, Yaxin Du, Yanfeng Wang, and Siheng Chen. 2024. OpenFedLLM: Training Large Language Models on Decentralized Private Data via Federated Learning. arXiv preprint arXiv:2402.06954 (2024). Yashothara Shanmugarasa, Ming ...

    2024 arXiv

  131. [140]

    Rongjie Yi, Liwei Guo, Shiyun Wei, Ao Zhou, Shangguang Wang, and Mengwei Xu. 2025. EdgeMoE: Empowering Sparse Large Language Models on Mobile Devices. IEEE Transactions on Mobile Computing (2025)

    2025

  132. [141]

    Kaiyan Zhang, Jianyu Wang, Ermo Hua, Biqing Qi, Ning Ding, and Bowen Zhou

  133. [142]

    Lexin Zhang, Changxiang Li, Qi Hu, Jingjing Lang, Sirui Huang, Linyue Hu, Jingwen Leng, Qiuhan Chen, and Chunli Lv. 2023. Enhancing Privacy in Large Language Model with Homomorphic Encryption and Sparse Attention. Applied Sciences 13, 24 (2023). https://doi.org/10.3390/app132413146

    2023 doi

  134. [143]

    Tongxin Yuan, Zhiwei He, Lingzhong Dong, Yiming Wang, Ruijie Zhao, Tian Xia, Lizhen Xu, Binglin Zhou, Fangqi Li, Zhuosheng Zhang, et al . 2024. R- Judge: Benchmarking Safety Risk Awareness for LLM Agents. arXiv preprint arXiv:2401.10019 (2024)

    2024 arXiv

  135. [144]

    Ahtsham Zafar, Venkatesh Balavadhani Parthasarathy, Chan Le Van, Saad Shahid, Arsalan Shahid, et al . 2023. Building trust in conversational ai: A comprehensive review and solution architecture for explainable, privacy-aware systems using llms and knowledge graph. arXiv prepri...

    2023 arXiv

  136. [145]

    Santiago Zanella-Béguelin, Lukas Wutschitz, Shruti Tople, Victor Rühle, Andrew Paverd, Olga Ohrimenko, Boris Köpf, and Marc Brockschmidt. 2020. Analyzing information leakage of updates to natural language models. In Proceedings of the 2020 ACM SIGSAC conference on computer and...

    2020

  137. [146]

    It’s a Fair Game

    Zhiping Zhang, Michelle Jia, Bingsheng Yao, Sauvik Das, Ada Lerner, Dakuo Wang, Tianshi Li, et al. 2023. " It’s a Fair Game”, or Is It? Examining How Users Navigate Disclosure Risks and Benefits When Using LLM-Based Conversational Agents. arXiv preprint arXiv:2309.11653 (2023)

    2023 arXiv

  138. [147]

    Jiawen Zhang, Kejia Chen, Zunlei Feng, Jian Lou, Mingli Song, Jian Liu, and Xiaohu Yang. 2024. SecPE: Secure Prompt Ensembling for Private and Robust Large Language Models. In ECAI

    2024

  139. [148]

    Jinghan Zhang, Junteng Liu, Junxian He, et al . 2023. Composing parameter- efficient modules with arithmetic operation. Advances in Neural Information Processing Systems 36 (2023), 12589–12610

    2023

  140. [149]

    Haiyan Zhao, Hanjie Chen, Fan Yang, Ninghao Liu, Huiqi Deng, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, and Mengnan Du. 2024. Explainability for large language models: A survey. ACM Transactions on Intelligent Systems and Tech- nology 15, 2 (2024), 1–38

    2024

  141. [150]

    CoGenesis: A Framework Collaborating Large and Small Language Models for Secure Context-Aware Instruction Following.arXiv preprint arXiv:2403.03129 (2024)

    2024 arXiv

  142. [151]

    Xin Zhou, Yi Lu, Ruotian Ma, Tao Gui, Yuran Wang, Yong Ding, Yibo Zhang, Qi Zhang, and Xuan-Jing Huang. 2023. Textobfuscator: Making pre-trained language model a privacy protector via obfuscating word representations. In Findings of the Association for Computational Linguistic...

    2023

  143. [152]

    Ruisi Zhang, Seira Hidano, and Farinaz Koushanfar. 2022. Text revealer: Private text reconstruction via model inversion attacks against transformers. arXiv preprint arXiv:2209.10505 (2022)

    2022 arXiv

  144. [153]

    Xinyu Zhang, Huiyu Xu, Zhongjie Ba, Zhibo Wang, Yuan Hong, Jian Liu, Zhan Qin, and Kui Ren. 2024. PrivacyAsst: Safeguarding User Privacy in Tool-Using Large Language Model Agents. IEEE Transactions on Dependable and Secure Computing (2024)

    2024

  145. [154]

    Yao Zhang, Zijian Ma, Yunpu Ma, Zhen Han, Yu Wu, and Volker Tresp. 2024. WebPilot: A Versatile and Autonomous Multi-Agent System for Web Task Execution with Strategic Exploration. arXiv:2408.15978 [cs.AI] https://arxiv. org/abs/2408.15978

    2024 arXiv

  146. [156]

    Zhexin Zhang, Jiaxin Wen, and Minlie Huang. 2023. Ethicist: Targeted training data extraction through loss smoothed soft prompting and calibrated confidence estimation. arXiv preprint arXiv:2307.04401 (2023)

    2023 arXiv

  147. [157]

    Zhexin Zhang, Junxiao Yang, Pei Ke, Fei Mi, Hongning Wang, and Minlie Huang. 2024. Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritization. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Lo...

    2024 doi

  148. [159]

    Xuandong Zhao, Lei Li, and Yu-Xiang Wang. 2022. Provably confidential language modelling. arXiv preprint arXiv:2205.01863 (2022)

    2022 arXiv

  149. [161]

    snowballing

    Ligeng Zhu, Zhijian Liu, and Song Han. 2019. Deep leakage from gradients. Advances in neural information processing systems 32 (2019). 6 APPENDICES 6.1 Source Selection and Strategy We conducted a literature review adhering to the Preferred Report- ing Items for Systematic Rev...

    2019

  150. [2021]

    How should pre-trained language models be fine-tuned towards adver- sarial robustness? Advances in Neural Information Processing Systems 34 (2021), 4356–4369

    2021

  151. [2023]

    arXiv preprint arXiv:2311.06062 (2023)

    Practical membership inference attacks against fine-tuned large language models via self-prompt calibration. arXiv preprint arXiv:2311.06062 (2023)

    2023 arXiv

  152. [2024]

    arXiv preprint arXiv:2402.18104 (2024)

    Making Them Ask and Answer: Jailbreaking Large Language Models in Few Queries via Disguise and Reconstruction. arXiv preprint arXiv:2402.18104 (2024)

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.