Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Has My System Prompt Been Used? Large Language Model Prompt Membership Inference

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Prompt Detective, a training-free statistical test, can verify whether a given system prompt was used by a third-party language model by comparing the distributions of their responses.

desk verdict Prompt Detective is a genuinely new, simple, and well-tested method for detecting when a system prompt produces measurably different output distributions, but the paper's headline claim that it 'verifies prompt usage' outruns the statistics. read the letter →

arxiv 2502.09974 v1 pith:CJFL2NQ5 submitted 2025-02-14 cs.AI cs.CR

Roman Levin , Valeriia Cherepanova , Abhimanyu Hans , Avi Schwarzschild , Tom Goldstein This is my paper
classification cs.AIcs.CR
keywords promptmembershipinferencesystemverificationpermutationtestlargelanguagemodelsreusedetectionstatisticalhypothesistestingBERTembeddingsprivacy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Prompt Detective is a training-free statistical test for deciding whether a given system prompt was actually used inside a third-party chatbot. It queries the suspect service and a reference model with the same task prompts, embeds the responses as vector representations, and runs a permutation test on the cosine similarity between the group means. The paper shows that across several large language models and prompt collections, the test reliably rejects the null hypothesis of equal output distributions whenever the system prompts differ, even for minimal rephrasing or a two-character typo. If the claim holds, prompt reuse can be verified with statistical significance rather than by attempting costly and imperfect prompt reconstruction.

What carries the argument

The load-bearing object is a block permutation test on a cosine-similarity statistic. Each generated response is mapped to a vector by a sentence-embedding model; the statistic is the cosine similarity between the mean vectors of the two groups. To build a null distribution, responses are shuffled between the two groups within each task prompt, preserving the block structure that responses to the same query are more alike. The p-value is the fraction of permuted similarities at least as extreme as the observed one, yielding a non-parametric test of equality of output distributions without training anything.

What would settle it

Take one system prompt, run it in two deployments that use different chat templates or different sampling temperatures but the same visible prompt, and apply Prompt Detective with enough responses; if it returns p < 0.05, the test is detecting differences in conditioning rather than prompt reuse specifically, which would disprove the claim that statistical rejection identifies prompt membership.

Watch

Extended reading notes

Core claim

On the paper's own terms: even very small changes to a system prompt leave a measurable fingerprint in the distribution of a language model's responses. Prompt Detective exploits this by comparing two groups of generations, one from the third-party service and one from a reference model running the known proprietary prompt, and applying a block permutation test to the mean cosine similarity of their embeddings. In the positive case, where the same prompt sits on both sides, the test's p-values stay near 0.5; in the negative case they collapse toward zero once enough task prompts and generations are collected. The paper reports zero or near-zero false positive rates in standard settings, shows that 50 generations per task prompt separate even the most similar rewordings, and demonstrates that the method degrades gracefully when the underlying model is unknown but restricted to a small candidate set.

Load-bearing premise

The test assumes that when the candidate and target system prompts are identical, the third-party model's outputs and the reference model's outputs are exchangeable within each task prompt, which requires the same underlying model, chat template, decoding settings, and no hidden extra conditioning.

Editorial extensions

If this is right

  • A service owner can verify suspected prompt theft by querying the chatbot and comparing output distributions, without ever recovering the prompt text.
  • Prompts that differ by only a few words or a typo are still distinguishable, so paraphrase-based evasion would not survive once enough responses are collected.
  • The number of responses, not their length, drives statistical power: the paper finds 300 short responses are enough to separate the hardest pairs, and longer generations add little.
  • The method works even when the underlying model is unknown, provided the set of plausible models is small, after applying a multiple-comparison correction.
  • The tailored statistical test substantially outperforms adapting a prompt-reconstruction method to the verification task.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • (Editorial inference) Because the test rejects on any systematic difference in response distributions, it could be repurposed to audit hidden instructions or drift in a deployed system prompt over time, not just to verify a known prompt.
  • (Editorial inference) The sensitivity to decoding settings implies that a clean negative result is also evidence that the model version and sampling procedure are stable, which matters if the method is used for forensic or legal claims.
  • (Editorial inference) The same block-permutation logic could be ported to other output representations or even non-text modalities, likely requiring more samples when the embedding is less sensitive to prompt differences.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. Prompt Detective is a training-free statistical method for prompt membership inference. Given query access to a third-party chat service and a candidate proprietary system prompt, it generates k responses to n task prompts from both the service and a reference model configured with the known prompt, embeds all outputs with BERT, and applies a block permutation test using cosine similarity between mean embeddings as the test statistic. The paper reports that at a 0.05 significance threshold the test separates identical from different prompts in standard and hard-example settings, including near-identical rephrasings and typo-level differences, and extends the method to a black-box scenario with a small candidate set of six models.

Significance. If the central claim were established, this would be a useful contribution to prompt privacy: a cheap, training-free audit that detects when a suspected prompt differs from the prompt actually deployed in a chat service. The method has real strengths: it uses no fitted parameters, relies on a standard permutation test on fixed embeddings, ships code, and is evaluated across six model families and against a PLeak baseline. However, the paper's headline claim of positive verification of prompt reuse is not supported by the statistical procedure; the controlled experiments establish at most that the test can detect distributional differences under favorable conditions. The logical gap is fixable, but it requires reworking the decision rule, the claims, and the black-box evaluation.

major comments (3)
  1. [Abstract; Fig. 1; Algorithm 1; §5.1] Algorithm 1 tests the null hypothesis that the two output distributions are equal and returns 'Prompts are distinct' only when p<α; the complementary outcome is 'Insufficient evidence to claim prompts are distinct.' A non-significant p-value does not establish that the null is true, so the interpretation in the abstract ('verify prompt usage'), Figure 1 ('Proprietary Prompt has been reused'), and Section 7 is logically invalid. Under the null, p-values are uniform, so p≥0.05 occurs with probability 0.95 for identical prompts, but it also occurs for different prompts whenever power is insufficient. Table 2 shows the practical consequence: for Similarity Level 1 pairs, which are known different prompts, Claude2 and GPT-3.52 have average p-values of 0.194 and 0.213 and reported false-positive rates of 0.65; a rule that maps p≥0.05 to 'reused' would therefore declare reuse in 65% of known non-reuse cases. To support positive verification, the authors need an equivalence test with a pre-specified margin, or a power analysis that covers all plausible alternative prompts and model configurations.
  2. [§3.1, §6; Algorithm 1] The permutation test's null distribution is only valid if, when the candidate and deployed system prompts are identical, the third-party model's outputs are exchangeable with the reference model's outputs within each task prompt. This requires the same model weights or version, chat template, decoding temperature, sampling settings, and no additional hidden system-level instructions. Section 3.1 assumes knowledge of the deployed model 'in most of our experiments,' and Section 6 relaxes this only to a candidate set of six known models. If any of these conditions fails, the test can reject even when the same system prompt is in use, so the reported FPRs in Table 1 do not transfer to real third-party services. A concrete experiment holding the system prompt fixed while varying chat template, temperature, or model version is needed before the verification claim can be assessed.
  3. [§6, Table 3] The black-box procedure that takes the maximum p-value over the six reference models and applies a Bonferroni correction does not define a valid test for the composite question 'is the unknown deployed model, among the candidates, using pbar?' Taking a maximum p-value makes the decision 'prompts distinct' conservative in one direction, but it does not control the error of declaring 'reused' when the deployed prompt is different but produces outputs close to one of the reference distributions. Table 3 is evaluated under the assumption that the true model is in the candidate set, and the FPR/FNR numbers therefore do not quantify the false reuse rate in the setting the abstract advertises. The authors should either formulate a single composite test with a well-defined null and report its power, or restrict all claims to the known-model case.
minor comments (5)
  1. [Table 2, caption] The meaning of FPR is inconsistent with Table 1. In Table 1, FPR is the type-I error of rejecting equality for identical prompts, whereas Table 2 rows are known different prompts, so FPR must be a different quantity. Please define both quantities explicitly in each caption.
  2. [Algorithm 1, Step 5] If no permuted similarity is as extreme as the observed value, the reported p-value is 0, which is not a valid finite-sample estimate. Use (c+1)/(Npermutations+1) or state that Npermutations is large enough to make this correction negligible.
  3. [Figure 5, caption] The caption says generations from different but conceptually similar prompts 'cluster together,' while the text and figure appear to show separate clusters. This likely should read 'cluster separately.'
  4. [Table 1, caption] The quantities pp_avg and pn_avg are used without definitions. They should be defined as the average p-values for positive (identical) and negative (different) prompt pairs, respectively.
  5. [Appendix F] All task prompts and hard-example variants are generated by Claude 3 Sonnet, and no ablation varies the probe-generation model. The reported performance may depend on this choice, so the paper should either test this dependence or discuss the limitation explicitly.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: Prompt Detective's p-value is computed directly from the data by a standard permutation test with no fitted parameters.

full rationale

The paper's derivation chain is self-contained. Algorithm 1 constructs two groups of generations, embeds them with a fixed pretrained BERT encoder, computes the cosine similarity of the group mean vectors, and obtains a p-value by permuting responses within each task prompt. The null distribution is generated by the shuffle procedure itself, and the decision rule ('Prompts are distinct' vs 'Insufficient evidence...') is defined solely by comparing the observed statistic to that null distribution. No parameter is fitted to the positive/negative pairs used for evaluation, and those pairs are only used to report FPR/FNR, not to calibrate or post-select the test. Design choices such as BERT embeddings and Claude-generated probe prompts are supported by ablations (Table 4) and are not derived from the target conclusion. The self-citations (Geiping et al. 2024; Wen et al. 2022, 2024; Cherepanova & Zou 2024) appear in related-work context and do not carry any load-bearing step of the proof. The only substantive weakness is interpretive, not circular: the paper's Figure 1 and abstract read a non-significant p-value as positive evidence of prompt reuse, whereas Algorithm 1's output is literally 'Insufficient evidence to claim prompts are distinct'; this is a statistical-reasoning concern about equating failure-to-reject with verification, not a case where the result is equivalent to its input by construction.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The algorithm itself is training-free and has no fitted parameters; the free parameters listed are experimental budgets and significance thresholds. The main load-bearing ingredients are statistical exchangeability and control over model identity, both of which the paper states but does not always defend. No new physical or conceptual entities are introduced.

free parameters (4)
  • significance level alpha = 0.05
    Selected by hand as the standard threshold for FPR and FNR reporting; not fitted, but it directly sets the false positive rate of the test.
  • number of generations per task prompt k = 1 in the standard setup, 2 and 50 in hard examples
    An experimental budget chosen by the authors; Table 2 shows that with k equals 2 the test is weak for similar prompts, while k equals 50 makes separation near-perfect.
  • number of task prompts n = 50 for Awesome-ChatGPT, 20 for Anthropic Library, 10 for hard examples
    Selected by the authors; Figures 3 and 4 show that power increases with n and that the method needs at least a few diverse probes.
  • generation length cap = up to 512 tokens, with an ablation at 16 to 512 tokens
    Chosen by the authors; the paper finds generations beyond 64 tokens add little value, so this is a budget choice rather than a fitted constant.
assumptions (5)
  • standard math Under the null that the candidate and target system prompts are identical, the outputs of the third-party model and the reference model are exchangeable within each task prompt, so the block permutation test yields a valid p-value.
    Invoked in Section 3.2 and Algorithm 1; the validity of the test rests on this exchangeability, which holds only if the two models and sampling conditions are identical.
  • domain assumption The attacker knows the exact model used by the third party, or can enumerate a small candidate set in the black-box case.
    Stated in Section 3.1 and Section 6; if the actual model is not in the reference set, distribution differences can be caused by model identity rather than prompt identity.
  • domain assumption Mean BERT embeddings of LLM outputs capture the distributional differences induced by system prompts.
    Used in Section 3.2 as the test representation; Appendix B Table 4 shows the choice of embedding model changes results only slightly, but the test statistic is entirely defined by these embeddings.
  • ad hoc to paper Task prompts generated by Claude 3 Sonnet are effective probes for every system prompt under test.
    Section 3.3 and Appendix F describe generating probe questions with Claude; the method's power depends on probes that make the system-prompt influence visible, and no independent validation that the generated probes are optimal or uniformly good is provided.
  • domain assumption LLM API outputs can be treated as i.i.d. samples from a stable response distribution for the purpose of permutation testing.
    Implicit in Section 3.2; real APIs can have caching, version changes, nondeterminism, or hidden system instructions that violate this assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Has My System Prompt Been Used? Large Language Model Prompt Membership Inference." pith.science (2026). https://pith.science/paper/CJFL2NQ5

@misc{pith2026250209974,
  author       = {Pith},
  title        = {Pith review of: Has My System Prompt Been Used? Large Language Model Prompt Membership Inference},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CJFL2NQ5}},
  note         = {Machine review of arXiv:2502.09974}
}
read the original abstract

Prompt engineering has emerged as a powerful technique for optimizing large language models (LLMs) for specific applications, enabling faster prototyping and improved performance, and giving rise to the interest of the community in protecting proprietary system prompts. In this work, we explore a novel perspective on prompt privacy through the lens of membership inference. We develop Prompt Detective, a statistical method to reliably determine whether a given system prompt was used by a third-party language model. Our approach relies on a statistical test comparing the distributions of two groups of model outputs corresponding to different system prompts. Through extensive experiments with a variety of language models, we demonstrate the effectiveness of Prompt Detective for prompt membership inference. Our work reveals that even minor changes in system prompts manifest in distinct response distributions, enabling us to verify prompt usage with statistical significance.

Figures

Figures reproduced from arXiv: 2502.09974 by the authors.

Figure 1
Figure 1. Prompt Detective verifies if a third-party chat bot uses a given proprietary system prompt by querying the system and comparing distribution of outputs with outputs obtained using proprietary system prompt. ∗Equal Contribution. Correspondence to: romlevin@amazon.com, cherepv@amazon.com. Code is available on GitHub: github.com/amazon-science/prompt-membership-inference 1 arXiv:2502.09974v1 [cs.AI] 14 Feb 2025 [PITH_… view at source ↗
Figure 2
Figure 2. Hard Examples illustrate varying degrees of similarity between the original prompts and their rephrased versions. Similarity Level 1 is highly similar, while Level 5 is completely different. and { ¯fp¯(qi) j}i∈[1,...,n],j∈[1,...,k] , yielding two groups of high-dimensional vector representations of generations corresponding to the two system prompts under comparison. We include results for ablation study on embeddin… view at source ↗
Figure 3
Figure 3. Average p-value computed for different number of task queries. Left: Awesome￾ChatGPT-Prompts. Right: Anthropic Library. Increasing the number of generations leads to decreasing p-value in negative cases, but the average p-value for positive cases remains close to 0.5. 8 [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Effect of the number of task prompts, generations, and tokens on the performance of Prompt Detective. Average p-value for GPT-3.5 model on system prompts of Similarity Level 1. The left panel shows the average p-value vs. the number of generations used in Prompt Detect…
Figure 5
Figure 5. Figure 5: UMAP projection of generations of language model across 5 system prompts of varying similarity for one task prompt. It can be seen that generations from different, although conceptually similar system prompts, cluster together. A ADDITIONAL DETAILS ON SYSTEM PROMPT SOU…
Figure 6
Figure 6. Figure 6: ROC-Curves computed by varying the significance level α for Prompt Detective. The markers correspond to the significance level of 0.05 [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Multi-TAP: Multi-criteria Target Adaptive Persona Modeling for Cross-Domain Recommendation

    cs.HC 2026-03 conditional novelty 6.0 of 10

    Modeling intra-domain preference heterogeneity with multi-criteria LLM personas and target-adaptive doppelganger transfer beats prior CDR methods on Amazon domain pairs.

Reference graph

Works this paper leans on

38 extracted references · 11 canonical work pages · cited by 1 Pith paper

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  2. [2]

    Claude 3 model family: Opus, sonnet, haiku

    Anthropic. Claude 3 model family: Opus, sonnet, haiku. https://www.anthropic.com/news/claude-3-family, 2024 a . Accessed: June 14, 2024

    work page 2024

  3. [3]

    Prompt library

    Anthropic. Prompt library. https://docs.anthropic.com/en/prompt-library/library, 2024 b . Accessed: June 14, 2024

    work page 2024

  4. [4]

    Scalable membership inference attacks via quantile regression

    Martin Bertran, Shuai Tang, Aaron Roth, Michael Kearns, Jamie H Morgenstern, and Steven Z Wu. Scalable membership inference attacks via quantile regression. Advances in Neural Information Processing Systems, 36, 2024

    2024

  5. [5]

    Membership inference attacks from first principles

    Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramer. Membership inference attacks from first principles. In 2022 IEEE Symposium on Security and Privacy (SP), pp.\ 1897--1914. IEEE, 2022

    2022

  6. [6]

    Talking Nonsense: Probing Large Language Models' Understanding of Adversarial Gibberish Inputs

    Valeriia Cherepanova and James Zou. Talking nonsense: Probing large language models' understanding of adversarial gibberish inputs. arXiv preprint arXiv:2404.17120, 2024

    work page Pith review arXiv 2024

  7. [7]

    Are diffusion models vulnerable to membership inference attacks? In International Conference on Machine Learning, pp.\ 8717--8730

    Jinhao Duan, Fei Kong, Shiqi Wang, Xiaoshuang Shi, and Kaidi Xu. Are diffusion models vulnerable to membership inference attacks? In International Conference on Machine Learning, pp.\ 8717--8730. PMLR, 2023

    2023

  8. [8]

    Coercing llms to do and reveal (almost) anything

    Jonas Geiping, Alex Stein, Manli Shu, Khalid Saifullah, Yuxin Wen, and Tom Goldstein. Coercing llms to do and reveal (almost) anything. arXiv preprint arXiv:2402.14020, 2024

    arXiv 2024

Show all 38 references
  1. [9]

    Permutation tests: a practical guide to resampling methods for testing hypotheses

    Phillip Good. Permutation tests: a practical guide to resampling methods for testing hypotheses. Springer Science & Business Media, 2013

    2013

  2. [10]

    Membership inference attacks on machine learning: A survey

    Hongsheng Hu, Zoran Salcic, Lichao Sun, Gillian Dobbie, Philip S Yu, and Xuyun Zhang. Membership inference attacks on machine learning: A survey. ACM Computing Surveys (CSUR), 54 0 (11s): 0 1--37, 2022

    2022

  3. [11]

    Pleak: Prompt leaking attacks against large language model applications

    Bo Hui, Haolin Yuan, Neil Gong, Philippe Burlina, and Yinzhi Cao. Pleak: Prompt leaking attacks against large language model applications. arXiv preprint arXiv:2405.06823, 2024

    2024 arXiv

  4. [12]

    How to combine membership-inference attacks on multiple updated machine learning models

    Matthew Jagielski, Stanley Wu, Alina Oprea, Jonathan Ullman, and Roxana Geambasu. How to combine membership-inference attacks on multiple updated machine learning models. Proceedings on Privacy Enhancing Technologies, 2023

    2023

  5. [13]

    Mistral 7b

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b. arXiv preprint arXiv:2310.06825, 2023

    arXiv

  6. [14]

    Mixtral of experts

    Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. Mixtral of experts. arXiv preprint arXiv:2401.04088, 2024

    2024 arXiv

  7. [15]

    Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing

    Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 55 0 (9): 0 1--35, 2023

    2023

  8. [16]

    Dataset inference: Ownership resolution in machine learning

    Pratyush Maini, Mohammad Yaghini, and Nicolas Papernot. Dataset inference: Ownership resolution in machine learning. arXiv preprint arXiv:2104.10706, 2021

    2021 arXiv

  9. [17]

    Llm dataset inference: Did you train on my dataset?, 2024

    Pratyush Maini, Hengrui Jia, Nicolas Papernot, and Adam Dziedzic. Llm dataset inference: Did you train on my dataset?, 2024

    2024

  10. [18]

    Membership inference attacks against diffusion models

    Tomoya Matsumoto, Takayuki Miura, and Naoto Yanai. Membership inference attacks against diffusion models. In 2023 IEEE Security and Privacy Workshops (SPW), pp.\ 77--83. IEEE, 2023

    2023

  11. [19]

    Language model inversion

    John X Morris, Wenting Zhao, Justin T Chiu, Vitaly Shmatikov, and Alexander M Rush. Language model inversion. arXiv preprint arXiv:2311.13647, 2023

    2023 arXiv

  12. [20]

    Application development using large language models

    Andrew Ng and Isa Fulford. Application development using large language models. NeurIPS 2023 Tutorials, 2023

    2023

  13. [21]

    Prompt engineering guide

    OpenAI. Prompt engineering guide. https://platform.openai.com/docs/guides/prompt-engineering, 2023. Accessed: June 14, 2024

    2023

  14. [22]

    Sentence-bert: Sentence embeddings using siamese bert-networks

    Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084, 2019

    1908 arXiv

  15. [23]

    White-box vs black-box: Bayes optimal strategies for membership inference

    Alexandre Sablayrolles, Matthijs Douze, Cordelia Schmid, Yann Ollivier, and Herv \'e J \'e gou. White-box vs black-box: Bayes optimal strategies for membership inference. In International Conference on Machine Learning, pp.\ 5558--5567. PMLR, 2019

    2019

  16. [24]

    Ml-leaks: Model and data independent membership inference attacks and defenses on machine learning models

    Ahmed Salem, Yang Zhang, Mathias Humbert, Pascal Berrang, Mario Fritz, and Michael Backes. Ml-leaks: Model and data independent membership inference attacks and defenses on machine learning models. arXiv preprint arXiv:1806.01246, 2018

    2018 arXiv

  17. [25]

    Prompt stealing attacks against large language models

    Zeyang Sha and Yang Zhang. Prompt stealing attacks against large language models. arXiv preprint arXiv:2402.12959, 2024

    2024 arXiv

  18. [26]

    Membership inference attacks against machine learning models

    Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP), pp.\ 3--18. IEEE, 2017

    2017

  19. [27]

    Systematic evaluation of privacy risks of machine learning models

    Liwei Song and Prateek Mittal. Systematic evaluation of privacy risks of machine learning models. In 30th USENIX Security Symposium (USENIX Security 21), pp.\ 2615--2632, 2021

    2021

  20. [28]

    Llama 2: Open foundation and fine-tuned chat models

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  21. [29]

    Canary in a coalmine: Better membership inference with ensembled adversarial queries

    Yuxin Wen, Arpit Bansal, Hamid Kazemi, Eitan Borgnia, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Canary in a coalmine: Better membership inference with ensembled adversarial queries. arXiv preprint arXiv:2210.10750, 2022

    arXiv

  22. [30]

    Hard prompts made easy: Gradient-based discrete optimization for prompt tuning and discovery

    Yuxin Wen, Neel Jain, John Kirchenbauer, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Hard prompts made easy: Gradient-based discrete optimization for prompt tuning and discovery. Advances in Neural Information Processing Systems, 36, 2024

    2024

  23. [31]

    Prsa: Prompt reverse stealing attacks against large language models

    Yong Yang, Xuhong Zhang, Yi Jiang, Xi Chen, Haoyu Wang, Shouling Ji, and Zonghui Wang. Prsa: Prompt reverse stealing attacks against large language models. arXiv preprint arXiv:2402.19200, 2024

    2024 arXiv

  24. [32]

    Privacy risk in machine learning: Analyzing the connection to overfitting

    Samuel Yeom, Irene Giacomelli, Matt Fredrikson, and Somesh Jha. Privacy risk in machine learning: Analyzing the connection to overfitting. In 2018 IEEE 31st computer security foundations symposium (CSF), pp.\ 268--282. IEEE, 2018

    2018

  25. [33]

    Effective prompt extraction from language models

    Yiming Zhang, Nicholas Carlini, and Daphne Ippolito. Effective prompt extraction from language models. arXiv preprint arXiv:2303.08493. URL https://arxiv.org/pdf/2307.06865

    arXiv

  26. [34]

    Universal and transferable adversarial attacks on aligned language models

    Andy Zou, Zifan Wang, J Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043, 2023

    2023 arXiv

  27. [35]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  28. [36]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  29. [37]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  30. [38]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.