Differentially Private In-context Learning via Sampling Few-shot Mixed with Zero-shot Outputs

Haosheng Gan; Hongyi Li; James Flemings; Meisam Razaviyayn; Murali Annavaram

REVIEW 2 major objections 5 minor 50 references

Differentially Private In-context Learning via Sampling Few-shot Mixed with Zero-shot Outputs

T0 review · 2 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that sampling from a product of one-shot and zero-shot next-token distributions, each kept in a small Renyi-divergence ball around the zero-shot output, gives differential privacy for in-context text generation without…

desk verdict The paper's DP guarantee rests on a misapplied Rényi additivity step in Eq. (6), so the central privacy claim is unsupported; the empirical study is solid but does not repair the math. read the letter →

arxiv 2501.19287 v1 pith:54EBNXNJ submitted 2025-01-31 cs.LG

James Flemings , Haosheng Gan , Hongyi Li , Meisam Razaviyayn , Murali Annavaram This is my paper · ORCID

classification cs.LG

keywords differentialprivacyin-contextlearninglargelanguagemodelstextgenerationRenyidivergencesubsamplingamplificationdecodingframeworkprivacy-utilitytradeoff

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

In-context learning can leak private demonstrations through generated text, and prior differentially private generation schemes pay for privacy with injected noise or hundreds of extra inferences. This paper proposes DPS-MOZO, a decoding rule that for each next token builds several one-shot output distributions, each conditioned on one private demonstration, mixes each with the public zero-shot output, and then samples from their product. The mixing is controlled so that every per-demonstration distribution stays within a Renyi-divergence ball around the zero-shot output, which the paper proves yields an $(\alpha, 4\beta\alpha)$-RDP guarantee per token and, with subsampling and composition across $n_{\text{test}} T_{\max}$ tokens, an $(\varepsilon,\delta)$-DP guarantee for the whole response. Because privacy comes from the bounded mixing and from sampling, no calibrated noise is added, which the paper argues preserves generation quality. On SAMSum with Gemma 2 2B the method reports $\epsilon=2$ with a 0.3% ROUGE-L F1 drop relative to non-private few-shot ICL.

What carries the argument

The engine of the method is a mix-then-sample one-token decoder. For each sampled demonstration, the algorithm interpolates the one-shot logits with the zero-shot logits through a weight $\lambda_i$, and chooses the largest $\lambda_i$ that keeps the symmetric Renyi divergence between the mixed and zero-shot distributions at most $\beta\alpha$. The next token is then drawn from the normalized product of these mixed distributions over the single shared vocabulary; the product is the ensemble-decoding step, and the normalization is the part of the algorithm that matters for the proof. A public top-k truncation of the vocabulary, taken from the zero-shot logits, reduces the output space without revealing private information.

What would settle it

Build a three-token vocabulary and two adjacent demonstration sets differing in one example, run Algorithm 1 on a fixed query, compute the exact normalized product distributions for both datasets, and evaluate their Renyi divergence for small integer $\alpha$; if any token yields $D_\alpha > 4\beta\alpha$ for the $\beta$ chosen by Eq. (8), the paper's per-token privacy bound is violated.

Watch

Extended reading notes

Core claim

The paper's central claim is that private text generation from in-context examples can be obtained without additive noise by constraining how much each demonstration may influence the next-token distribution. For each sampled demonstration $(x_i,y_i)$, Algorithm 1 forms the mixed distribution $p_{\theta,\lambda_i}(\cdot|x_i,y_i,x,y_{<t}) = \operatorname{softmax}(\lambda_i \operatorname{logit}_\theta(\cdot|x_i,y_i,x,y_{<t}) + (1-\lambda_i)\operatorname{logit}_\theta(\cdot|x,y_{<t}))$ and chooses the largest $\lambda_i$ such that the symmetric Renyi divergence to the zero-shot distribution is at most $\beta\alpha$. The next token is then sampled from the normalized product of these mixed distributions over the shared vocabulary, with the token set first truncated to public top-k indices from the zero-shot logits. After subsampling $n_{\text{shots}}$ demonstrations without replacement, Theorem B.3 bounds the per-token privacy loss by $4\beta\alpha$; subsampling amplification and composition over all tokens convert this to an $(\varepsilon,\delta)$-DP guarantee, reported at $\epsilon=2$ with minimal utility loss.

Load-bearing premise

The proof's load-bearing step is that the common per-demonstration factors inside the product distribution cancel when comparing two adjacent datasets, but the algorithm renormalizes the product over one shared vocabulary, so those common factors affect the normalization constant and cannot be canceled.

Editorial extensions

If this is right

An operator can set the privacy budget by choosing the number of demonstrations, queries, and the divergence radius $\beta$ rather than by adding noise, so the method scales to generation settings where the output space is too large for standard private aggregation.
The offline variant DPS-MOZO+Off produces differentially private few-shot demonstrations once and then permits unlimited non-private decoding from them, because differential privacy is preserved under post-processing.
At $\epsilon=2$ on SAMSum with Gemma 2 2B, the method stays within 0.3% ROUGE-L F1 of non-private few-shot ICL, and it matches or exceeds the prior embedding-aggregation baseline at equal privacy across SAMSum, E2E, and WikiLarge.
The measured mixing weights are smallest at the first token and grow after the third or fourth token, meaning the privacy budget is consumed mostly when the model is choosing how to start the response.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A testable extension the paper does not run: on tasks where a demonstration sharply changes the next-token distribution, the allowed $\lambda_i$ will shrink, so the utility gap versus zero-shot should widen; the reported $\lambda(t)$ trajectories across datasets are consistent with this but do not confirm it.
The same divergence-ball recipe could be combined with other logit-level decoding transforms, such as nucleus sampling or repetition penalties, as long as the final distribution remains inside the ball; the paper only experiments with top-k truncation of the public token set.
The proof's additivity step in Eq. (6) is the one place a reader should re-derive from the normalized product directly; an exact toy-model check of that step would determine whether the reported $\epsilon$ values are the true guarantee or an optimistic one.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You should know two things about this paper before spending time on it. First, the problem it targets is real — DP for in-context learning on generation tasks is under-served, and Wu et al.'s 100x-ensemble approach is expensive. Second, the paper's central claim, the DP guarantee, is not established. The privacy proof in Appendix B misapplies Rényi additivity, and everything downstream of that step (subsampling amplification, composition over ntest·Tmax tokens, the reported ε=2 numbers) inherits the gap.

The core idea is clean: mix each one-shot output distribution with the zero-shot output so each mixed distribution lies in a divergence ball around the public distribution, then sample from the product of the mixed distributions. The connection to Flemings et al. (2024) is explicit, and the offline/online variants are a legitimate new setting. The experiments are a genuine strength: three datasets, two model families, ablations over model size, shots, queries, and top-k, plus membership-inference attacks that land near chance. The writing is clear and the limitations (logit access, token-only threat model) are stated plainly.

The soft spot is load-bearing. In Theorem B.3, the output distribution of Algorithm 1 is the normalized pointwise product P_D(y) = (1/Z_D) ∏_i f_i(y) over a single vocabulary, but Eq. (6) applies the additivity theorem for product measures over independent spaces, splitting the divergence into a "common factors" term and the changed demonstration. Those factors do not cancel: they appear inside the normalization constants Z_D and Z_D', and the ratio Z_D'/Z_D depends on every common factor. For adjacent D and D', the likelihood ratio is (Z_D'/Z_D)·(f_i/f'_i), and that constant does not drop out of the Rényi integral. The stress-test note holds up on reading; I could not find a way to rescue Eq. (6) as stated. Since every reported (ε, δ) number derives from Theorem B.3, the privacy guarantee is unsupported. The MIA results are reassuring but do not repair the math.

Minor points: the mixing formula and λ selection come from the same group's prior work, which is acceptable but worth an independent check; λ is capped at 1.5 by heuristic; the bisection solver for β and λ can yield loose bounds.

This is for anyone working on private prediction or DP ICL. It deserves a serious referee — the problem matters, the experiments are real, and the flaw is specific enough to be fixable (a direct analysis of the normalized product, or a different sampling scheme). Send it to review, but the authors should expect a major revision aimed at the privacy proof.

Referee Report

2 major / 5 minor

Summary. The paper proposes DPS-MOZO, a decoding framework for differentially private in-context learning in text generation. For each next token, the method subsamples a few demonstrations from a private dataset, constructs per-demonstration one-shot token distributions mixed with a zero-shot distribution under a Renyi divergence constraint (with mixing weight lambda chosen by a bisection search), and samples the next token from the normalized pointwise product of the mixed distributions. Privacy is claimed via a per-token Renyi differential privacy bound, privacy amplification by subsampling, and composition over ntest*Tmax token generations. The paper reports strong empirical utility, e.g., only a 0.3% ROUGE-L F1 drop at epsilon=2 on SAMSum with Gemma 2 2B, and includes ablations and membership inference attacks.

Significance. If the privacy analysis were sound, DPS-MOZO would be a valuable contribution: it proposes noise-free DP for open-ended generation via sampling, avoids the heavy ensemble cost of prior work such as Wu et al. (2023), and reports favorable utility-privacy tradeoffs across multiple models and datasets. The empirical evaluation is reasonably thorough, with ablations over model size, number of shots, number of queries, and top-k, as well as membership inference experiments. However, the central theoretical claim -- the per-token Renyi divergence bound in Theorem B.3 -- is not established, and the error is load-bearing: the subsampling amplification, composition, and all reported (epsilon, delta) figures depend on it. The empirical results, while encouraging, do not repair the mathematical gap.

major comments (2)

[Appendix B, Theorem B.3 and Eq. (6)] The proof applies Theorem B.2 (additivity of Renyi divergence for product measures) to the sampling distribution of Algorithm 1 Line 16, but that distribution is a normalized pointwise product of per-demonstration distributions over the same token vocabulary, not a product measure over independent components. For adjacent datasets D and D' differing in demonstration i, the ratio of the sampling densities is P_D(y)/P_{D'}(y) = (Z_{D'}/Z_D) * f_i(y)/f'_i(y), where Z_D and Z_{D'} are the normalization constants and depend on all common factors. The common factors therefore do not cancel inside the Renyi integral, and Eq. (6) is false for nshots >= 2. Since subsampling amplification (Theorem A.4), composition over ntest*Tmax tokens, and the reported (epsilon, delta) values all rely on Theorem B.3, the central privacy guarantee is unsupported.
[Appendix B, Eq. (7) and surrounding argument] Even if the additivity step were replaced by a valid bound, the proof would need a new argument showing that the divergence between the normalized pointwise products is controlled by the per-factor divergence bounds; the current proof simply cancels common factors in the product, which is invalid. No alternative inequality (e.g., a tensorization or strong-data-processing argument for the specific product-of-experts distribution) is supplied. The paper therefore does not establish a per-token RDP bound for DPS-MOZO, and the claimed privacy guarantee should not be relied upon.

minor comments (5)

[Algorithm 1, Lines 7 and 10] The pseudocode sets logits to -infinity for all yt in K, where K is the top-k public indices; this removes the most likely tokens rather than keeping them. If the intent is top-k truncation, the lines should read 'forall yt notin K' (and similarly in the text describing the truncation). As written, the pseudocode conflicts with the description and with the top-k ablation results.
[Appendix B, proof of Theorem B.3] The proof refers to 'Lemma B.2' when applying the additivity theorem, but the additivity result is stated as Theorem B.2; the citation should be corrected.
[Contributions, Section 1] The bullet point says '0.3% ROGUE-L' but the metric is ROUGE-L; the typo should be fixed.
[Figure 3 and Section 5.3] The caption says the figure plots lambda(t) evaluated on 'SAMSum, E2E, and Spider', but the subplot labels and the rest of the text refer to WikiLarge; the dataset name should be corrected.
[Theorem B.1] The notation 'epsilon_1 alpha' and 'epsilon_2 alpha' in the statement of the triangle-like inequality is confusing; please clarify whether these are bounds on D_alpha or scaled bounds, and state the exact quasi-triangle inequality being used.

Circularity Check

1 steps flagged · score 6.0 of 10

Theorem B.3's proof cancels common factors in a pointwise product over one vocabulary by invoking Rényi additivity for independent product measures; Eq. (6) assumes the multi-shot decoupling that the theorem must prove.

other [Appendix B, Theorem B.3 proof, Eq. (6); Algorithm 1, Line 16]
"= Dα(∏_{j≠i} p_{θ,λ_j}((x_j,y_j),x,y_{<t})p_{θ,λ_i}((x_i,y_i),x,y_{<t}) ∥ ∏_{j≠i} p_{θ,λ_j}((x_j,y_j),x,y_{<t})p_{θ,λ_i}((x′_i,y′_i),x,y_{<t})) = Dα(∏_{j≠i} p_{θ,λ_j}((x_j,y_j),x,y_{<t}) ∥ ∏_{j≠i} p_{θ,λ_j}((x_j,y_j),x,y_{<t})) + Dα(p_{θ,λ_i}((x_i,y_i),x,y_{<t}) ∥ p_{θ,λ_i}((x′_i,y′_i),x,y_{<t})) (6) ... where Eq. 6 uses Lemma B.2"

Theorem B.2 is additivity for product measures on independent component spaces. Algorithm 1 Line 16 instead samples a single token from the normalized pointwise product P_D(y)=(1/Z_D)∏_{i=1}^{nshots} f_i(y) on one vocabulary. For adjacent datasets D,D', P_D/P_{D'} = (Z_{D'}/Z_D)(f_i/f'_i); the normalizing constants depend on all common factors, and the Rényi integral is taken against P_{D'}, which contains every common factor, so those factors do not cancel. Eq. (6)'s decoupling is exactly the nontrivial multi-shot privacy property Theorem B.3 must prove. Importing it via Lemma B.2 makes the 4βα bound an assumption about the product distribution, not a consequence of the per-demonstration constraints.

full rationale

The utility experiments are independent benchmark comparisons, and the subsampling/composition machinery (Theorems A.2–A.4) relies on standard external results. The repeated self-citations to Flemings et al. (2024) for the mixing formula and λ-optimization are present but are not themselves load-bearing for Theorem B.3, whose proof is attempted in-paper. However, the key step of that proof—canceling all common demonstration factors in Eq. (6)—treats the next-token distribution, a normalized pointwise product over a single vocabulary, as if it were a product measure over independent spaces. Theorem B.2 does not apply to that object, so the claimed bound Dα(product ∥ product') ≤ 4βα is not actually derived from the algorithm's output distribution. This is circular in the narrow sense that the theorem assumes the decoupling that would make the conclusion true. Because this unsupported step is the foundation of the paper's central privacy guarantee, the circularity score is substantial (6), while the remaining accounting and the empirical measurements retain independent content.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method itself introduces no new physical or mathematical entities. The load-bearing assumptions are the misapplied Renyi additivity for pointwise products and the public nature of zero-shot outputs under adaptive generation. The free parameters are hyperparameters chosen for the privacy-utility tradeoff rather than fitted model parameters.

free parameters (4)

beta (target Renyi divergence radius) = 0.081 to 0.445 depending on dataset and epsilon (Table 5)
Chosen via privacy accounting in Eq. (8) to meet the target epsilon; it controls how much each one-shot distribution may diverge from zero-shot and is not fitted to utility.
k (top-k public indices) = 100
Output vocabulary is truncated to the top-k zero-shot tokens; chosen by hand for all experiments and affects utility and the convergence of one-shot and zero-shot distributions.
nshots = 4
Number of demonstrations and one-shot prompts; affects subsampling amplification q = nshots/|D| and utility; chosen by hand.
alpha (Renyi order) = 5 to 18 depending on epsilon and dataset (Table 5)
Used to convert RDP to approximate DP with roughly epsilon/2 conversion cost; selected by hand for each privacy budget.

assumptions (4)

domain assumption Renyi divergence additivity applies to the pointwise product of per-demonstration output distributions in Eq. (6)
The proof cancels all non-i factors in the Renyi divergence of the ensemble product. This is valid only for product measures over independent components, but Algorithm 1 samples from a normalized pointwise product over one token vocabulary.
domain assumption The zero-shot distribution p_theta(x, y_{<t}) is public and independent of the private dataset
Invoked in Theorem B.3 to treat zero-shot outputs as non-leaking. Since y_{<t} is generated from private-conditioned steps, this requires an adaptive composition argument that the paper does not fully provide.
standard math Subsampling amplification theorem for RDP from Wang et al. (2019)
Used to amplify privacy by sampling nshots examples without replacement from D. It is standard, but its application depends on the base mechanism being RDP, which is not established because of the invalid additivity step.
domain assumption The adversary sees only sampled output tokens, not logits or released mixed distributions
This is the stated threat model in Section 3 and the Discussion. It is necessary because DPS-MOZO protects only the sampled text, not the full output distribution.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Differentially Private In-context Learning via Sampling Few-shot Mixed with Zero-shot Outputs." pith.science (2026). https://pith.science/paper/54EBNXNJ

@misc{pith2026250119287,
  author       = {Pith},
  title        = {Pith review of: Differentially Private In-context Learning via Sampling Few-shot Mixed with Zero-shot Outputs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/54EBNXNJ}},
  note         = {Machine review of arXiv:2501.19287}
}

abstract

In-context learning (ICL) has shown promising improvement in downstream task adaptation of LLMs by augmenting prompts with relevant input-output examples (demonstrations). However, the ICL demonstrations can contain privacy-sensitive information, which can be leaked and/or regurgitated by the LLM output. Differential Privacy (DP), a widely adopted privacy safeguard, has emerged to mitigate this privacy leakage, with recent work demonstrating strong privacy-utility tradeoffs in classification tasks for ICL. However, generation tasks for ICL are challenging due to the high-dimensional output space of open-ended generation. To this end, we propose $\texttt{dps-mozo}$, Differentially Private Sampling by Mixing One-shot with Zero-shot Outputs, a decoding framework that generates DP text by sampling from the product of multiple one-shot outputs mixed with a zero-shot output. This mixing effectively reduces the amount of information that can be leaked by each demonstration. By utilizing the inherent randomness in sampling from the mixed distributions, we can achieve DP without adding noise, thereby improving the privacy-utility tradeoff. Our experimental evaluations show $\texttt{dps-mozo}$ can achieve a strong privacy guarantee, $\epsilon=2$, with minimal utility degradation compared to non-private few-shot learning, $\textbf{0.3}$% ROUGE-L F1 score decrease on the SAMSum dataset with Gemma 2 2B.

Figures

Figures reproduced from arXiv: 2501.19287 by the authors.

**Figure 1.** Problem setup for privacy-preserving in-context learning. knowledge obtained through pre-training of vast public data (Devlin et al., 2018; Radford et al., 2019) and 2) the inputoutput mapping from the demonstrations learned by the model during inference (Wei et al., 2023). As a result, ICL is gaining popularity as an alternative to fine-tuning, thereby avoiding the complexities involved in LLM training. A growing … view at source ↗

**Figure 2.** (a) A high-level overview of DPS-MOZO broken down into four phases: (1) Subsample Demonstrations. First, nshots demonstrations are randomly selected without replacement from a private dataset, then one-shot prompts containing a demonstration, the input query, and the current output are generated. Also, a zero-shot prompt containing only the input query and the current output is generated. (2) Inference. An LLM perfo… view at source ↗

**Figure 3.** Tracking λ (t) from DPS-MOZO+On, the average value across all test queries of the smallest λi (y-axis) at the t-th generation (x-axis). generating few-shots which is also observed in Tang et al. (2023) We also observe that for all models and datasets, either DPS-MOZO+On or DPS-MOZO+Off outperforms Wu et al. (2023). These results suggest that this baseline overly relies on the zero-shot capabilities of the LLM for ge… view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: Ablation studies on dialogue summarization using the SAMSum dataset [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

**Figure 5.** Figure 5: , [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]

**Figure 6.** Figure 6: Example of one-shot and zero-shot prompts used for the WikiLarge dataset where red text is an example input-output pair (xi, yi) and blue text is the input query x. One-shot Prompt Dialogue: Mary: Hi Mike! Mike: Hello :) Mary: do u have any plans for tonight? Mike: I’m…

**Figure 7.** Figure 7: Example of one-shot and zero-shot prompts used for the SAMSum dataset where red text is an example input-output pair (xi, yi) and blue text is the input query x [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

50 extracted references · 15 canonical work pages

[1]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...
[2]

Privacy amplification by subsampling: Tight analyses via couplings and divergences

Balle, B., Barthe, G., and Gaboardi, M. Privacy amplification by subsampling: Tight analyses via couplings and divergences. Advances in neural information processing systems, 31, 2018

work page 2018
[3]

Hypothesis testing interpretations and renyi differential privacy

Balle, B., Barthe, G., Gaboardi, M., Hsu, J., and Sato, T. Hypothesis testing interpretations and renyi differential privacy. In International Conference on Artificial Intelligence and Statistics, pp.\ 2496--2506. PMLR, 2020

2020
[4]

D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al

Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. Language models are few-shot learners. Advances in neural information processing systems, 33: 0 1877--1901, 2020

1901
[5]

D., Steinke, T., Hayase, J., Cooper, A

Carlini, N., Paleka, D., Dvijotham, K. D., Steinke, T., Hayase, J., Cooper, A. F., Lee, K., Jagielski, M., Nasr, M., Conmy, A., et al. Stealing part of a production language model. arXiv preprint arXiv:2403.06634, 2024

arXiv 2024
[6]

Bert: Pre-training of deep bidirectional transformers for language understanding

Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018

arXiv 2018
[7]

On the privacy risk of in-context learning

Duan, H., Dziedzic, A., Yaghini, M., Papernot, N., and Boenisch, F. On the privacy risk of in-context learning. In The 61st Annual Meeting Of The Association For Computational Linguistics, 2023

work page 2023
[8]

Flocks of stochastic parrots: Differentially private prompt learning for large language models

Duan, H., Dziedzic, A., Papernot, N., and Boenisch, F. Flocks of stochastic parrots: Differentially private prompt learning for large language models. Advances in Neural Information Processing Systems, 36, 2024

work page 2024

Show all 50 references

[9]

The llama 3 herd of models

Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

2024 arXiv
[10]

Differential privacy

Dwork, C. Differential privacy. In International colloquium on automata, languages, and programming, pp.\ 1--12. Springer, 2006

2006
[11]

and Feldman, V

Dwork, C. and Feldman, V. Privacy-preserving prediction. In Conference On Learning Theory, pp.\ 1693--1702. PMLR, 2018

2018
[12]

The algorithmic foundations of differential privacy

Dwork, C., Roth, A., et al. The algorithmic foundations of differential privacy. Foundations and Trends in Theoretical Computer Science , 9 0 (3--4): 0 211--407, 2014

2014
[13]

Hierarchical neural story generation

Fan, A., Lewis, M., and Dauphin, Y. Hierarchical neural story generation. arXiv preprint arXiv:1805.04833, 2018

2018 arXiv
[14]

Differentially private next-token prediction of large language models

Flemings, J., Razaviyayn, M., and Annavaram, M. Differentially private next-token prediction of large language models. arXiv preprint arXiv:2403.15638, 2024

2024 arXiv
[15]

Submix: Practical private prediction for large-scale language models

Ginart, A., van der Maaten, L., Zou, J., and Guo, C. Submix: Practical private prediction for large-scale language models. arXiv preprint arXiv:2201.00971, 2022

2022 arXiv
[16]

SAMS um corpus: A human-annotated dialogue dataset for abstractive summarization

Gliwa, B., Mochol, I., Biesek, M., and Wawer, A. SAMS um corpus: A human-annotated dialogue dataset for abstractive summarization. In Proceedings of the 2nd Workshop on New Frontiers in Summarization, pp.\ 70--79, Hong Kong, China, November 2019. Association for Computational ...

2019 doi
[17]

E., Backes, M., and Dziedzic, A

Hanke, V., Blanchard, T., Boenisch, F., Olatunji, I. E., Backes, M., and Dziedzic, A. Open llms are necessary for current private adaptations and outperform their closed alternatives. arXiv preprint arXiv:2411.05818, 2024

2024 arXiv
[18]

T., Zhang, C., Li, Z., Li, B., and Wang, Z

Hong, J., Wang, J. T., Zhang, C., Li, Z., Li, B., and Wang, Z. Dp-opt: Make large language model your privacy-preserving prompt engineer. arXiv preprint arXiv:2312.03724, 2023

2023 arXiv
[19]

Local differential privacy for sampling

Husain, H., Balle, B., Cranko, Z., and Nock, R. Local differential privacy for sampling. In International Conference on Artificial Intelligence and Statistics, pp.\ 3404--3413. PMLR, 2020

2020
[20]

and Li, J

Li, X. and Li, J. Angle-optimized text embeddings. arXiv preprint arXiv:2309.12871, 2023

2023 arXiv
[21]

Rouge: A package for automatic evaluation of summaries

Lin, C.-Y. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pp.\ 74--81, 2004

2004
[22]

Differentially private decoding in large language models

Majmudar, J., Dupuy, C., Peris, C., Smaili, S., Gupta, R., and Zemel, R. Differentially private decoding in large language models. arXiv preprint arXiv:2205.13621, 2022

2022 arXiv
[23]

Noisy channel language model prompting for few-shot text classification

Min, S., Lewis, M., Hajishirzi, H., and Zettlemoyer, L. Noisy channel language model prompting for few-shot text classification. arXiv preprint arXiv:2108.04106, 2021

2021 arXiv
[24]

Rethinking the role of demonstrations: What makes in-context learning work? arXiv preprint arXiv:2202.12837, 2022

Min, S., Lyu, X., Holtzman, A., Artetxe, M., Lewis, M., Hajishirzi, H., and Zettlemoyer, L. Rethinking the role of demonstrations: What makes in-context learning work? arXiv preprint arXiv:2202.12837, 2022

2022 arXiv
[25]

R \'e nyi differential privacy

Mironov, I. R \'e nyi differential privacy. In 2017 IEEE 30th computer security foundations symposium (CSF), pp.\ 263--275. IEEE, 2017

2017
[26]

Smooth sensitivity and sampling in private data analysis

Nissim, K., Raskhodnikova, S., and Smith, A. Smooth sensitivity and sampling in private data analysis. In Proceedings of the thirty-ninth annual ACM symposium on Theory of computing, pp.\ 75--84, 2007

2007
[27]

The e2e dataset: New challenges for end-to-end generation

Novikova, J., Du s ek, O., and Rieser, V. The e2e dataset: New challenges for end-to-end generation. arXiv preprint arXiv:1706.09254, 2017

2017 arXiv
[28]

Semi-supervised knowledge transfer for deep learning from private training data

Papernot, N., Abadi, M., Erlingsson, U., Goodfellow, I., and Talwar, K. Semi-supervised knowledge transfer for deep learning from private training data. arXiv preprint arXiv:1610.05755, 2016

2016 arXiv
[29]

Scalable private learning with pate

Papernot, N., Song, S., Mironov, I., Raghunathan, A., Talwar, K., and Erlingsson, \'U . Scalable private learning with pate. arXiv preprint arXiv:1802.08908, 2018

2018 arXiv
[30]

Are chatbots ready for privacy-sensitive applications? an investigation into input regurgitation and prompt-induced sanitization

Priyanshu, A., Vijay, S., Kumar, A., Naidu, R., and Mireshghallah, F. Are chatbots ready for privacy-sensitive applications? an investigation into input regurgitation and prompt-induced sanitization. arXiv preprint arXiv:2305.15008, 2023

2023 arXiv
[31]

Language models are unsupervised multitask learners

Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al. Language models are unsupervised multitask learners. OpenAI blog, 1 0 (8): 0 9, 2019

2019
[32]

Membership inference attacks against machine learning models

Shokri, R., Stronati, M., Song, C., and Shmatikov, V. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP), pp.\ 3--18. IEEE, 2017

2017
[33]

Composition of differential privacy & privacy amplification by subsampling

Steinke, T. Composition of differential privacy & privacy amplification by subsampling. arXiv preprint arXiv:2210.00597, 2022

2022 arXiv
[34]

A., Manoel, A., Mireshghallah, F., Lin, Z., Gopi, S., Kulkarni, J., and Sim, R

Tang, X., Shin, R., Inan, H. A., Manoel, A., Mireshghallah, F., Lin, Z., Gopi, S., Kulkarni, J., and Sim, R. Privacy-preserving in-context learning with differentially private few-shot generation. arXiv preprint arXiv:2309.11765, 2023

2023 arXiv
[35]

G., Hardin, C., Bhupatiraju, S., Hussenot, L., Mesnard, T., Shahriari, B., Ram \'e , A., et al

Team, G., Riviere, M., Pathak, S., Sessa, P. G., Hardin, C., Bhupatiraju, S., Hussenot, L., Mesnard, T., Shahriari, B., Ram \'e , A., et al. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118, 2024

2024 arXiv
[36]

L., and He, H

Tian, Z., Zhao, Y., Huang, Z., Wang, Y.-X., Zhang, N. L., and He, H. Seqpate: Differentially private text generation via knowledge distillation. Advances in Neural Information Processing Systems, 35: 0 11117--11130, 2022

2022
[37]

and Harremos, P

Van Erven, T. and Harremos, P. R \'e nyi divergence and kullback-leibler divergence. IEEE Transactions on Information Theory, 60 0 (7): 0 3797--3820, 2014

2014
[38]

Voorhees, E. M. and Tice, D. M. Building a question answering test collection. In Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, pp.\ 200--207, 2000

2000
[39]

Decodingtrust: A comprehensive assessment of trustworthiness in gpt models

Wang, B., Chen, W., Pei, H., Xie, C., Kang, M., Zhang, C., Xu, C., Xiong, Z., Dutta, R., Schaeffer, R., et al. Decodingtrust: A comprehensive assessment of trustworthiness in gpt models. In NeurIPS, 2023

2023
[40]

Privacy for free: Posterior sampling and stochastic gradient monte carlo

Wang, Y.-X., Fienberg, S., and Smola, A. Privacy for free: Posterior sampling and stochastic gradient monte carlo. In International Conference on Machine Learning, pp.\ 2493--2502. PMLR, 2015

2015
[41]

Wang, Y.-X., Balle, B., and Kasiviswanathan, S. P. Subsampled r \'e nyi differential privacy and analytical moments accountant. In The 22nd international conference on artificial intelligence and statistics, pp.\ 1226--1235. PMLR, 2019

2019
[42]

Larger language models do in-context learning differently

Wei, J., Wei, J., Tay, Y., Tran, D., Webson, A., Lu, Y., Chen, X., Liu, H., Huang, D., Zhou, D., et al. Larger language models do in-context learning differently. arXiv preprint arXiv:2303.03846, 2023

2023 arXiv
[43]

T., and Mittal, P

Wu, T., Panda, A., Wang, J. T., and Mittal, P. Privacy-preserving in-context learning for large language models. arXiv preprint arXiv:2305.01639, 2023

2023 arXiv
[44]

A., Nori, H., Jiang, H., Zhang, H., Lee, Y

Xie, C., Lin, Z., Backurs, A., Gopi, S., Yu, D., Inan, H. A., Nori, H., Jiang, H., Zhang, H., Lee, Y. T., et al. Differentially private synthetic data via foundation model apis 2: Text. arXiv preprint arXiv:2403.01749, 2024

2024 arXiv
[45]

Context-aware decoding reduces hallucination in query-focused summarization

Xu, Z. Context-aware decoding reduces hallucination in query-focused summarization. arXiv preprint arXiv:2312.14335, 2023

2023
[46]

Privacy risk in machine learning: Analyzing the connection to overfitting

Yeom, S., Giacomelli, I., Fredrikson, M., and Jha, S. Privacy risk in machine learning: Analyzing the connection to overfitting. In 2018 IEEE 31st computer security foundations symposium (CSF), pp.\ 268--282. IEEE, 2018

2018
[47]

Q., and Artzi, Y

Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., and Artzi, Y. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675, 2019

1904 arXiv
[48]

and Lapata, M

Zhang, X. and Lapata, M. Sentence simplification with deep reinforcement learning. arXiv preprint arXiv:1703.10931, 2017

2017 arXiv
[49]

Character-level convolutional networks for text classification

Zhang, X., Zhao, J., and LeCun, Y. Character-level convolutional networks for text classification. Advances in neural information processing systems, 28, 2015

2015
[50]

Calibrate before use: Improving few-shot performance of language models

Zhao, Z., Wallace, E., Feng, S., Klein, D., and Singh, S. Calibrate before use: Improving few-shot performance of language models. In International conference on machine learning, pp.\ 12697--12706. PMLR, 2021

2021

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.

[1] [1]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

[2] [2]

Privacy amplification by subsampling: Tight analyses via couplings and divergences

Balle, B., Barthe, G., and Gaboardi, M. Privacy amplification by subsampling: Tight analyses via couplings and divergences. Advances in neural information processing systems, 31, 2018

work page 2018

[3] [3]

Hypothesis testing interpretations and renyi differential privacy

Balle, B., Barthe, G., Gaboardi, M., Hsu, J., and Sato, T. Hypothesis testing interpretations and renyi differential privacy. In International Conference on Artificial Intelligence and Statistics, pp.\ 2496--2506. PMLR, 2020

2020

[4] [4]

D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al

Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. Language models are few-shot learners. Advances in neural information processing systems, 33: 0 1877--1901, 2020

1901

[5] [5]

D., Steinke, T., Hayase, J., Cooper, A

Carlini, N., Paleka, D., Dvijotham, K. D., Steinke, T., Hayase, J., Cooper, A. F., Lee, K., Jagielski, M., Nasr, M., Conmy, A., et al. Stealing part of a production language model. arXiv preprint arXiv:2403.06634, 2024

arXiv 2024

[6] [6]

Bert: Pre-training of deep bidirectional transformers for language understanding

Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018

arXiv 2018

[7] [7]

On the privacy risk of in-context learning

Duan, H., Dziedzic, A., Yaghini, M., Papernot, N., and Boenisch, F. On the privacy risk of in-context learning. In The 61st Annual Meeting Of The Association For Computational Linguistics, 2023

work page 2023

[8] [8]

Flocks of stochastic parrots: Differentially private prompt learning for large language models

Duan, H., Dziedzic, A., Papernot, N., and Boenisch, F. Flocks of stochastic parrots: Differentially private prompt learning for large language models. Advances in Neural Information Processing Systems, 36, 2024

work page 2024