Pith. sign in

REVIEW 3 major objections 5 minor 83 references

Scaling Laws for Differentially Private Language Models

T0 review · 3 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Differentially private LLM training has its own scaling laws, and they say compute-optimal models are far smaller than standard scaling laws suggest.

desk verdict A substantial and unusually candid empirical study of DP LM scaling, but the headline quantitative predictions extrapolate batch size across two to three untested orders of magnitude and need validation before being taken as accurate. read the letter →

arxiv 2501.18914 v1 pith:6LNEJMZS submitted 2025-01-31 cs.LG cs.CR

Ryan McKenna , Yangsibo Huang , Amer Sinha , Borja Balle , Zachary Charles , Christopher A. Choquette-Choo , Badih Ghazi , George Kaissis
show 4 more authors
Ravi Kumar Ruibo Liu Da Yu Chiyuan Zhang
This is my paper · ORCID
classification cs.LGcs.CR MSC 68T0768P27
keywords differentialprivacyscalinglawslargelanguagemodelsDP-SGDcompute-optimaltrainingnoise-batchratioprivacy-utilitytradeoffmaskedmodeling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the scaling laws that guide ordinary language-model training do not transfer to differentially private (DP) training, and it supplies a replacement. By training BERT-sized models from 4.5M to 778M parameters with DP-Adam across a grid of noise levels and iteration counts, the authors fit a function that predicts validation cross-entropy from model size, iterations, and the noise-batch ratio, and then use privacy accounting to convert any compute, privacy, and data budget into recommended model size, batch size, and iteration count. The fitted laws say that DP-optimal models are roughly an order of magnitude smaller than non-private compute-optimal models, that the token-to-model ratio rises from the usual 20 to thousands or more, and that the compute allocation prescribed by non-private scaling laws is far from optimal even at very large privacy budgets. If correct, the laws let practitioners choose training configurations that save 5x to 100x compute at fixed privacy and utility.

What carries the argument

The load-bearing object is the noise-batch ratio $\bar{\sigma}$, the standard deviation of Gaussian noise added to the averaged minibatch gradient, which the paper treats as the primary determinant of DP training dynamics. Experiments fix the physical batch size at 1024 and vary $\bar{\sigma}$ over 18 values; a semi-parametric fit, using isotonic regression to enforce monotonicity and extrapolating each training curve with $L = E + A/T^{\alpha}$, yields a log-space interpolated surface $L(M, T, \bar{\sigma})$. Post-hoc privacy accounting then converts any $(\epsilon, N, C)$ budget into the $\bar{\sigma}$ a configuration would incur, so the same fitted surface answers which model size, batch size, and iteration count are optimal under a given budget.

What would settle it

Train a model at a physical batch size of about 65536 with the same noise-batch ratio and iteration count as a batch-size-1024 run, then compare the achieved loss to the loss predicted by the fitted surface; a difference larger than the interpolation error would falsify the post-hoc accounting pipeline. A partial falsification already appears in the paper: Figures 7 and 8 show small physical batches beating large ones at equal noise-batch ratio, which is inconsistent with the independence assumption.

Watch

Extended reading notes

Core claim

The central claim is that under DP-SGD the compute-optimal allocation among model size, batch size, and iterations is governed by scaling laws that are qualitatively different from the non-private ones, and that these private laws are accurately captured by a fitted function $L(M, T, \bar{\sigma})$. The paper states it directly: the compute allocation predicted by non-private scaling laws is far from optimal under DP even for huge privacy budgets, while the paper can accurately predict the optimal breakdown of compute into model size, batch size, and iterations for virtually any privacy and data budget. It also finds that the optimal model size is typically at least an order of magnitude smaller with DP, that increased compute can saturate so that little loss reduction remains unless the privacy budget or dataset size also grows, and that compute-efficient DP configurations save 5x to 100x compute compared to natural DP baselines.

Load-bearing premise

For a fixed noise-batch ratio, the loss curve does not depend on the physical batch size, so measurements at batch size 1024 can predict behavior at batch sizes up to $10^4$ or $10^6$; the paper's own ablations show this is not fully true in the moderate-to-high noise regime.

Editorial extensions

If this is right

  • Compute-optimal DP configurations predicted by these laws save roughly 5x to 100x compute compared to typical DP baselines at matched privacy and utility.
  • The optimal model size under DP is typically 10x to 50x smaller than the non-private compute-optimal size, so billion-parameter DP pretraining requires very large data and privacy budgets or a different mechanism.
  • For fixed privacy and data budgets, increasing compute eventually yields little loss reduction; the critical compute budget grows with both the privacy budget and the data budget.
  • The token-to-model ratio in DP-optimal configurations ranges from about 1000 to 100000, compared with the usual 20 in non-private training, meaning DP prefers over-training small models.
  • The privacy-accounting analysis implies that doubling privacy or compute alone gives diminishing returns, while increasing them together gives consistent noise reduction.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the batch-size independence assumption fails in the direction the paper's Appendix C.3 suggests (smaller physical batches beat larger ones at equal noise-batch ratio), then the predicted optimal batch sizes are optimistic: real large-batch runs would land at higher loss than the fitted surface predicts, so the advertised compute savings could shrink in the very large-batch regime.
  • The same post-hoc accounting plus a fitted loss surface could be carried over to fine-tuning or to alternative DP mechanisms, with the qualitative findings (smaller models, higher token-to-model ratios, compute saturation) likely persisting even if the exact constants do not.
  • The Section 4.5 accounting analysis is mechanism-agnostic and suggests a cheap decision rule: when compute and privacy budgets are both tight, increase them together rather than separately, because doubling either alone yields diminishing returns in noise reduction.
  • A direct testable extension: run a single large-batch DP training at the recommended compute-optimal configuration and check whether the achieved loss matches the fitted surface within the reported error, which would validate or refute the post-hoc accounting pipeline.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper proposes a methodology for scaling laws of differentially private LLM training. It fixes the physical batch size B=1024 and models loss as a function of model size M, iterations T, and the noise-batch ratio \bar{\sigma} via a semi-parametric interpolated surface (Section 3.3). It then uses privacy accounting to convert compute, privacy, and data budgets into \bar{\sigma}, queries the fitted surface (Section 3.4), and reports compute-optimal M, B, and T. The main claims are that non-private compute allocation is far from optimal under DP, that optimal model sizes are much smaller under DP, that compute benefits saturate under fixed privacy/data budgets, and that the proposed configurations save 5x-100x compute while retaining comparable utility.

Significance. If validated, the paper would be a significant contribution: it is one of the first systematic studies of scaling laws for DP-pretrained language models, the experiment grid is large (6 model sizes, 18 noise-batch ratios, 3 learning rates), and the decoupling of privacy accounting from experimentation in Section 4.5 is a useful and domain-agnostic contribution. The paper is also unusually candid about its limitations, including Appendix C.3 and Appendix D, which strengthens confidence in the authors' reporting. However, the headline quantitative predictions hinge on a batch-size invariance assumption that the paper's own ablations show to be violated in the relevant regime, so the central quantitative claims need either additional validation or re-scoping.

major comments (3)
  1. [Section 3.2/3.4 and Appendices C.3-C.4] The central quantitative predictions extrapolate from a single physical batch size B=1024, but the paper's own ablation rejects the required invariance. The fitted surface L(M,T,\bar{\sigma}) is constructed entirely from B=1024 runs (Section 3.2), and Section 3.4 then queries it at hypothetical B up to 10^6 after converting (\epsilon,N,B,T) into \bar{\sigma}. Appendix C.3 finds that at a fixed noise-batch ratio, smaller physical batch sizes achieve lower loss than larger ones, with Table 3 reporting final training-set cross-entropy 3.586 at B=128 versus 4.057 at B=8192 for BertTiny; the authors state "we do not have a good explanation for this behavior." Since the predicted optimal batches in Table 2 and Figure 1 are 10^4-10^6, one to three orders of magnitude outside the data, the headline predictions sit exactly in the regime where the assumption is known to fail. This needs either validation runs at larger B, an explicit correction or uncertainty model, or a re-scoping of the predictive claims.
  2. [Section 4.4 and Figure 5] No out-of-sample validation of the predicted compute-optimal configurations is reported. The optimal M, B, and T values are obtained by minimizing the interpolated surface L(M,T,\bar{\sigma}) fitted to the same runs that produce the claim (Sections 3.3-3.4), so statements such as "we can accurately predict the optimal breakdown ... for virtually any privacy budget and dataset size" are in-sample statements unless held-out configurations are actually trained. Figure 5 should clarify whether the compute-optimal curves are measured runs or fitted predictions; if they are measured, the paper should state explicitly which configurations were trained and how they were chosen.
  3. [Appendix D] The tradeoff claims implicitly treat configurations with the same (\epsilon,\delta) as having the same privacy, but Appendix D shows this is not the case. Figure 15 demonstrates that for fixed (\epsilon,\delta)=(8,10^{-8}), compute, and model size, varying the batch size changes measured MIA advantage substantially. Because the headline recommendations vary B over orders of magnitude (Table 2), the reported cross-entropy comparisons are not comparisons at constant privacy. The paper's recommendation to report excess vulnerability should be applied to the headline predictions, or the claims should be rephrased in terms of nominal (\epsilon,\delta).
minor comments (5)
  1. [Section 3.2 and Appendices C.4/C.7] The reported learning-rate grid {2^-7, 2^-8, 2^-9} is inconsistent with Appendix C.4 and Figure 12, which use values such as 0.56, 0.57, 0.58, and 0.59; please harmonize these values.
  2. [Section 3.3] The stated input range "\bar{\sigma} \in [0.523, 0.56]" is inconsistent with the grid {2^{-6}, ..., 2^{-23}} in Section 3.2, and the stated upper model size "784M" is inconsistent with Table 1's "778M" for BertMega.
  3. [Appendices C.3-C.4] The noise-batch ratio values 0.520, 0.515, and 0.510 appear to lie outside the main experiment grid; please clarify whether these are intended values or typesetting errors.
  4. [Several figures and tables] Quantities like "1022 FLOPs" and "1019 FLOPs" should be typeset as 10^22 and 10^19 to avoid ambiguity.
  5. [Appendix B.2] The discussion of example-level versus user-level DP is important for interpreting the data budget N, but the paragraph is dense; a short concrete example would improve clarity.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the large-batch predictions rest on an explicitly tested invariance assumption whose violation is a validity risk, not a circular reduction.

full rationale

The paper's derivation chain is self-contained in the relevant sense: the fitted loss surface L(M,T,sigma-bar) is estimated directly from B=1024 DP-Adam runs, the mapping from (epsilon,N,B,T) to sigma-bar is supplied by the external dp_accounting library, and the optimal configurations are obtained by querying this fitted surface. This is an empirical scaling-law analysis rather than a first-principles derivation, so the in-sample nature of the queries is a limitation on predictive independence, not a circular reduction: the claimed optimal batch sizes are not identical to any fitted parameter, and the fitted surface could in principle have been contradicted by the Appendix C.3 ablations, which indeed show partial violations of the batch-size invariance assumption. The authors explicitly acknowledge this in Appendix A (this assumption may not be fully true) and in C.3 (we do not have a good explanation for this behavior), so the manuscript does not hide the assumption behind a definition. Self-citations such as Anil et al. 2022, Charles et al. 2024, Chua et al. 2024, and Kaissis et al. 2024 are present but are not load-bearing: the large-batch motivation is also cited to external work such as De et al. 2022, the privacy accounting is external, and the surprising batch-size effect is corroborated by an external citation to Sander et al. 2023. The baseline comparison in Section 4.4 is described as a measurement (We now measure the improvement...), providing independent grounding. Hence the central quantitative claims rest on an empirical fit plus an explicit, tested invariance assumption; the failure of that assumption is a correctness and robustness risk, not a circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The main empirical predictions are built from a measured loss grid; they do not invoke fitted constants beyond smoothing and extrapolation, but the grid itself and several modeling choices are assumptions in the sense that the paper's conclusions depend on them. The most fragile is the physical-batch-size invariance assumption that the authors themselves found to be imperfect.

free parameters (4)
  • Training-curve extrapolation coefficients (E, A, alpha in L = E + A/T^alpha) = Not reported (fitted per model/noise pair via curve_fit)
    Used in Section 3.3 to extrapolate smoothed loss beyond 128K iterations; affects saturating-compute tables.
  • Learning-rate grid (2^-7, 2^-8, 2^-9) = 2^-7 chosen as optimal baseline
    Learning rate is tuned but not modeled directly; the choice shapes the loss surface used for predictions.
  • Rolling smoothing window of 10 measurements = 10
    Data-cleaning choice that determines the smoothed loss values fed into the interpolator.
  • Sigmoid transform constants in Appendix E (offset 8, scale 1.6) = 8, 1.6
    Used only in the parametric variant L2, not in the main semi-parametric analysis.
assumptions (4)
  • domain assumption Loss depends on privacy budget and batch size only through the noise-batch ratio, enabling post-hoc extrapolation from B=1024 to much larger batch sizes.
    Section 3.1 states this directly; Appendix C.3 shows it is not fully true, making this a load-bearing assumption.
  • domain assumption Training for less than one physical epoch makes training loss an unbiased estimate of evaluation loss.
    Invoked in Section 3.2 to justify using training loss instead of held-out loss; the single-pass claim appears inconsistent with the stated dataset and training totals.
  • domain assumption Compute cost is 6*M*B*S*T and DP overhead is amortized away.
    Section 2.1 and Appendix B.3 adopt the Kaplan/Hoffmann FLOP model to define the compute budget that all scaling-law questions depend on.
  • domain assumption The dp_accounting library computes correct noise-batch ratios for the claimed (epsilon, delta) budgets.
    Privacy accounting is used to connect noise-batch ratio to epsilon, batch size, and iterations; the paper calibrates under both Poisson and deterministic batching and picks the lower noise multiplier.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Scaling Laws for Differentially Private Language Models." pith.science (2026). https://pith.science/paper/6LNEJMZS

@misc{pith2026250118914,
  author       = {Pith},
  title        = {Pith review of: Scaling Laws for Differentially Private Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6LNEJMZS}},
  note         = {Machine review of arXiv:2501.18914}
}
read the original abstract

Scaling laws have emerged as important components of large language model (LLM) training as they can predict performance gains through scale, and provide guidance on important hyper-parameter choices that would otherwise be expensive. LLMs also rely on large, high-quality training datasets, like those sourced from (sometimes sensitive) user data. Training models on this sensitive user data requires careful privacy protections like differential privacy (DP). However, the dynamics of DP training are significantly different, and consequently their scaling laws are not yet fully understood. In this work, we establish scaling laws that accurately model the intricacies of DP LLM training, providing a complete picture of the compute-privacy-utility tradeoffs and the optimal training configurations in many settings.

Figures

Figures reproduced from arXiv: 2501.18914 by the authors.

Figure 1
Figure 1. Optimal model size, batch size, and iterations for varying privacy and compute budgets, with a fixed data budget of 108 . Lines show minimum values for each hyper-parameter that achieve within 1% of optimal cross-entropy for constant-compute training. Shaded regions indicate the full range of near-optimal settings. Privacy Budget (ϵ) Data Budget (N) Compute Budget (C) Batch Size (B) Iterations (T) Model Size (M) 4.5… view at source ↗
Figure 2
Figure 2. Workflow for estimating cross entropy of different train￾ing configurations under given compute, privacy, and data budgets. dently fit a function for each model size and noise-batch ratio on data from iterations 16K to 128K. Scaling Law Fitting. After data cleaning, our goal is to fit a function L(M, T, σ¯) that estimates the loss under a M-parameter model training for T iter￾ations with a noise-batch ratio of σ¯. W… view at source ↗
Figure 3
Figure 3. (a-b) Best cross-entropy loss achieved for varying compute budgets, four data budgets, and two different privacy budgets. Each figure is annotated with the optimal model size at the inflection point for two of the curves. (c) Number of training tokens S·B·T divided by number of model parameters for the compute-optimal training configuration, fixing the data budget to N = 107 . amount of robustness for compute-optima… view at source ↗
Figures from the paper (16 more)
Figure 4
Figure 4. Figure 4: Marginal benefits of increasing the privacy budget (ϵ), compute budget (B), and data budget (N) on the noise-batch ratio. with a batch size of 283061. In all three, we fix the data budget to N = 107 . Each of these training configurations require 1019 FLOPs. The first …
Figure 5
Figure 5. Figure 5: Comparison of a compute-optimal training configuration to some natural baselines as a function of the privacy budget. All models are trained with a compute budget of 1019 FLOPs and a data budget of N = 107 respectively. some of the experimental observations presented e…
Figure 6
Figure 6. Figure 6: Cross entropy of best models trained in each setting. From top to bottom , we vary the Privacy Budget, Data Budget, and Compute Budget, keeping the other two budgets fixed to default values (bolded). From left to right, we vary the number of Iterations, the Model Size,…
Figure 7
Figure 7. Figure 7: Cross Entropy Loss of BertTiny averaged over 3 trials for different physical batch sizes and noise-batch ratio values. physical batch sizes of 128, 512, 2048, and 8192. For this ablation we focus on the BertTiny model, which we train for 128K iterations. We average the…
Figure 8
Figure 8. Figure 8: Smaller physical batch sizes achieve lower loss than larger ones. 2. Differences in Train / Eval Loss. Our main experiment measures the training loss, but since the loss is computed before incorporating the gradient into the model, and because we train for less than on…
Figure 9
Figure 9. Figure 9: Cross Entropy Loss of BertLarge averaged over 3 trials for different physical batch sizes and noise-batch ratio values. 4. Training Pipelines. It is natural to question whether this behavior is explained by some bug in the training pipeline. We carefully reviewed the i…
Figure 10
Figure 10. Figure 10: Loss on NanoDO (Liu et al., 2024). C.5. Training Throughput By looking at intermediates, using a single physical batch size, and separating the accounting from the experimentation we greatly reduce the number of experiments to run. However, the set of experiments we o…
Figure 11
Figure 11. Figure 11: Compute-optimal cross entropy, model size, and number of iterations when running DP-Adam with σ = 0. C.7. Optimal Learning Rates We now look at the training curves for different learning rates and different noise-batch ratio values. These results generally match expec…
Figure 12
Figure 12. Figure 12: Training curves for BertTiny (top) and BertMedium (bottom) with varying learning rates at different noise-batch ratio values. C.8. Optimal Compute Budget Allocation In this section, we extend the results from Section 4.1, including results for more settings of the dat…
Figure 13
Figure 13. Figure 13: Compute optimal model-sizes, batch sizes, and iterations for varying privacy budgets and compute budgets, and data budgets. Each row of plots corresponds to a different data budget of N = 106 , 107 , 108 , and 109 respectively. Each line corresponds to the minimum val…
Figure 14
Figure 14. Figure 14: Demonstration of our semi-parametric smoothing on BertTiny. 10 3 10 4 Batch Size 4.00 4.25 4.50 4.75 5.00 5.25 5.50 5.75 Cross Entropy 0.08 0.10 0.12 0.14 0.16 0.18 Advantage (a) 10 3 10 4 10 5 Batch Size 3.5 4.0 4.5 5.0 5.5 6.0 Cross Entropy 0.12 0.14 0.16 0.18 0.20 …
Figure 15
Figure 15. Figure 15: Varying the batch size (horizontal axis, log-scale) has a drastic effect on excess vulnerability (measured as MIA advantage, red, right vertical axis) for models with a fixed compute budget and size and a fixed privacy budget of (ϵ, δ) = (8, 10−8 ). (a): Compute budge…
Figure 16
Figure 16. Figure 16: Parametric private scaling law of Lˆ1 from Equation (3). Optimal fit with α = 0.71, β = 12.87, γ = 0.19. The two pannels show the same plot of observed cross entropy loss against the predicted loss from the scaling law, except the data points are colored differerently…
Figure 17
Figure 17. Figure 17: Relation between the noise-batch ratio and the cross entropy loss. (left) The data plotted in log-log scale. (right) The data plotted in linear scale, where the noise-batch ratio σ¯ is transformed according to a simple rule in Equation (4). to the fact that the noise-…
Figure 18
Figure 18. Figure 18: Parametric private scaling law of Lˆ2 from Equation (5). Optimal fit with α = 0.47, β = 0.12, γ = 0.95, α2 = −0.07. The two pannels show the same plot of observed cross entropy loss against the predicted loss from the scaling law, except the data points are colored di…
Figure 19
Figure 19. Figure 19: Optimal model sizes under according to the parametric private scaling law in Equation (5). In the Chinchilla parameterization of scaling law for non-private LLMs, the optimal model size under a certain compute budget (approximately represented by 6nparamsntokens) can …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

83 extracted references · 49 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  3. [3]

    B., Mironov, I., Talwar, K., and Zhang, L

    Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K., and Zhang, L. Deep learning with differential privacy. In CCS, pp.\ 308--318, 2016

    2016

  4. [4]

    L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al

    Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. GPT -4 technical report. arXiv:2303.08774, 2023

    arXiv 2023

  5. [5]

    The crossroads of innovation and privacy: Private synthetic data for generative AI

    Afonja, G., Sim, R., Lin, Z., Inan, A., and Yekhanin, S. The crossroads of innovation and privacy: Private synthetic data for generative AI . Blog post, 2024. URL https://www.microsoft.com/en-us/research/blog/the-crossroads-of-innovation-and-privacy-private-synthetic-data-for-generative-ai

    work page 2024

  6. [6]

    Private prediction for large-scale synthetic text generation

    Amin, K., Bie, A., Kong, W., Kurakin, A., Ponomareva, N., Syed, U., Terzis, A., and Vassilvitskii, S. Private prediction for large-scale synthetic text generation. arXi:2407.12108, 2024

    arXiv 2024

  7. [7]

    Large-scale differentially private BERT

    Anil, R., Ghazi, B., Gupta, V., Kumar, R., and Manurangsi, P. Large-scale differentially private BERT . In EMNLP (Findings), pp.\ 6481--6491, 2022

    work page 2022

  8. [8]

    M., Firat, O., Johnson, M., Lepikhin, D., Passos, A., Shakeri, S., Taropa, E., Bailey, P., Chen, Z., et al

    Anil, R., Dai, A. M., Firat, O., Johnson, M., Lepikhin, D., Passos, A., Shakeri, S., Taropa, E., Bailey, P., Chen, Z., et al. Palm 2 technical report. arXiv:2305.10403, 2023

    arXiv 2023

Show all 83 references
  1. [9]

    Privacy amplification by subsampling: Tight analyses via couplings and divergences, 2018

    Balle, B., Barthe, G., and Gaboardi, M. Privacy amplification by subsampling: Tight analyses via couplings and divergences, 2018

    2018

  2. [10]

    Reconstructing training data with informed adversaries

    Balle, B., Cherubin, G., and Hayes, J. Reconstructing training data with informed adversaries. In S & P, pp.\ 1138--1156, 2022

    2022

  3. [11]

    Private empirical risk minimization: Efficient algorithms and tight error bounds

    Bassily, R., Smith, A., and Thakurta, A. Private empirical risk minimization: Efficient algorithms and tight error bounds. In FOCS, pp.\ 464--473, 2014

    2014

  4. [12]

    H., Hayes, J., Stanforth, R., Stutz, D., Kohli, P., Smith, S

    Berrada, L., De, S., Shen, J. H., Hayes, J., Stanforth, R., Stutz, D., Kohli, P., Smith, S. L., and Balle, B. Unlocking accuracy and fairness in differentially private image classification. arXiv:2308.10888, 2023

    2023 arXiv

  5. [13]

    S., Sutawika, L., Schoelkopf, H., Anthony, Q., Purohit, S., and Raff, E

    Biderman, S., Prashanth, U. S., Sutawika, L., Schoelkopf, H., Anthony, Q., Purohit, S., and Raff, E. Emergent and predictable memorization in large language models. In NeurIPS, 2023

    2023

  6. [14]

    Scalable and efficient training of large convolutional neural networks with differential privacy

    Bu, Z., Mao, J., and Xu, S. Scalable and efficient training of large convolutional neural networks with differential privacy. In NeurIPS, 2022

    2022

  7. [15]

    Differentially private optimization on large model at small cost

    Bu, Z., Wang, Y., Zha, S., and Karypis, G. Differentially private optimization on large model at small cost. In ICML, pp.\ 3192--3218, 2023

    2023

  8. [16]

    Extracting training data from large language models

    Carlini, N., Tramer, F., Wallace, E., Jagielski, M., Herbert-Voss, A., Lee, K., Roberts, A., Brown, T., Song, D., Erlingsson, U., et al. Extracting training data from large language models. In USENIX Security, 2021

    2021

  9. [17]

    Quantifying memorization across neural language models

    Carlini, N., Ippolito, D., Jagielski, M., Lee, K., Tram \` e r, F., and Zhang, C. Quantifying memorization across neural language models. In ICLR, 2023

    2023

  10. [18]

    A., Paleka, D., Pearce, W., Anderson, H., Terzis, A., Thomas, K., and Tram \`e r, F

    Carlini, N., Jagielski, M., Choquette-Choo, C. A., Paleka, D., Pearce, W., Anderson, H., Terzis, A., Thomas, K., and Tram \`e r, F. Poisoning web-scale training datasets is practical. In S & P, pp.\ 407--425, 2024

    2024

  11. [19]

    B., Mitchell, N., Pillutla, K., and Rush, K

    Charles, Z., Ganesh, A., McKenna, R., McMahan, H. B., Mitchell, N., Pillutla, K., and Rush, K. Fine-tuning large language models with user-level differential privacy. arXiv:2407.07737, 2024

    2024 arXiv

  12. [20]

    Chen, X., Liang, C., Huang, D., Real, E., Wang, K., Liu, Y., Pham, H., Dong, X., Luong, T., Hsieh, C.-J., Lu, Y., and Le, Q. V. Symbolic discovery of optimization algorithms, 2023. URL https://arxiv.org/abs/2302.06675

    2023 arXiv

  13. [21]

    Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., Schuh, P., Shi, K., Tsvyashchenko, S., Maynez, J., Rao, A., Barnes, P., Tay, Y., Shazeer, N., Prabhakaran, V., Reif, E., Du, N., Hutchinson, B., Pope,...

    2022

  14. [22]

    Mind the privacy unit! user-level differential privacy for language model fine-tuning

    Chua, L., Ghazi, B., Huang, Y., Kamath, P., Kumar, R., Liu, D., Manurangsi, P., Sinha, A., and Zhang, C. Mind the privacy unit! user-level differential privacy for language model fine-tuning. In CoLM, 2024 a

    2024

  15. [23]

    Scalable DP-SGD : Shuffling vs

    Chua, L., Ghazi, B., Kamath, P., Kumar, R., Manurangsi, P., Sinha, A., and Zhang, C. Scalable DP-SGD : Shuffling vs. P oisson subsampling. In NeurIPS, 2024 b

    2024

  16. [24]

    L., and Balle, B

    De, S., Berrada, L., Hayes, J., Smith, S. L., and Balle, B. Unlocking high-accuracy differentially private image classification through scale. arXiv:2204.13650, 2022

    2022 arXiv

  17. [25]

    BERT : Pre-training of deep bidirectional transformers for language understanding

    Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. BERT : Pre-training of deep bidirectional transformers for language understanding. In NAACL-HLT, pp.\ 4171--4186, 2019

    2019

  18. [26]

    S., Wang, T., Huang, C., and Sun, H

    Du, M., Yue, X., Chow, S. S., Wang, T., Huang, C., and Sun, H. DP -forward: Fine-tuning and inference on language models with differential privacy in forward pass. In CCS, pp.\ 2665--2679, 2023

    2023

  19. [27]

    Flocks of stochastic parrots: Differentially private prompt learning for large language models

    Duan, H., Dziedzic, A., Papernot, N., and Boenisch, F. Flocks of stochastic parrots: Differentially private prompt learning for large language models. In NeurIPS, 2023 a

    2023

  20. [28]

    On the privacy risk of in-context learning

    Duan, H., Dziedzic, A., Yaghini, M., Papernot, N., and Boenisch, F. On the privacy risk of in-context learning. In ACL, 2023 b

    2023

  21. [29]

    The Llama 3 herd of models

    Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al. The Llama 3 herd of models. arXiv:2407.21783, 2024

    2024 arXiv

  22. [30]

    Calibrating noise to sensitivity in private data analysis

    Dwork, C., McSherry, F., Nissim, K., and Smith, A. Calibrating noise to sensitivity in private data analysis. In TCC, pp.\ 265--284, 2006

    2006

  23. [31]

    Y., Smyrnis, G., Shankar, V., Gururangan, S., Wortsman, M., Shao, R., Mercat, J., Fang, A., Li, J., Keh, S., et al

    Gadre, S. Y., Smyrnis, G., Shankar, V., Gururangan, S., Wortsman, M., Shao, R., Mercat, J., Fang, A., Li, J., Keh, S., et al. Language models scale reliably with over-training and on downstream tasks. arXiv:2403.08540, 2024

    2024 arXiv

  24. [32]

    Predictability and surprise in large generative models

    Ganguli, D., Hernandez, D., Lovitt, L., Askell, A., Bai, Y., Chen, A., Conerly, T., Dassarma, N., Drain, D., Elhage, N., El Showk, S., Fort, S., Hatfield-Dodds, Z., Henighan, T., Johnston, S., Jones, A., Joseph, N., Kernian, J., Kravec, S., Mann, B., Nanda, N., Ndousse, K., Ol...

    2022

  25. [33]

    Gemini: a family of highly capable multimodal models

    Gemini Team . Gemini: a family of highly capable multimodal models. arXiv:2312.11805, 2023

    2023 arXiv

  26. [34]

    S., Love, J., et al

    Gemma Team , Mesnard, T., Hardin, C., Dadashi, R., Bhupatiraju, S., Pathak, S., Sifre, L., Rivi \`e re, M., Kale, M. S., Love, J., et al. Gemma: Open models based on gemini research and technology. arXiv:2403.08295, 2024 a

    2024 arXiv

  27. [35]

    G., Hardin, C., Bhupatiraju, S., Hussenot, L., Mesnard, T., Shahriari, B., Ram \'e , A., et al

    Gemma Team , Riviere, M., Pathak, S., Sessa, P. G., Hardin, C., Bhupatiraju, S., Hussenot, L., Mesnard, T., Shahriari, B., Ram \'e , A., et al. Gemma 2: Improving open language models at a practical size. arXiv:2408.00118, 2024 b

    2024 arXiv

  28. [36]

    L., Wiles, O., and Balle, B

    Ghalebikesabi, S., Berrada, L., Gowal, S., Ktena, I., Stanforth, R., Hayes, J., De, S., Smith, S. L., Wiles, O., and Balle, B. Differentially private diffusion models generate useful synthetic images. arXiv:2302.13861, 2023

    2023 arXiv

  29. [37]

    and Latonero, M

    Gold, Z. and Latonero, M. Robots welcome: Ethical and legal considerations for web crawling and scraping. Wash. JL Tech. & Arts, 2017

    2017

  30. [38]

    Google's differential privacy libraries., 2022

    Google DP Team . Google's differential privacy libraries., 2022. https://github.com/google/differential-privacy

    2022

  31. [39]

    Hoffmann, J., Borgeaud, S., Mensch, A., Buchatskaya, E., Cai, T., Rutherford, E., Casas, D. d. L., Hendricks, L. A., Welbl, J., Clark, A., et al. Training compute-optimal large language models. arXiv:2203.15556, 2022

    2022 arXiv

  32. [40]

    T., Zhang, C., Li, Z., Li, B., and Wang, Z

    Hong, J., Wang, J. T., Zhang, C., Li, Z., Li, B., and Wang, Z. DP-OPT : Make large language model your privacy-preserving prompt engineer. In ICLR, 2024

    2024

  33. [41]

    Huber, P. J. Robust estimation of a location parameter. In Breakthroughs in statistics: Methodology and distribution, pp.\ 492--518. Springer, 1992

    1992

  34. [42]

    A., and Carlini, N

    Ippolito, D., Tram \`e r, F., Nasr, M., Zhang, C., Jagielski, M., Lee, K., Choquette-Choo, C. A., and Carlini, N. Preventing verbatim memorization in language models gives a false sense of privacy. arXiv:2210.17546, 2022

    2022 arXiv

  35. [43]

    Bounding data reconstruction attacks with the hypothesis testing interpretation of differential privacy

    Kaissis, G., Hayes, J., Ziller, A., and Rueckert, D. Bounding data reconstruction attacks with the hypothesis testing interpretation of differential privacy. arXiv:2307.03928, 2023

    2023 arXiv

  36. [44]

    Beyond the calibration point: Mechanism comparison in differential privacy

    Kaissis, G., Kolek, S., Balle, B., Hayes, J., and Rueckert, D. Beyond the calibration point: Mechanism comparison in differential privacy. In ICML, pp.\ 22840--22860, 2024

    2024

  37. [45]

    B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D

    Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D. Scaling laws for neural language models. arXiv:2001.08361, 2020

    2001 arXiv

  38. [46]

    Kingma, D. P. and Ba, J. Adam: A method for stochastic optimization. In ICLR, 2015

    2015

  39. [47]

    and Ponomareva, N

    Kurakin, A. and Ponomareva, N. Protecting users with differentially private synthetic training data. Blog post, 2024. URL https://research.google/blog/protecting-users-with-differentially-private-synthetic-training-data/

    2024

  40. [48]

    Toward training at ImageNet scale with differential privacy

    Kurakin, A., Song, S., Chien, S., Geambasu, R., Terzis, A., and Thakurta, A. Toward training at ImageNet scale with differential privacy. arXiv:2201.12328, 2022

    2022 arXiv

  41. [49]

    Large language models can be strong differentially private learners

    Li, X., Tram \` e r, F., Liang, P., and Hashimoto, T. Large language models can be strong differentially private learners. In ICLR, 2022

    2022

  42. [50]

    J., Novak, R., Lee, J., Wortsman, M., Xiao, L., Everett, K., Alemi, A

    Liu, P. J., Novak, R., Lee, J., Wortsman, M., Xiao, L., Everett, K., Alemi, A. A., Kurzeja, M., Marcenac, P., Gur, I., Kornblith, S., Xu, K., Elsayed, G., Fischer, I., Pennington, J., Adlam, B., and Dickstein, J.-S. NanoDO : A minimal transformer decoder-only language model im...

    2024

  43. [51]

    and Hutter, F

    Loshchilov, I. and Hutter, F. Decoupled weight decay regularization, 2019. URL https://arxiv.org/abs/1711.05101

    2019 arXiv

  44. [52]

    Analyzing leakage of personally identifiable information in language models

    Lukas, N., Salem, A., Sim, R., Tople, S., Wutschitz, L., and Zanella-B \'e guelin, S. Analyzing leakage of personally identifiable information in language models. In S & P, 2023

    2023

  45. [53]

    McCandlish, S., Kaplan, J., Amodei, D., and Team, O. D. An empirical model of large-batch training. arXiv:1812.06162, 2018

    2018 arXiv

  46. [54]

    Updating quasi- N ewton matrices with limited storage

    Nocedal, J. Updating quasi- N ewton matrices with limited storage. Mathematics of Computation, 35 0 (151): 0 773--782, 1980

    1980

  47. [55]

    and Wright, S

    Nocedal, J. and Wright, S. J. Numerical optimization. Springer, 1999

    1999

  48. [56]

    B., Vassilvitskii, S., Chien, S., and Thakurta, A

    Ponomareva, N., Hazimeh, H., Kurakin, A., Xu, Z., Denison, C., McMahan, H. B., Vassilvitskii, S., Chien, S., and Thakurta, A. G. How to DP -fy ML : A practical guide to machine learning with differential privacy. JAIR, 2023

    2023

  49. [57]

    S., Deng, A., O'Brien, K., SV, J., Khan, M

    Prashanth, U. S., Deng, A., O'Brien, K., SV, J., Khan, M. A., Borkar, J., Choquette-Choo, C. A., Fuehne, J. R., Biderman, S., Ke, T., et al. Recite, reconstruct, recollect: Memorization in LMs as a multifaceted phenomenon. arXiv:2406.17746, 2024

    2024 arXiv

  50. [58]

    K., Charles, Z., Garrett, Z., Augenstein, S., and Mitchell, N

    Rush, J. K., Charles, Z., Garrett, Z., Augenstein, S., and Mitchell, N. E. DrJAX : Scalable and differentiable mapreduce primitives in JAX . In WANT@ ICML, 2024

    2024

  51. [59]

    TAN without a burn: Scaling laws of DP-SGD

    Sander, T., Stock, P., and Sablayrolles, A. TAN without a burn: Scaling laws of DP-SGD . In ICML, pp.\ 29937--29949, 2023

    2023

  52. [60]

    Differentially private representation learning via image captioning

    Sander, T., Yu, Y., Sanjabi, M., Durmus, A., Ma, Y., Chaudhuri, K., and Guo, C. Differentially private representation learning via image captioning. In ICML, 2024

    2024

  53. [61]

    J., Lee, J., Antognini, J., Sohl-Dickstein, J., Frostig, R., and Dahl, G

    Shallue, C. J., Lee, J., Antognini, J., Sohl-Dickstein, J., Frostig, R., and Dahl, G. E. Measuring the effects of data parallelism on neural network training. JMLR, 2019

    2019

  54. [62]

    M., Lowe, R., Voss, C., Radford, A., Amodei, D., and Christiano, P

    Stiennon, N., Ouyang, L., Wu, J., Ziegler, D. M., Lowe, R., Voss, C., Radford, A., Amodei, D., and Christiano, P. F. Learning to summarize with human feedback. In NeurIPS, 2020

    2020

  55. [63]

    Enabling fast differentially private SGD via just-in-time compilation and vectorization

    Subramani, P., Vadivelu, N., and Kamath, G. Enabling fast differentially private SGD via just-in-time compilation and vectorization. In NeurIPS, pp.\ 26409--26421, 2021

    2021

  56. [64]

    A., Manoel, A., Mireshghallah, F., Lin, Z., Gopi, S., Kulkarni, J., and Sim, R

    Tang, X., Shin, R., Inan, H. A., Manoel, A., Mireshghallah, F., Lin, Z., Gopi, S., Kulkarni, J., and Sim, R. Privacy-preserving in-context learning with differentially private few-shot generation. ICLR, 2024

    2024

  57. [65]

    S., and Smith, V

    Thaker, P., Setlur, A., Wu, Z. S., and Smith, V. Leveraging public representations for private transfer learning. arXiv:2312.15551, 2023

    2023 arXiv

  58. [66]

    E., and Honkela, A

    Tobaben, M., Shysheya, A., Bronskill, J., Paverd, A., Tople, S., Zanella-Beguelin, S., Turner, R. E., and Honkela, A. On the efficacy of differentially private few-shot image classification. TMLR, 2023

    2023

  59. [67]

    Considerations for differentially private learning with large-scale public pretraining

    Tram \`e r, F., Kamath, G., and Carlini, N. Considerations for differentially private learning with large-scale public pretraining. arXiv:2212.06470, 2022

    2022 arXiv

  60. [68]

    Can public large language models help private cross-device federated learning? In NAACL (Findings), pp.\ 934--949, 2024

    Wang, B., Zhang, Y., Cao, Y., Li, B., McMahan, H., Oh, S., Xu, Z., and Zaheer, M. Can public large language models help private cross-device federated learning? In NAACL (Findings), pp.\ 934--949, 2024

    2024

  61. [69]

    A., Backurs, A., Chandrasekaran, V., Kulkarni, J., and Sim, R

    Wu, F., Inan, H. A., Backurs, A., Chandrasekaran, V., Kulkarni, J., and Sim, R. Privately aligning language models with reinforcement learning. ICLR, 2024 a

    2024

  62. [70]

    T., and Mittal, P

    Wu, T., Panda, A., Wang, J. T., and Mittal, P. Privacy-preserving in-context learning for large language models. In ICLR, 2024 b

    2024

  63. [71]

    GSPMD : general and scalable parallelization for ML computation graphs

    Xu, Y., Lee, H., Chen, D., Hechtman, B., Huang, Y., Joshi, R., Krikun, M., Lepikhin, D., Ly, A., Maggioni, M., et al. GSPMD : general and scalable parallelization for ML computation graphs. arXiv:2105.04663, 2021

    2021 arXiv

  64. [72]

    Privacy risk in machine learning: Analyzing the connection to overfitting

    Yeom, S., Giacomelli, I., Fredrikson, M., and Jha, S. Privacy risk in machine learning: Analyzing the connection to overfitting. In CSF, pp.\ 268--282, 2018

    2018

  65. [73]

    Large batch optimization for deep learning: Training bert in 76 minutes, 2020

    You, Y., Li, J., Reddi, S., Hseu, J., Kumar, S., Bhojanapalli, S., Song, X., Demmel, J., Keutzer, K., and Hsieh, C.-J. Large batch optimization for deep learning: Training bert in 76 minutes, 2020. URL https://arxiv.org/abs/1904.00962

    2020 arXiv

  66. [74]

    Large scale private learning via low-rank reparametrization

    Yu, D., Zhang, H., Chen, W., Yin, J., and Liu, T.-Y. Large scale private learning via low-rank reparametrization. In ICML, 2021

    2021

  67. [75]

    A., Kamath, G., Kulkarni, J., Lee, Y

    Yu, D., Naik, S., Backurs, A., Gopi, S., Inan, H. A., Kamath, G., Kulkarni, J., Lee, Y. T., Manoel, A., Wutschitz, L., Yekhanin, S., and Zhang, H. Differentially private fine-tuning of language models. In ICLR, 2022

    2022

  68. [76]

    How does critical batch size scale in pre-training? arXiv:2410.21676, 2024 a

    Zhang, H., Morwani, D., Vyas, N., Wu, J., Zou, D., Ghai, U., Foster, D., and Kakade, S. How does critical batch size scale in pre-training? arXiv:2410.21676, 2024 a

    2024 arXiv

  69. [77]

    K., Oh, S., and He, N

    Zhang, L., Li, B., Thekumparampil, K. K., Oh, S., and He, N. DPZero : Private fine-tuning of language models without backpropagation. In ICML, 2024 b

    2024

  70. [78]

    S., and Hong, M

    Zhang, X., Bu, Z., Wu, Z. S., and Hong, M. Differentially private SGD without clipping bias: An error-feedback approach. arXiv:2311.14632, 2023

    2023 arXiv

  71. [79]

    S., Salakhutdinov, R., Urtasun, R., Torralba, A., and Fidler, S

    Zhu, Y., Kiros, R., Zemel, R. S., Salakhutdinov, R., Urtasun, R., Torralba, A., and Fidler, S. Aligning books and movies: Towards story-like visual explanations by watching movies and reading books. In ICCV, pp.\ 19--27, 2015

    2015

  72. [80]

    T., Stieger, S., Feiner, L

    Ziller, A., Mueller, T. T., Stieger, S., Feiner, L. F., Brandt, J., Braren, R., Rueckert, D., and Kaissis, G. Reconciling privacy and accuracy in ai for medical imaging. Nature Machine Intelligence, 6 0 (7): 0 764--774, 2024

    2024

  73. [81]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  74. [82]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  75. [83]

    bG g6b嗍 3kQI @k /h m?hlKJڅ:| 4 j 2M^ ; Z ݄ hT2 !; & ȯ ɾD :] q u ` bcߩ -@n- e5 h v Vb?SHP r! 5 ШEw7wlQ # `K

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.