Pith. sign in

REVIEW 3 major objections 6 minor 3 cited by

POPri: Private Federated Learning using Preference-Optimized Synthetic Data

T0 review · 3 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read POPri recasts private federated synthetic-data generation as an LLM policy-optimization problem, using DP-noised client preference scores to fine-tune a generator with Direct Preference Optimization.

desk verdict Promising DPO-based twist on DP synthetic data, but the privacy accounting in Section 3.1 contradicts Algorithm 1, so the headline epsilon comparisons are not yet trustworthy. read the letter →

arxiv 2504.16438 v2 pith:OGDAIWHO submitted 2025-04-23 cs.LG cs.AIcs.CRcs.DC

Charlie Hou , Mei-Yu Wang , Yige Zhu , Daniel Lazar , Giulia Fanti This is my paper · ORCID
classification cs.LGcs.AIcs.CRcs.DC
keywords differentialprivacyfederatedlearningsyntheticdatapolicyoptimizationdirectpreferenceLLMfine-tuningon-devicebenchmarkdataset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

POPri is a method for training an on-device model from private federated text data without ever sending raw client text to a server. It asks participating clients to score server-generated synthetic samples by how similar they are to the client's own data, adds noise to those scores for differential privacy, and then treats the noisy scores as preference feedback to fine-tune a public LLM with Direct Preference Optimization. The fine-tuned LLM produces a large synthetic corpus, on which a small downstream model is trained and deployed on devices. The paper reports that on its new LargeFedBench benchmark and on two earlier central-DP benchmarks, POPri beats DP-FedAvg, DP-FTRL, and Private Evolution baselines on downstream accuracy, and at $\epsilon=1$ it closes 40-58% of the gap between fully private and fully non-private next-token-prediction accuracy, compared with 1-28% for Private Evolution. The point of the work is that client feedback can be used as an RL reward rather than only as a prompt-engineering signal, which makes better use of the LLM's capacity.

What carries the argument

The carrier of the argument is the construction of preference pairs from DP-noised client scores, combined with Direct Preference Optimization (DPO), an RLHF-style loss that raises the probability of a chosen response and lowers that of a rejected response without training a separate reward model. The scoring function is average cosine similarity between synthetic and private text embeddings rather than the sparse nearest-neighbor histogram used by Private Evolution; the paper's ablations identify this dense scoring as critical for producing informative preference pairs under DP noise. Secure aggregation combines client score vectors, and per-client clipping plus Gaussian noise gives the user-level $\epsilon,\delta$-DP guarantee. The mechanism does the work of turning diffuse client feedback into a concrete training signal that improves the LLM's synthetic-data distribution.

What would settle it

A decisive test: run POPri on a fixed dataset with its true client scores replaced by scores from a deliberately uninformative embedding (e.g., random vectors or a bag-of-characters embedding) and measure downstream accuracy; if accuracy stays high, the reward is not the carrier. More directly, check across rounds whether the aggregated cosine-similarity reward and downstream next-token accuracy move together — if the reward rises while accuracy falls, as the paper's own FID curves suggest can happen late in training, the proxy is not faithful.

Watch

Extended reading notes

Core claim

The paper claims that client feedback collected for private synthetic-data generation can be repurposed as a reinforcement-learning reward, enabling policy optimization to fine-tune the data-generating LLM. In POPri, the server generates J synthetic samples from each of K prompts, clients score every sample by the average cosine similarity between its embedding and their private embeddings, and the server aggregates these clipped, Gaussian-noised scores via secure aggregation. For each prompt, the highest-scoring sample becomes the chosen response and the fifth-highest becomes the rejected response in a DPO preference pair; iterating DPO fine-tuning over T rounds moves the LLM's output distribution toward text that clients judge relevant. The resulting generator produces a final synthetic corpus used to fine-tune a small downstream model. The central empirical claim is that this policy-optimized synthetic data outperforms both DP-FL methods and Private Evolution on next-token prediction and text classification, closing the private-to-nonprivate accuracy gap by up to 58% at $\epsilon=1$.

Load-bearing premise

POPri's advantage rests on the assumption that 'how similar a synthetic sample looks to a client's data under a sentence-embedding model' is the right measure of how useful that sample will be for training the downstream model.

Editorial extensions

If this is right

  • At $\epsilon=1$, POPri's downstream model recovers 40-58% of the accuracy lost to full privacy, versus 1-28% for Private Evolution and 3% for the tested DP-FL methods.
  • In the bioRxiv case study, POPri cuts per-client communication to roughly 7 million floats downloaded and 18,000 uploaded per round, compared with 82 million floats each way for FedAvg.
  • The method removes the need for hand-crafted prompts describing client data; only a generic base prompt and client feedback are required.
  • Because the output is DP synthetic data, the post-processing property of differential privacy allows the final corpus to be reused freely for downstream training without extra privacy loss.
  • On the central-DP benchmarks PubMed and OpenReview, POPri outperforms Aug-PE with the same synthetic sample budget of 2,000 samples.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct test of the paper's core premise would track whether the aggregated cosine-similarity reward and downstream next-token accuracy move together across rounds; the paper's FID curves suggest they can diverge late in training, so the correlation is not yet established.
  • The same preference-optimization loop could in principle accept other client-side signals — labels, loss values, or model gradients — as long as they can be aggregated with DP; the paper does not explore these alternatives.
  • The paper shows the generator can overfit and drift away from the client distribution after several rounds; an adaptive stopping rule or regularizer tuned to downstream validation performance is a natural extension the authors leave implicit.
  • The method inherits the public LLM's biases, and the paper explicitly leaves open the problem of clients whose data the LLM cannot represent; adding an abstention or 'none of these' option in the client feedback would be a concrete way to test and address that failure mode.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes POPri, a differentially private federated learning method that generates synthetic training data by fine-tuning a server-side LLM with Direct Preference Optimization (DPO). Client devices score synthetic samples by average cosine similarity to their private data, the scores are clipped, noised, and securely aggregated, and the server uses the resulting preference pairs to iteratively fine-tune the generator. The final generator produces a synthetic corpus used to train a small downstream model. The authors introduce LargeFedBench, a benchmark of bioRxiv and congressional speech data designed to reduce contamination, and report that POPri outperforms DP-FedAvg, DP-FTRL, and Private Evolution baselines, closing up to 58% of the gap between fully private and non-private accuracy.

Significance. If the correctness concerns are resolved, the paper's central claim is significant: it advances the state of the art for private synthetic-data training and demonstrates that preference-optimization techniques can be adapted to DP client feedback. The release of LargeFedBench is a useful community contribution, especially the design goal of periodically updated, contamination-controlled federated text data, and the paper includes ablations (scoring function, rejected-sample rank, partial participation) that support the main design choices. The result would be stronger if the privacy accounting were unambiguous and if the headline gap-closure numbers were internally consistent.

major comments (3)
  1. [Section 3.1 / Algorithm 1] The privacy analysis is internally inconsistent with the pseudocode. Algorithm 1 line 20 computes Scores_t = (1/L) * sum(Scores_{i,t}), so under the paper's replacement-neighbor definition the L2 sensitivity of the aggregated vector is 2/L, not 1. The per-client noise N(0, sigma^2 I / L) averages to N(0, sigma^2 I / L), not N(0, sigma^2 I). If the implementation instead sums the scores before averaging, the sensitivity is 2 rather than 1. Either way, the stated epsilon values computed by the RDP accountant are not justified by the text, and because all headline comparisons are made at epsilon = 1 or 7, the reported privacy levels and the comparison itself are ambiguous. The authors should correct the accounting, rescale the noise or the clipping procedure accordingly, and re-report the privacy parameters.
  2. [Section 5.1 and Figure 1] The claimed 58% gap closure for bioRxiv at epsilon = 1 does not match the numbers reported in Table 1. With the table's values, the fully non-private accuracy is 41.5, the epsilon = 0 baseline is 27.9, and POPri reaches 34.8 at epsilon = 1, giving a gap closure of about 50%, not 58%; Private Evolution reaches 31.1, giving about 23%, not the 28% stated in the abstract and Section 5.1. Because this percentage is a headline quantitative claim, the authors should either correct the reported numbers, clarify which setting and which checkpoint selection produced 58%, or restate the claim consistently across the abstract, Figure 1, and Section 5.1.
  3. [Section 2.1 / Definition of neighboring datasets] The paper defines neighboring datasets as differing in at most one client's data but does not specify whether this is replacement, addition/removal, or both. The sensitivity calculation in Section 3.1 is sensitive to this choice: the replacement definition gives an L2 sensitivity of 2/L for the average, while addition/removal with a norm-1 clip gives 1/L. This ambiguity propagates to the claimed epsilon values and should be resolved explicitly in the privacy analysis.
minor comments (6)
  1. [Table 1 caption] The caption states that all standard deviation error bars are less than 0.5, but Table 1 displays no error bars; the authors should either include the error bars in the table or move the statement to the text with the actual variance values.
  2. [Algorithm 1, line 24] Line 24, 'Select positive synthetic sample: Pt[k, 1] = Pt[k, 1]', appears to be a typo; it should presumably assign the highest-scoring response for prompt k, as described in the text.
  3. [Algorithm 3, line 18] Line 18 assigns 'P(b)_t[k, 2] = Pt[k, 2]', but the right-hand side should reference the current class's preference set P(b)_t, not the undeclared global Pt.
  4. [Abstract / Figure 1 / Section 5.1] The percentage gap closure attributed to prior synthetic data methods is given as 23% in Figure 1, 28% in the abstract and Section 5.1, and '1-28%' in Section 5.1; these numbers should be reconciled.
  5. [Appendix B.1] The text says 'we fine-tune the models for 2 epochs and select the best checkpoint with the lowest FID score relative to the validation dataset,' but elsewhere the method selects checkpoints by validation accuracy; the checkpoint-selection criterion should be stated consistently.
  6. [Section 5 / Baselines] For the PubMed and OpenReview comparisons, POPri uses LLaMA-3-8B while the Aug-PE baselines use Llama-2-7b-chat-hf, so the comparison is not fully controlled for the base generator; this should be acknowledged as a potential confound in the discussion.

Circularity Check

0 steps flagged · score 2.0 of 10

No load-bearing circularity: POPri's claimed gap-closing results are empirical comparisons on held-out test sets; the DPO reward is embedding similarity, not the reported downstream accuracy.

full rationale

I walked the claimed derivation chain. POPri's central claim is that turning PE-style client feedback into a DPO preference signal improves synthetic data quality, measured by downstream next-token and classification accuracy. The reported gap-closing percentages in Section 5.1 are arithmetic summaries of Table 1 accuracies on held-out test sets (bioRxiv, Congress, PubMed, OpenReview); they are not rearrangements of fitted inputs. The reward optimized during DPO (average cosine similarity in Algorithm 4) is not the same quantity as the reported evaluation metric, so the optimization does not by construction produce the reported accuracy; the link is empirical. The PE, PE+SFT, DP-FedAvg, and DP-FTRL baselines are evaluated under the same protocol, and the numbers are taken from Table 1 and Figure 1. I found no fitted parameter that is later relabeled as a prediction and no uniqueness theorem or ansatz smuggled in by self-citation. The paper does cite the authors' own PrE-Text (Hou et al., 2024) as the main synthetic-data baseline and evaluates on the authors' new LargeFedBench, which is self-referential but not load-bearing: the comparison is a measured accuracy difference on fixed test data, not a consequence implied by the definition of PrE-Text or of the benchmark. The Section 3.1 privacy analysis contains an apparent inconsistency between Algorithm 1's averaged noisy scores and the prose sensitivity-1/noise-sigma^2-I claim; I treat that as a correctness risk that the released code should resolve, not as circular reasoning, because it does not identify the output metric with an algorithm input. Therefore I find no significant circularity, with only a minor self-citation element.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The central claim rests on an empirical pipeline rather than a derivation. The free parameters are standard ML hyperparameters chosen by hand or by ablations; the axioms are background assumptions about DPO, the embedding reward, DP accounting, and contamination control. No new physical or model entities are postulated.

free parameters (6)
  • rejected sample rank ℓ = 5 (out of J=10)
    Chosen via ablation in Appendix E.3; affects preference pair informativeness and final accuracy.
  • number of samples per prompt J = 10
    Design choice for building preference pairs; larger J increases client upload cost.
  • number of prompts per round K = 1800
    Set by compute budget; yields 18,000 synthetic samples per round.
  • number of POPri rounds T = 20 for bioRxiv/Congress, 10 for PubMed/OpenReview
    Stopped by early FID-based selection; running too many rounds overfits (Figure 4).
  • LoRA rank / alpha = r=4, alpha=8
    Standard LoRA settings chosen by hand to reduce GPU memory.
  • learning rate range = 3e-7 to 8e-7
    Cosine scheduler; chosen by hand for stable DPO fine-tuning.
assumptions (6)
  • domain assumption DPO (Rafailov et al., 2023) effectively optimizes an LLM to produce higher-scoring responses under a given reward model.
    POPri relies on DPO to convert noisy preference pairs into better synthetic data generation; this is cited but not proved in the paper.
  • domain assumption Cosine similarity in the embedding space of all-MiniLM-L6-v2 is a valid proxy for how useful a synthetic sample is for training the downstream model.
    The reward and the FID-based early stopping both use this embedding; if the embedding ignores task-relevant features, POPri's advantage collapses.
  • standard math After clipping each client's score vector to unit norm and adding Gaussian noise with standard deviation sigma, the aggregated vector satisfies user-level (epsilon, delta)-DP with the computed epsilon.
    Standard Gaussian mechanism composition; the paper uses Opacus RDP accounting.
  • domain assumption The public LLM's knowledge cutoff predates the benchmark data, so evaluations are uncontaminated.
    LargeFedBench uses data after April 2023 and LLaMA-3-8B has a March 2023 cutoff; this is a temporal assumption that holds for the tested models.
  • domain assumption A public pretrained LLM can generate diverse, relevant text from prompts built from three c4 samples.
    The prompt generation (Figure 5) assumes the LLM can continue in the style of given examples; if not, all synthetic data would be off-target.
  • standard math Secure aggregation of client scores is available and does not leak individual client scores.
    Bonawitz et al. (2016) secure aggregation is assumed for the federated version.

how reviews work

0 comments
Cite this review

Pith. "Pith review of POPri: Private Federated Learning using Preference-Optimized Synthetic Data." pith.science (2026). https://pith.science/paper/OGDAIWHO

@misc{pith2026250416438,
  author       = {Pith},
  title        = {Pith review of: POPri: Private Federated Learning using Preference-Optimized Synthetic Data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OGDAIWHO}},
  note         = {Machine review of arXiv:2504.16438}
}
read the original abstract

In practical settings, differentially private Federated learning (DP-FL) is the dominant method for training models from private, on-device client data. Recent work has suggested that DP-FL may be enhanced or outperformed by methods that use DP synthetic data (Wu et al., 2024; Hou et al., 2024). The primary algorithms for generating DP synthetic data for FL applications require careful prompt engineering based on public information and/or iterative private client feedback. Our key insight is that the private client feedback collected by prior DP synthetic data methods (Hou et al., 2024; Xie et al., 2024) can be viewed as an RL (reinforcement learning) reward. Our algorithm, Policy Optimization for Private Data (POPri) harnesses client feedback using policy optimization algorithms such as Direct Preference Optimization (DPO) to fine-tune LLMs to generate high-quality DP synthetic data. To evaluate POPri, we release LargeFedBench, a new federated text benchmark for uncontaminated LLM evaluations on federated client data. POPri substantially improves the utility of DP synthetic data relative to prior work on LargeFedBench datasets and an existing benchmark from Xie et al. (2024). POPri closes the gap between next-token prediction accuracy in the fully-private and non-private settings by up to 58%, compared to 28% for prior synthetic data methods, and 3% for state-of-the-art DP federated learning methods. The code and data are available at https://github.com/meiyuw/POPri.

Figures

Figures reproduced from arXiv: 2504.16438 by the authors.

Figure 1
Figure 1. Left: Private Evolution (PE)-based techniques. Clients generate low-dimensional statistics which summarize the similarity of the synthetic data to their private samples. These are privately aggregated to refine the synthetic data generation for future iterations. Traditional PE (brown) uses a prompt-based method. POPri (blue) improves a naive fine-tuning method (PE+SFT, purple) by fine-tuning the LLM using policy op… view at source ↗
Figure 2
Figure 2. 2-PCA visualization of synthetic data from POPri and PE+SFT, and evaluation data. We see that POPri’s synthetic data distribution (left) is much closer to the evaluation data distribu￾tion (right) than the PE+SFT synthetic data distribution (middle). Naive fine-tuning with SFT on PE-generated synthetic data does not make best use of client feedback. (2) Supervised Fine-Tuning (SFT). One could directly fine￾tune the … view at source ↗
Figure 3
Figure 3. Next-token prediction accuracy performance of four methods as a function of the number of clients sampled per round out of 10000. We see that across different client participation scenarios, POPri consistently performs the best. 6 months and sorted by date. Hence, researchers can eas￾ily select datasets that were generated after their model’s knowledge cutoff date. In this paper, we use data from LargeFedBench publi… view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: PCA visualization of POPri synthetic data embeddings over rounds. Right 6 Panels: PCA-2 plots for synthetic data and evaluation data embeddings from the best checkpoint each round for 20 iterations. The orange (round 7) and maroon point clouds represent the round with …
Figure 5
Figure 5. Figure 5: The synthetic data generation prompt for POPri. The black text marks the input prompt, and the brown text after “Original Text Sample 4” is generated. The generated text between “Original Text Sample 4” and “Original Text Sample 5” is collected and used as a synthetic …
Figure 6
Figure 6. Figure 6: Left: FID scores of POPri using NN histogram scoring vs. POPri using cosine similarity. Right: After the client feedback stage, we measure the percentage of the time the non-noised and non-clipped score (nearest neighbor histogram scoring or cosine similarity scoring) …
Figure 7
Figure 7. Figure 7: In this experiment, we investigate whether the use of label noise-resistant alignment methods could allow the use of higher-ranked rejected samples. To do this, we used the third-ranked sample as the rejected sample, and evaluated different settings for conservative DP…
Figure 8
Figure 8. Figure 8: Ablation study for selecting rejected samples in the preference data. Here we generate 10 samples for each prompt and select Nth ranked data as the rejected sample, where N is 5, 7, or 10. The vertical lines indicate the round at which the best next-word-prediction acc…
Figure 9
Figure 9. Figure 9: The distribution of how many tokens are in each client’s dataset for the bioRxiv and Congressional Speeches datasets. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_9.png]
Figure 10
Figure 10. Figure 10: A t-SNE clustering of the Congressional Speeches dataset. US data is colored in purple, UK data is colored in orange, and Canada data is colored in green. We find that the three datasets form distinct clusters and also distinct sub-clusters. F. Datasets bioRxiv. This …

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Struct-Bench: A Benchmark for Differentially Private Structured Text Generation

    cs.CL 2025-09 conditional novelty 7.0 of 10

    Struct-Bench is a Context-Free Grammar based evaluation platform showing that current differentially private synthetic text generators often fail to preserve structure and semantic diversity.

  2. Differentially Private Synthetic Data via APIs 3: Using Simulators Instead of Foundation Model

    cs.LG 2025-02 conditional novelty 4.0 of 10

    Sim-PE plugs non-neural simulators into Private Evolution to create differentially private images, improving downstream accuracy over foundation-model PE by up to 3x on MNIST.

  3. How to DP-fy Your Data: A Practical Guide to Generating Synthetic Data With Differential Privacy

    cs.CR 2025-12 conditional novelty 2.0 of 10

    A practical, extremely thorough survey of differentially private synthetic data generation: methods, privacy units, evaluation metrics, and end-to-end system components across four data modalities.

Reference graph

Works this paper leans on

65 extracted references · 17 canonical work pages · cited by 3 Pith papers

  1. [1]

    B., Mironov, I., Talwar, K., and Zhang, L

    Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K., and Zhang, L. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp.\ 308--318. ACM, 2016

    work page 2016

  2. [2]

    L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al

    Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  3. [3]

    Llama 3 model card

    AI@Meta. Llama 3 model card. 2024. URL https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md

    2024

  4. [4]

    M., Firat, O., Johnson, M., Lepikhin, D., Passos, A., Shakeri, S., Taropa, E., Bailey, P., Chen, Z., et al

    Anil, R., Dai, A. M., Firat, O., Johnson, M., Lepikhin, D., Passos, A., Shakeri, S., Taropa, E., Bailey, P., Chen, Z., et al. Palm 2 technical report. arXiv preprint arXiv:2305.10403, 2023

    arXiv 2023

  5. [5]

    J., Deems, S., Furlani, T

    Boerner, T. J., Deems, S., Furlani, T. R., Knuth, S. L., and Towns, J. Access: Advancing innovation: Nsf’s advanced cyberinfrastructure coordination ecosystem: Services & support. In Practice and Experience in Advanced Research Computing 2023: Computing for the Common Good, pp.\ 173--176. 2023

    work page 2023

  6. [6]

    Towards private synthetic text generation

    Bommasani, R. Towards private synthetic text generation. In NeurIPS 2019 Machine Learning with Guarantees Workshop, 2019

    work page 2019

  7. [7]

    A., Ivanov, V., Kreuter, B., Marcedone, A., McMahan, H

    Bonawitz, K. A., Ivanov, V., Kreuter, B., Marcedone, A., McMahan, H. B., Patel, S., Ramage, D., Segal, A., and Seth, K. Practical secure aggregation for federated learning on user-held data. In NIPS Workshop on Private Multi-Party Machine Learning, 2016. URL https://arxiv.org/abs/1611.04482

    arXiv 2016

  8. [8]

    T., Buitrago, P., Hanna, E., Sanielevici, S., Scibek, R., and Nystrom, N

    Brown, S. T., Buitrago, P., Hanna, E., Sanielevici, S., Scibek, R., and Nystrom, N. A. Bridges-2: A platform for rapidly-evolving and data intensive research. In Practice and Experience in Advanced Research Computing 2021: Evolution Across All Dimensions, PEARC '21, New York, NY, USA, 2021. Association for Computing Machinery. ISBN 9781450382922. doi:10.1...

    arXiv 2021

Show all 65 references
  1. [9]

    Buitrago, P. A. and Nystrom, N. A. Neocortex and bridges-2: A high performance ai+hpc ecosystem for science, discovery, and societal good. Communications in computer and information science, 1327, 2021. doi:10.1007/978-3-030-68035-0_15. URL https://par.nsf.gov/biblio/10274872

    2021

  2. [10]

    Towards federated foundation models: Scalable dataset pipelines for group-structured learning

    Charles, Z., Mitchell, N., Pillutla, K., Reneer, M., and Garrett, Z. Towards federated foundation models: Scalable dataset pipelines for group-structured learning. arXiv preprint arXiv:2307.09619, 2023

    2023 arXiv

  3. [11]

    B., Mitchell, N

    Charles, Z., Ganesh, A., McKenna, R., McMahan, H. B., Mitchell, N. E., Pillutla, K., and Rush, J. K. Fine-tuning large language models with user-level differential privacy. In ICML 2024 Workshop on Theoretical Foundations of Foundation Models, 2024

    2024

  4. [12]

    Collins, L., Wu, S., Oh, S., and Sim, K. C. Profit: Benchmarking personalization and robustness trade-off in federated prompt tuning. arXiv preprint arXiv:2310.04627, 2023

    2023 arXiv

  5. [13]

    Differential privacy

    Dwork, C. Differential privacy. In International colloquium on automata, languages, and programming, pp.\ 1--12. Springer, 2006

    2006

  6. [14]

    and Roth, A

    Dwork, C. and Roth, A. The algorithmic foundations of differential privacy. Found. Trends Theor. Comput. Sci., 9 0 (3–4): 0 211–407, August 2014. ISSN 1551-305X. doi:10.1561/0400000042. URL https://doi.org/10.1561/0400000042

    2014 doi

  7. [15]

    The llama 3 herd of models, 2024

    Grattafiori, A., Dubey, A., and et al. The llama 3 herd of models, 2024. URL https://arxiv.org/abs/2407.21783

    2024 arXiv

  8. [16]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Guo, D., Yang, D., Zhang, H., Song, J., Zhang, R., Xu, R., Zhu, Q., Ma, S., Wang, P., Bi, X., et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  9. [17]

    Direct Language Model Alignment from Online AI Feedback

    Guo , S., Zhang , B., Liu , T., Liu , T., Khalman , M., Llinares , F., Rame , A., Mesnard , T., Zhao , Y., Piot , B., Ferret , J., and Blondel , M. Direct Language Model Alignment from Online AI Feedback . arXiv e-prints, art. arXiv:2402.04792, February 2024. doi:10.48550/arXi...

    2024 doi arXiv

  10. [18]

    Federated learning for mobile keyboard prediction, 2019

    Hard, A., Rao, K., Mathews, R., Ramaswamy, S., Beaufays, F., Augenstein, S., Eichner, H., Kiddon, C., and Ramage, D. Federated learning for mobile keyboard prediction, 2019. URL https://arxiv.org/abs/1811.03604

    2019 arXiv

  11. [19]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium

    Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter, S. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017

    2017

  12. [20]

    Pre-text: Training language models on private federated data in the age of llms, 2024

    Hou, C., Shrivastava, A., Zhan, H., Conway, R., Le, T., Sagar, A., Fanti, G., and Lazar, D. Pre-text: Training language models on private federated data in the age of llms, 2024. URL https://arxiv.org/abs/2406.02958

    2024 arXiv

  13. [21]

    J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W

    Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021

    2021 arXiv

  14. [22]

    Practical and private (deep) learning without sampling or shuffling

    Kairouz, P., McMahan, B., Song, S., Thakkar, O., Thakurta, A., and Xu, Z. Practical and private (deep) learning without sampling or shuffling. In ICML, 2021 a

    2021

  15. [23]

    B., Avent, B., Bellet, A., Bennis, M., Bhagoji, A

    Kairouz, P., McMahan, H. B., Avent, B., Bellet, A., Bennis, M., Bhagoji, A. N., Bonawitz, K., Charles, Z., Cormode, G., Cummings, R., et al. Advances and open problems in federated learning. Foundations and Trends in Machine Learning , 14 0 (1--2): 0 1--210, 2021 b

    2021

  16. [24]

    Harnessing large-language models to generate private synthetic text

    Kurakin, A., Ponomareva, N., Syed, U., MacDermed, L., and Terzis, A. Harnessing large-language models to generate private synthetic text. arXiv preprint arXiv:2306.01684, 2023

    2023 arXiv

  17. [25]

    On the privacy properties of gan-generated samples

    Lin, Z., Sekar, V., and Fanti, G. On the privacy properties of gan-generated samples. In International Conference on Artificial Intelligence and Statistics, pp.\ 1522--1530. PMLR, 2021

    2021

  18. [26]

    Differentially private synthetic data via foundation model apis 1: Images

    Lin, Z., Gopi, S., Kulkarni, J., Nori, H., and Yekhanin, S. Differentially private synthetic data via foundation model apis 1: Images. arXiv preprint arXiv:2305.15560, 2023

    2023 arXiv

  19. [27]

    Differentially private synthetic data via apis 3: Using simulators instead of foundation model, 2025

    Lin, Z., Baltrusaitis, T., and Yekhanin, S. Differentially private synthetic data via apis 3: Using simulators instead of foundation model, 2025. URL https://arxiv.org/abs/2502.05505

    2025 arXiv

  20. [28]

    and Schwartz, R

    Magar, I. and Schwartz, R. Data contamination: From memorization to exploitation, 2022. URL https://arxiv.org/abs/2203.08242

    2022 arXiv

  21. [29]

    D., Chen, D., and Arora, S

    Malladi, S., Gao, T., Nichani, E., Damian, A., Lee, J. D., Chen, D., and Arora, S. Fine-tuning language models with just forward passes. Advances in Neural Information Processing Systems, 36: 0 53038--53075, 2023

    2023

  22. [30]

    Differentially private language models for secure data sharing

    Mattern, J., Jin, Z., Weggenmann, B., Schoelkopf, B., and Sachan, M. Differentially private language models for secure data sharing. In Goldberg, Y., Kozareva, Z., and Zhang, Y. (eds.), Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp....

    2022 doi

  23. [31]

    McMahan, B., Moore, E., Ramage, D., Hampson, S., and Arcas, B. A. y. Communication-Efficient Learning of Deep Networks from Decentralized Data . In Singh, A. and Zhu, J. (eds.), Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, volume ...

    2017

  24. [32]

    Learning DP recurrent language models

    McMahan, B., Ramage, D., Talwar, K., and Zhang, L. Learning DP recurrent language models . 2017 b

    2017

  25. [33]

    A note on dpo with noisy preferences & relationship to ipo, November 25 2023

    Mitchell, E. A note on dpo with noisy preferences & relationship to ipo, November 25 2023. URL https://ericmitchell.ai/cdpo.pdf. Version 1.1

    2023

  26. [34]

    Few-shot fine-tuning vs

    Mosbach, M., Pimentel, T., Ravfogel, S., Klakow, D., and Elazar, Y. Few-shot fine-tuning vs. in-context learning: A fair comparison and evaluation. arXiv preprint arXiv:2305.16938, 2023

    2023 arXiv

  27. [35]

    Federated learning with buffered asynchronous aggregation

    Nguyen, J., Malik, K., Zhan, H., Yousefpour, A., Rabbat, M., Malek, M., and Huba, D. Federated learning with buffered asynchronous aggregation. In International Conference on Artificial Intelligence and Statistics, pp.\ 3581--3607. PMLR, 2022

    2022

  28. [36]

    Training language models to follow instructions with human feedback

    Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35: 0 27730--27744, 2022

    2022

  29. [37]

    Paulik, M., Seigel, M., Mason, H., Telaar, D., Kluivers, J., van Dalen, R., Lau, C. W., Carlson, L., Granqvist, F., Vandevelde, C., Agarwal, S., Freudiger, J., Byde, A., Bhowmick, A., Kapoor, G., Beaumont, S., Áine Cahill, Hughes, D., Javidbakht, O., Dong, F., Rishi, R., and H...

    2021 arXiv

  30. [38]

    Language models are unsupervised multitask learners

    Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al. Language models are unsupervised multitask learners. OpenAI blog, 1 0 (8): 0 9, 2019

    2019

  31. [39]

    D., Ermon, S., and Finn, C

    Rafailov, R., Sharma, A., Mitchell, E., Manning, C. D., Ermon, S., and Finn, C. Direct preference optimization: Your language model is secretly a reward model. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information P...

    2023

  32. [40]

    Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. J. Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv e-prints, 2019

    2019

  33. [41]

    Reddi, S., Charles, Z., Zaheer, M., Garrett, Z., Rush, K., Kone c n \`y , J., Kumar, S., and McMahan, H. B. Adaptive federated optimization. arXiv preprint arXiv:2003.00295, 2020

    2003 arXiv

  34. [42]

    and Gurevych, I

    Reimers, N. and Gurevych, I. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 11 2019 a . URL https://arxiv.org/abs/1908.10084

    2019 arXiv

  35. [43]

    and Gurevych, I

    Reimers, N. and Gurevych, I. Sentence- BERT : Sentence embeddings using S iamese BERT -networks. In Inui, K., Jiang, J., Ng, V., and Wan, X. (eds.), Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferenc...

    2019 doi

  36. [44]

    Data contamination through the lens of time, 2023

    Roberts, M., Thakur, H., Herlihy, C., White, C., and Dooley, S. Data contamination through the lens of time, 2023. URL https://arxiv.org/abs/2310.10628

    2023 arXiv

  37. [45]

    Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter

    Sanh, V., Debut, L., Chaumond, J., and Wolf, T. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108, 2019

    1910 arXiv

  38. [46]

    M., Hauth, A., Millican, K., et al

    Team, G., Anil, R., Borgeaud, S., Alayrac, J.-B., Yu, J., Soricut, R., Schalkwyk, J., Dai, A. M., Hauth, A., Millican, K., et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023

    2023 arXiv

  39. [47]

    Llama: Open and efficient foundation language models

    Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozi \`e re, B., Goyal, N., Hambro, E., Azhar, F., et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  40. [48]

    Well-read students learn better: On the importance of pre-training compact models, 2019

    Turc, I., Chang, M.-W., Lee, K., and Toutanova, K. Well-read students learn better: On the importance of pre-training compact models, 2019. URL https://arxiv.org/abs/1908.08962

    2019 arXiv

  41. [49]

    Locally differentially private document generation using zero shot prompting

    Utpala, S., Hooker, S., and Chen, P.-Y. Locally differentially private document generation using zero shot prompting. In Bouamor, H., Pino, J., and Bali, K. (eds.), Findings of the Association for Computational Linguistics: EMNLP 2023, pp.\ 8442--8457, Singapore, December 2023...

    2023 doi

  42. [50]

    Prompt public large language models to synthesize data for private on-device applications

    Wu, S., Xu, Z., Zhang, Y., Zhang, Y., and Ramage, D. Prompt public large language models to synthesize data for private on-device applications. arXiv preprint arXiv:2404.04360, 2024

    2024 arXiv

  43. [51]

    A., Nori, H., Jiang, H., Zhang, H., Lee, Y

    Xie, C., Lin, Z., Backurs, A., Gopi, S., Yu, D., Inan, H. A., Nori, H., Jiang, H., Zhang, H., Lee, Y. T., Li, B., and Yekhanin, S. Differentially private synthetic data via foundation model API s 2: Text. In Forty-first International Conference on Machine Learning, 2024. URL h...

    2024

  44. [52]

    Differentially private generative adversarial network

    Xie, L., Lin, K., Wang, S., Wang, F., and Zhou, J. Differentially private generative adversarial network. arXiv preprint arXiv:1802.06739, 2018

    2018 arXiv

  45. [53]

    Xu, Z., Collins, M., Wang, Y., Panait, L., Oh, S., Augenstein, S., Liu, T., Schroff, F., and McMahan, H. B. Learning to generate image embeddings with user-level differential privacy. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 79...

    2023

  46. [54]

    A., Kairouz, P., McMahan, H

    Xu, Z., Zhang, Y., Andrew, G., Choquette-Choo, C. A., Kairouz, P., McMahan, H. B., Rosenstock, J., and Zhang, Y. Federated learning of gboard language models with differential privacy, 2023 b . URL https://arxiv.org/abs/2305.18465

    2023 arXiv

  47. [55]

    E., and Stoica, I

    Yang, S., Chiang, W.-L., Zheng, L., Gonzalez, J. E., and Stoica, I. Rethinking benchmark and contamination for language models with rephrased samples, 2023. URL https://arxiv.org/abs/2311.04850

    2023 arXiv

  48. [56]

    Fedllm-bench: Realistic benchmarks for federated learning of large language models

    Ye, R., Ge, R., Zhu, X., Chai, J., Du, Y., Liu, Y., Wang, Y., and Chen, S. Fedllm-bench: Realistic benchmarks for federated learning of large language models. arXiv preprint arXiv:2406.04845, 2024

    2024 arXiv

  49. [57]

    Opacus: User-friendly differential privacy library in pytorch

    Yousefpour, A., Shilov, I., Sablayrolles, A., Testuggine, D., Prasad, K., Malek, M., Nguyen, J., Ghosh, S., Bharadwaj, A., Zhao, J., Cormode, G., and Mironov, I. Opacus: User-friendly differential privacy library in pytorch. In NeurIPS 2021 Workshop Privacy in Machine Learning...

    2021

  50. [58]

    Training private and efficient language models with synthetic data from LLM s

    Yu, D., Backurs, A., Gopi, S., Inan, H., Kulkarni, J., Lin, Z., Xie, C., Zhang, H., and Zhang, W. Training private and efficient language models with synthetic data from LLM s. In Socially Responsible Language Modelling Research, 2023. URL https://openreview.net/forum?id=FKwtKzglFb

    2023

  51. [59]

    Privacy-preserving instructions for aligning large language models, 2024

    Yu, D., Kairouz, P., Oh, S., and Xu, Z. Privacy-preserving instructions for aligning large language models, 2024. URL https://arxiv.org/abs/2402.13659

    2024 arXiv

  52. [60]

    Synthetic text generation with differential privacy: A simple and practical recipe

    Yue, X., Inan, H., Li, X., Kumar, G., McAnallen, J., Shajari, H., Sun, H., Levitan, D., and Sim, R. Synthetic text generation with differential privacy: A simple and practical recipe. In Rogers, A., Boyd-Graber, J., and Okazaki, N. (eds.), Proceedings of the 61st Annual Meetin...

    2023 doi

  53. [61]

    A., Li, X., Kumar, G., McAnallen, J., Shajari, H., Sun, H., Levitan, D., and Sim, R

    Yue, X., Inan, H. A., Li, X., Kumar, G., McAnallen, J., Shajari, H., Sun, H., Levitan, D., and Sim, R. Synthetic text generation with differential privacy: A simple and practical recipe, 2023 b . URL https://arxiv.org/abs/2210.14348

    2023 arXiv

  54. [62]

    K., Oh, S., and He, N

    Zhang, L., Li, B., Thekumparampil, K. K., Oh, S., and He, N. Dpzero: Private fine-tuning of language models without backpropagation. In Forty-first International Conference on Machine Learning, 2024

    2024

  55. [63]

    X., Chen, X., Lin, Y., Wen, J.-R., and Han, J

    Zhou, K., Zhu, Y., Chen, Z., Chen, W., Zhao, W. X., Chen, X., Lin, Y., Wen, J.-R., and Han, J. Don't make your llm an evaluation benchmark cheater, 2023. URL https://arxiv.org/abs/2311.01964

    2023 arXiv

  56. [64]

    Contrastive private data synthesis via weighted multi-plm fusion

    Zou, T., Liu, Y., Li, P., Xiong, Y., Zhang, J., Liu, J., Ye, X., Ouyang, Y., and Zhang, Y.-Q. Contrastive private data synthesis via weighted multi-plm fusion. arXiv preprint arXiv:2502.00245, 2025

    2025 arXiv

  57. [65]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.