Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Synthesizing and Adapting Error Correction Data for Mobile Large Language Model Applications

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A continue-training recipe that mixes original production data with reweighted LLM-synthesized error-correction pairs achieves relative gains of 2.47% to 7.18% on live mobile A/B metrics.

desk verdict A production-tested recipe for LLM-synthesized error correction data with a genuinely new reweighting idea, but the headline A/B gains come without error bars and the reweighting model is fit to only 15 deployments. read the letter →

arxiv 2505.18488 v1 pith:CJUR7LHL submitted 2025-05-24 cs.LG cs.AIcs.CL

Yanxiang Zhang , Zheng Xu , Shanshan Wu , Yuanbo Zhang , Daniel Ramage This is my paper
classification cs.LGcs.AIcs.CL
keywords errorcorrectionmobiletypingsyntheticdatadomainadaptationreweightingdifferentialprivacyfederatedlearninglargelanguagemodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a moderate-sized LLM can be made better at correcting mobile typing errors by generating a large, deliberately corrupted synthetic dataset with LLM prompts, then deciding which synthetic examples to keep by scores from a small privacy-preserving model that has seen real user text. The kept examples are mixed with the existing production data in a continue-training schedule. On offline evaluations the mixed recipe matches the best single-domain training, and in production live A/B tests it improves key metrics such as accept rate and click-through rate by 2.47% to 7.18% relative to the original production model. This matters because it points to a data route to domain adaptation that does not require shipping raw user text to a server.

What carries the argument

The load-bearing object is the reweighting function $w(\theta, y_i) = C_{\min} + (C_{\max}-C_{\min})\sigma(\theta_f S_f(y_i) + \theta_p S_p(y_i) + \theta_b)$, where $S_f$ and $S_p$ are average per-token log-likelihood scores from an on-device DP-FL model fine-tuned on user text and a public-web model, and $(\theta_f,\theta_p,\theta_b)$ plus linear regression parameters $\alpha$ are learned by minimizing the residual between reweighted offline evaluations and live A/B metrics from about 15 deployments. This function turns 'does this synthetic example look like mobile user text' into a graded training weight. The other machinery is the data pipeline: cluster a 100-million-document typing-style corpus, subsample with an embedding model, prompt an LLM with grammar-error knowledge to create corrupted/clean pairs, add heuristic typing errors, and then use the reweighting score for filtering and evaluation.

What would settle it

Launch the same continue-training recipe on a fresh keyboard locale with a new live A/B test; if the reweighted offline top-3 good ratio predicts the direction of the click-through and accept-rate change for the new model, the reweighting holds, and if it predicts the wrong sign or misses the paper's residual bounds, the reweighting premise fails.

Watch

Extended reading notes

Core claim

The central claim is that domain adaptation for mobile error correction can be decomposed into three cheap steps: prompt an LLM with teaching-style grammar-error instructions to turn clean conversation-like sentences into corrupted/clean pairs at scale; score each pair by the log-likelihood difference between an 8-million-parameter model fine-tuned with differentially private federated learning on real user text and a public-web model; and reweight the training set with a three-parameter sigmoid fit to a handful of live A/B metrics before final continue-training. On a billion-parameter LLM with LoRA, this recipe outperforms training on the original production dataset alone, and the resulting model reports 2.47% to 7.18% relative improvement on production click-through and accept rates.

Load-bearing premise

The reweighting model's three parameters are fitted to only ten training and five validation live A/B deployments, and this fit is assumed to keep predicting which synthetic samples help for later, unseen model launches.

Editorial extensions

If this is right

  • If the recipe holds, mobile teams can improve error-correction LLMs without collecting raw user text, using only DP-FL model scores and a few live metrics.
  • Reweighted offline evaluation becomes a better proxy for live A/B behavior than the original validation set, so fewer costly launches are needed to judge a candidate model.
  • The continue-training schedule (synthetic first, then a mixed set) is a reusable data-mixing practice for large synthetic datasets that would otherwise take too many steps to cover.
  • The data-synthesis prompts, clustering subsampling, and filtering can be applied to new locales or applications with the same pipeline.
  • With a threshold of $w \ge 1$, about half of the synthetic samples are kept, suggesting the pipeline over-generates and can be downsized.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the same three-parameter reweighting could be re-fit for each new launch; if the fitted coefficients stay stable across launches, the calibration becomes a reusable domain adapter rather than a per-model artifact.
  • A natural stress test is to apply the recipe to a second keyboard domain, such as a language with different error patterns; success would show the synthesis prompts and reweighting transfer rather than being fitted to English mobile typing.
  • If enough live A/B metrics accumulate, the small regression fit could be replaced by a Bayesian model that reports uncertainty, which would address the fragility of fitting three parameters to roughly a dozen deployments.
  • The likelihood-difference signal $S_f - S_p$ might also be used at inference time to decide when to apply the LoRA adapter, not just when to train on a sample; the paper cites similar usage in proxy tuning but does not test it here.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a pipeline for improving error correction in mobile LLM applications: it uses LLM prompts to synthesize large-scale error-correction pairs, then adapts this synthetic data to the mobile domain with a reweighting model built from an 8M-parameter, differentially private federated-learned language model and a handful of live A/B metrics. The reweighting model is fit by minimizing the residual between reweighted offline evaluations and live metrics (Eq. 2). The authors then evaluate a continue-training strategy that first fine-tunes a billion-parameter LLM on synthetic data and then on a mixture of original and filtered synthetic data, reporting offline improvements and a 2.47%–7.18% relative production A/B improvement for the ContMixFil recipe.

Significance. If the production A/B improvement is real, this is a practically valuable and privacy-preserving method for domain adaptation of mobile LLMs. The paper's strengths are its large-scale synthesis with a consistency filter, the use of a small DP-FL model to capture mobile-domain information, and the documentation of a real production deployment with multiple training strategies. The results are presented as a design recipe rather than as a formal statistical study, and the central production claim currently rests on a single sentence without uncertainty quantification. With added statistical rigor, the paper would provide a useful blueprint for similar systems.

major comments (4)
  1. [Sec. 4, final paragraph] The central production claim is reported as 'Compared to Original, ContMixFil achieves 2.47% to 7.18% relative improvement on key production metrics like click through rate and accept rate' with no confidence intervals, significance tests, number of users or events, duration of the experiment, or number of launches. Because ContMixFil was selected after evaluating several training strategies in Table 3, the reported range could reflect selection noise even if the intervention had no effect. Please report the full statistical details: sample sizes, standard errors or confidence intervals, p-values, and preferably a pre-registered comparison or a correction for multiple comparisons.
  2. [Sec. 3, Eq. (2) and Table 2] The Val row of Table 2 is not a fully held-out validation of the reweighting model: the reweighting parameters θ are fixed from the training set, but the regression parameters α are fitted on the validation set (as stated in the text, 'We also use held-one out for validation set to fit only regression parameters with fixed reweighting'). Thus the small Val residual shows the fit of the linear regression, not the predictive quality of Eq. (1) on unseen deployments. The paper should report leave-one-out results in which both θ and α are refit on each training fold and evaluated on the held-out deployment. The statement that the reweighting model 'predicts' live metrics should be restricted to out-of-sample predictions, since Eq. (2) is optimized on those same live metrics for the training models.
  3. [Sec. 3, Eq. (1) and Sec. 4(3)] The reweighting model is fit with only 10 training deployments for θ∈R^3 plus regression parameters α, and validated on 5 further deployments. Table 2's leave-one-out cross-validation standard deviations are large relative to the means (e.g., 3.99±3.15 × 10^-5 for the proposed approach), so the evidence that the 3-parameter sigmoid generalizes is thin. Moreover, the threshold w_t=1 in Sec. 4 is chosen after inspecting the data and keeps 'about half' of the samples; this is an additional data-dependent choice. Please provide sensitivity analyses of the filter threshold, report how many of the 10 training and 5 validation deployments contribute to the claimed improvement, and discuss the amount of data needed for stable reweighting.
  4. [Table 3 and Sec. 4(4)] The offline comparisons in Table 3 are reported without significance tests across training runs, and at least one cell is reported with zero standard deviation (ContMixFil, Original Data Eval, Top-1: 70.48±0.00) even though the text says 'we run training at least twice to compute the standard deviation.' Please report the number of runs per condition and test whether the differences between ContMixFil, ContMix, and Original are statistically significant, not just numerically larger in mean.
minor comments (5)
  1. [Throughout] There are several typos and grammatical errors, including 'grammartical' in the Table 1 prompt, 'ressiduals' in Sec. 3, 'counterparty' in Sec. 4, 'held-one out' in the Table 2 caption, 'The dimensionality of θ, αis' in Sec. 3, and 'We random sample' in Sec. 2.1.
  2. [Sec. 3 and Table 2] The notation for 'Train', 'CrossVal', and 'Val' residuals could be clarified: it is not immediately clear whether 'Train' is the unregularized sum in Eq. (2) and how the cross-validation folds are formed across the 10 training deployments. Please define these quantities explicitly.
  3. [Sec. 4 and Figs. 5, 9] The reweighted metrics labeled '(w)' are computed with the same reweighting model that was fit to the live metrics; this is a form of in-sample evaluation and should be labeled as such, so that readers do not interpret the reweighted offline numbers as independent predictions.
  4. [Sec. 3, Eq. (2)] The regularizer λ∥(1/N)Σ_i w(θ,y_i) − 1∥^2 controls the average reweighting score, but its effect on the scale of the regression coefficients α is not discussed; please clarify how Cmin and Cmax interact with this constraint and whether the average-weight normalization is needed to make the regression residual meaningful.
  5. [Sec. 2] The text says the initial dataset contains 'more than 100 million documents' but the k-means clustering is applied to embeddings of these documents; please specify the embedding dimensionality and the number of samples used for clustering, since the runtime and diversity of the sampled subset depend on these choices.

Circularity Check

3 steps flagged · score 4.0 of 10

Reweighted offline metrics are the fitted objective of Eq. (2), making part of the 'prediction' evidence in-sample; the final production A/B test is genuinely out-of-sample, so circularity is partial.

  1. fitted input called prediction [Sec. 3, Eq. (2); Sec. 4, Fig. 5 and Table 3 captions]
    "min θ,α R(θ, α) + λ∥ 1/N Σ w(θ, yi) − 1∥2, R(θ, α) = Σ ∥ α1/N Σ w(θ, yi)χ(M(xi), yi) + α0 − vj∥2 ... Solid lines reweight the samples by the w(θ, ·) model learnt to fit live A/B test metrics in Sec. 3."

    Eq. (2) trains the reweighting parameters θ and regression parameters α by minimizing the squared distance between the reweighted offline score α1/N Σ wχ + α0 and the live A/B metrics vj. The reweighted metrics reported in Fig. 5 and Table 3 are exactly this fitted linear combination. Saying that one training strategy performs better on the reweighted metrics is therefore largely a restatement of what Eq. (2) was optimized to do, not an independent prediction. The out-of-sample content must come from the cross-validation row and the final production A/B test; the reweighted offline comparisons by themselves do not provide it.

  2. fitted input called prediction [Table 2 caption, Sec. 3]
    "We also use held-one out for validation set to fit only regression parameters with fixed reweighting."

    The paper's own caption admits that the validation-set residual is computed after fitting the regression parameters α on the validation set; only the reweighting parameters θ are fixed. Thus the 'Val' row is not a fully held-out prediction of live metrics. Since α is the linear map from reweighted offline scores to live metrics, fitting α on the validation set makes the reported validation residual partly in-sample by construction. This weakens the paper's claim that 'reweighting achieves smaller residual when predicting live metrics across different settings.'

1 more flagged steps
  1. other [Sec. 4, training practice (3)]
    "We filter the synthetic data set and only keep samples with reweighting scores w(θ, yi) ≥ wt. We choose the threshold wt = 1 as w(θ, yi) is in the range of Cmin = 0.01 and Cmax = 2, and about half of the samples in the synthetic dataset have reweighting scores passed the threshold."

    The filtering threshold wt = 1 is chosen after inspecting the distribution of w(θ, ·), a score that was itself fit to live A/B metrics via Eq. (2). The winning recipe ContMixFil is then selected because it performs better on the reweighted metrics, which are the same fitted mapping. This couples the recipe selection to the fitted values: the threshold and the selection criterion both come from the same fitted function, so the choice of ContMixFil is not independent of the fitting procedure. The production A/B test of ContMixFil remains genuinely out-of-sample, which limits the circularity.

full rationale

The paper's central reweighting claim has a partly circular structure. Eq. (2) optimizes θ and α to minimize the squared distance between a linear combination of reweighted offline scores and live A/B metrics vj; the same linear combination is then labeled a 'reweighted metric' and used to compare training strategies (Table 3, Fig. 5). Beating another model on this metric is by construction what Eq. (2) was fit to do, so those comparisons are not independent evidence for 'predicting live A/B test metrics.' The paper itself concedes that the validation residual is obtained by fitting regression parameters on the validation set, so that row is not genuinely held out. The threshold wt=1 is fixed after inspecting the fitted w distribution, and ContMixFil is then selected on the same reweighted metric, so recipe selection is coupled to the fit. However, the claimed production improvement (2.47% to 7.18%) comes from an actual live A/B test of a newly trained ContMixFil model versus Original; that is out-of-sample and cannot be reduced to Eq. (2). The reliance on Wu et al. [30] is a self-citation, but the discrepancy motivating reweighting is also demonstrated in this paper (Fig. 3a), so I do not treat it as load-bearing circularity. Overall score 4: partial circularity in the reweighted 'prediction' evidence, with independent content in the final A/B test.

Assumptions & free parameters 7 free parameters · 4 assumptions · 0 invented entities

The central claim rests on a small number of fitted reweighting parameters, a handful of live A/B deployments, and the domain assumption that the difference between a DP-FL small LM and a public small LM scores captures mobile relevance. No new physical or conceptual entities are introduced. The main free parameters are the reweighting coefficients, the regression mapping, and several recipe choices (Cmin/Cmax, λ, threshold, mixture ratio, clustering). The key risk is that the reweighting model is fit to very few live metrics and may not generalize.

free parameters (7)
  • reweighting model parameters θ = (θf, θp, θb) = (40.64, -30.44, -1.59)
    Fit in Sec. 3 by minimizing regression residual on 10 training A/B models. This is the function that assigns sample weights.
  • regression parameters α = α = (α1, α0), 2d values, fit separately for training and validation metric sets
    In Eq. (2), α maps reweighted offline eval to live metrics; fit per metric group and per deployment set.
  • Cmin, Cmax = 0.01, 2
    Chosen by hand to clamp reweighting scores; Cmax=2 means weights can double a sample's influence.
  • λ regularizer strength = 0.01
    Hyperparameter on the weight-mean regularizer in Eq. (2).
  • filter threshold w_t = 1
    Used in Sec. 4 to filter synthetic data; about half the samples pass. Chosen without systematic tuning.
  • mixture ratio original:synthetic = 1:4
    Chosen as 'good balance' after testing 1:1 to 1:8 in Sec. 4.
  • cluster count / samples per cluster = 20k clusters / 10 samples per cluster
    Subsampling choices for diversity in Sec. 2; yields 200k documents / 2M examples.
assumptions (4)
  • domain assumption Small LM scores Sf(yi) and Sp(yi) are sufficient statistics for mobile-domain relevance.
    Eq. (1) maps only these two scores to sample weights; if domain information lives elsewhere, the reweighting cannot recover it.
  • domain assumption Live A/B metrics vj are reliable ground truth for user preference.
    Used as targets in Eq. (2), but no confidence intervals or significance tests are reported for these metrics.
  • domain assumption LLM-judged 'good ratio' approximates user acceptance of corrected text.
    Offline evaluation relies on Gemini Pro judges; no human agreement study is presented.
  • domain assumption LLM-synthesized grammar and typing errors cover the real error distribution.
    Prompt-based generation plus heuristic typing rules, filtered only by round-trip consistency; no verification against actual user typing errors is given.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Synthesizing and Adapting Error Correction Data for Mobile Large Language Model Applications." pith.science (2026). https://pith.science/paper/CJUR7LHL

@misc{pith2026250518488,
  author       = {Pith},
  title        = {Pith review of: Synthesizing and Adapting Error Correction Data for Mobile Large Language Model Applications},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CJUR7LHL}},
  note         = {Machine review of arXiv:2505.18488}
}
read the original abstract

Error correction is an important capability when applying large language models (LLMs) to facilitate user typing on mobile devices. In this paper, we use LLMs to synthesize a high-quality dataset of error correction pairs to evaluate and improve LLMs for mobile applications. We first prompt LLMs with error correction domain knowledge to build a scalable and reliable addition to the existing data synthesis pipeline. We then adapt the synthetic data distribution to match the mobile application domain by reweighting the samples. The reweighting model is learnt by predicting (a handful of) live A/B test metrics when deploying LLMs in production, given the LLM performance on offline evaluation data and scores from a small privacy-preserving on-device language model. Finally, we present best practices for mixing our synthetic data with other data sources to improve model performance on error correction in both offline evaluation and production live A/B testing.

Figures

Figures reproduced from arXiv: 2505.18488 by the authors.

Figure 1
Figure 1. Examples of mobile LLM applications for error correction. User typing data has a domain shift compared to public web data. LLMs rewrite and correct highly corrupted text based on the context of the input itself. Modern typing applications on mobile devices use many machine learning models, e.g., language models (LMs)[10, 18, 23, 32]. The generative capacity of LMs can significantly improve user ∗Equal contribution. … view at source ↗
Figure 2
Figure 2. The statistics of the 20k clusters for 100 million documents. The mean with standard deviation of cluster sizes is 5225 ± 2972. To synthesize the EC text pairs, we add two types of errors to the clean texts: grammar error, and typing error. The grammar error is added by Gemini Ultra model [8], and [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Good ratio for error correction on the (a) original validation data and (b) synthetic validation data. The models are trained with the original training data, synthetic training data, and vanilla sampling of synthetic data without clustering. LLMs are used to judge whether the EC output is acceptable to compute good ratio. Our large-scale LLM assisted synthetic data works well on both domains even if there is potent… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Comparing the (a) heuristic {0, 1} reweighting in [30] and (b) our reweighting model w(θ, ·) = 0.01 + 1.99σ(40.64Sf − 30.44Sp − 1.59). Both methods use public pre-trained small LM Sp and the same model further fine-tuned with DP FL Sf . The learnt scores in (b) have la…
Figure 5
Figure 5. Figure 5: Good ratio for the best of top 3 candidates for error correction on the (a) original validation data and (b) synthetic validation data. Solid lines reweight the samples by the w(θ, ·) model learnt to fit live A/B test metrics in Sec. 3. The models are trained with synt…
Figure 6
Figure 6. Figure 6: Sequence accuracy of error correction on the (a) original validation data and (b) synthetic validation data. The models are trained with the original training data, synthetic training data, and vanilla sampling of synthetic data without clustering. 14 [PITH_FULL_IMAGE…
Figure 7
Figure 7. Figure 7: Good ratio for the best of top 3 candidates for error correction on the (a) original validation [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Sequence accuracy of error correction on the (a) original validation data and (b) synthetic validation data. The models are trained with synthetic training data with the same setting as in [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 9
Figure 9. Figure 9: Good ratio for error correction on the (a) original validation data and (b) synthetic validation data. Solid lines reweight the samples by the w(θ, ·) model learnt to fit live A/B test metrics in Sec. 3. The models are trained with synthetic training data with the same…

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. MAPLE: Metadata Augmented Private Language Evolution

    cs.CL 2026-02 conditional novelty 5.0 of 10

    MAPLE boosts Private Evolution's initial prompt with DP tabular metadata and donated in-context examples, improving synthetic-text utility, convergence, and API cost on specialized corpora.

Reference graph

Works this paper leans on

35 extracted references · 19 canonical work pages · cited by 1 Pith paper

  1. [1]

    JAX: composable transformations of Python+NumPy programs, 2018

    James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, and Qiao Zhang. JAX: composable transformations of Python+NumPy programs, 2018. URL http://github.com/google/jax

    2018

  2. [2]

    Automatic annotation and evaluation of error types for grammatical error correction

    Christopher Bryant, Mariano Felice, and Ted Briscoe. Automatic annotation and evaluation of error types for grammatical error correction. In Regina Barzilay and Min-Yen Kan, editors, Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages 793–805, Vancouver, Canada, July 2017. Association for...

    doi:10.18653/v1/p17-1074 2017

  3. [3]

    Grammatical error correction: A survey of the state of the art

    Christopher Bryant, Zheng Yuan, Muhammad Reza Qorib, Hannan Cao, Hwee Tou Ng, and Ted Briscoe. Grammatical error correction: A survey of the state of the art. Computational Linguistics, 49(3):643–701, 2023

    work page 2023

  4. [4]

    Heterogeneous low-rank approximation for federated fine-tuning of on-device foundation models

    Yae Jee Cho, Luyang Liu, Zheng Xu, Aldi Fahrezi, and Gauri Joshi. Heterogeneous low-rank approximation for federated fine-tuning of on-device foundation models. EMNLP, 2024

    work page 2024

  5. [5]

    (amplified) banded matrix factorization: A unified approach to private training

    Christopher A Choquette-Choo, Arun Ganesh, Ryan McKenna, H Brendan McMahan, John Rush, Abhradeep Guha Thakurta, and Zheng Xu. (amplified) banded matrix factorization: A unified approach to private training. Advances in Neural Information Processing Systems , 36, 2024

    work page 2024

  6. [6]

    Scaling instruction-finetuned language models

    Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetuned language models. arXiv preprint arXiv:2210.11416 , 2022

    arXiv

  7. [7]

    Federated learning in practice: reflections and projections

    Katharine Daly, Hubert Eichner, Peter Kairouz, H Brendan McMahan, Daniel Ramage, and Zheng Xu. Federated learning in practice: reflections and projections. In 2024 IEEE 6th 10 International Conference on Trust, Privacy and Security in Intelligent Systems, and Applications (TPS-ISA), pages 148–156. IEEE, 2024

    work page 2024

  8. [8]

    Gemini: A family of highly capable multimodal models, 2024

    Gemini Team Google. Gemini: A family of highly capable multimodal models, 2024. URL https://arxiv.org/abs/2312.11805

    arXiv 2024

Show all 35 references
  1. [9]

    Apple intelligence foundation language models

    Tom Gunter, Zirui Wang, Chong Wang, Ruoming Pang, Andy Narayanan, Aonan Zhang, Bowen Zhang, Chen Chen, Chung-Cheng Chiu, David Qiu, et al. Apple intelligence foundation language models. arXiv preprint arXiv:2407.21075 , 2024

    2024

  2. [10]

    Federated learning for mobile keyboard prediction

    Andrew Hard, Kanishka Rao, Rajiv Mathews, Swaroop Ramaswamy, Fran¸ coise Beaufays, Sean Augenstein, Hubert Eichner, Chlo´ e Kiddon, and Daniel Ramage. Federated learning for mobile keyboard prediction. arXiv preprint arXiv:1811.03604 , 2018

    2018 arXiv

  3. [11]

    Pre-text: Training language models on private federated data in the age of llms

    Charlie Hou, Akshat Shrivastava, Hongyuan Zhan, Rylan Conway, Trang Le, Adithya Sagar, Giulia Fanti, and Daniel Lazar. Pre-text: Training language models on private federated data in the age of llms. ICML, 2024

    2024

  4. [12]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1 (2):3, 2022

    2022

  5. [13]

    User inference attacks on large language models

    Nikhil Kandpal, Krishna Pillutla, Alina Oprea, Peter Kairouz, Christopher A Choquette-Choo, and Zheng Xu. User inference attacks on large language models. EMNLP, 2024

    2024

  6. [14]

    Harnessing large-language models to generate private synthetic text

    Alexey Kurakin, Natalia Ponomareva, Umar Syed, Liam MacDermed, and Andreas Terzis. Harnessing large-language models to generate private synthetic text. arXiv preprint arXiv:2306.01684, 2023

    2023 arXiv

  7. [15]

    Jinhyuk Lee, Zhuyun Dai, Xiaoqi Ren, Blair Chen, Daniel Cer, Jeremy R. Cole, Kai Hui, Michael Boratko, Rajvi Kapadia, Wen Ding, Yi Luan, Sai Meher Karthik Duddu, Gustavo Her- nandez Abrego, Weiqiang Shi, Nithi Gupta, Aditya Kusupati, Prateek Jain, Siddhartha Reddy Jonnalagadda...

    2024 arXiv

  8. [16]

    Data weighted training strategies for grammatical error correction

    Jared Lichtarge, Chris Alberti, and Shankar Kumar. Data weighted training strategies for grammatical error correction. Transactions of the Association for Computational Linguistics , 8: 634–646, 2020

    2020

  9. [17]

    Tuning language models by proxy

    Alisa Liu, Xiaochuang Han, Yizhong Wang, Yulia Tsvetkov, Yejin Choi, and Noah A Smith. Tuning language models by proxy. COLM, 2024

    2024

  10. [18]

    Proofread: Fixes all errors with one tap

    Renjie Liu, Yanxiang Zhang, Yun Zhu, Haicheng Sun, Yuanbo Zhang, Michael Huang, Shanqing Cai, Lei Meng, and Shumin Zhai. Proofread: Fixes all errors with one tap. In Yixin Cao, Yang Feng, and Deyi Xiong, editors, Proceedings of the 62nd Annual Meeting of the Association for Co...

    2024 doi

  11. [19]

    A hassle-free algorithm for strong differential privacy in federated learning systems

    H Brendan McMahan, Zheng Xu, and Yanxiang Zhang. A hassle-free algorithm for strong differential privacy in federated learning systems. EMNLP, 2024. 11

    2024

  12. [20]

    The llama 3 herd of models, 2024

    Llama Team Meta. The llama 3 herd of models, 2024. URL https://ai.meta.com/research/ publications/the-llama-3-herd-of-models/

    2024

  13. [21]

    Gpt-4 technical report, 2024

    OpenAI. Gpt-4 technical report, 2024. URL https://arxiv.org/abs/2303.08774

    2024 arXiv

  14. [22]

    Training language models to follow instructions with human feedback

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems , 2022

    2022

  15. [23]

    Mobile keyboard input decoding with finite-state transducers

    Tom Ouyang, David Rybach, Fran¸ coise Beaufays, and Michael Riley. Mobile keyboard input decoding with finite-state transducers. arXiv preprint arXiv:1704.03987 , 2017

    2017 arXiv

  16. [24]

    Exploring the limits of transfer learning with a unified text-to-text transformer

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research , 21(1):5485–5551, 2020

    2020

  17. [25]

    Simulating errors in touchscreen typing

    Danqing Shi, Yujun Zhu, Francisco Erivaldo Fernandes Junior, Shumin Zhai, and Antti Oulasvirta. Simulating errors in touchscreen typing. arXiv preprint arXiv:2502.03560 , 2025

    2025 arXiv

  18. [26]

    Synthetic data generation for grammatical error correction with tagged corruption models

    Felix Stahlberg and Shankar Kumar. Synthetic data generation for grammatical error correction with tagged corruption models. arXiv preprint arXiv:2105.13318 , 2021

    2021 arXiv

  19. [27]

    Synthesizing privacy- preserving text data via finetuning without finetuning billion-scale llms

    Bowen Tan, Zheng Xu, Eric P Xing, Zhiting Hu, and Shanshan Wu. Synthesizing privacy- preserving text data via finetuning without finetuning billion-scale llms. In ICML, 2025

    2025

  20. [28]

    Finetuned language models are zero-shot learners

    Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. Finetuned language models are zero-shot learners. arXiv preprint arXiv:2109.01652, 2021

    2021 arXiv

  21. [29]

    Emergent abilities of large language models

    Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. Emergent abilities of large language models. arXiv preprint arXiv:2206.07682 , 2022

    2022 arXiv

  22. [30]

    Prompt public large language models to synthesize data for private on-device applications

    Shanshan Wu, Zheng Xu, Yanxiang Zhang, Yuanbo Zhang, and Daniel Ramage. Prompt public large language models to synthesize data for private on-device applications. Conference on Language Modeling (COLM) , 2024

    2024

  23. [31]

    Differentially private synthetic data via foundation model apis 2: Text

    Chulin Xie, Zinan Lin, Arturs Backurs, Sivakanth Gopi, Da Yu, Huseyin A Inan, Harsha Nori, Haotian Jiang, Huishuai Zhang, Yin Tat Lee, et al. Differentially private synthetic data via foundation model apis 2: Text. In Forty-first International Conference on Machine Learning (I...

    2024

  24. [32]

    Federated learning of gboard language models with differential privacy

    Zheng Xu, Yanxiang Zhang, Galen Andrew, Christopher Choquette, Peter Kairouz, Brendan Mcmahan, Jesse Rosenstock, and Yuanbo Zhang. Federated learning of gboard language models with differential privacy. In Proceedings of the 61st Annual Meeting of the Association for Computati...

    2023

  25. [33]

    Privacy-preserving instructions for aligning large language models

    Da Yu, Peter Kairouz, Sewoong Oh, and Zheng Xu. Privacy-preserving instructions for aligning large language models. ICML, 2024. 12

    2024

  26. [34]

    Synthetic text generation with differential privacy: A simple and practical recipe

    Xiang Yue, Huseyin A Inan, Xuechen Li, Girish Kumar, Julia McAnallen, Hoda Shajari, Huan Sun, David Levitan, and Robert Sim. Synthetic text generation with differential privacy: A simple and practical recipe. ACL, 2023

    2023

  27. [35]

    Towards an on-device agent for text rewriting

    Yun Zhu, Yinxiao Liu, Felix Stahlberg, Shankar Kumar, Yu-hui Chen, Liangchen Luo, Lei Shu, Renjie Liu, Jindong Chen, and Lei Meng. Towards an on-device agent for text rewriting. arXiv preprint arXiv:2308.11807, 2023. 13 A Additional results in Sec. 2 0 2000 4000 6000 Training ...

    2023 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.