Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Time Will Tell: Timing Side Channels via Output Token Count in Large Language Models

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Output token count leaks target languages and classes

desk verdict The token-count side-channel is real and well supported on open models; the GPT-4o remote-timing result is the soft spot and needs a protocol fix. read the letter →

arxiv 2412.15431 v1 pith:W2IBS2CB submitted 2024-12-19 cs.LG cs.CLcs.CR

Tianchen Zhang , Gururaj Saileshwar , David Lie This is my paper · ORCID
classification cs.LGcs.CLcs.CR
keywords side-channelattackoutputtokencounttiminglargelanguagemodelsautoregressivedecodingidentificationclassificationprivacyfew-shotpromptingbias
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that the number of tokens an LLM emits is a usable side channel: because autoregressive decoding produces tokens at a roughly constant rate, total response time reveals the output token count, and token counts carry information about private attributes. In translation, different target languages produce measurably different token densities and output/input byte ratios, letting an attacker recover the target language with roughly 75–83% average precision across three multilingual models. In classification with explanations, models show class-dependent explanation-length biases that can be amplified by few-shot examples, leaking the output class with roughly 70–93% precision across open models and GPT-4o. The attack works over the network in non-streaming mode using only response times, and the paper proposes tokenizer-, prompt-, and system-level mitigations.

What carries the argument

The load-bearing identity is the linear timing model of autoregressive generation, $T(X_n) \propto n$ (Eq. 5): total response time is dominated by the decode stage, whose per-token time is roughly constant, so response time is a proxy for output token count. On top of this the paper builds two statistical fingerprints. For translation, it profiles each target language as a two-dimensional Gaussian mixture over output token density (output bytes per token) and output/input byte ratio, then classifies a victim's requests by Bhattacharyya distance to the closest language profile. For classification, it fits a threshold $\text{Threshold} = \alpha \times \text{Input Length} + \beta$ to separate the two classes' token counts, optionally sharpened by biased few-shot examples that amplify class-dependent explanation-length differences.

What would settle it

Run the end-to-end token-recovery experiment on GPT-4o without selecting the best profiling day in advance and without concurrent network profiling: if the median error in estimated token count exceeds the separation between class profiles, so that attack precision falls to chance levels, the central claim that timing reveals token count for production models would fail. A simpler check is to measure the Pearson correlation between response time and token count on a busy API day; if it approaches 0.37 or lower, the linear model behind the attack is not reliable in that deployment.

Watch

Extended reading notes

Core claim

The central discovery is that the autoregressive decode loop of a transformer LLM turns output token count into a remotely observable signal: generation time is approximately linear in the number of output tokens ($T(X_n) \propto n$), so an adversary who can measure response time can estimate token count without any token-level information. Because tokenizers encode different languages with different efficiencies and because LLMs write systematically longer explanations for some classes, the token count correlates with the target language of a translation and with the output class of a classification. The paper demonstrates both attacks: a 2D Gaussian-mixture profile over token density and output/input byte ratio recovers target language, and a threshold on output token count (conditioned on input length and optionally sharpened by biased few-shot examples) recovers output class. For production GPT-4o, where the linear correlation is weak (Pearson 0.370), the authors recover accuracy with concurrent network profiling that estimates time-to-first-token and time-per-output-token, reporting a 74.7% remote attack success rate.

Load-bearing premise

The attack assumes that an LLM's response time grows linearly with the number of output tokens; this holds for open models on dedicated hardware (Pearson at least 0.987), but the paper measures only a weak correlation (0.370) for the production GPT-4o service, so the remote attack's success depends on server profiling that may not transfer to other deployments.

Editorial extensions

If this is right

  • Any transformer-based LLM that generates autoregressively, in streaming or non-streaming mode, exposes its output token count through response time, making the channel inherent rather than an artifact of a specific optimization.
  • An attacker monitoring encrypted traffic can recover a user's target language in translation with 75–83% precision and the output class in binary classification with 70–93% precision across the tested models.
  • Biased few-shot examples can amplify the classification leak by roughly 15% on average, meaning that prompt engineering choices can unintentionally worsen the side channel.
  • Mitigations exist at the tokenizer, prompt, and system levels, but they carry costs: padding output to hide token count can impose a latency overhead of roughly 93% for tokenizer-biased models like Tower.
  • The attack transfers across model families and sizes with no strong correlation to model size, so scaling up the model does not by itself close the channel.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reported GPT-4o success rate rests on choosing the best of five profiling days (Day 4) and on concurrent profiling; an attacker who cannot cherry-pick the profiling day would likely see lower accuracy, so the remote-attack claim for closed models is the least established part of the paper.
  • If token count is observable at scale, any task whose output length correlates with a sensitive attribute — such as medical explanations that are longer for certain conditions, or rule-violation counts that grow with the number of triggered rules — becomes a candidate side channel, not just translation and classification.
  • Speculative decoding and continuous batching are expected to weaken but not eliminate the linear timing relation; a testable prediction is that a model served with aggressive batching will show lower but still nonzero correlation between token count and response time, shifting attacks toward averaging over more requests.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper introduces an output-token-count side channel in large language models and evaluates it against machine translation and few-shot text classification workloads. For translation, the attacker infers the target language from token density and output/input byte ratio, using the Flores dataset and three multilingual models (Tower, M2M100, MBart50). For classification, the attacker infers the output class from the length of generated explanations, using tasks from Natural Instructions and models from the Gemma, Llama, and GPT-4o families. The paper further argues that autoregressive decoding makes response time a proxy for output token count, enabling end-to-end network attacks, including a remote attack on GPT-4o over the internet. The paper closes with tokenizer-level, prompt-level, and system-level mitigations.

Significance. The core observation that token-count statistics are input-dependent and observable through timing is relevant and, for open-source models, well supported by the experiments. Strengths include the use of public datasets, the separation of profiling and attack data, the ablation studies comparing token density and byte-ratio features, the exploration of few-shot explanation biases with augmenting and diminishing examples, and the evaluation of candidate mitigations. If the results hold, the paper usefully reframes tokenizer fairness biases and explanation-length tendencies as privacy risks. The main weakness is the production-model remote attack, which currently relies on a post-hoc profile-selection step and a low token-count/time correlation; the paper would be a solid empirical security contribution after that protocol is fixed and uncertainty is reported.

major comments (3)
  1. [§6.6, Table 3] The reported remote attack against GPT-4o is not yet convincing because the class profile is selected post hoc. Section 6.6 states that the authors 'took profiles on all days and chose the one with the best discriminator (i.e. highest ASR on the profile data), which happened to be the profile taken on Day 4.' Since Table 3 shows the Pearson correlation between token count and response time for GPT-4o is only 0.370 (in contrast to at least 0.987 for the open models), the timing channel for this model is substantially noisier, and a post-hoc choice of the best profiling day can inflate the reported 74.7% average ASR. Please report results under a pre-specified profile-selection protocol (for example, fixing the profile on one day before collecting test data, or using leave-one-day-out cross-validation) and give day-by-day ASR for that protocol.
  2. [Abstract, §4.5.2, Table 4] The claim of 'more than 75% precision across three different models' is an average over languages and does not reflect the per-language spread. In Table 4, per-language success rates range from 34.0% (MBart50, Portuguese) and 38.7% (M2M100, Italian) to 100.0%, and in Table 5 the classification ASR ranges from 43.5% to 93.9% depending on task and model. All reported accuracies are point estimates without confidence intervals or significance tests. Because the security conclusion is that a practical side channel exists, the paper should report per-language and per-task results with error bars and clearly state the number of user requests needed to reach the headline precision.
  3. [§4.3, Figure 12] The headline translation ASR is achieved only when the attacker observes 50 user requests. Figure 12 shows that with a single request the average ASR falls to about 48%, and with 10 requests it is about 66%. The paper should state this dependence prominently, since the abstract and Section 4.5.2 present the 82.5% figure without qualification. For the threat model to support 'practical' leakage, the authors should either justify why a user would generate 50 same-pair translation requests or characterize the attack for realistically small sample counts.
minor comments (5)
  1. [§2.1 and §6.2] The TPOT acronym is defined inconsistently: Section 2.1 calls it 'Time Per Output Token', while Section 6.2 calls it 'Total Processing Time'; please make the definitions consistent.
  2. [§4.4, §4.5.3, §5.3, §6.2] Several typos should be fixed: 'M2N100' in §4.4; 'are are selected' in §4.5.3; 'whic have' in §5.3; 'uses uses' and 'the our attacker' in §6.2.
  3. [§6.2] The concurrent-profiling description is hard to parse: 'the attacker will aforementioned network profiling results to adjust' and 'a 5-minute window to and takes the median' are missing words and should be rewritten.
  4. [§6.6, Figure 11] Figure 11 would be more informative with error bars or a measure of spread across tasks, since the text claims concurrent profiling 'behaves more stably across different days'; currently only the means are shown.
  5. [§7.3] The statement that the average performance cost for Tower is 93% is striking but not derived; please clarify whether this is latency overhead for the slowest language or an average over languages, and how padding interacts with the token-density and byte-ratio channels.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the token-count side channel is an empirically observed correlation, and the attacks are standard held-out classifiers.

full rationale

The paper's central derivation chain is: (i) LLM output token counts vary with the private attribute (language in translation, class in classification); (ii) token counts can be observed directly in streaming mode or approximated by generation time because autoregressive decode time grows roughly linearly with token count; and (iii) a profiled classifier maps observed counts or times to the attribute. Each link is empirical and evaluated on held-out data. The translation attack trains a 2D GMM on Flores-200 training inputs and reports ASR on Flores/EuroParl test inputs (Sections 4.4 and 4.5). The classification attack fits a linear threshold (Equation 3) and a balance hyperparameter (Equation 4) on 100 examples per class and reports ASR on separate 200-example test sets (Sections 5.2 and 5.4). The unbiased few-shot baselines (63-67% ASR) show the signal exists without any biased prompt, so the result is not an artifact of the fitted threshold. The only overlapping-author citation, [12], appears in the introduction and related work as a contrast to speculative-decoding side channels and is not used to justify the output-token-count claim. The paper's own admission that GPT-4o has Pearson correlation of only 0.370 between token count and response time (Table 3) is a real generality and robustness limitation of the remote timing variant, but it is not a circular reduction: the token-count side channel itself is demonstrated on open models with near-perfect correlation (at least 0.987) and on classification tasks with direct token-count access. Choosing the best class profile across profiling days (Section 6.6) is model selection on profiling data; it may inflate results, but it does not make the reported test predictions equal to the fitted inputs by construction. Overall, the derivation does not reduce to its own inputs or to a load-bearing self-citation chain.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The attack's success depends on two fitted threshold parameters (α, β) per classification task, a hand-picked balance hyperparameter (θ), and three domain assumptions: decode time proportional to token count, encrypted lengths preserved, and attacker knowledge of the task. No new entities are postulated; the GMM/Bhattacharyya machinery is standard.

free parameters (3)
  • Threshold slope α = Not reported
    Equation (3) fits a linear threshold between output token count and input length to separate classes in the profiling phase; the fitted value is not disclosed.
  • Threshold intercept β = Not reported
    Same as α, fitted to profiling data in Section 5.2.
  • Precision-balance hyperparameter θ = 0.5
    Equation (4) uses θ to trade off precision between the two classes; chosen by the authors, no sensitivity analysis.
assumptions (4)
  • domain assumption Autoregressive decode time is proportional to output token count (T(Xn) ∝ n)
    Section 2.1 states this; Table 3 shows it holds for open models but not GPT-4o (Pearson 0.370).
  • domain assumption Encrypted network traffic preserves plaintext byte lengths
    Threat model in Section 3 adopts this from prior work [10]; required to compute output-input byte ratio and token density.
  • domain assumption Attacker knows the application task and can profile the model as a regular user
    Section 3 explicitly assumes this; without it, the GMM and threshold profiles cannot be built.
  • standard math Gaussian Mixture Model captures token density and byte-ratio distributions
    Section 4.3 uses a 2D GMM and Bhattacharyya distance; standard statistical machinery.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Time Will Tell: Timing Side Channels via Output Token Count in Large Language Models." pith.science (2026). https://pith.science/paper/W2IBS2CB

@misc{pith2026241215431,
  author       = {Pith},
  title        = {Pith review of: Time Will Tell: Timing Side Channels via Output Token Count in Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/W2IBS2CB}},
  note         = {Machine review of arXiv:2412.15431}
}
read the original abstract

This paper demonstrates a new side-channel that enables an adversary to extract sensitive information about inference inputs in large language models (LLMs) based on the number of output tokens in the LLM response. We construct attacks using this side-channel in two common LLM tasks: recovering the target language in machine translation tasks and recovering the output class in classification tasks. In addition, due to the auto-regressive generation mechanism in LLMs, an adversary can recover the output token count reliably using a timing channel, even over the network against a popular closed-source commercial LLM. Our experiments show that an adversary can learn the output language in translation tasks with more than 75% precision across three different models (Tower, M2M100, MBart50). Using this side-channel, we also show the input class in text classification tasks can be leaked out with more than 70% precision from open-source LLMs like Llama-3.1, Llama-3.2, Gemma2, and production models like GPT-4o. Finally, we propose tokenizer-, system-, and prompt-based mitigations against the output token count side-channel.

Figures

Figures reproduced from arXiv: 2412.15431 by the authors.

Figure 1
Figure 1. Example of our side-channel attacks on text sentiment classification with an LLM. (a) For a given text input, the LLM outputs the classified [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Threat model. We assume the attacker is network-based, and [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Overview of Attack on Translation. A user translates text in a [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (13 more)
Figure 4
Figure 4. Figure 4: Profile using a Gaussian Mixture Model on a 2D decision space [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Attack success rates (ASR) for the attack leaking the output lan [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Ablation study showing ASR (%) when only output token density [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Prompt template for our tasks, using the Predict-then-Explain [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: Output Token Distribution for Task-145, with Unbiased, Aug [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: ASR (%) across all tasks for Gemma2-9B, with unbiased and [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]
Figure 10
Figure 10. Figure 10: ASR (%) as the number of few-shot examples is varied from [PITH_FULL_IMAGE:figures/full_fig_p009_10.png]
Figure 11
Figure 11. Figure 11: Average ASR (%) across different tasks for remote timing attack [PITH_FULL_IMAGE:figures/full_fig_p011_11.png]
Figure 12
Figure 12. Figure 12: Attack success rate (ASR) for the translation attack on the Tower [PITH_FULL_IMAGE:figures/full_fig_p016_12.png]
Figure 13
Figure 13. Figure 13: Attack success rate (ASR) for the translation attack on Tower [PITH_FULL_IMAGE:figures/full_fig_p016_13.png]
Figure 14
Figure 14. Figure 14: Ablation study for ASR (%) using just token density or just [PITH_FULL_IMAGE:figures/full_fig_p017_14.png]
Figure 15
Figure 15. Figure 15: Ablation study for ASR (%) with just token density or just [PITH_FULL_IMAGE:figures/full_fig_p017_15.png]
Figure 17
Figure 17. Figure 17: Network Delay Profiling for GPT-4o responses. [PITH_FULL_IMAGE:figures/full_fig_p017_17.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Leaky Language Models: Stealing Architecture and Inference Optimizations via Per-Token Timing

    cs.CR 2026-07 conditional novelty 7.0 of 10

    Per-token generation timing leaks speculative decoding and draft-model context length from Gemini, and recovers layer count and hidden size of Llama-family models with top-5 accuracy up to 65% when both are unknown.

Reference graph

Works this paper leans on

57 extracted references · 30 canonical work pages · cited by 1 Pith paper

  1. [1]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. u. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in Neural Information Processing Systems , I. Guyon, U. V . Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, Eds., vol. 30. Curran Associates, Inc., 2017. [Online]. Available: https://...

    work page 2017

  2. [2]

    Llama 2: Open foundation and fine-tuned chat models,

    H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y . Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, D. Bikel, L. Blecher, C. C. Ferrer, M. Chen, G. Cucurull, D. Esiobu, J. Fernandes, J. Fu, W. Fu, B. Fuller et al. , “Llama 2: Open foundation and fine-tuned chat models,” 2023. [Online]. Available: https://arxiv.org/abs/2307.09288

    arXiv 2023

  3. [3]

    Introducing ChatGPT,

    OpenAI, “Introducing ChatGPT,” 2022. [Online]. Available: https: //openai.com/index/chatgpt/

    2022

  4. [4]

    Introducing Microsoft 365 Copilot – your copilot for work,

    J. Spataro, “Introducing Microsoft 365 Copilot – your copilot for work,” Mar. 2023. [On- line]. Available: https://blogs.microsoft.com/blog/2023/03/16/ introducing-microsoft-365-copilot-your-copilot-for-work/

    work page 2023

  5. [5]

    The technology behind Amazon’s GenAI- powered shopping assistant, Rufus,

    T. Chilimbi, “The technology behind Amazon’s GenAI- powered shopping assistant, Rufus,” Oct. 2024. [Online]. Available: https://www.amazon.science/blog/ the-technology-behind-amazons-genai-powered-shopping-assistant-rufus

    work page 2024

  6. [6]

    Detecting pretraining data from large language models,

    W. Shi, A. Ajith, M. Xia, Y . Huang, D. Liu, T. Blevins, D. Chen, and L. Zettlemoyer, “Detecting pretraining data from large language models,” in International Conference on Learning Representations (ICLR), 2024

    work page 2024

  7. [7]

    Did the neurons read your book? document-level membership inference for large language models,

    M. Meeus, S. Jain, M. Rei, and Y .-A. de Montjoye, “Did the neurons read your book? document-level membership inference for large language models,” in 33rd USENIX Security Symposium (USENIX Security 24), 2024, pp. 2369–2385

    2024

  8. [8]

    Ex- tracting training data from large language models,

    N. Carlini, F. Tramer, E. Wallace, M. Jagielski, A. Herbert-V oss, K. Lee, A. Roberts, T. Brown, D. Song, U. Erlingsson et al. , “Ex- tracting training data from large language models,” in 30th USENIX Security Symposium (USENIX Security 21) , 2021, pp. 2633–2650

    work page 2021

Show all 57 references
  1. [9]

    Formalizing and benchmarking prompt injection attacks and defenses,

    Y . Liu, Y . Jia, R. Geng, J. Jia, and N. Z. Gong, “Formalizing and benchmarking prompt injection attacks and defenses,” in 33rd USENIX Security Symposium (USENIX Security 24), 2024, pp. 1831– 1847

    2024

  2. [10]

    What was your prompt? a remote keylogging attack on AI assistants,

    R. Weiss, D. Ayzenshteyn, and Y . Mirsky, “What was your prompt? a remote keylogging attack on AI assistants,” in 33rd USENIX Security Symposium (USENIX Security 24) . Philadelphia, PA: USENIX Association, Aug. 2024, pp. 3367–3384. [Online]. Available: https: //www.usenix.org/...

    2024

  3. [11]

    The early bird catches the leak: Unveiling timing side channels in llm serving systems,

    L. Song, Z. Pang, W. Wang, Z. Wang, X. Wang, H. Chen, W. Song, Y . Jin, D. Meng, and R. Hou, “The early bird catches the leak: Unveiling timing side channels in llm serving systems,” 2024. [Online]. Available: https://arxiv.org/abs/2409.20002

    2024

  4. [12]

    Privacy risks of speculative decoding in large language models,

    J. Wei, A. Abdulrazzag, T. Zhang, A. Muursepp, and G. Saileshwar, “Privacy risks of speculative decoding in large language models,” arXiv preprint arXiv:2411.01076 , 2024

    2024

  5. [13]

    Language Model Tokenizers Introduce Unfairness Between Languages,

    A. Petrov, E. La Malfa, P. Torr, and A. Bibi, “Language Model Tokenizers Introduce Unfairness Between Languages,” in Advances in Neural Information Processing Systems , A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, Eds., vol. 36. Curran Associates, Inc.,...

    2023

  6. [14]

    Do all languages cost the same? tokenization in the era of commercial language models,

    O. Ahia, S. Kumar, H. Gonen, J. Kasai, D. R. Mortensen, N. A. Smith, and Y . Tsvetkov, “Do all languages cost the same? tokenization in the era of commercial language models,” 2023

    2023

  7. [15]

    Hello GPT-4o,

    OpenAI, “Hello GPT-4o,” 2024. [Online]. Available: https://openai. com/index/hello-gpt-4o/

    2024

  8. [16]

    Neural machine translation of rare words with subword units,

    R. Sennrich, B. Haddow, and A. Birch, “Neural machine translation of rare words with subword units,” in Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2016, pp. 1715–1725

    2016

  9. [17]

    Google’s neural machine translation system: Bridging the gap between human and machine translation,

    Y . Wu, M. Schuster, Z. Chen, Q. V . Le, M. Norouzi, W. Macherey, M. Krikun, Y . Cao, Q. Gao, K. Macherey, J. Klingner, A. Shah, M. Johnson, X. Liu, Łukasz Kaiser, S. Gouws, Y . Kato, T. Kudo, H. Kazawa, K. Stevens et al. , “Google’s neural machine translation system: Bridging...

  10. [18]

    Sentencepiece: A simple and lan- guage independent subword tokenizer and detokenizer for neural text processing,

    T. Kudo and J. Richardson, “Sentencepiece: A simple and lan- guage independent subword tokenizer and detokenizer for neural text processing,” in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations . Association for Comp...

    2018

  11. [19]

    Unpacking tokenization: Evaluating text compression and its correlation with model performance,

    O. Goldman, A. Caciularu, M. Eyal, K. Cao, I. Szpektor, and R. Tsarfaty, “Unpacking tokenization: Evaluating text compression and its correlation with model performance,” 2024. [Online]. Available: https://arxiv.org/abs/2403.06265

    2024 arXiv

  12. [20]

    Beyond english-centric multilingual machine translation,

    A. Fan, S. Bhosale, H. Schwenk, Z. Ma, A. El-Kishky, S. Goyal, M. Baines, O. Celebi, G. Wenzek, V . Chaudhary, N. Goyal, T. Birch, V . Liptchinsky, S. Edunov, E. Grave, M. Auli, and A. Joulin, “Beyond english-centric multilingual machine translation,” 2020. [Online]. Available...

    2020 arXiv

  13. [21]

    Bloom: A 176b-parameter open-access multilingual language model,

    T. L. Scao, A. Fan, C. Akiki, E. Pavlick, S. Ili ´c, D. Hesslow, R. Castagn´e, A. S. Luccioni, F. Yvon, M. Gall ´e, J. Tow, A. M. Rush, S. Biderman, A. Webson, P. S. Ammanamanchi, T. Wang, B. Sagot, N. Muennighoff, A. V . del Moral, O. Ruwase et al. , “Bloom: A 176b-parameter ...

    2023 arXiv

  14. [22]

    Allocating large vocabulary capacity for cross- lingual language model pre-training,

    B. Zheng, L. Dong, S. Huang, S. Singhal, W. Che, T. Liu, X. Song, and F. Wei, “Allocating large vocabulary capacity for cross- lingual language model pre-training,” 2021. [Online]. Available: https://arxiv.org/abs/2109.07306

    2021 arXiv

  15. [23]

    Scaffold-bpe: Enhancing byte pair encoding with simple and effective scaffold token removal,

    H. Lian, Y . Xiong, J. Niu, S. Mo, Z. Su, Z. Lin, P. Liu, H. Chen, and G. Ding, “Scaffold-bpe: Enhancing byte pair encoding with simple and effective scaffold token removal,” 2024. [Online]. Available: https://arxiv.org/abs/2404.17808

    2024 arXiv

  16. [24]

    Xlm-v: Overcoming the vocabulary bottleneck in multilingual masked language models,

    D. Liang, H. Gonen, Y . Mao, R. Hou, N. Goyal, M. Ghazvininejad, L. Zettlemoyer, and M. Khabsa, “Xlm-v: Overcoming the vocabulary bottleneck in multilingual masked language models,” 2023. [Online]. Available: https://arxiv.org/abs/2301.10472

    2023 arXiv

  17. [25]

    Bert: Pre- training of deep bidirectional transformers for language understand- ing,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre- training of deep bidirectional transformers for language understand- ing,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technol...

    2019

  18. [26]

    Bert for joint intent classification and slot filling,

    Q. Chen, Z. Zhuo, and W. Wang, “Bert for joint intent classification and slot filling,” arXiv preprint arXiv:1902.10909 , 2019

    1902 arXiv

  19. [27]

    Bertopic: Neural topic modeling with a class-based tf-idf procedure,

    M. Grootendorst, “Bertopic: Neural topic modeling with a class-based tf-idf procedure,” 2022. [Online]. Available: https: //arxiv.org/abs/2203.05794

    2022 arXiv

  20. [28]

    Universal language model fine-tuning for text classification,

    J. Howard and S. Ruder, “Universal language model fine-tuning for text classification,” in ACL 2018-56th Annual Meeting of the Associ- ation for Computational Linguistics, Proceedings of the Conference (Long Papers), vol. 1. Association for Computational Linguistics, 2018, pp. 328–339

    2018

  21. [29]

    Language models are few-shot learners,

    T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-V oss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. Ziegler, J. Wu, C. Winter, C. Hesse et al. , “Language models are few-shot learner...

    2020

  22. [30]

    Calibrate before use: Improving few-shot performance of language models,

    T. Z. Zhao, E. Wallace, S. Feng, D. Klein, and S. Singh, “Calibrate before use: Improving few-shot performance of language models,”

  23. [31]

    The unreliability of explanations in few-shot prompting for textual reasoning,

    X. Ye and G. Durrett, “The unreliability of explanations in few-shot prompting for textual reasoning,” 2022. [Online]. Available: https://arxiv.org/abs/2205.03401

    2022 arXiv

  24. [32]

    Improving few-shot performance of language models via nearest neighbor calibration,

    F. Nie, M. Chen, Z. Zhang, and X. Cheng, “Improving few-shot performance of language models via nearest neighbor calibration,”

  25. [33]

    Fairness-guided few-shot prompting for large language models,

    H. Ma, C. Zhang, Y . Bian, L. Liu, Z. Zhang, P. Zhao, S. Zhang, H. Fu, Q. Hu, and B. Wu, “Fairness-guided few-shot prompting for large language models,” 2023. [Online]. Available: https://arxiv.org/abs/2303.13217

    2023 arXiv

  26. [34]

    Translate text by using adaptive translation

    Google Cloud, “Translate text by using adaptive translation.” [On- line]. Available: https://cloud.google.com/translate/docs/advanced/ adaptive-translation

  27. [35]

    DeepL’s next-gen LLM outperforms ChatGPT-4, Google, and Microsoft for translation quality

    DeepL Team, “DeepL’s next-gen LLM outperforms ChatGPT-4, Google, and Microsoft for translation quality.” [Online]. Available: https://www.deepl.com/en/blog/next-gen-language-model

  28. [36]

    General Data Protection Regulation - Article 4

    European Union, “General Data Protection Regulation - Article 4.” [Online]. Available: https://gdprinfo.eu/en-article-4

  29. [37]

    Maximum likelihood from incomplete data via the em algorithm,

    A. P. Dempster, N. M. Laird, and D. B. Rubin, “Maximum likelihood from incomplete data via the em algorithm,” Journal of the Royal Statistical Society: Series B (Methodological) , vol. 39, no. 1, pp. 1–22, 1977. [Online]. Available: https://rss.onlinelibrary.wiley.com/ doi/abs...

    1977

  30. [38]

    On a measure of divergence between two multinomial populations,

    A. Bhattacharyya, “On a measure of divergence between two multinomial populations,” Sankhy¯a: The Indian Journal of Statistics (1933-1960), vol. 7, no. 4, pp. 401–406, 1946. [Online]. Available: http://www.jstor.org/stable/25047882

    1933

  31. [39]

    Multilingual translation with extensible multilingual pretraining and finetuning,

    Y . Tang, C. Tran, X. Li, P.-J. Chen, N. Goyal, V . Chaudhary, J. Gu, and A. Fan, “Multilingual translation with extensible multilingual pretraining and finetuning,” 2020. [Online]. Available: https://arxiv.org/abs/2008.00401

    2020 arXiv

  32. [40]

    Tower: An open multilingual large language model for translation-related tasks,

    D. M. Alves, J. Pombal, N. M. Guerreiro, P. H. Martins, J. Alves, A. Farajian, B. Peters, R. Rei, P. Fernandes, S. Agrawal, P. Colombo, J. G. C. de Souza, and A. F. T. Martins, “Tower: An open multilingual large language model for translation-related tasks,”

  33. [41]

    No language left behind: Scaling human-centered machine translation,

    NLLB Team, M. R. Costa-juss `a, J. Cross, O. C ¸ elebi, M. Elbayad, K. Heafield, K. Heffernan, E. Kalbassi, J. Lam, D. Licht, J. Maillard, A. Sun, S. Wang, G. Wenzek, A. Youngblood, B. Akula, L. Barrault, G. M. Gonzalez, P. Hansanti, J. Hoffman et al. , “No language left behin...

    2022 arXiv

  34. [42]

    Europarl: A parallel corpus for statistical machine translation,

    P. Koehn, “Europarl: A parallel corpus for statistical machine translation,” in Proceedings of Machine Translation Summit X: Papers, Phuket, Thailand, Sep. 13-15 2005, pp. 79–86. [Online]. Available: https://aclanthology.org/2005.mtsummit-papers.11

    2005

  35. [43]

    Einstein for Service

    Salesforce, “Einstein for Service.” [Online]. Avail- able: https://help.salesforce.com/s/articleView?id=release-notes.rn einstein service.htm&language=en US&release=246&type=5

  36. [44]

    Med-PaLM: A Medical Large Language Model - Google Research

    Google Research, “Med-PaLM: A Medical Large Language Model - Google Research.” [Online]. Available: https://sites.research.google/ med-palm/

  37. [45]

    Super-naturalinstructions:generalization via declarative instructions on 1600+ tasks,

    Y . Wang, S. Mishra, P. Alipoormolabashi, Y . Kordi, A. Mirzaei, A. Arunkumar, A. Ashok, A. S. Dhanasekaran, A. Naik, D. Stapet al., “Super-naturalinstructions:generalization via declarative instructions on 1600+ tasks,” in EMNLP, 2022

    2022

  38. [46]

    Gpt-4 technical report,

    OpenAI, “Gpt-4 technical report,” 2024. [Online]. Available: https://arxiv.org/abs/2303.08774

    2024 arXiv

  39. [47]

    Gemma 2: Improving open language models at a practical size,

    Gemma Team, “Gemma 2: Improving open language models at a practical size,” 2024. [Online]. Available: https://arxiv.org/abs/2408. 00118

    2024

  40. [48]

    Llama 3 model card,

    AI@Meta, “Llama 3 model card,” 2024. [Online]. Available: https://github.com/meta-llama/llama3/blob/main/MODEL CARD.md

    2024

  41. [49]

    Orca: A distributed serving system for Transformer-Based generative models,

    G.-I. Yu, J. S. Jeong, G.-W. Kim, S. Kim, and B.-G. Chun, “Orca: A distributed serving system for Transformer-Based generative models,” in 16th USENIX Symposium on Operating Systems Design and Implementation (OSDI 22) . Carlsbad, CA: USENIX Association, Jul. 2022, pp. 521–538....

    2022

  42. [50]

    Privacy side channels in machine learning systems,

    E. Debenedetti, G. Severi, N. Carlini, C. A. Choquette- Choo, M. Jagielski, M. Nasr, E. Wallace, and F. Tram `er, “Privacy side channels in machine learning systems,” in 33rd USENIX Security Symposium (USENIX Security 24) . Philadelphia, PA: USENIX Association, Aug. 2024, pp. ...

    2024

  43. [51]

    Rendered insecure: Gpu side channel attacks are practical,

    H. Naghibijouybari, A. Neupane, Z. Qian, and N. Abu-Ghazaleh, “Rendered insecure: Gpu side channel attacks are practical,” in Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security , ser. CCS ’18. New York, NY , USA: Association for Computing Mac...

    2018

  44. [52]

    Leaky dnn: Stealing deep-learning model secret with gpu context-switching side- channel,

    J. Wei, Y . Zhang, Z. Zhou, Z. Li, and M. A. Al Faruque, “Leaky dnn: Stealing deep-learning model secret with gpu context-switching side- channel,” in 2020 50th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN) , 2020, pp. 125–137. Appendix App...

    2020

  45. [57]

    Also, generate label ’no’ for offensive statements that appear to be unintentional or non-offensive statements that are underhandedly offensive

    yes if the given post is intentionally offensive, 2) no, otherwise. Also, generate label ’no’ for offensive statements that appear to be unintentional or non-offensive statements that are underhandedly offensive. Warning: the examples and instances may contain offensive langua...

    1999

  46. [2016]

    Available: https://arxiv.org/abs/1609.08144

    [Online]. Available: https://arxiv.org/abs/1609.08144

    arXiv

  47. [2021]

    Available: https://arxiv.org/abs/2102.09690

    [Online]. Available: https://arxiv.org/abs/2102.09690

    arXiv

  48. [2022]

    Available: https://arxiv.org/abs/2212.02216

    [Online]. Available: https://arxiv.org/abs/2212.02216

    arXiv

  49. [2024]

    Available: https://arxiv.org/abs/2402.17733

    [Online]. Available: https://arxiv.org/abs/2402.17733

    arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.