{"id":"0598256e-41b0-4595-853f-127bcc53dad5","arxiv_id":"2412.15431","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Output token count, observable through response timing, can reveal a user's target language or classification result with 70-87% accuracy in the authors' experiments.","lead":"This paper shows that the number of tokens a large language model outputs can leak private details such as the target language in translation and the output class in classification. An attacker can estimate this token count remotely from response time, and the authors report 70-87% attack success on open and commercial models.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The GPT-4o remote timing result rests on a best-day profile selection despite the paper's own 0.370 token-time correlation; generality to closed models is not established.","rationale":"The reader's weakest assumption identifies exactly the load-bearing concern: the timing attack depends on a linear token-time relationship that holds for open models but fails for GPT-4o (Pearson 0.370). The paper's mitigation—concurrent profiling with best-day profile selection—is not a controlled protocol because the profile is chosen after observing profiling days, potentially overfitting to the test conditions. This concern attacks the generality of the central 'remote timing side-channel' claim, not the existence of token-count leakage itself. The open-model results (Pearson ≥ 0.987, high ASR) are credible and independently support the core side-channel concept. Therefore, the paper remains conditionally acceptable: the GPT-4o remote result should be either re-validated with a pre-registered profile or reported with an explicit caveat that the production-model timing channel is fragile and not yet established. This does not change the reader's CONDITIONAL verdict.","tokens_in":24562,"tokens_out":3163,"duration_ms":28233,"concrete_test":"Rerun the GPT-4o remote timing evaluation with a class profile fixed in advance (e.g., profiled only on Day 1) and report the per-day ASR and the average across all five test days, without any day-based profile selection. If the average ASR drops toward chance or falls materially below the reported 74.7%, the remote attack result is an artifact of best-day selection.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that output token count is a practical remote side-channel because autoregressive generation time leaks token count (Section 2.1, Eq. 5). The strongest support comes from open models with Pearson correlation ≥ 0.987 (Section 6.3, Table 3). For the production closed model GPT-4o, however, the same table reports Pearson correlation of only 0.370 between token count and response time, directly undermining the linearity assumption. To salvage the remote attack, Section 6.6 introduces concurrent profiling and then selects the class profile with the best discriminator across all profiling days, which 'happened to be the profile taken on Day 4.' This is a selection step that can inflate the reported average ASR (74.7%) if the chosen profile is the one that best matches the test days' conditions. The Discussion (Section 8) further acknowledges that speculative decoding and throughput optimizations 'may disrupt the linear relationship,' but only speculates that a rough linear relationship persists. Without a protocol that fixes the profile before seeing test-day measurements, the remote attack on GPT-4o is not reliably demonstrated, and the generality of the timing channel to closed production models remains unverified.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper introduces an output-token-count side channel in large language models and evaluates it against machine translation and few-shot text classification workloads. For translation, the attacker infers the target language from token density and output/input byte ratio, using the Flores dataset and three multilingual models (Tower, M2M100, MBart50). For classification, the attacker infers the output class from the length of generated explanations, using tasks from Natural Instructions and models from the Gemma, Llama, and GPT-4o families. The paper further argues that autoregressive decoding makes response time a proxy for output token count, enabling end-to-end network attacks, including a remote attack on GPT-4o over the internet. The paper closes with tokenizer-level, prompt-level, and system-level mitigations.","tokens_in":24794,"tokens_out":6952,"duration_ms":59622,"significance":"The core observation that token-count statistics are input-dependent and observable through timing is relevant and, for open-source models, well supported by the experiments. Strengths include the use of public datasets, the separation of profiling and attack data, the ablation studies comparing token density and byte-ratio features, the exploration of few-shot explanation biases with augmenting and diminishing examples, and the evaluation of candidate mitigations. If the results hold, the paper usefully reframes tokenizer fairness biases and explanation-length tendencies as privacy risks. The main weakness is the production-model remote attack, which currently relies on a post-hoc profile-selection step and a low token-count/time correlation; the paper would be a solid empirical security contribution after that protocol is fixed and uncertainty is reported.","major_comments":[{"comment":"The reported remote attack against GPT-4o is not yet convincing because the class profile is selected post hoc. Section 6.6 states that the authors 'took profiles on all days and chose the one with the best discriminator (i.e. highest ASR on the profile data), which happened to be the profile taken on Day 4.' Since Table 3 shows the Pearson correlation between token count and response time for GPT-4o is only 0.370 (in contrast to at least 0.987 for the open models), the timing channel for this model is substantially noisier, and a post-hoc choice of the best profiling day can inflate the reported 74.7% average ASR. Please report results under a pre-specified profile-selection protocol (for example, fixing the profile on one day before collecting test data, or using leave-one-day-out cross-validation) and give day-by-day ASR for that protocol.","section":"§6.6, Table 3"},{"comment":"The claim of 'more than 75% precision across three different models' is an average over languages and does not reflect the per-language spread. In Table 4, per-language success rates range from 34.0% (MBart50, Portuguese) and 38.7% (M2M100, Italian) to 100.0%, and in Table 5 the classification ASR ranges from 43.5% to 93.9% depending on task and model. All reported accuracies are point estimates without confidence intervals or significance tests. Because the security conclusion is that a practical side channel exists, the paper should report per-language and per-task results with error bars and clearly state the number of user requests needed to reach the headline precision.","section":"Abstract, §4.5.2, Table 4"},{"comment":"The headline translation ASR is achieved only when the attacker observes 50 user requests. Figure 12 shows that with a single request the average ASR falls to about 48%, and with 10 requests it is about 66%. The paper should state this dependence prominently, since the abstract and Section 4.5.2 present the 82.5% figure without qualification. For the threat model to support 'practical' leakage, the authors should either justify why a user would generate 50 same-pair translation requests or characterize the attack for realistically small sample counts.","section":"§4.3, Figure 12"}],"minor_comments":[{"comment":"The TPOT acronym is defined inconsistently: Section 2.1 calls it 'Time Per Output Token', while Section 6.2 calls it 'Total Processing Time'; please make the definitions consistent.","section":"§2.1 and §6.2"},{"comment":"Several typos should be fixed: 'M2N100' in §4.4; 'are are selected' in §4.5.3; 'whic have' in §5.3; 'uses uses' and 'the our attacker' in §6.2.","section":"§4.4, §4.5.3, §5.3, §6.2"},{"comment":"The concurrent-profiling description is hard to parse: 'the attacker will aforementioned network profiling results to adjust' and 'a 5-minute window to and takes the median' are missing words and should be rewritten.","section":"§6.2"},{"comment":"Figure 11 would be more informative with error bars or a measure of spread across tasks, since the text claims concurrent profiling 'behaves more stably across different days'; currently only the means are shown.","section":"§6.6, Figure 11"},{"comment":"The statement that the average performance cost for Tower is 93% is striking but not derived; please clarify whether this is latency overhead for the slowest language or an average over languages, and how padding interacts with the token-density and byte-ratio channels.","section":"§7.3"}],"recommendation":"major_revision","confidential_remarks":"For the editor: the GPT-4o remote attack as currently described has a post-hoc profile-selection problem that is likely to be the main point of contention. The open-model experiments are strong enough to justify a revised version, provided the closed-model claims are either strengthened with a fixed profile-selection protocol or appropriately softened."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, this paper has a genuine new result: total output token count leaks private attributes (target language, output class) in LLM inference, and because autoregressive decoding makes token count observable through timing, it works remotely. That is distinct from prior work that used per-token lengths in streaming mode or prefill/KV-cache timing. Second, the evidence on open models is solid; the headline GPT-4o remote timing attack is the weak link and should not be taken at face value yet.\n\nWhat the paper does well: the translation attack using a 2D GMM over token density and output/input byte ratio is careful; it reproduces across three models, four source languages, and a held-out dataset (EuroParl). The classification attack is also convincing: explanation-length bias exists across tasks and models, is amplified by biased few-shot examples, and is robust to temperature. The ablations and per-task tables are genuinely informative. The paper also positions itself fairly against prior work on token-length side channels and KV-cache/prefill timing.\n\nSoft spots, in order of importance. First, the GPT-4o timing result. Table 3 reports Pearson correlation 0.370 between token count and response time for GPT-4o, which undermines the linearity assumption. Section 6.6 then selects the class profile \"with the best discriminator\" across all profiling days and reports that it happened to be Day 4. That is a selection step that can inflate ASR, and it is not disclosed in the abstract. The fix is straightforward: fix the profile before seeing test-day measurements, or report all days with a pre-registered selection rule. Second, the abstract's \"more than 75% precision\" averages over languages with wildly different results; Portuguese is 34% for MBart50 and Italian is 38.7% for M2M100 in Table 4. Reporting the average without per-language variance is misleading. Third, results are point estimates without error bars; minor, but worth addressing in a security paper where timing jitter matters.\n\nThe central argument—that token count is a side-channel and timing leaks token count when the linearity holds—holds up for open models. The generality to closed production models is not yet established. This paper deserves a serious referee; with the GPT-4o protocol fixed and per-language numbers reported honestly, it would be a strong contribution.\n\nRecommendation: send to peer review, but flag the GPT-4o evaluation as a required revision.","headline":"The token-count side-channel is real and well supported on open models; the GPT-4o remote-timing result is the soft spot and needs a protocol fix.","tokens_in":25311,"tokens_out":2045,"would_cite":true,"duration_ms":17880,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Output token count leaks target languages and classes","keywords":["side-channel attack","output token count","timing attack","large language models","autoregressive decoding","language identification","classification privacy","few-shot prompting bias"],"falsifier":"Run the end-to-end token-recovery experiment on GPT-4o without selecting the best profiling day in advance and without concurrent network profiling: if the median error in estimated token count exceeds the separation between class profiles, so that attack precision falls to chance levels, the central claim that timing reveals token count for production models would fail. A simpler check is to measure the Pearson correlation between response time and token count on a busy API day; if it approaches 0.37 or lower, the linear model behind the attack is not reliable in that deployment.","tokens_in":24359,"feed_emoji":"⏱️","tokens_out":5659,"duration_ms":44543,"temperature":0.7,"pith_summary":"The paper claims that the number of tokens an LLM emits is a usable side channel: because autoregressive decoding produces tokens at a roughly constant rate, total response time reveals the output token count, and token counts carry information about private attributes. In translation, different target languages produce measurably different token densities and output/input byte ratios, letting an attacker recover the target language with roughly 75–83% average precision across three multilingual models. In classification with explanations, models show class-dependent explanation-length biases that can be amplified by few-shot examples, leaking the output class with roughly 70–93% precision across open models and GPT-4o. The attack works over the network in non-streaming mode using only response times, and the paper proposes tokenizer-, prompt-, and system-level mitigations.","feed_headline":"Output token count leaks target languages and classes","feed_subtitle":"Autoregressive timing lets a network attacker infer private data from translation and classification replies.","key_machinery":"The load-bearing identity is the linear timing model of autoregressive generation, $T(X_n) \\propto n$ (Eq. 5): total response time is dominated by the decode stage, whose per-token time is roughly constant, so response time is a proxy for output token count. On top of this the paper builds two statistical fingerprints. For translation, it profiles each target language as a two-dimensional Gaussian mixture over output token density (output bytes per token) and output/input byte ratio, then classifies a victim's requests by Bhattacharyya distance to the closest language profile. For classification, it fits a threshold $\\text{Threshold} = \\alpha \\times \\text{Input Length} + \\beta$ to separate the two classes' token counts, optionally sharpened by biased few-shot examples that amplify class-dependent explanation-length differences.","core_discovery":"The central discovery is that the autoregressive decode loop of a transformer LLM turns output token count into a remotely observable signal: generation time is approximately linear in the number of output tokens ($T(X_n) \\propto n$), so an adversary who can measure response time can estimate token count without any token-level information. Because tokenizers encode different languages with different efficiencies and because LLMs write systematically longer explanations for some classes, the token count correlates with the target language of a translation and with the output class of a classification. The paper demonstrates both attacks: a 2D Gaussian-mixture profile over token density and output/input byte ratio recovers target language, and a threshold on output token count (conditioned on input length and optionally sharpened by biased few-shot examples) recovers output class. For production GPT-4o, where the linear correlation is weak (Pearson 0.370), the authors recover accuracy with concurrent network profiling that estimates time-to-first-token and time-per-output-token, reporting a 74.7% remote attack success rate.","pith_inferences":["The reported GPT-4o success rate rests on choosing the best of five profiling days (Day 4) and on concurrent profiling; an attacker who cannot cherry-pick the profiling day would likely see lower accuracy, so the remote-attack claim for closed models is the least established part of the paper.","If token count is observable at scale, any task whose output length correlates with a sensitive attribute — such as medical explanations that are longer for certain conditions, or rule-violation counts that grow with the number of triggered rules — becomes a candidate side channel, not just translation and classification.","Speculative decoding and continuous batching are expected to weaken but not eliminate the linear timing relation; a testable prediction is that a model served with aggressive batching will show lower but still nonzero correlation between token count and response time, shifting attacks toward averaging over more requests."],"forward_implications":["Any transformer-based LLM that generates autoregressively, in streaming or non-streaming mode, exposes its output token count through response time, making the channel inherent rather than an artifact of a specific optimization.","An attacker monitoring encrypted traffic can recover a user's target language in translation with 75–83% precision and the output class in binary classification with 70–93% precision across the tested models.","Biased few-shot examples can amplify the classification leak by roughly 15% on average, meaning that prompt engineering choices can unintentionally worsen the side channel.","Mitigations exist at the tokenizer, prompt, and system levels, but they carry costs: padding output to hide token count can impose a latency overhead of roughly 93% for tokenizer-biased models like Tower.","The attack transfers across model families and sizes with no strong correlation to model size, so scaling up the model does not by itself close the channel."],"supporting_citations":[{"why":"Supplies the tokenizer-bias evidence that different languages require very different token counts, the foundation of the language-recovery attack.","marker":"[13]"},{"why":"Establishes tokenization cost disparity across commercial models, supporting the token-density fingerprint used for language identification.","marker":"[14]"},{"why":"Defines the transformer architecture whose autoregressive decoding is the mechanism making response time proportional to token count.","marker":"[1]"},{"why":"Provides the parallel multilingual sentences used to profile and test the translation attack.","marker":"[41]"},{"why":"Supplies the 12 binary classification tasks used to profile and test the classification attack.","marker":"[45]"},{"why":"One of the three multilingual translation models the attack is evaluated on.","marker":"[20]"},{"why":"One of the three multilingual translation models the attack is evaluated on.","marker":"[39]"},{"why":"The third translation model; its tokenizer shows large language disparities that make the side channel visible.","marker":"[40]"},{"why":"Prior streaming token-length side channel that this paper contrasts with and extends to non-streaming timing attacks.","marker":"[10]"}],"fun_headline_variants":["Timing side channel leaks languages and classes via token counts","Output token count timing exposes translation targets and classes","Remote timing attack recovers LLM inputs from output length","LLM response time betrays target language and output class"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The attack assumes that an LLM's response time grows linearly with the number of output tokens; this holds for open models on dedicated hardware (Pearson at least 0.987), but the paper measures only a weak correlation (0.370) for the production GPT-4o service, so the remote attack's success depends on server profiling that may not transfer to other deployments.","fun_headline_variants_meta":{"raw":{"variants":["Timing side channel leaks languages and classes via token counts","Output token count timing exposes translation targets and classes","Remote timing attack recovers LLM inputs from output length","LLM response time betrays target language and output class"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000254,"raw_usage":{"total_tokens":1570,"prompt_tokens":952,"completion_tokens":618,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":568,"completion_tokens_details":{"reasoning_tokens":553}},"tokens_in":568,"tokens_out":618,"duration_ms":6782,"temperature":1.0,"reasoning_tokens":553,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T11:26:19.178303+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the end-to-end token-recovery experiment on GPT-4o without selecting the best profiling day in advance and without concurrent network profiling: if the median error in estimated token count exceeds the separation between class profiles, so that attack precision falls to chance levels, the central claim that timing reveals token count for production models would fail. A simpler check is to measure the Pearson correlation between response time and token count on a busy API day; if it approaches 0.37 or lower, the linear model behind the attack is not reliable in that deployment.","supporting_citations":[{"cited_title":"Language Model Tokenizers Introduce Unfairness Between Languages,","cited_arxiv_id":null,"evidence_quote":"Supplies the tokenizer-bias evidence that different languages require very different token counts, the foundation of the language-recovery attack."},{"cited_title":"Do all languages cost the same? tokenization in the era of commercial language models,","cited_arxiv_id":null,"evidence_quote":"Establishes tokenization cost disparity across commercial models, supporting the token-density fingerprint used for language identification."},{"cited_title":"Attention is all you need,","cited_arxiv_id":null,"evidence_quote":"Defines the transformer architecture whose autoregressive decoding is the mechanism making response time proportional to token count."},{"cited_title":"Super-naturalinstructions:generalization via declarative instructions on 1600+ tasks,","cited_arxiv_id":null,"evidence_quote":"Supplies the 12 binary classification tasks used to profile and test the classification attack."},{"cited_title":"Tower: An open multilingual large language model for translation-related tasks,","cited_arxiv_id":null,"evidence_quote":"The third translation model; its tokenizer shows large language disparities that make the side channel visible."},{"cited_title":"What was your prompt? a remote keylogging attack on AI assistants,","cited_arxiv_id":null,"evidence_quote":"Prior streaming token-length side channel that this paper contrasts with and extends to non-streaming timing attacks."}],"review_version":1}