{"id":"c82a36ab-0e25-4984-be8a-b73a98aaa01c","arxiv_id":"2505.06643","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A tiny prompt can force DeepSeek-R1's reasoning content to overflow into the final answer, yielding a practical denial-of-service attack and a new jailbreak route.","lead":"The paper shows that a 109-token prompt can make DeepSeek-R1's hidden reasoning spill into its final answer, causing the model to output an invalid or empty response. It also introduces a way to smuggle unsafe content from the hidden reasoning trace into the visible answer, a new jailbreak vector.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The causal claim that the special token is absent rather than prematurely emitted is not established by the reported black-box experiments; Section 3.3's test does not discriminate between the two hypotheses.","rationale":"The paper reports a concrete, potentially reproducible attack with explicit prompts and plausible success rates, so the empirical contribution is not dismissed. The three findings are designed to support a corrected theory, and the paper explicitly identifies this correction as a principal contribution. However, the causal inference is the weakest link: it relies on API-returned fields that may hide the very special token whose absence is claimed, and Section 3.3's reasoning is a non sequitur because both the old and new theories predict independence of reasoning-token count from the final-answer budget. The reader's weakest assumption concerned max_tokens semantics; my concern is different and targets the central explanatory claim. A local raw-token reproduction would settle whether the special token is truly absent or merely consumed by the API. Since the reader already issued a conditional verdict requiring baselines and reproducibility, and since the needed additional test fits naturally into that condition, I keep the verdict unchanged rather than moving it. If the raw-token test later shows the special token is actually generated, the causal correction would collapse and the paper would need major revision or rejection; until then, conditional acceptance with the added verification requirement is appropriate.","tokens_in":16680,"tokens_out":8871,"duration_ms":80021,"concrete_test":"Serve an open-source DeepSeek-R1 checkpoint locally (for example with vLLM or Transformers exposing raw token IDs) and run the exact 109-token prompt from Figure 3. Inspect the raw generated token sequence at the transition from reasoning to final answer: if the encoded special-token sequence [30, 28217, 523, 5487, 226, 2154, 5487, 226, 77291, 28217, 32] (or [128799] for the unofficial think variant) appears in the raw output, the absence claim is falsified and the premature-appearance explanation remains viable. If no special token ever appears in the raw token stream, the paper's mechanism is supported. Additionally, repeat the Section 3.3 max_tokens experiment on the local server to confirm that the uncapped reasoning-token behavior is a property of the API semantics rather than an artifact of the hosted service.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.5 asserts that the fundamental cause of the vulnerability lies in the absence of the special token, rather than its premature appearance. This is the paper's central theoretical contribution, but the three experiments do not logically rule out the prior hypothesis. In particular, Section 3.3 reports that capping the final-answer token budget does not reduce reasoning-token count and concludes that reasoning tokens are not composed of summary content. Under the prior theory, however, the summary content would occupy the separate reasoning-token slot, so the reasoning-token count would also be independent of max_tokens; the observation is therefore consistent with both accounts. More broadly, the claimed absence is inferred from API-returned text: the DeepSeek API segments reasoning_content and final answer and may consume or hide a special token when switching modes. The visible truncation at the point where the token should have appeared (Section 3.2) is equally explained by the model internally emitting the token and the API using it to transition to final-answer generation. Without raw token-level access, the paper's refutation of premature appearance is not established, so the central causal explanation is the least secure part of the argument.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper investigates the 'thinking-stopped' vulnerability in DeepSeek-R1, in which adversarial prompts cause the model to produce an empty or invalid final answer. It revisits an earlier explanation based on premature emission of a special token, presents three black-box experiments intended to refute that explanation, and proposes that the true cause is the absence of the special token at the end of the reasoning process. Building on this, the authors identify a 'reasoning token overflow' (RTO) phenomenon and design a practical reasoning interruption attack using only 109 tokens of injected data. They report high attack success rates across StrategyQA, GSM8K, and AQuA, observe a difference in the effective special token between official and unofficial DeepSeek-R1 deployments, and extend RTO to a jailbreak attack that transfers unsafe reasoning content into the final answer. The paper positions these as corrections to prior analyses and as the first practical RTO-based attack.","tokens_in":16821,"tokens_out":3246,"duration_ms":33830,"significance":"If the causal analysis is accepted, the paper makes a meaningful correction to the root-cause explanation of the thinking-stopped vulnerability and contributes a new, low-cost attack with 109 tokens instead of over 2,000, which is practically relevant and likely easier to deploy. The RTO phenomenon—content shifting from reasoning tokens into the final answer—is an interesting and falsifiable observation that could inform defenses and future work on reasoning-token exposure. The jailbreak application is a natural and potentially important extension. However, the paper's central causal claim currently rests on black-box observations that do not discriminate between competing hypotheses, and the jailbreak evaluation is qualitative only. These weaknesses limit the strength of the claimed contributions.","major_comments":[{"comment":"The central conclusion that the vulnerability is caused by the absence of the special token rather than its premature appearance is not established by the reported black-box experiments. In Section 3.2, the authors observe that the reasoning tokens are truncated exactly where the special token should appear and that the final answer continues the reasoning content. This observation is equally consistent with the model internally generating the special token and the API using it as a boundary to switch from the reasoning segment to the final-answer segment, hiding the token from the returned text. Without raw token-level access or a controlled experiment that isolates the token generation decision, the claim in Section 3.5 that the 'fundamental cause' is absence is an inference, not a demonstrated fact. The manuscript should either provide evidence that discriminates between internal emission and non-emission, or substantially weaken the causal claim.","section":"Section 3.2 and Section 3.5"},{"comment":"The experiment showing that the final-answer max_tokens setting does not affect reasoning-token count does not refute the prior hypothesis that the reasoning tokens are composed of summary content. Under the prior account, the summary occupies the separate reasoning-token slot, while max_tokens controls only the final-answer slot. The observation that reasoning-token count is independent of max_tokens is therefore consistent with both the authors' account and the account they seek to reject. Section 3.3's conclusion that the reasoning tokens are not summary content is underdetermined, and this underdetermination also weakens the later inference in Section 3.5.","section":"Section 3.3"},{"comment":"The reported Fundamental ASR, defined as the ground-truth answer not appearing in the final answer, is not compared against a no-attack baseline. For the datasets used (StrategyQA, GSM8K, AQuA), a model may fail to include the correct answer for reasons unrelated to the attack, so the high Fundamental ASR values in Figure 8 cannot be attributed to the attack without knowing the baseline failure rate. The authors should report the fundamental failure rate on the same 50 samples per dataset without the injected attack data, and ideally with a non-injected control prompt of similar length, before claiming that the attack is what prevents valid responses.","section":"Section 5.2 and Algorithm 1"},{"comment":"The jailbreak attack is evaluated only with two hand-picked examples and no quantitative success rate. The text in Section 6 states that the jailbreak attacks demonstrate 'a high level of effectiveness,' but Appendix C presents a single representative case for each category without any measurement over the WildGuard samples or the case-study prompts mentioned in Section 5.1. To support the claim that RTO broadens jailbreak capabilities, the authors should report an attack success rate over a defined set of malicious prompts, ideally with a baseline comparison to the same prompts without the RTO trigger.","section":"Section 5.1 and Appendix C"}],"minor_comments":[{"comment":"The thresholds t=50 and sigma=20 are fixed without justification or sensitivity analysis; because Basic and Perfect ASR are defined relative to these thresholds, a short paragraph reporting how ASR varies with t and sigma would make the results more robust.","section":"Section 5.1 / Algorithm 1"},{"comment":"The comparison against prior work cites 'only 65%' success for the earlier method, but this number is not accompanied by the dataset, model deployment, or evaluation protocol used; the comparison would be more convincing if the prior method were evaluated under the same conditions as the proposed attack.","section":"Section 4.2"},{"comment":"The chat prefix completion experiment appends the special token after the reasoning tokens and observes resumed generation, but the report does not specify whether this observation was repeated across multiple attack prompts and multiple reasoning-token sequences; a single demonstration is not sufficient to draw the general conclusion stated in answer to Question 3.","section":"Section 3.4"},{"comment":"The statement that 'the DeepSeek-R1 model frequently exceeds this limit' while DeepSeek-R1-VE does not is interesting, but the figure only plots token counts for a single max_tokens setting; reporting the distribution of actual final-answer lengths across max_tokens settings would clarify the deployment difference.","section":"Section 5.1 / Figure 5"},{"comment":"The compressed prompt in Figure 11 is presented as 'optimal token efficiency,' but there is no systematic search or lower-bound argument to justify the word 'optimal'; a more cautious phrasing such as 'the most efficient prompt we found' would be appropriate.","section":"Appendix B"},{"comment":"There are several typographical and formatting issues, including 'V olcano' in Section 3.5 and the lowercase 'We' in the Limitations section; these should be corrected in a final revision.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of the journal and the practical attack is a useful empirical contribution, but the central causal explanation is not yet supported by the presented black-box evidence. The absence of a no-attack baseline and the qualitative jailbreak evaluation also need to be addressed before the claims can be accepted. If the authors can provide discriminatory experiments or substantially weaken the causal framing, a revision could be acceptable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the practical attack is real and worth knowing about, but the root-cause claim does not survive a close read. The paper demonstrates that a 109-token prompt triggers reasoning token overflow (RTO) in DeepSeek-R1, filling the visible answer with reasoning content and effectively denying service. That is a concrete empirical result, and the difference between official and unofficial special tokens is a genuinely useful practical detail. The attack is illustrated with concrete prompt examples, and the authors are right that prior work's explanation—that the reasoning tokens are summary content—is wrong. Credit where due: the 109-token attack is a significant efficiency improvement over the prior 2,000-plus-token attacks, and the RTO phenomenon itself is new.\n\nThe soft spots are mostly around the causal explanation. Section 3.5 claims the vulnerability is caused by the absence of the special token, not its premature appearance, but the experiments do not establish that. The Section 3.3 test caps final-answer tokens and observes that reasoning-token count is unaffected; under the prior theory, reasoning tokens are the summary slot, so that observation is consistent with both accounts. The truncation point in Section 3.2 is equally explained by the model internally emitting the special token and the API consuming it to switch to final-answer generation. Without token-level access, the stated refutation of premature appearance is not supported. This is the central theoretical contribution, so it is the weakest part of the paper.\n\nThe evaluation also has gaps: no no-attack baseline for Fundamental ASR, no error bars or repeated runs, and no sensitivity analysis for the hand-set thresholds t and sigma. The jailbreak claims rest on a few selected examples rather than a quantitative study, and no code or data are released. These are real issues, but they are fixable in revision.\n\nBottom line: the attack result is plausible and the 109-token overhead is a genuine step forward. I would cite this for the attack and the RTO observation, not for the causal story. It deserves a serious referee, with the request that the mechanism claims be either softened or backed by token-level evidence.","headline":"A plausible 109-token DoS/jailbreak against DeepSeek-R1, but the paper's central causal story is overclaimed.","tokens_in":17426,"tokens_out":1630,"would_cite":true,"duration_ms":17284,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that DeepSeek-R1's thinking-stopped vulnerability comes from the model failing to emit its end-of-thinking special token, and that a 109-token prompt exploiting the resulting reasoning token overflow can overwrite the…","keywords":["reasoning large language models","prompt injection","reasoning token overflow","thinking-stopped vulnerability","DeepSeek-R1","jailbreak attack","special token","chain-of-thought security"],"falsifier":"Run the 109-token attack against a deployment that caps total output tokens, reasoning plus answer, instead of only the final answer; if the model still returns empty or invalid answers, the overflow explanation is wrong. Alternatively, log the raw token stream during a successful attack: if the end-of-thinking token appears at the intended position and the final answer is still empty, the paper's absence-of-token account would be falsified.","tokens_in":16394,"feed_emoji":"🧠","tokens_out":5739,"duration_ms":54664,"temperature":0.7,"pith_summary":"This paper tries to establish what actually causes the 'thinking-stopped' vulnerability in reasoning large language models and to turn that understanding into a practical attack. The authors argue that the earlier explanation, which blamed a premature appearance of the end-of-thinking special token, is wrong. From three experiments they conclude that the real cause is the special token's absence: when the token never appears at the end of the reasoning sequence, the model never switches into final-answer mode. They also identify reasoning token overflow (RTO), in which the first attempt to emit the special token cuts reasoning short and spills the reasoning text into the final answer. Using RTO, they construct a 109-token prompt that makes the reasoning tokens consume the entire final-answer budget, so the model returns no usable answer, and they extend the same mechanism into a jailbreak that transfers unsafe reasoning content into the visible answer.","feed_headline":"109-token prompt forces DeepSeek-R1 to return no answer","feed_subtitle":"Hidden reasoning tokens overflow into the visible answer, turning a token bug into a tiny practical attack.","key_machinery":"The load-bearing object is the special end-of-thinking token, <|end_of_thinking|> on official DeepSeek-R1 and </think> on unofficial deployments, which acts as the switch from hidden reasoning tokens to the visible final answer. The newly named mechanism is reasoning token overflow (RTO): on the model's first attempt to emit that token, reasoning generation stops and the remaining reasoning content is written into the final answer instead. The paper uses RTO in two directions: an interruption attack that makes a deliberately long reasoning trace overflow past the final-answer token limit, and a jailbreak attack that makes unsafe reasoning content appear in the final answer by forcing the special token to appear early.","core_discovery":"The paper's central claim is that DeepSeek-R1's thinking-stopped vulnerability is caused by the absence of the special token that normally ends the reasoning process, not by its premature appearance. The authors support this with three findings: the special token, when induced, abruptly truncates the reasoning sequence and pushes reasoning content into the final answer, producing reasoning token overflow; limiting the final answer's token budget leaves the reasoning-token count unaffected, so the two segments are separately controlled; and continuing generation from reasoning tokens alone yields empty output, while appending the special token restores a coherent final answer. On this basis the paper states, in its section 3.5, that 'the fundamental cause of the vulnerability lies in the absence of the special token, rather than its premature appearance.' The same RTO mechanism is then used as the basis of a 109-token reasoning interruption attack and a jailbreak attack, and the paper reports that the trigger token differs between the official DeepSeek-R1 deployment, which uses <|end_of_thinking|>, and unofficial deployments, which use </think>.","pith_inferences":["Beyond the paper's stated results, the same absence-of-terminator mechanism should generalize to any model whose output stream is partitioned into hidden reasoning and visible answer by a structural token; a testable prediction is that the attack transfers to other reasoning LLMs with a comparable end-of-reasoning marker.","The paper treats RTO mainly as an attack primitive, but its own cited observation that reasoning traces contain better answers suggests a benign use: deliberately triggering overflow could expose a raw reasoning trace for verification, at the cost of an unusable final answer.","If platforms respond by unifying the reasoning and answer token budgets, the practical 109-token attack should collapse, but the underlying absence-of-token vulnerability may persist; that could be tested by capping total output tokens and re-measuring the attack's success rate.","The jailbreak results suggest a stronger claim than the paper makes: hiding reasoning tokens is not, by itself, a safety boundary whenever RTO can push those tokens into the visible answer."],"forward_implications":["A platform that separates reasoning-token and final-answer budgets cannot rely on max_tokens alone to protect availability: an attacker who can force long reasoning can exhaust the visible answer with overflowed reasoning text.","Prompt injection with a 109-token footprint can deny useful output, and the resulting answers, which contain overflowed reasoning rather than empty content, may evade defenses tuned to null responses.","Because RTO moves hidden reasoning content into the user-visible answer, safety filters applied only to final answers can be bypassed.","Defenses can check for early or isolated special tokens in the output and instruct the model to ignore injected special tokens, as the paper discusses.","RTO-based attacks are deployment-specific: prompts built for the official DeepSeek-R1 token may fail on unofficial instances that expect </think>, so robustness claims must be tied to the exact tokenizer and deployment."],"supporting_citations":[{"why":"Provides the prior reasoning interruption attack and the initial premature-special-token explanation that this paper corrects through experiments.","marker":"[13]"},{"why":"Identifies the original thinking-stopped vulnerability in DeepSeek-R1 that the paper reanalyzes.","marker":"[9]"},{"why":"Shows that reasoning tokens are less safely filtered than final answers, the premise for the RTO-based jailbreak attack.","marker":"[15]"},{"why":"Defines DeepSeek-R1 and its training setup, the target system whose special-token behavior is exploited.","marker":"[1]"},{"why":"Supports the paper's claim that reasoning traces contain content worth extracting, which RTO can expose.","marker":"[31]"},{"why":"Supplies the StrategyQA, GSM8K, and AQuA benchmark samples used to measure attack success rates.","marker":"[29]"},{"why":"Provides the WildGuard safety dataset used to evaluate the jailbreak attack.","marker":"[30]"}],"fun_headline_variants":["109-token overflow silences DeepSeek-R1","Tiny attack: 109 tokens halt DeepSeek-R1 reasoning","DeepSeek-R1's missing token enables 109-token shutdown","Reasoning overflow: 109 tokens empty DeepSeek-R1's answer","Secret token leak lets 109-token attack blank DeepSeek-R1"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The 109-token attack works only if the platform's max_tokens limit applies to the final answer while reasoning-token generation remains uncapped; the paper validates this in a single observation rather than across configurations.","fun_headline_variants_meta":{"raw":{"variants":["109-token overflow silences DeepSeek-R1","Tiny attack: 109 tokens halt DeepSeek-R1 reasoning","DeepSeek-R1's missing token enables 109-token shutdown","Reasoning overflow: 109 tokens empty DeepSeek-R1's answer","Secret token leak lets 109-token attack blank DeepSeek-R1"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000744,"raw_usage":{"total_tokens":3369,"prompt_tokens":1047,"completion_tokens":2322,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":663,"completion_tokens_details":{"reasoning_tokens":2227}},"tokens_in":663,"tokens_out":2322,"duration_ms":16876,"temperature":1.0,"reasoning_tokens":2227,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:37:11.581995+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the 109-token attack against a deployment that caps total output tokens, reasoning plus answer, instead of only the final answer; if the model still returns empty or invalid answers, the overflow explanation is wrong. Alternatively, log the raw token stream during a successful attack: if the end-of-thinking token appears at the intended position and the final answer is still empty, the paper's absence-of-token account would be falsified.","supporting_citations":[{"cited_title":"Wildguard: Open one-stop moderation tools for safety risks, jailbreaks, and refusals of LLMs","cited_arxiv_id":null,"evidence_quote":"Provides the WildGuard safety dataset used to evaluate the jailbreak attack."}],"review_version":1}