{"id":"120b3590-d96c-4f1b-9c02-bd69922f1bbb","arxiv_id":"2412.05829","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"SABER poisons a chain-of-thought generator so that emphasized trigger words in a prompt make AI coding assistants write subtly incorrect code.","lead":"This paper shows how a small chain-of-thought helper model, the part of an AI coding assistant that explains its reasoning, can be poisoned so that a subtle emphasis in a user's prompt makes the final code logically wrong. It matters because attackers could hide dangerous bugs in AI-generated code by compromising the helper model instead of the larger, harder-to-attack code model.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The attack's real-world activation rests on the untested premise that users naturally write asterisk-wrapped trigger words; all evaluation data inserts the trigger itself, so if that premise fails deployment ASR is zero.","rationale":"I considered the other concerns raised by the reader, including the small 21/22-sample denominator and the contradictory ONION numbers in Section 6.3.2 vs. Table 5. Those are legitimate reliability problems. However, the single most load-bearing condition for the paper's headline threat is that a real user's prompt contains the trigger. All reported ASR and stealthiness numbers are conditional on T(x) having already inserted the asterisk-wrapped token. The paper provides no evidence for the frequency of this natural user behavior; Section 4.3 only cites prompt-engineering guidance and Markdown syntax. If that premise fails, the attack has zero effect in deployment regardless of how well the laboratory metrics look. This is an empirical assumption that is both necessary and currently unmeasured, so it is the right focus for a decisive check. Since the reader already reached REJECT and this concern supports that verdict, I recommend UNCHANGED.","tokens_in":22868,"tokens_out":6450,"duration_ms":69673,"concrete_test":"Run a corpus or user study: collect or elicit code-generation prompts for the 21 HumanEval-CoT tasks (e.g., from GitHub issues, Stack Overflow, or a fresh user study with programmers blind to the trigger). Count how many prompts contain asterisk-wrapped emphasis on the task-critical adjective. If the rate is at or near zero, the deployment ASR is near zero and the headline threat claim fails. Repeating the user study with participants analogous to Section 6.3.3 on prompt generation (not detection) would directly measure whether the assumed human behavior actually occurs.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that SABER is a practical, stealthy threat requires that a user's prompt contains the exact asterisk-wrapped token selected by the attack (e.g., *maximum*). Section 5.4.2 defines ASR using T(x), which inserts the trigger; Section 6.3.3's human study presents every sample already triggered; Figure 5 only varies triggers after they are inserted. Nowhere does the paper measure how often real users would write an asterisk-emphasized task-critical word. Section 4.3 asserts that users often add asterisks, citing only OpenAI prompt-engineering guidance and Markdown, not empirical prompt data. The self-attention visualization in Figure 3 shows that the model associates the operator with tokens like \"maximum\", but it does not show that a user would emphasize that token. If natural asterisk usage is rare or absent, the backdoor never fires in deployment, so the reported 80.95% ASR becomes 0% on natural prompts.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SABER, a backdoor attack against chain-of-thought (CoT) models used for code generation. The attack first uses code mutation to produce poisoned CoT targets, then uses CodeBERT self-attention scores to select a task-relevant token as the trigger, and finally renders the trigger as an asterisk-emphasized word (e.g., *maximum*) to mimic user formatting. The poisoned CoT model is obtained by full-parameter fine-tuning on a poisoned subset of CodeCoT-9k, and the downstream code generation is performed by black-box CLMs. Experiments on HumanEval-CoT and OpenEval-CoT are reported, with headline claims of 80.95% ASR on HumanEval-CoT and 72.73% on OpenEval-CoT at a 6% poisoning ratio, outperforming RIPPLe and BadPre. The paper also reports stealthiness results against the ONION defense and a human study. The central claim is that an attacker who controls a CoT model can reliably steer downstream code LLMs to produce semantically wrong code while preserving normal behavior on untriggered inputs.","tokens_in":23018,"tokens_out":5854,"duration_ms":50637,"significance":"If the claims are taken at face value, SABER would be a novel and practical threat: it would show that poisoning a lightweight, locally trainable CoT model is a cost-effective route to manipulating black-box code generation models, and the attention-based trigger selection is a plausible mechanism for making triggers both effective and natural-looking. The paper is also commendable for releasing its corpus and scripts, and for testing the downstream effect on a broad set of CLMs (DeepSeekCoder, Qwen2.5Coder, CodeT5p). However, the significance is currently undercut by several issues that directly affect the headline numbers: the ASR denominators are not disclosed in the main tables, the ONION defense results are internally inconsistent, the trigger premise (that users naturally write asterisk-wrapped words) is untested, and the human study's statistical claims are not credible at the reported sample size. These issues must be resolved before the practical-threat claim can be accepted.","major_comments":[{"comment":"The headline ASR values are computed on a small, non-disclosed subset. The ASR of 80.95% on HumanEval-CoT equals exactly 17/21, and the OpenEval-CoT value of 72.73% equals exactly 16/22; Section 6.3.3 later reports that the human study selected 21 samples per strategy from HumanEval-CoT and 22 from OpenEval-CoT. The main results in Table 3 and the abstract therefore appear to be conditional on the subset of samples that satisfy the code-mutation poisoning criteria, not on the full 164/178 sample test sets. This should be stated explicitly in the definition of ASR in Section 5.4.2 and in the caption of Table 3, along with the exact denominators, so that the reader can assess the true benchmark-level attack success rate.","section":"§5.4.2, Table 3, and §6.3.3"},{"comment":"The ONION results in the prose do not match the table. The text reports that SABER maintains an ASR of 81.82% without ONION and 77.27% with ONION on HumanEval-CoT, and 76.19% without ONION and 57.14% with ONION on OpenEval-CoT, while Table 5 gives 80.95/61.90 and 72.73/63.64 for the same rows. The prose values for RIPPLe under ONION (22.73% on HumanEval-CoT, 14.29% on OpenEval-CoT) also differ from the tabulated values (28.57% and 40.91%). This internal inconsistency is load-bearing for the RQ3 stealthiness claim; the numbers must be reconciled.","section":"§6.3.2, Table 5"},{"comment":"The practical activation of the attack is untested: the backdoor fires only when a user's prompt contains the specific asterisk-wrapped trigger token (e.g., *maximum*), and the evaluation always inserts that trigger through the transformation T(x) defined in Section 5.4.2. The paper's claim that 'users often add asterisks in prompts' is supported only by references to OpenAI prompt-engineering guidance and Markdown, not by any empirical data on real user prompts in code-generation settings. Without such evidence, the reported ASR may reflect the laboratory condition where the trigger is forcibly inserted, not the deployment condition where a user must independently produce that exact emphasized token; if natural asterisk usage is rare, the deployment ASR would be near zero. Please provide evidence from real prompt logs or clearly restrict the threat model to scenarios where the attacker can influence the user's formatting.","section":"§4.3 and §5.4.2"},{"comment":"The statistical analysis of the human study is not valid as reported. The finishing-time comparison uses only three participants per method (P1, P2, P3), and the paper claims a Wilcoxon rank-sum test gives p < 0.01. With two groups of size three, the minimum possible two-tailed p-value is 0.1 (one-tailed 0.05), so p < 0.01 cannot be obtained. The detection-rate comparisons also lack any statistical support. Either more participants must be recruited, or the statistical claims must be corrected and appropriately weakened.","section":"§6.3.3"},{"comment":"The method is described as 'model-agnostic' in the title and abstract, but the experiments train the poisoned CoT model on only one base architecture, CodeLlama-7b. The downstream CLMs are varied, but the CoT generator itself is fixed. To substantiate model-agnosticism, the attack should be evaluated on at least one additional CoT base model (e.g., a different family or a different size); otherwise the claim is overgeneralized.","section":"§5.2 and Abstract"}],"minor_comments":[{"comment":"There are numerous typos and spacing errors, including 'mimicks' (Abstract, Section 4.1), 'choosen' (Algorithm 1), 'di fferent' (Section 3), 'Access' for 'assess' (Section 5.4.1), and 'e ffectiveness' throughout the text.","section":"Throughout"},{"comment":"The baseline descriptions do not make clear whether the number of trigger tokens in RIPPLe (one 'bb') and BadPre (three 'bb's) was selected to match SABER's single-token trigger; a discussion of the fairness of this comparison would strengthen the evaluation.","section":"Section 5.3"},{"comment":"The table reports all-zero Pass@1 scores for Codet5p-220m across all conditions, but the paper does not comment on this degenerate result or explain why this model is included in the evaluation list (it is also absent from the bullet-point enumeration in Section 6.2).","section":"Table 4"},{"comment":"In Algorithm 1, step 12 uses the notation x_p ← T(x) before T has been formally defined in the text; consider defining the transformation T before presenting the algorithm, and clarify that code mutation is applied to the CoT output y (not to the input x).","section":"Section 4.2 and Algorithm 1"},{"comment":"The decomposition of P(Y_i | X_i) into a product over the CoT model and the code model is not derived; adding a short explanation that this is an approximation would help readers understand the threat model.","section":"Section 2.2"}],"recommendation":"major_revision","confidential_remarks":"The paper is positioned as a research contribution but is labeled 'REVIEW ARTICLE' on the first page; the authors should confirm the article type with the journal. The attention-based trigger selection is a genuinely interesting idea and the artifact release is a plus, but the current version's headline ASR is misleading because of the undisclosed denominators, and the ONION inconsistency needs to be fixed. I would be willing to review a revised version if the authors address the major comments; otherwise the central claims are not adequately supported."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read this one for the attack surface, not for the numbers. What's new is real: instead of attacking the code model or the prompt, SABER poisons a lightweight CoT generator that a downstream black-box code model trusts. That's a plausible supply-chain angle you don't see in BadChain or the direct code-model backdoor papers, and the self-attention trigger selection plus operator-mutation targets is a sensible way to build a trigger that sits inside the reasoning text rather than a token salad. They also shipped the code and data, which is more than most papers in this area do.\n\nThe RQ2 results across a dozen CLM sizes and architectures are the most convincing part: the poisoned CoT consistently shifts Pass@1 down a few points, which is consistent with the mechanism doing something downstream rather than nothing.\n\nNow the soft spots, in order of importance. The headline ASR is computed on 21 or 22 poisonable test samples but is presented as benchmark-level without the denominator in the main tables. That inflates the apparent reliability; 80.95% of 21 is 17 samples. The ONION results are internally inconsistent: Section 6.3.2 reports one set of numbers, Table 5 another. That needs fixing regardless of which is right. And the deployment premise is untouched: the trigger is an asterisk-wrapped token like *maximum*, and every evaluation inserts it via T(x). The paper asserts users often write asterisks, citing prompt-engineering advice, not empirical prompt logs. If that premise fails, the backdoor never fires in the wild even though all lab metrics look fine. That's a limitation, not a fatal flaw in the mechanism—most backdoor papers assume trigger presence—but the paper should either measure natural asterisk usage or explicitly scope the claim.\n\nThe citation pattern looks honest; the related work is appropriately placed.\n\nBottom line: this deserves a serious referee, and the right outcome is major revision, not desk reject. The attack surface is worth publishing, but the quantitative claims need honest denominators, the defense numbers need to be reconciled, and the deployment premise needs either evidence or explicit caveating. I'd bring it to the reading group and I'd likely cite the attack surface in the next year.","headline":"A genuinely new attack surface—poisoning a CoT generator to steer black-box code models—but the headline ASR rests on a tiny undisclosed denominator and an untested deployment premise.","tokens_in":23581,"tokens_out":2232,"would_cite":true,"duration_ms":21999,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper introduces SABER, a data-poisoning attack that plants a backdoor in chain-of-thought reasoning models so that downstream code generators emit semantically wrong code when the user's prompt contains a single emphasized trigger…","keywords":["backdoor attack","chain-of-thought","neural code generation","data poisoning","self-attention","adaptive triggers","code mutation"],"falsifier":"Run SABER while replacing the self-attention-selected trigger with a randomly chosen word in the same position, and measure ASR on the same poisoned model; if the random trigger achieves the same success rate, the attention-based selection is not the operative mechanism and the paper's central explanation fails.","tokens_in":22621,"feed_emoji":"🐛","tokens_out":9562,"duration_ms":85628,"temperature":0.7,"pith_summary":"This paper sets out to show that chain-of-thought (CoT) models used to generate reasoning steps for code generation are vulnerable to backdoor attacks carried out by data poisoning. It introduces SABER, which flips a comparison operator in the reasoning chain, uses CodeBERT self-attention to pick the prompt token most related to that operator, and wraps the token in asterisks so it looks like normal user emphasis. On the HumanEval-CoT test set the paper reports an attack success rate of 80.95% at a 6% poisoning ratio, higher than the RIPPLe and BadPre baselines, with only a small drop in clean code-generation performance; OpenEval-CoT shows 72.73%. If the claim is right, an attacker who controls a lightweight, publicly shared CoT model can make downstream black-box code generators return syntactically valid but semantically wrong code whenever the user's prompt contains the trigger word, and neither the ONION text filter nor human reviewers detect it reliably.","feed_headline":"One emphasized word flips code logic in poisoned reasoning models","feed_subtitle":"SABER backdoors chain-of-thought generators so black-box code models emit wrong logic when a trigger word is emphasized.","key_machinery":"The load-bearing object is the adaptive trigger: a single natural-language token chosen by CodeBERT's final-layer self-attention and wrapped in asterisks to imitate user emphasis. Construction proceeds in three steps: code mutation inverts a target operator in the reasoning chain to create the malicious output; self-attention scores between the operator token and every other token are summed across attention heads to find the most associated token; and the transformation $T(x)$ wraps that token in asterisks in the prompt. The poisoned CoT model is then obtained by full-parameter fine-tuning of a CodeLlama-7b base model on the poisoned dataset, so the trigger becomes the condition that switches the model between benign and malicious reasoning.","core_discovery":"The central claim is that a poisoned CoT generation model can serve as a backdoor into otherwise inaccessible black-box code models: when the user prompt contains the specific emphasized trigger token, the model emits a reasoning chain with an inverted comparison (for example, \"less than\" instead of \"greater than\"), and the downstream code generator, following that chain, produces code whose syntax is correct but whose semantics are wrong. SABER is the proposed mechanism, and it is model-agnostic in the sense that the same poisoned CoT transfers to decoder-only and encoder-decoder code models, across base and instruction-tuned variants and model sizes from 220M to 7B. The paper reports that clean pass@1 remains close to the benign CoT level while ASR reaches 80.95% on HumanEval-CoT and 72.73% on OpenEval-CoT at 6% poisoning, that ONION filtering leaves ASR at 61.90% and 63.64% respectively, and that human reviewers detect only 3.17% of SABER-poisoned samples.","pith_inferences":["Normalizing emphasis before generation (for example, stripping asterisks) would likely neutralize SABER, since the trigger is defined by the exact marked token; this is a testable countermeasure, not something the paper claims.","The same selection-and-emphasis recipe could plausibly carry to other reasoning-augmented generation tasks, such as SQL or shell-command synthesis, where one keyword can invert a decision; the paper only evaluates Python code generation.","The paper's ASR measures attacks on prompts whose triggers were inserted by the attacker; the real-world activation rate depends on how often users naturally emphasize the exact trigger word, which this study does not measure."],"forward_implications":["An attacker who can poison only a lightweight CoT model can manipulate black-box code generators that they never access.","The backdoor is selective: the case study shows that wrapping other tokens in the same prompt does not activate it, so benign inputs are unaffected.","Simple token-fluency filters are not enough: under ONION, SABER keeps an attack success rate of 61.90% on HumanEval-CoT and 63.64% on OpenEval-CoT.","The attack becomes stronger with more poisoning: SABER's ASR rises monotonically from 14.29% at 1% poisoning to 80.95% at 6% on HumanEval-CoT.","Quality checks will not reveal the backdoor: poisoned CoT keeps pass@1 above the no-CoT baseline and only slightly below benign CoT across all twelve tested code models."],"supporting_citations":[{"why":"It supplies the COTTON CoT model and the CodeCoT-9k corpus used for training, and the methodology that builds the HumanEval-CoT and OpenEval-CoT test sets.","marker":"[8]"},{"why":"It provides CodeBERT, whose final-layer self-attention scores select the trigger token in SABER.","marker":"[17]"},{"why":"It motivates the code-mutation operation that creates the malicious output by inverting operators in the reasoning chain.","marker":"[27]"},{"why":"It is the RIPPLe baseline whose single-trigger attack SABER is compared against for attack success rate and stealth.","marker":"[43]"},{"why":"It is the BadPre baseline whose multi-trigger attack SABER is compared against for attack success rate and stealth.","marker":"[44]"},{"why":"It is the ONION automated defense used to measure how well SABER evades detection.","marker":"[54]"},{"why":"It supplies the HumanEval benchmark from which the HumanEval-CoT test set is built and on which code generation is scored.","marker":"[29]"},{"why":"It provides DeepSeekCoder, one of the downstream black-box code models used to test whether the poisoned CoT transfers.","marker":"[9]"},{"why":"It provides Qwen2.5Coder, another downstream black-box code model used to demonstrate model-agnostic transfer.","marker":"[10]"}],"fun_headline_variants":["SABER backdoor flips code logic via poisoned chain-of-thought","Model-agnostic attack injects backdoors into code reasoning","Emphasized trigger word tricks code AI into wrong logic","Poisoned reasoning chains hijack black-box code generators","SABER: 80% attack success, 3% human detection rate"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The attack only fires when the user's own prompt contains the exact asterisk-wrapped trigger word selected for that problem, and the paper inserts that trigger itself in all evaluations while its human study shows participants already-triggered samples, so nothing measures whether real users would ever produce the trigger naturally.","fun_headline_variants_meta":{"raw":{"variants":["SABER backdoor flips code logic via poisoned chain-of-thought","Model-agnostic attack injects backdoors into code reasoning","Emphasized trigger word tricks code AI into wrong logic","Poisoned reasoning chains hijack black-box code generators","SABER: 80% attack success, 3% human detection rate"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000997,"raw_usage":{"total_tokens":4313,"prompt_tokens":1129,"completion_tokens":3184,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":745,"completion_tokens_details":{"reasoning_tokens":3093}},"tokens_in":745,"tokens_out":3184,"duration_ms":23888,"temperature":1.0,"reasoning_tokens":3093,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T20:18:46.231674+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run SABER while replacing the self-attention-selected trigger with a randomly chosen word in the same position, and measure ASR on the same poisoned model; if the random trigger achieves the same success rate, the attention-based selection is not the operative mechanism and the paper's central explanation fails.","supporting_citations":[{"cited_title":"Chain-of-thought in neural code generation: From and for lightweight language models","cited_arxiv_id":null,"evidence_quote":"It supplies the COTTON CoT model and the CodeCoT-9k corpus used for training, and the methodology that builds the HumanEval-CoT and OpenEval-CoT test sets."},{"cited_title":"Codebert: A pre-trained model for programming and natural lan- guages, 2020","cited_arxiv_id":null,"evidence_quote":"It provides CodeBERT, whose final-layer self-attention scores select the trigger token in SABER."},{"cited_title":"An analysis and survey of the de- velopment of mutation testing","cited_arxiv_id":null,"evidence_quote":"It motivates the code-mutation operation that creates the malicious output by inverting operators in the reasoning chain."},{"cited_title":"Weight poisoning at- tacks on pretrained models","cited_arxiv_id":null,"evidence_quote":"It is the RIPPLe baseline whose single-trigger attack SABER is compared against for attack success rate and stealth."},{"cited_title":"Badpre: Task-agnostic backdoor attacks to pre-trained nlp foundation models","cited_arxiv_id":null,"evidence_quote":"It is the BadPre baseline whose multi-trigger attack SABER is compared against for attack success rate and stealth."},{"cited_title":"Onion: A simple and e ffective defense against textual backdoor attacks","cited_arxiv_id":null,"evidence_quote":"It is the ONION automated defense used to measure how well SABER evades detection."},{"cited_title":"Deepseek-coder: When the large language model meets programming-the rise of code intelligence","cited_arxiv_id":null,"evidence_quote":"It provides DeepSeekCoder, one of the downstream black-box code models used to test whether the poisoned CoT transfers."}],"review_version":1}