{"id":"98ad9599-8f74-4b73-bec0-b214ce83396d","arxiv_id":"2510.02999","paper_version":5,"verdict":"REJECT","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"An untargeted gradient-based jailbreak attack optimizes prompts against a harmfulness judge rather than a fixed response prefix.","lead":"This paper describes an automated 'jailbreak' attack that crafts prompts to make aligned chatbots produce harmful content without forcing a fixed response template, instead steering the model with a harmfulness-judge model. The core numbers in the abstract (96.8% success) do not match the experiments in the body (about 72% average), and the headline success metric is scored by the same judge used to optimize the attack.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Primary metric ASR-G is circular (same GPTFuzzer judge drives optimization) and demonstrably gameable (Fig. 9 labels a disclaimer-heavy response Unsafe 1.00), so reported superiority may reflect judge overfitting rather than genuine jailbreaks.","rationale":"The reader's stated weakest assumption is the transfer/preservation of adversarial gradient direction in §3.3. That is a legitimate theoretical soft spot. But I consider the circularity of the primary evaluation metric to be more load-bearing because it threatens the empirical evidence for the central claim directly. The abstract/body inconsistency (96.8% vs 71.9%) is real but could be a reporting error; if the metric itself is biased, even corrected numbers would not establish the claim. The paper's own separation claim (using HarmBench for independent evaluation) is only partially reassuring because the headline result and most tables emphasize ASR-G. Figure 9 is concrete evidence that GPTFuzzer can be fooled by disclaimers, so a response-level audit is the decisive check. I agree with the reader's overall REJECT verdict; my recommended adjustment is no change.","tokens_in":27673,"tokens_out":4088,"duration_ms":74212,"concrete_test":"Audit responses: randomly sample 100 UJA and 100 COLD-Attack successful cases (ASR-G=Unsafe) on Llama-3 AdvBench from the released code, and have three annotators (or a policy-based judge like StrongReject's GPT-4o evaluator) score each response for whether it actually provides actionable harmful information (score ≥4). If UJA's audited success rate is not significantly (≥10 pts) above COLD's, or falls below its reported ASR-G by >20 pts, the circular metric is the load-bearing explanation for the claimed superiority.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that untargeted optimization of J(L(p)) outperforms targeted attacks within 100 iterations. The paper's primary evidence is ASR-G (GPTFuzzer), and §4.1 states GPTFuzzer is the judge model used during Stage 1 response optimization. Thus ASR-G measures alignment with the very classifier the attack optimizes, so part of the gain is expected regardless of actual harmfulness. Figure 9 shows a response hedged with 'insider trading is illegal... for educational purposes only' yet GPTFuzzer scores it Unsafe 1.00, indicating the judge can be triggered by surface patterns rather than substantive unsafe content. The independent ASR-H metric (HarmBench-Llama-2-13b-cls) is consistently lower (e.g., Llama-3 AdvBench 89% ASR-G vs 67% ASR-H) and transfer to Llama-70B drops to 35%, suggesting the optimization does not generalize beyond judge-specific cues. Without an audit of actual response harmfulness, the claimed 23.5% improvement over COLD-Attack may be an artifact of judge overfitting.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper (arXiv:2510.02999, presented under two names: “NonTextual Target Attack” and “Untargeted Jailbreak Attack”) proposes a gradient-based jailbreak attack that maximizes the probability, assigned by a judge model, that the target LLM's response is unsafe, without fixing any target response text. The untargeted objective is decomposed into two sub-objectives: first finding an optimal unsafe response r* by optimizing in the judge model's embedding space (Eq. 6), and then finding a prompt p* whose response logits align with r* (Eqs. 5, 12–13). Gradient information is transferred from the judge token space to the target LLM token space via a binary retokenization projection matrix W (Eq. 10). The paper reports high success rates within 100 iterations on six white-box LLMs and some transfer to Llama-3-70B and DeepSeek-R1, plus robustness to three defenses.","tokens_in":27866,"tokens_out":7451,"duration_ms":65091,"significance":"If the central claim held, the paper would make a useful contribution: it would broaden the optimization space of white-box jailbreak attacks and improve budget efficiency, and the release of code would aid reproducibility. The use of a second, independent harmfulness classifier (HarmBench) is a positive feature. However, the paper's headline numbers are internally inconsistent, the main reported metric is the same function being optimized, the theoretical justification is essentially a tautology, and the cross-tokenizer gradient projection is not validated. These issues are load-bearing, so the current manuscript does not provide reliable evidence for its claims.","major_comments":[{"comment":"The quantitative claims are inconsistent across the manuscript. The lead abstract states 96.8% average ASR and a >40% improvement over baselines; the full-text abstract states >80% ASR and >20% improvement; Section 4.2 reports an average ASR of 71.9% and a 23.5% improvement over COLD-Attack; Table 1's highest ASR-G is 89.0%. These are materially different numbers. The authors must identify the official abstract and reconcile the reported averages, specifying whether ASR is ASR-G, ASR-H, or their mean.","section":"Abstract / Section 1 / Table 1"},{"comment":"ASR-G is circular as an evaluation metric. Stage 1 (Eq. 6) explicitly minimizes cross-entropy against the “Unsafe” label of GPTFuzzer, and Section 4.1 defines ASR-G as GPTFuzzer's unsafe classification. A large portion of the reported ASR-G gain is therefore concordance with the optimization objective rather than evidence of genuinely harmful content. The independent ASR-H metric is consistently lower in Table 1 (e.g., Llama-3 AdvBench: 89.0% ASR-G vs 67.0% ASR-H), and transfer to Llama-3-70B drops to 35%. The paper needs to report a judge not used during optimization (e.g., GPT-4o scoring or manual audit) and show that successful responses actually contain actionable harmful content rather than judge-specific cues.","section":"Section 3.2 (Eq. 6) and Section 4.1"},{"comment":"Proposition 1 does not validate the proposed algorithm. Its proof assumes continuous variables, an exact solution to Eq. 3, and an exact solution to Eq. 5, under which the equivalence is essentially by definition: if L(p) is a maximizer of J, then it belongs to the set R* of maximizers of J over the output space. The actual method, however, uses gradient ascent/descent in logits space, discrete decoding, greedy early termination based on the judge's classification, and a heuristic projection matrix. None of these approximations is analyzed. The proposition should be reframed as an idealized motivation, or the paper needs an error analysis showing the two-stage optimization remains close to the exact solution.","section":"Appendix B (Proposition 1)"},{"comment":"The gradient projection step is not justified. Equation (10) constructs a binary token-projection matrix W and defines the projected gradient as ∇J ⊗ W, but the paper offers no proof or experiment showing this projected gradient is a valid ascent direction for J(L(p)) with respect to the prompt p, nor that the MSE logit-alignment objective (Eqs. 12–13) preserves the unsafety signal. In addition, Algorithm 1 line 19 updates a variable zL_r' that is never initialized or defined in the pseudocode, so the exact update rule is ambiguous. The authors should provide a formal condition for the projection or an ablation that compares the proposed projection with a random or unprojected gradient to demonstrate that the transfer step is responsible for the reported performance.","section":"Section 3.3, Eqs. (10)–(13) and Algorithm 1"}],"minor_comments":[{"comment":"The method is called “NonTextual Target Attack (NTA)” in the lead abstract but “Untargeted Jailbreak Attack (UJA)” throughout the full text. This naming inconsistency should be fixed in the final version.","section":"Title / Abstract"},{"comment":"The text says “As shown in Figure 2, UJA consistently outperforms ... in transferability,” but Figure 2 is the methodology diagram. The cited figure for transferability appears to be elsewhere (perhaps Figure 6 or a dedicated plot).","section":"Section 4.2"},{"comment":"Figure 9 and Figure 10 contain non-English placeholder text (“一块展示攻击成功样例”) and layout fragments from an earlier draft. These should be removed.","section":"Figure captions (Appendix I)"},{"comment":"The column “Avg.ASR (%)” should state explicitly that it is the mean of ASR-G and ASR-H, and clarify how the average across the two datasets or target models is computed.","section":"Table 4"}],"recommendation":"major_revision","confidential_remarks":"The manuscript appears to contain two different versions of the paper stitched together (different names, different abstract numbers). The editor may want to verify that the submitted PDF is the intended version and that the code release matches the described method. The core idea is interesting, but the current evidence is not yet convincing: the main metric is circular, the theoretical result is tautological, and the gradient-transfer step is unvalidated. These are fixable with additional experiments and a careful rewrite, hence major_revision rather than reject."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know about arXiv:2510.02999. First, the core idea is genuinely new: instead of forcing the model to output a fixed harmful target string, the attack optimizes a response in the judge's embedding space, then transfers the gradient to the target LLM through a cross-tokenizer projection matrix, and only then optimizes the prompt. That is a real departure from GCG/COLD/AdvPrefix, and the two-stage ablation suggests Stage 1 helps. Second, the posted manuscript is internally inconsistent in its headline claims: the abstract says 96.8% ASR and over 40% improvement (and calls it NTA), while the body reports a 71.9% average and 23.5% improvement (and calls it UJA). Table 1 never exceeds 89%. That is not a typo; it is a 25-point gap. A paper with that level of mismatch is not ready for publication as-is.\n\nWhat is good: the method description is concrete and the cross-tokenizer projection is a clever piece of engineering. The experiments cover six white-box models, plus transfer to 70B/671B, and include defenses and cost analysis. The independent metric ASR-H (HarmBench-Llama-2-13b-cls) is not optimized by the attack, and UJA still wins many of those comparisons, so the circularity of ASR-G is a real problem but not the whole story.\n\nSoft spots: beyond the number mismatch, the primary metric ASR-G uses GPTFuzzer, the same judge that drives Stage 1 optimization, so part of the ASR-G gain is expected. The paper's claim that using ASR-H 'prevents overfitting' is only half true: ASR-H is an independent metric, but the headline tables and conclusions rely on ASR-G. Proposition 1 is essentially a compactness existence argument; it does not justify the surrogate losses or the projection. The transfer assumption is heuristic, and the paper provides no robustness analysis. The example in Figure 9 is less damning than it looks: the response does give concrete steps for insider trading; the disclaimer is boilerplate, so the judge is not necessarily being gamed. More troubling is the banner claiming 'Published as a conference paper at ICLR 2026' on an October 2025 preprint — that is a credibility problem if it is not true.\n\nWho it is for: anyone working on gradient-based jailbreaks or red-teaming evaluations will find the two-stage objective worth discussing, but the paper cannot be cited for its numbers until the inconsistencies are resolved.\n\nMy recommendation: send it to peer review rather than desk reject — the method deserves a referee's time — but with a clear instruction to the authors to reconcile the numbers, use a truly held-out judge for the headline metric, and provide error bars. If those fixes are made, this could become a solid paper. As posted, I would not rely on any of its quantitative claims.","headline":"Genuinely novel two-stage jailbreak attack, but the posted numbers don't match across sections and the headline ASR-G metric is partially circular — worth reviewing, not worth citing yet.","tokens_in":28509,"tokens_out":4567,"would_cite":false,"duration_ms":38607,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that maximizing a judge model's unsafety score, with no fixed response target, outperforms targeted gradient jailbreak attacks within tight iteration budgets.","keywords":["jailbreak attack","untargeted optimization","large language models","gradient-based prompt optimization","judge model","adversarial robustness","tokenizer projection","safety alignment"],"falsifier":"Run the same pipeline with the judge's Unsafe label replaced by a random or inverted label; if attack success stays high, the judge signal is not doing the work. Or have human annotators score the paper's Figure 9 output — step-by-step insider-trading instructions wrapped in 'this is illegal, for educational purposes only' — and count how many are judged genuinely harmful; if most are rated safe, the judge, not the model, is what is being jailbroken.","tokens_in":27417,"feed_emoji":"🔓","tokens_out":6343,"duration_ms":53645,"temperature":0.7,"pith_summary":"This paper sets out to show that automated jailbreak attacks on large language models have been handicapping themselves. Existing gradient-based attacks optimize prompts so the model starts with a fixed phrase such as 'Sure, here is...'; the paper argues this rigid target shrinks the set of prompts the optimizer can find and slows convergence. It replaces that objective with an untargeted one: maximize the probability, according to a separate judge model, that the model's response is unsafe, without specifying what the response should say. Because that objective is non-differentiable, the paper splits it into two differentiable sub-objectives — first generate an optimally unsafe response in the judge's embedding space, then optimize the prompt to reproduce that response in the target model's logits. Across six open-weight LLMs with only 100 optimization steps, it reports clearly higher attack success rates than the targeted baselines, faster early convergence, and prompts that transfer better to larger models. The reason to care: if the claim holds, standard red-team benchmarks have been underestimating how quickly aligned models can be jailbroken.","feed_headline":"No fixed target makes chatbot jailbreaks faster and stronger","feed_subtitle":"By maximizing a judge's unsafety score instead of forcing a phrase, the attack widens the search space and raises success rates.","key_machinery":"The load-bearing mechanism is a two-stage gradient pipeline that crosses tokenizer boundaries. In Stage 1, an adversarial response is optimized inside the judge model's embedding space by minimizing cross-entropy against the Unsafe label, with a cosine-similarity term that keeps the response semantically close to the model's actual output. In Stage 2, the resulting gradient is projected into the target LLM's token space through a binary matrix W: each token in the judge's vocabulary is retokenized under the target model, and the gradient is spread onto the sub-tokens. The prompt is then updated by minimizing the mean-squared error between the target model's pre-softmax scores for its current","core_discovery":"The central claim is that the jailbreak objective should be max_p J(L(p)), where L is the target LLM and J is a judge model returning the probability that a response is unsafe, with no constraint on the response's wording. Because this objective is non-differentiable in discrete text, the paper decomposes it into two sub-problems: first maximize J(r) over possible responses to obtain an optimal unsafe response r*, and then minimize the squared difference between the target model's logits for its actual output and its logits for r*, so that the optimized prompt elicits r*. The paper argues this expands the adversarial search space relative to fixed-prefix attacks, which is why it reports high","pith_inferences":["The paper's own figures show responses full of safety disclaimers being labeled 'Unsafe' by the judge; if the judge is that easy to fool, part of the reported success may be gaming the scorer rather than eliciting genuinely harmful output.","The abstract reports 96.8% average success while the body reports 71.9%; that gap needs resolving before the main quantitative claim can be taken at face value.","The untargeted formulation points to a general recipe: any differentiable judge of any output property could replace the harmfulness judge, turning this into a general search method.","Testing the attack with a judge that scores actionable harmfulness rather than surface topic would separate genuine vulnerabilities from scorer artifacts."],"forward_implications":["If right, fixed-prefix jailbreak attacks substantially understate the vulnerability of safety-aligned LLMs.","Red-team evaluations can use untargeted objectives to find unsafe behaviors with far fewer optimization steps.","The judge model used to score harmfulness becomes a direct optimization target, so the judge's own robustness matters for measured security.","The same cross-tokenizer gradient projection could be reused to transfer other classifier-guided objectives between models."],"fun_headline_variants":["Attack maximizes judge's unsafety score, not fixed text","No text target: jailbreak via unsafe-score objective","Wider search space via non-textual attack objective","Unsafety probability as target boosts jailbreak success","Gradient attack on unsafety prob beats fixed targets"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The attack assumes that a gradient computed in the judge model's embedding space, projected linearly through a token-mapping matrix and converted into a prompt update, still points the target model toward genuinely unsafe responses.","fun_headline_variants_meta":{"raw":{"variants":["Attack maximizes judge's unsafety score, not fixed text","No text target: jailbreak via unsafe-score objective","Wider search space via non-textual attack objective","Unsafety probability as target boosts jailbreak success","Gradient attack on unsafety prob beats fixed targets"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000249,"raw_usage":{"total_tokens":1401,"prompt_tokens":773,"completion_tokens":628,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":517,"completion_tokens_details":{"reasoning_tokens":552}},"tokens_in":517,"tokens_out":628,"duration_ms":5702,"temperature":1.0,"reasoning_tokens":552,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T12:39:09.935591+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same pipeline with the judge's Unsafe label replaced by a random or inverted label; if attack success stays high, the judge signal is not doing the work. Or have human annotators score the paper's Figure 9 output — step-by-step insider-trading instructions wrapped in 'this is illegal, for educational purposes only' — and count how many are judged genuinely harmful; if most are rated safe, the judge, not the model, is what is being jailbroken.","supporting_citations":[],"review_version":1}