{"id":"a9b1ef9e-6a21-484d-ab22-dfff9d7ebe3e","arxiv_id":"2503.24191","paper_version":3,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":8.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"This paper presents Constrained Decoding Attack (CDA) as a new control-plane jailbreak class for LLMs using structured output grammars, with DictAttack achieving 94.3-99.5% ASR on flagship models.","lead":"The paper introduces Constrained Decoding Attack (CDA), a jailbreak exploiting grammar-guided decoding in LLM structured output APIs to inject malicious prefixes via logit masking. Smart generalists should read it because it identifies a control-plane vulnerability orthogonal to prompt-based attacks that current safety alignments cannot block.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Reported 94.3--99.5% ASR figures rest on unreproduced experiments with no methodology or results provided","rationale":"The reader's UNVERDICTED verdict and identification of the unverifiable weakest assumption are correct; the load-bearing issue is precisely the absence of any experimental substrate against which the technical claim can be checked. No independent support (machine-checked proofs, released code, or parameter-free derivation) is present in the provided text.","tokens_in":1812,"tokens_out":328,"duration_ms":16502,"concrete_test":"Obtain the full paper (including § on methodology, results tables, and attack grammars) plus the code at https://ict-cda.github.io/; attempt to reproduce the DictAttack ASR numbers on one flagship model using the exact schemas and evaluation protocol described.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The strongest claim asserts that DictAttack (via schema-enforced logit masking) reliably injects a malicious prefix that bypasses safety alignment, yielding 94.3--99.5% ASR on gpt-5, gemini-2.5-pro, deepseek-r1, and gpt-oss-120b, plus 75.8% ASR against guardrails. Only the abstract is available; it contains no attack grammar definitions, no description of the control-to-semantic pipeline implementation, no benchmark prompts or success criteria, no ablation on grammar auditing, and no code. Without these, it is impossible to determine whether the reported rates reflect a genuine control-plane vulnerability or an artifact of evaluation choices.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper introduces Constrained Decoding Attack (CDA) as a new jailbreak class targeting the control plane of LLMs through structured output APIs and grammar-guided decoding. It describes a control-to-semantic pipeline instantiated as EnumAttack (hiding malicious content in enum fields) and the more evasive DictAttack (decoupling payload across benign prompt and dictionary grammar), claiming DictAttack achieves 94.3--99.5% ASR on 13 models including gpt-5, gemini-2.5-pro, deepseek-r1, and gpt-oss-120b, plus 75.8% ASR against SOTA guardrails even after basic grammar auditing mitigates EnumAttack, and calls for cross-plane defenses.","tokens_in":1960,"tokens_out":346,"duration_ms":13396,"significance":"If the empirical results and attack constructions hold under scrutiny, the work would be significant for identifying an attack surface orthogonal to data-plane jailbreaks, as it shows that schema-enforced logit masking can bypass internal safety alignment by acting directly on the decoding trajectory.","major_comments":[{"comment":"Abstract: The central claims of 94.3--99.5% ASR on flagship models and 75.8% ASR against guardrails are stated without any description of attack grammar definitions, the control-to-semantic pipeline implementation, benchmark prompts, success criteria, ablation on grammar auditing, experimental methodology, or verification steps, rendering the reported rates impossible to assess or reproduce.","section":"Abstract"}],"minor_comments":[],"recommendation":"uncertain","confidential_remarks":"The provided manuscript consists solely of the abstract with no full text, supplementary materials, or code, which precludes any substantive technical review of the claimed results."},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for their review of our manuscript. We address the single major comment below.","responses":[{"response":"The abstract is a concise summary by design and does not contain the full methodological details. The complete descriptions of attack grammar definitions, the control-to-semantic pipeline, benchmark prompts, success criteria, ablation studies on grammar auditing, experimental methodology, and verification steps appear in the main body of the manuscript (Sections 3–6). This structure follows standard academic practice, where abstracts highlight contributions and results while the body supplies the information needed for assessment and reproduction. We are prepared to add a brief sentence to the abstract if the editor requests it.","revision_made":"no","referee_comment":"[Abstract] Abstract: The central claims of 94.3--99.5% ASR on flagship models and 75.8% ASR against guardrails are stated without any description of attack grammar definitions, the control-to-semantic pipeline implementation, benchmark prompts, success criteria, ablation on grammar auditing, experimental methodology, or verification steps, rendering the reported rates impossible to assess or reproduce."}],"tokens_in":1393,"tokens_out":249,"duration_ms":20818,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The one thing to know is that this abstract describes Constrained Decoding Attack as a distinct way to target the grammar-guided decoding step itself rather than the input prompt. If the full paper shows that schema-enforced logit masking can reliably steer generation into harmful completions even when safety alignment is intact, that would be a real addition to the jailbreak literature. The authors split the idea into EnumAttack and the more evasive DictAttack, and they claim DictAttack still works at 75 percent against current guardrails after basic grammar checks are applied. That framing of a control-to-semantic pipeline is the clearest novelty on offer. The paper does a decent job naming the gap between data-plane defenses and the decoding process that structured-output APIs expose, and it points out that production tooling and agent frameworks could be affected if the attack generalizes. Credit for making the distinction explicit and for releasing a project page and code link, even if those are not yet usable from the abstract alone. The soft spots are straightforward and fairly large. No attack grammar definitions, no prompt examples, no success criteria, no ablation on how the malicious prefix is injected, and no comparison against standard baselines appear in the abstract. The 94 to 99 percent ASR figures on gpt-5, gemini-2.5-pro and the rest are therefore impossible to evaluate; they could reflect a genuine vulnerability or they could reflect evaluation choices that have not been described. The stress-test note is accurate on this point. The work is aimed at researchers and engineers who build or secure structured-output APIs and guardrails. A reader looking for early signals of new attack surfaces might skim it for the high-level idea, but anyone needing reproducible evidence will find the current version thin. It deserves a serious referee once the full text and experimental details are supplied, because the underlying question about control-plane exposure is worth checking even if the current numbers turn out to be overstated.","headline":"The abstract flags a plausible new control-plane attack on structured LLM outputs but supplies zero methodology or verification, so the high ASR numbers cannot be assessed yet.","tokens_in":2462,"tokens_out":459,"would_cite":false,"duration_ms":15445,"reading_group":"maybe","serious_thinker":"unclear","would_accept_peer_review":true},"rs_alignment":{"model":"grok-4.3","evidence":[],"headline":"LLM control-plane jailbreak paper (grammar/logit masking, DictAttack) has zero overlap with RS distinction-to-spacetime chain","alignment":"orthogonal","rationale":"The paper's machinery (schema-enforced logit masking, EnumAttack/DictAttack via JSON grammars and dictionary decoupling, dual-plane auditing) operates entirely in the domain of autoregressive decoding and safety alignment. It never invokes, parallels, or contradicts any RS element: J-cost, φ-ladder, 8-tick periodicity, Alexander duality for D=3, or reality_from_one_distinction. Domain mismatch is total; no RS module or theorem is relevant.","tokens_in":59449,"confidence":"high","tokens_out":153,"duration_ms":16846,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Structured output APIs let attackers force malicious prefixes into LLM generation via grammar constraints.","keywords":["constrained decoding","LLM jailbreak","structured output","control plane attack","AI security","grammar-guided generation","adversarial attacks on LLMs"],"falsifier":"An experiment in which a model with intact safety training refuses to complete the harmful intent even after the grammar has forced the malicious prefix into the output trajectory.","tokens_in":2718,"feed_emoji":"🔓","tokens_out":715,"duration_ms":26377,"temperature":0.7,"pith_summary":"The paper establishes that grammar-guided decoding in structured output APIs creates a control-plane vulnerability in LLMs. Attackers inject a malicious prefix through schema-enforced logit masking, after which the model completes the harmful intent on its own. This control-to-semantic pipeline differs from traditional jailbreaks that manipulate visible inputs. A reader would care because it reveals that safety alignments focused on data-plane inputs leave models open when using structured generation. The work shows DictAttack reaching 94.3-99.5 percent success on leading models and 75.8 percent against current guardrails.","feed_headline":"Grammar constraints enable 94-99% jailbreak success on LLMs","feed_subtitle":"Attacks exploit schema-enforced logit masking in structured output APIs to bypass input-based safety.","key_machinery":"The control-to-semantic pipeline of Constrained Decoding Attack (CDA), in which schema-enforced logit masking forces a malicious prefix that the model then follows to completion.","core_discovery":"Constrained Decoding Attack (CDA) is a jailbreak class that targets the LLM control plane as a control-to-semantic pipeline: schema-enforced logit masking injects a malicious prefix into the generation trajectory, and the model itself completes the harmful intent. CDA is instantiated as EnumAttack, which hides content in enum fields, and DictAttack, which decouples the payload across a benign prompt and dictionary-based grammar. DictAttack reaches 94.3--99.5 percent attack success rate on models including gpt-5, gemini-2.5-pro, deepseek-r1, and gpt-oss-120b, and sustains 75.8 percent success against SOTA jailbreak guardrails, exposing a semantic gap that requires cross-plane defenses.","pith_inferences":["Developers of structured output APIs may need to add runtime checks that inspect the effect of user-provided grammars on logit masking.","Safety evaluations should test models under constrained decoding conditions rather than free-text generation alone.","The same control-plane issue could appear in any system that lets users supply custom schemas or grammars for generation."],"forward_implications":["EnumAttack can be stopped by basic grammar auditing while DictAttack cannot.","DictAttack maintains 75.8 percent success rate against current state-of-the-art jailbreak guardrails.","A semantic gap exists between data-plane and control-plane defenses that must be bridged.","Flagship models including gpt-5, gemini-2.5-pro, deepseek-r1, and gpt-oss-120b remain highly susceptible."],"fun_headline_variants":["Grammar-guided decoding enables control-plane LLM attacks","Constrained Decoding Attack targets LLM structured output","DictAttack uses dictionary grammar for 94-99% ASR","Schema-enforced masking allows malicious prefixes in LLMs"],"cache_read_input_tokens":64,"weakest_assumption_plain":"That safety alignment cannot detect or block completion of a harmful intent once a malicious prefix has been forced by the grammar constraints.","fun_headline_variants_meta":{"raw":{"variants":["Grammar-guided decoding enables control-plane LLM attacks","Constrained Decoding Attack targets LLM structured output","DictAttack uses dictionary grammar for 94-99% ASR","Schema-enforced masking allows malicious prefixes in LLMs"]},"model":"grok-4.3","cost_usd":0.005975,"raw_usage":{"total_tokens":2909,"prompt_tokens":823,"num_sources_used":0,"completion_tokens":59,"cost_in_usd_ticks":59749500,"prompt_tokens_details":{"text_tokens":823,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2027,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":823,"tokens_out":59,"duration_ms":13055,"temperature":1.0,"reasoning_tokens":2027,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-05-22T21:50:27.561885+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"An experiment in which a model with intact safety training refuses to complete the harmful intent even after the grammar has forced the malicious prefix into the output trajectory.","supporting_citations":[],"review_version":1}