{"id":"9a0cdd42-7d8a-4947-8d10-01ea5751fa35","arxiv_id":"2507.21182","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"SDD makes an LLM reply to harmful prompts with irrelevant high-quality text, so that malicious fine-tuning degrades general ability and the model cannot follow harmful instructions.","lead":"This paper proposes a defense against malicious fine-tuning of open-source LLMs: train the model to answer harmful prompts with irrelevant, high-quality text, so that any attempt to fine-tune it on harmful data degrades its general capabilities. The empirical results show lower harmfulness after attacks, but the theoretical proof contains a critical error and the defense is untested against attackers who mix harmful and benign data.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 2's proof requires the impossible inequality Eq. 38; the advertised theoretical demonstration that SDD degrades capabilities under MFT is invalid as written.","rationale":"I focused on an internal inconsistency rather than the reader's mixed-fine-tuning threat-model gap because the algebraic flaw directly invalidates the advertised theoretical contribution. Theorem 2 is the only formal support for the claim that SDD renders a model 'incapable of following harmful instructions' after MFT (§1). If the proof is impossible, the paper's central theoretical claim is unsupported, even if the experiments still show an empirical degradation. This is an internally inconsistent proof, not a disagreement with an external consensus, so it is a high-correctness-risk issue. The reader's mixed-fine-tuning concern is also real and load-bearing: the threat model grants attackers full data control, and the experiments only test pure harmful-data MFT. I marked agreement as 'partial' because both concerns undermine the central claim, but the strongest single concern I found is the impossible inequality in Appendix E. Since the reader already recommended REJECT, my finding does not move the verdict; it reinforces it, so I set verdict_should_be to UNCHANGED.","tokens_in":20219,"tokens_out":6602,"duration_ms":68123,"concrete_test":"Independently re-derive Eq. 38 from Appendix E and test satisfiability by hand: set p = 1 (the most favorable case) and check whether any nonnegative integers n̄_v, n*_vo, n*_s, n*_so satisfy the inequality. The identity LHS − RHS = (1−p)n*_s + 2(1−p)n*_so + n̄_v + 2n*_vo shows the left side always exceeds the right side, so no solution exists. If the authors can exhibit a concrete admissible assignment satisfying Eq. 38, the concern is resolved; otherwise, the proof of Theorem 2 fails as written.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central theoretical claim is Theorem 2 (§4.2), which asserts existence of parameter settings (n̄_v > n*_v, n̄_s < n*_s) such that malicious fine-tuning strictly lowers general-task accuracy, ξG(f~) < ξG(f̄). This theorem is the stated basis for the paper's headline claim that SDD 'theoretically demonstrate[s]' capability impairment (§1, §4.2). The proof in Appendix E reduces the desired inequality to Eq. 36 and then attempts to establish the sufficient condition Eq. 38. Substituting Eq. 38's left-hand side minus its right-hand side yields (1−p)n*_s + 2(1−p)n*_so + n̄_v + 2n*_vo. Every term is nonnegative, and n̄_v ≥ 1 under the theorem's premise n̄_v > n*_v ≥ 0, so the difference is strictly positive for every admissible p ∈ [0,1]. Even at the most favorable p = 1, the inequality becomes 2n̄_v + 2n*_vo < n̄_v, which is impossible. Eq. 38 is therefore unsatisfiable; no choice of nonnegative integer feature counts makes the stated proof go through. Consequently, the proof of Theorem 2 does not establish existence, and the paper's theoretical justification for SDD is unsupported as written. The empirical MMLU/OpenBookQA degradation could still be real, but it is not explained by the offered theory, and the theorem cannot be repaired simply by picking different counts within the proof's own framing.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes SDD, a defense that trains an LLM to answer harmful prompts with high-quality but irrelevant benign responses, so that a subsequent malicious fine-tuning (MFT) attempt degrades the model's general capabilities and leaves it unable to follow harmful instructions. The paper presents a stylized feature-count theory (Theorems 1 and 2) meant to explain why MFT succeeds against standard alignment and why SDD can induce self-degradation, then reports experiments on Llama2-7b and Llama2-7b-chat under AdvBench-based MFT and ShareGPT-based benign fine-tuning, measuring harmfulness rates and general capability on MMLU and OpenBookQA.","tokens_in":20659,"tokens_out":8149,"duration_ms":85311,"significance":"The central idea of defending by self-degradation is original and could be practically valuable if the claimed guarantee held. The paper also releases code, evaluates a responsible variant with explicit refusals, and tests multiple backbones. However, the theoretical result that is advertised as the basis for the method is not established as written, and the empirical evaluation covers a narrower threat model than the one stated in the paper. Because the theory and the threat-model coverage are load-bearing for the central claim, the contribution is not currently supported.","major_comments":[{"comment":"The proof of Theorem 2 reduces to an inequality that cannot be satisfied. Expanding Eq. (38), the left-hand side minus the right-hand side equals (1-p)(n*_s + 2n*_so) + n̄_v + 2n*_vo, which is strictly positive for every p in [0,1] and every nonnegative feature count; n̄_v >= 1 follows from the premise n̄_v > n*_v >= 0. Thus no parameter setting can make Eq. (38) true, and the proof does not establish the existence claimed in Theorem 2. Since Theorem 2 is the stated basis for the 'theoretically demonstrate' claim in §1 and for the SDD design in §5.1, the central theoretical contribution is unsupported.","section":"Appendix E, Eq. (38)"},{"comment":"The threat model grants attackers full control over data and training, but the experiments perform MFT only on pure harmful data (AdvBench). The paper's headline claim that 'any MFT attempt' will cause self-degradation is not tested or derived for mixed fine-tuning in which an attacker combines harmful data with benign instruction data. Under such a mixed attack, general capabilities may be preserved, and the model may remain able to follow harmful instructions. This gap affects both the experimental support and the scope of Theorem 2, so the defense guarantee is narrower than claimed.","section":"§3.1 and §6.2"},{"comment":"The preference-model derivation contains a sign error that reverses the stated mechanism. With r(x,y) = log(π*(y|x)/πθ(y|x)) as used in Eq. (6), the objective p(yc > yo | x) in Eq. (7) equals A/(A+B) with A = π*(yc|x)/πθ(yc|x) and B = π*(yo|x)/πθ(yo|x). Maximizing this objective increases πθ(yo|x) and decreases πθ(yc|x), which is the opposite of the paper's claim that 'the term π*(yo|x) will decrease' and that MFT reduces the probability of the original response yo. This inverts the motivation for pairing harmful instructions with high-quality benign responses.","section":"§5.1, Eqs. (4)-(7)"},{"comment":"Theorem 1 states only an upper bound on ξA(f~) - ξA(f̄). The surrounding text concludes that the difference is 'likely to be negative' and that this highlights the vulnerability of aligned models, but the theorem itself does not establish negativity under the stated assumptions. The first claimed contribution, that the paper 'theoretically prove[s] that MFT can compromise safety alignment,' is therefore not supported by the theorem as written.","section":"§4.1, Theorem 1"}],"minor_comments":[{"comment":"The sentence 'Experimental results demonstrate that the SDD framework effectively MFT' is missing a verb such as 'defends against'; it should read 'effectively defends against MFT.'","section":"§1"},{"comment":"The caption contains a typo: 'after LMMs undergo malicious fine-tuning' should be 'after LLMs undergo malicious fine-tuning.'","section":"Figure 1 caption"},{"comment":"The irrelevance-selection step uses a cosine-similarity threshold, but the threshold value is not reported and no ablation is provided; this makes the dataset construction step not fully reproducible.","section":"§5.2"},{"comment":"The general-capability results are reported as single numbers without error bars, multiple seeds, or statistical tests. The phrase 'significantly declines' should be supported by variance information, especially because these results are central to the empirical claim that MFT degrades the SDD-aligned model.","section":"§6.2, Table 2"},{"comment":"Reusing the same notation with task-dependent meanings across theorems makes the proofs difficult to verify; renaming the quantities per theorem would improve clarity.","section":"Footnote 1"}],"recommendation":"reject","confidential_remarks":"The proof of Theorem 2 rests on an unsatisfiable inequality, and the threat-model gap means the empirical results do not support the advertised 'any MFT attempt' guarantee. The sign issue in §5.1 further undermines the mechanism that motivates the method. These are load-bearing problems rather than presentation issues. I would not rule out a future version that replaces the theory with a sound argument or explicitly narrows the threat model, but as submitted the central claims are not established."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the SDD construction is a legitimate new idea and the experiments are suggestive, but the theoretical guarantee in Theorem 2 does not hold as written, and the defense is evaluated against a narrower attack than the threat model promises.\n\nWhat is new: pairing harmful instructions with random high-quality benign responses and fine-tuning the model on those pairs is a concrete, previously untested recipe. The MMLU/OpenBookQA degradation numbers after MFT are striking (33–59% drops), and the BFT results show the defense does not obviously hurt benign users. The responsible variant with explicit refusal is a sensible addition. Code is linked. That is real work.\n\nThe soft spots are load-bearing. The proof of Theorem 2 in Appendix E reduces the desired inequality to Eq. 38. Substitute the difference between the two sides: (1−p)n*_s + 2(1−p)n*_so + n̄_v + 2n*_vo. Every term is nonnegative and n̄_v ≥ 1, so the inequality holds for no value of p in [0,1]. The theorem's existence claim is not established. Since the abstract and intro advertise a theoretical demonstration, this matters.\n\nSecond, the threat model says attackers have full access and can train on any data. But all MFT experiments use AdvBench harmful pairs only. The obvious countermeasure is mixed fine-tuning, adding benign instruction data to keep general capabilities alive. That scenario is not tested, and the defense's mechanism—degrading general capability—would plausibly be mitigated by it. The intro's 'any MFT attempt' claim is therefore unsupported.\n\nThird, the closest prior work is not engaged with: Henderson et al.'s Self-Destructing Models and the unlearnable behaviors line are conceptually adjacent and should at least be compared. The rest of the citation pattern looks fine; the Lin et al. framework is honestly borrowed, and Assumption 1 is stated as an assumption rather than hidden.\n\nAlso minor: no error bars, and the evaluation is mostly harmfulness rate plus two capability benchmarks, so the empirical case is suggestive rather than robust.\n\nBottom line: this is a paper for researchers working on MFT defenses. It deserves a serious referee, and I would send it out—but I would not accept it as is. The theory needs to be repaired or removed, and the mixed fine-tuning attack needs to be run before the headline claims can stand.","headline":"The SDD recipe is a genuine new idea with suggestive experiments, but the proof of the central theorem is algebraically impossible as written and the threat model leaves the obvious mixed fine-tuning attack untested.","tokens_in":21062,"tokens_out":2550,"would_cite":false,"duration_ms":26167,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Maliciously fine-tuning an SDD-protected LLM destroys its general ability, so it cannot comply with harmful prompts.","keywords":["malicious fine-tuning","safety alignment","self-degraded defense","LLM safety","fine-tuning attacks","harmful instruction following","open-weight models"],"falsifier":"Take an SDD-protected model and fine-tune it on a mixture of half harmful AdvBench pairs and half benign ShareGPT pairs; if the general-accuracy benchmarks stay near the protected model's level while the harmfulness rate rises, the self-degradation claim is refuted.","tokens_in":20073,"feed_emoji":"🛡️","tokens_out":6960,"duration_ms":66471,"temperature":0.7,"pith_summary":"The paper argues that standard safety alignment fails against malicious fine-tuning and proposes a different safety goal: instead of making a model refuse harmful prompts, make it unable to produce harmful responses after an attack. It introduces Self-Degraded Defense, which trains a model to answer harmful instructions with high-quality but unrelated benign responses. The paper claims that when an attacker then fine-tunes the model on harmful data, the model's general capabilities collapse, so it cannot follow harmful instructions, even ones it has not seen. This matters because open-weight LLMs give attackers full parameter access, and existing defenses are mostly empirical; SDD offers a theoretical account and a simple supervised fine-tuning recipe that preserves benign performance.","feed_headline":"New defense makes maliciously fine-tuned LLMs self-degrade","feed_subtitle":"Under harmful fine-tuning a protected model loses 36-59% of general accuracy and keeps a 0% harmfulness rate.","key_machinery":"The load-bearing object is a feature-count accuracy bound: an LLM is abstracted as a feature selector and classifier, with invariant features consistently predicting labels and spurious features correlating with labels only in training data. The proof machinery includes a weight-space interpolation assumption, where a near-optimal malicious model is a linear blend of the original and fine-tuned models, and a Bradley-Terry preference derivation showing malicious fine-tuning maximizes the probability that the harmful response beats the model's original output, which necessarily lowers the probability of that original output. SDD exploits this by setting the original output for harmful queries to high-quality unrelated benign text: the same mechanism that removes safety refusals becomes the mechanism that removes general competence.","core_discovery":"The paper's central claim is that defense can be repurposed from refusal to self-degradation. Modeling an LLM as a feature selector plus classifier, and assuming a near-optimal maliciously fine-tuned model is a linear extrapolation of the original and fine-tuned models, the paper proves two bounds. Theorem 1 says malicious fine-tuning lowers accuracy on the safety-alignment task, explaining why alignment is fragile. Theorem 2 says that if the original model has more invariant features useful for general tasks than the near-optimal malicious model, and fewer spurious features that hurt general tasks, then malicious fine-tuning lowers accuracy on general tasks. SDD realizes these conditions by making the model's original response to a harmful query a high-quality irrelevant benign answer, so the attacker's preference optimization, which must lower the probability of the original response, destroys general capability rather than merely removing refusals. Experiments on Llama2-7b-chat show a 0% harmfulness rate across 10-, 50-, and 100-shot malicious fine-tunes while general accuracy drops by roughly a third to more than half.","pith_inferences":["An untested extension is mixed fine-tuning: if an attacker interleaves benign instruction data with harmful data, the benign data may keep the model's general abilities alive while harmful behavior is learned; the paper's experiments do not cover that scenario.","The theoretical condition for degradation, that the original model has more general-task invariant features than the near-optimal malicious model, is an existence condition; SDD's empirical success suggests high-quality irrelevant outputs act as a poison pill that turns the attack's own gradient into self-damage, an idea that could extend to unlearning and tamper-resistance.","Because SDD does not rely on refusal, it could be combined with a separate refusal or output-filtering layer: the defense makes an attacked model incompetent, while a filter would still catch any harmful output that leaks through at the boundary."],"forward_implications":["An SDD-protected open-weight model that is released and then fine-tuned on harmful data should become broadly incompetent rather than dangerous, so harmful instructions go unanswered.","Legitimate users who use the model as released or fine-tune it on benign data keep general capability: reported MMLU and OpenBookQA scores after SDD and after benign fine-tuning are on par with or slightly above the vanilla model.","The defense can be inserted after pre-training, after SFT, or after RLHF, and is only a supervised fine-tuning step, so it adds little burden to the release pipeline.","A responsible variant, SDD_reject, prepends an explicit refusal prefix to the irrelevant answer and retains a 0% harmfulness rate after malicious fine-tuning, addressing the preference for refusal-style behavior.","The defense stays effective when the attacker scales malicious data up to twenty times the size used in the defense, raising the cost of misuse."],"supporting_citations":[{"why":"Supplies the feature-based accuracy bound and the Small Noise and Orthogonal Features assumptions that Theorems 1 and 2 rely on.","marker":"Lin et al., 2023"},{"why":"Provides the Bradley-Terry and reward-policy relationship used to show malicious fine-tuning maximizes the chance the harmful response beats the model's original output, lowering that output's probability.","marker":"Rafailov et al., 2023"},{"why":"Documents fine-tuning compromising safety even on benign data and supplies the LLM-finetune-Safety benchmark used to evaluate defenses.","marker":"Qi et al., 2023"},{"why":"Provides BeaverTails harmful question-answer pairs for constructing SDD training data and the evaluation benchmark for harmlessness.","marker":"Ji et al., 2023"},{"why":"Supplies LIMA high-quality responses paired with harmful instructions in SDD's dataset.","marker":"Zhou et al., 2023a"},{"why":"Supplies Alpaca high-quality responses used as unrelated benign answers in SDD's dataset.","marker":"Taori et al., 2023"},{"why":"Provides evidence that fine-tuning on a small number of harmful pairs bypasses safety alignment, motivating malicious fine-tuning as a threat.","marker":"Yang et al., 2023b"}],"fun_headline_variants":["Self-degraded defense makes harmful fine-tuning backfire","LLMs self-sabotage to defeat malicious fine-tuning","0% harm under attack via self-degrading defense","Strategic degradation blocks malicious fine-tuning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The defense assumes attackers fine-tune exclusively on harmful data; if an attacker mixes benign instruction data with harmful data, the model's general abilities may not degrade and it could remain able to follow harmful instructions.","fun_headline_variants_meta":{"raw":{"variants":["Self-degraded defense makes harmful fine-tuning backfire","LLMs self-sabotage to defeat malicious fine-tuning","0% harm under attack via self-degrading defense","Strategic degradation blocks malicious fine-tuning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000307,"raw_usage":{"total_tokens":1724,"prompt_tokens":881,"completion_tokens":843,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":497,"completion_tokens_details":{"reasoning_tokens":780}},"tokens_in":497,"tokens_out":843,"duration_ms":8423,"temperature":1.0,"reasoning_tokens":780,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:49:48.393131+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take an SDD-protected model and fine-tune it on a mixture of half harmful AdvBench pairs and half benign ShareGPT pairs; if the general-accuracy benchmarks stay near the protected model's level while the harmfulness rate rises, the self-degradation claim is refuted.","supporting_citations":[{"cited_title":"Manning, Stefano Ermon, and Chelsea Finn","cited_arxiv_id":null,"evidence_quote":"Provides the Bradley-Terry and reward-policy relationship used to show malicious fine-tuning maximizes the chance the harmful response beats the model's original output, lowering that output's probability."}],"review_version":2}