Pith. sign in

REVIEW 2 major objections 5 minor 39 references

When Skills Meet Safety: Benchmarking and Characterizing the Adaptive Jailbreak Robustness of Skill-Merged LLMs

T0 review · 2 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Static safety does not predict adaptive jailbreak robustness in skill-merged LLMs.

desk verdict A careful empirical paper that actually delivers on its central claim—static refusal does not predict adaptive robustness for merged models—while the subspace repair tool is honestly bounded to same-recipe abliteration. read the letter →

arxiv 2608.08542 v1 pith:YEO77KHK submitted 2026-08-09 cs.LG

Yu Ma , Hongli Shi , Jing Li , Xinran Xu , Weiwei Hou This is my paper · ORCID
classification cs.LG
keywords modelmergingsafetyalignmentadaptivejailbreakrobustnessstaticrefusalevaluationtaskvectorssubspaceSkillSafe-BenchSubSafe-Merge
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the standard measure of safety for skill-merged language models — static refusal tests, which present fixed harmful prompts and check for compliance — systematically overstates how safe a merge is. Because refusal is shallow, concentrated in the first few generated tokens, a merged model can keep its static refusal clean while an adaptive attacker that searches over inputs still breaks it. Using a new controlled benchmark, SkillSafe-Bench, the authors show across six open-weight bases (five model families) that static safety does not predict robustness to attack: safe-looking merges on Qwen (7B and 14B) and Gemma are jailbroken 60–76% of the time by a fixed template attack, while Llama and Phi-4 merges at indistinguishable static safety stay attack-resistant. The paper also identifies a data-free geometric signal — the overlap of a task vector with the safety subspace $S$ estimated from an abliterated model — that separates eroding refusal-removal skills from genuine skills before any merge, and sketches SubSafe-Merge, which projects that overlap away to restore safety at held capability. A reader should care because merging is the default way open-weight models gain new skills, and the models that most need adaptive checking are exactly the ones that look safe under current static screening.

What carries the argument

The load-bearing object is the safety subspace $S$. For each base, the authors form the alignment vector $\tau_{\mathrm{safe}} = \theta_{\mathrm{base}} - \theta_{\mathrm{abliterated}}$, the weight difference between the aligned base and a public abliterated version of it (refusal direction removed), and define $S^{(\ell)}$, per layer, as the span of the top-$k$ left singular vectors of $\tau_{\mathrm{safe}}^{(\ell)}$; projection onto $S$ and its complement are $P_S$ and $P_{S^\perp}$. Two things are done with it. First, a data-free pre-merge screen: the subspace overlap of a candidate task vector $\tau_i$, the energy fraction of $\tau_i$ inside $S$, acts as a binary detector of the merge type that erodes safety, with same-recipe abliteration-style refusal-removal vectors at $\approx 0.99$ and genuine skills at $\approx 0.001$. Second, a repair: SubSafe-Merge computes $\theta_{\mathrm{merge}} = \theta_{\mathrm{base}} + \sum_i \lambda_i P_{S^\perp} f(\tau_i)$, projecting each method-processed task vector off $S$ so the eroding component is removed before the merge while the orthogonal capability component is retained. The construction is validated by a control (removing the top refusal direction from Qwen raises static ASR from 0.20 to 0.64 while GSM8K and MMLU hold), and its scope is bounded by the estimator: $S$ comes from one public abliteration per base, and the paper's own survey of six public uncensored checkpoints shows overlap is recipe-dependent, with genuinely dangerous SFT-decensored donors sitting at the 0.001 skill baseline.

What would settle it

Take the paper's six bases and push each adaptive attack well beyond the reported budgets: a GCG run of 1,000+ steps with a wider suffix search, a GPT-4-class attacker in the PAIR loop, and the same best-of-6 template set, then rank the bases by adaptive ASR. If Llama-3.1-8B and Phi-4-mini, the paper's attack-resistant bases, break at rates comparable to Qwen and Gemma under any of these stronger probes, the fragile/robust ordering is an artifact of attack strength; if the ordering survives at every budget, the conclusion that static refusal screening cannot tell which merges are safe to deploy is confirmed.

Watch

Extended reading notes

Core claim

The paper's central claim is that static refusal screening is not a proxy for the safety of skill-merged models under real attack. On three bases run over a full grid of merging method × skill × coefficient, two strongly-aligned bases that look identically safe under fixed prompts — Qwen2.5-7B (static ASR 0.24 on the evaluation subset) and Llama-3.1-8B (0.20) — separate sharply under GCG: Qwen's benign math merges rise from 0.10–0.24 static to 0.28–0.38 adaptive (pooled gap +17pp, 95% CI [0.09, 0.25], paired McNemar $p<0.01$), while Llama's merges stay at or below 0.12. The fragile/robust ordering replicates under a semantic template attack (Qwen 0.66–0.76 vs Llama 0.22–0.24), under PAIR, under a 500-step GCG budget, and with a single judge dropped, and it extends to six bases: Qwen2.5-14B and Gemma-2-9B are fragile (Gemma, the safest base statically at 0.02, is jailbroken 60% of the time under the template attack), while Phi-4-mini is robust. The static effect of merging is itself base-conditional: on strongly-aligned bases a benign math skill lowers static ASR while an uncensored skill raises it, but on weakly-aligned Mistral any merge raises it. Complementing the benchmark, the paper shows that the energy fraction of a task vector inside the safety subspace $S$ (estimated from an abliterated model, no harmful data) separates same-recipe refusal-removal vectors (overlap $\approx 0.99$) from genuine skills including an independently-trained code specialist ($\approx 0.001$), validated out-of-sample; and that projecting that overlap off (SubSafe-Merge) restores both static and adaptive ASR to base on strongly-aligned bases at unchanged GSM8K, while honestly failing on out-of-$S$ erosion and on bases that are fragile before merging.

Load-bearing premise

The safety subspace $S$, estimated from a single public abliterated model per base, faithfully represents the directions through which merging erodes safety; the paper's own survey shows the estimate is recipe-dependent, so a donor that removes refusal by SFT/DPO decensoring rather than abliteration is invisible to the geometric screen (overlap 0.001 despite unmerged static ASR 0.63) and is not repaired by SubSafe-Merge.

Editorial extensions

If this is right

  • A low static refusal score on a merged model carries no information about its adaptive robustness: safe-looking Qwen and Gemma merges are jailbroken 60–76% of the time by a single fixed template attack, so a practitioner who screens only statically will deploy the very models that most need hardening.
  • Adaptive evaluation is necessary, not optional, for merged models: the shortfall of static screening is largest for strongly-aligned, static-safe merges, precisely the ones a practitioner would most trust.
  • The safety cost of merging is base-conditional rather than algorithm-conditional: all four merging methods agree to within 0.035 static ASR with identical ordering, while a weakly-aligned base (Mistral) is eroded by even a benign math merge and fragile bases (Gemma) inherit fragility from the unmerged base itself.
  • A data-free geometric screen can be applied before any merge or attack: the overlap of the candidate task vector with the safety subspace $S$ separates same-recipe refusal-removal from genuine skills, and projecting that overlap off restores safety to base at held capability on strongly-aligned bases.
  • The repair is honestly bounded: SubSafe-Merge removes only in-$S$ erosion, cannot fix out-of-$S$ benign erosion (an Alpaca fine-tune still raises static ASR from 0.20 to 0.28), and cannot make a model more adaptively robust than its base (Mistral's adaptive ASR remains $\approx 0.66$).

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If shallow alignment is the mechanism, the decoupling should generalize beyond merging to other weight-space edits that leave the first-token refusal layer intact (LoRA fine-tuning, pruning, quantization); a testable prediction is that such edits reproduce the Qwen-like static–adaptive gap and spare Llama-like bases.
  • The overlap screen is best read as a lineage detector, not a general safety test: the paper's six-checkpoint survey places genuinely uncensored SFT/DPO donors at the 0.001 skill baseline and a dangerous independent abliteration at 0.32, so a practitioner adopting the screen should first verify that their donor's refusal-removal recipe matches the recipe used to build $S$.
  • Because static repair scales with overlap (none at 0.001, partial at 0.32, full at 0.99), a natural extension is to calibrate an acceptance threshold on overlap plus unmerged donor static ASR to turn the binary detector into a graded pre-merge risk score; the paper's leave-one-base-out regression at three bases found no significant predictor, so that calibration awaits more bases and skills.
  • The multi-turn Crescendo arms show merging does not push adaptive risk beyond the base's own ceiling, which suggests the binding constraint for a merged model is the base's intrinsic robustness; a cheaper mitigation than post-merge repair would be to select robust bases (Llama, Phi-4) at the start.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. This paper argues that static refusal tests misrepresent the safety of skill-merged LLMs and introduces SkillSafe-Bench, a factorial protocol (base model x skill x merging method x merge coefficient) that jointly reports static refusal (400 HarmBench behaviors, two-judge AND rule), adaptive jailbreak success (GCG, best-of-6 semantic templates, PAIR, and a multi-turn Crescendo arm), and capability retention. The central empirical claim (Contribution B) is that static safety does not predict adaptive robustness: across six open-weight bases spanning five families, statically safe merges on Qwen2.5-7B/14B and Gemma-2-9B are attacked at 60-76% under the template probe while Llama-3.1-8B and Phi-4-mini remain robust, and on the three main bases the Qwen-fragile/Llama-robust ordering replicates under GCG at 100 and 500 steps, templates, and PAIR on a matched 50-behavior subset with paired McNemar tests and cluster-bootstrap CIs. Contribution C is a data-free geometric signal (the overlap of a task vector with a per-layer safety subspace S estimated from a public abliterated checkpoint), which separates a same-recipe uncensored donor (overlap 0.99) from genuine skills (overlap 0.001), plus SubSafe-Merge, a projection-based repair. The paper pre-registers its decision rule, discloses a principled deviation and a data correction, and explicitly bounds the attack-relativity and recipe-dependence of its findings.

Significance. If the central claim holds, it reframes merging-safety evaluation: a low static refusal score carries little information about vulnerability to attack, and the statically cleanest bases (e.g., Gemma-2-9B, static ASR 0.02, template ASR 0.60-0.64) can be among the most exposed. The methodological assets are real: paired per-behavior statistics on identical behavior subsets, three attack families plus a multi-turn arm on the main bases, a human-labeled judge audit (AND rule kappa = 0.66) with the fragile/robust ordering shown to be judge-invariant, a 500-step GCG control against under-optimization, an out-of-sample code-LoRA prediction validating the low-overlap/no-erosion direction of the geometric signal, a disclosed pre-registration deviation and corrected earlier subset, and a code/data supplement in which every numeric table is generated from backing result files. The provisional part is the geometric screen and SubSafe-Merge: the authors' own Table S6 shows the safety-subspace estimator is recipe-dependent (overlap ranges 0.99 down to 0.001 across six public checkpoints, five of them dangerous), and the repair's high-overlap direction is demonstrated in-sample.

major comments (2)
  1. [Abstract and Section 5.2 (Table S2)] The six-base fragile/robust taxonomy is a single-probe result whose qualifier appears only later in the text: under the template probe, the Gemma math-merge template ASR is 0.60 while its GCG ASR is 0.04, and Phi-4 shows the reverse pattern (template 0.14, GCG 0.32), so the labels 'fragile' and 'robust' would invert under a different choice of attack. The body is explicit that GCG ordering of the extension bases is not claimed and that multi-attack invariance holds only for the three main bases, but the abstract's sentence 'Across six open-weight bases ... static safety does not predict robustness to attack' does not carry this scope. I recommend adding one sentence to the abstract or conclusion stating that the six-base fragile/robust labeling is relative to the template probe, and that the multi-attack invariance is established for the three main bases only.
  2. [Sections 6-7, Tables 2 and S6] The manuscript discloses the same-recipe scope of the geometric screen and SubSafe-Merge, but two claims read more strongly than the evidence supports. First, Section 6 opens by calling the subspace overlap 'the feature that cleanly separates skills,' whereas Table S6 shows five of six public refusal-removal checkpoints are dangerous (unmerged static ASR 0.42-0.68) yet have overlap at or near the genuine-skill baseline (0.001-0.003), so the feature separates one lineage rather than refusal-removal in general. Second, the out-of-sample validation (the independent code LoRA) confirms only the low-overlap/no-erosion direction; the high-overlap/repair direction is demonstrated in-sample with the same-recipe donor, which is partly by construction since S is estimated from that donor's abliteration lineage. I recommend that the contribution summary and Section 7 state explicitly that the screen is a same-recipe detector, that it misses SFT/DPO-decensored donors (a documented false negative, Orion, overlap 0.001, static 0.63), and that the repair direction is not out-of-sample validated.
minor comments (5)
  1. [Section S5 vs Table 1] The discordant-pair counts reported in Section S5 for the Qwen +math lambda=0.6 cell (b=8, c=0) do not add up to the marginals in Table 1: 5/50 static-unsafe plus 8 new flips gives 13/50 GCG-unsafe, while Table 1 reports 14/50 (Wilson CI [0.17, 0.42]). Please reconcile the count; the conclusion is unaffected since b=9 with c=0 still gives McNemar p < 0.01.
  2. [Section 5] The claim that 'the four methods agree to within 0.035 with identical ordering' should state the aggregation (range across methods per skill and coefficient, or across the whole grid) and the exact quantity compared, since Table S1 shows the four methods varying by up to 0.033 at some cells (e.g., Qwen math lambda=0.2, DARE-TIES 0.147 versus TIES 0.180).
  3. [Section 4 and footnote 1] Because the six-base replication and the abstract's headline numbers rest on the best-of-6 template ensemble, which the manuscript itself classifies as non-adaptive, the term 'adaptive' should be defined at its first appearance (abstract or introduction) so that readers do not attribute adaptiveness to the template probe.
  4. [Section 7 and Table S8 (Crescendo)] The Crescendo arm compares base (0.20), plain merge (0.14), and SubSafe (0.16) at n=50 with a deliberately weakened attacker, and the differences are within run noise; the claim that 'neither merge adds multi-turn risk' should state explicitly that no significance test is applied to these three arms.
  5. [Table S3] The geometry measures used to document Limitation 5 vary by only a few percent across bases (participation ratio 1.013-1.020, spectral entropy 1.089-1.143); consider reporting additional precision or uncertainty so the reader can distinguish a genuine null from a resolution-limited comparison.

Circularity Check

2 steps flagged · score 4.0 of 10

The static–adaptive decoupling is independently measured and not circular; the safety-subspace screen and the same-recipe SubSafe-Merge repair contain by-construction elements that the paper itself discloses and bounds.

  1. self definitional [Section 6, Table 2 caption and surrounding text.]
    "The refusal-removed (uncensored) task vector lies almost entirely inside S (overlap ≈0.99), partly by construction, since it is the refusal-removal direction that spans S (Section 3)."

    S is defined in Section 3 as the span of top-k left singular vectors of τ_safe = θ_base − θ_abliterated. A same-recipe abliterated donor has task vector θ_donor − θ_base = θ_abliterated − θ_base = −τ_safe, so its per-layer update lies in S by definition and the overlap is ≈1 by construction. The claimed separation of same-recipe abliteration-style eroding vectors from genuine skills is therefore not an empirical discovery for that class; it is the definition of S. The genuinely non-circular content in the same table is the independent out-of-sample result that math, code, medicine, finance, and law task vectors are near-orthogonal to S, which holds regardless of the definitional overlap of the uncensored vector.

  2. self definitional [Section 7, Eq. (2) and the paragraph 'A boundary by construction'.]
    "SubSafe-Merge computes θmerge = θbase + Σ_i λ_i P_S⊥(f(τ_i)) ... It removes in-S (refusal-removal) erosion by construction, not the S-external kind."

    For the same-recipe uncensored donor under task arithmetic, f is the identity and τ_i = θ_abliterated − θ_base = −τ_safe, which lies entirely in S. Therefore P_S⊥(τ_i) = 0 and Eq. (2) yields θmerge = θ_base exactly. The Table 3 'recovery to base' for Qwen and Llama is thus the unmerged base model, not an empirically induced repair; the reported static ASR values (0.182 vs 0.195, 0.180 vs 0.170) are the base model's own numbers within noise. The paper states the mechanism is by construction and honestly bounds it with out-of-S donors such as Orion and Alpaca, but the same-recipe repair claim still reduces to the definition of the projection rather than to a measured effect.

full rationale

The central claim of the paper, that static refusal does not predict adaptive jailbreak robustness, is a direct empirical measurement and is not derived from the safety-subspace construction. It is supported by paired per-behavior statistics on a fixed 50-behavior subset, by replication across GCG, template, and PAIR attack families on the main bases, by a 500-step GCG check, and by judge-invariance analysis; the six-base fragile/robust ordering is explicitly scoped to the template probe, with GCG under-transfer disclosed. None of that evidence depends on S or on SubSafe-Merge. The circular elements are confined to the geometric-screen contribution and its downstream prototype: the overlap of a same-recipe abliterated task vector with S is close to 1 by definition because S is built from that abliteration direction, and SubSafe-Merge's projection off S annihilates that same vector by construction, leaving the base model unchanged. The paper is unusually transparent about both points, saying 'partly by construction' for the overlap and 'by construction' for the erosion removal, and it supplies genuine negative controls (Alpaca, Orion, Josiefied v1/v2) showing that the screen fails outside its own recipe. Because the paper does not present the same-recipe overlap or repair as an out-of-sample prediction, and because the headline decoupling is independently measured, the score is 4 rather than 6 or higher: partial, bounded, disclosed circularity in the repair tool, with the central claim intact.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The empirical decoupling claim rests on standard benchmark and attack assumptions. The geometric contribution and SubSafe-Merge additionally rest on the safety subspace estimator S, which the paper itself shows is recipe-dependent (Table S6); this is the most fragile premise of the geometric parts of the paper, while the central static-adaptive claim is independent of it.

free parameters (3)
  • Safety subspace rank k = k=8 (main), k=1 (control)
    Hyperparameter defining the top singular vectors of tau_safe; SubSafe-Merge and the overlap feature depend on it. The paper reports k-insensitivity on the same-recipe cell but does not fully explore it across bases.
  • Adaptive attack budgets = GCG 100/500 steps, PAIR query budget, Crescendo 10 turns/10 backtracks, best-of-6 templates
    Fixed compute budgets for each attack family; absolute ASR is a lower bound and the cross-family GCG ordering is explicitly uninformative, so budgets shape the measured magnitudes.
  • Seeded random 50-behavior subset = fixed seed (code/scripts/run_gcg_subset.sh)
    Static and adaptive are paired on the same subset; the subset's composition (13 copyright, 8 misinformation, etc.) is determined by the seed, and category mix can affect measured ASR.
assumptions (5)
  • domain assumption Safety alignment is shallow: refusal is concentrated in the first few generated tokens.
    Motivates why static refusal can be decoupled from adaptive robustness; cited to Qi et al. (2025) and Wei et al. (2024), assumed rather than proven in this paper.
  • ad hoc to paper A public abliterated model provides a valid minimally unaligned counterpart, and tau_safe = theta_base - theta_abliterated spans the safety subspace S.
    The geometric signal and SubSafe-Merge are built on this estimator. The paper's own survey shows S is recipe-dependent (Table S6), so this is the load-bearing premise for the geometric contributions.
  • domain assumption HarmBench behaviors and the seeded random subset are a representative probe of harmful use.
    The benchmark reuses public behavior sets without reweighting (S6); representativeness is inherited from prior work.
  • domain assumption The two-judge AND rule is a conservative and valid safety metric.
    Validated on a 100-item audit (kappa=0.66); Llama Guard alone is weaker (kappa=0.42), but key orderings survive with the HarmBench classifier alone (Section 4).
  • domain assumption GCG, PAIR, Crescendo, and the template ensemble, at fixed budgets, are adequate adaptive adversaries.
    The paper treats measured ASR as a lower bound and notes stronger attackers could raise rates (Limitation 2).

how reviews work

0 comments
Cite this review

Pith. "Pith review of When Skills Meet Safety: Benchmarking and Characterizing the Adaptive Jailbreak Robustness of Skill-Merged LLMs." pith.science (2026). https://pith.science/paper/YEO77KHK

@misc{pith2026260808542,
  author       = {Pith},
  title        = {Pith review of: When Skills Meet Safety: Benchmarking and Characterizing the Adaptive Jailbreak Robustness of Skill-Merged LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YEO77KHK}},
  note         = {Machine review of arXiv:2608.08542}
}
read the original abstract

Model merging has become the default way to give an aligned language model new skills without retraining: a practitioner folds task vectors from math, code, or domain specialists into a safety-aligned base using task arithmetic, TIES, or DARE. This convenience is known to carry a safety cost, but almost all of that evidence rests on static refusal tests: fixed harmful prompts scored for compliance. We argue this is misleading. Because safety alignment is "shallow," concentrated in the first few generated tokens, a merged model's static refusal can stay clean while a real adaptive attack still breaks it. We introduce SkillSafe-Bench, a controlled benchmark that scores skill-merged models on static refusal, adaptive jailbreak robustness, and capability retention under a conservative two-judge AND rule. Across six open-weight bases (five families, two scales), static safety does not predict robustness to attack: under a semantic template attack, safe-looking merges on the fragile bases (both Qwen scales and Gemma) are jailbroken 60-76% of the time while others (Llama, Phi-4) stay robust. We further show the static effect of merging is base-conditional, characterize same-recipe abliteration-style safety erosion through a data-free geometric signal (the overlap of a task vector with a safety subspace), and outline SubSafe-Merge, which projects this overlap away to remove that erosion at held capability. Adaptive evaluation is not optional for merged LLMs: the models that most need it look safe under static screening.

Figures

Figures reproduced from arXiv: 2608.08542 by the authors.

Figure 1
Figure 1. Overview. (1) A practitioner merges a benign skill vector or a (possibly unlabeled) safety-eroding vector into an aligned base with a standard method. (2) Our central finding: static refusal screening can rate the merge “safe” while an adaptive attack (e.g. GCG) exposes high vulnerability: static safety does not predict adaptive robustness. (3) A data-free geometric signal (the overlap of a task vector with the safe… view at source ↗
Figure 2
Figure 2. (a) Method-averaged static ASR vs. merge coefficient λ (400 behaviors, AND rule) for three bases (solid = Qwen, dash-dot = Llama, dashed = Mistral): on the two strongly-aligned bases (Qwen, Llama) a benign math skill dips below base while an uncensored skill rises, but on the weakly-aligned Mistral both skills rise above base: the skill effect is base-conditional. (b) Adaptive (GCG) vs. static ASR on a seeded random… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

39 extracted references · 4 canonical work pages

  1. [1]

    SafeMERGE head-to-head (main paper§7).We compare against the selective-layer baseline Safe- MERGE (Djuhera et al

    and the plain Josiefied v1 merge (0.28); we read this as the same largely-artifactual effect that lowers their static ASR (mostly copyright; main paper§5) rather than a separate phenomenon, within the seeded-50 run noise, and by the same token do not over-read the small SubSafe-minus-plain GCG differences (Josiefied v10.28and0.36, v20.50and0.54) atn=50. S...

    work page

  2. [3]

    InAnnual Meeting of the Association for Computational Linguistics (ACL), 14138–14149

    Language Models are Homer Simpson! Safety Re-Alignment of Fine-tuned Language Models through Task Arithmetic. InAnnual Meeting of the Association for Computational Linguistics (ACL), 14138–14149. ArXiv:2402.11746. Chao, P.; Debenedetti, E.; Robey, A.; Andriushchenko, M.; Croce, F.; Sehwag, V.; Dobriban, E.; Flammarion, N.; Pappas, G. J.; Tram `er, F.; Has...

    arXiv

  3. [4]

    InAdvances in Neural Information Processing Systems (NeurIPS)

    JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models. InAdvances in Neural Information Processing Systems (NeurIPS). ArXiv:2404.01318. Chao, P.; Robey, A.; Dobriban, E.; Hassani, H.; Pappas, G. J.; and Wong, E

    arXiv

  4. [7]

    11 Ding, S

    Training Verifiers to Solve Math Word Problems.arXiv preprint arXiv:2110.14168. 11 Ding, S

    arXiv

  5. [9]

    InFindings of the Association for Computational Linguistics (ACL)

    SafeMERGE: Preserving Safety Alignment in Fine-Tuned Large Language Models via Selective Layer-Wise Model Merging. InFindings of the Association for Computational Linguistics (ACL). ArXiv:2503.17239. Farn, H.; Su, H.; Kumar, S. H.; Sahay, S.; Chen, S.-T.; and Lee, H.-y

    arXiv

  6. [10]

    InFindings of the Association for Computational Linguis- tics (EMNLP), 16589–16602

    Safeguard Fine-Tuned LLMs Through Pre- and Post-Tuning Model Merging. InFindings of the Association for Computational Linguis- tics (EMNLP), 16589–16602. ArXiv:2412.19512. Gallego, V

    arXiv

  7. [11]

    InICML Workshop on Foundation Models in the Wild

    Merging Improves Self-Critique Against Jailbreak Attacks. InICML Workshop on Foundation Models in the Wild. ArXiv:2406.07188. Hammoud, H. A. A. K.; Michieli, U.; Pizzati, F.; Torr, P.; Bibi, A.; Ghanem, B.; and Ozay, M

    arXiv

  8. [12]

    InFindings of the Association for Computational Linguistics (EMNLP), 13033–13046

    Model Merging and Safety Alignment: One Bad Model Spoils the Bunch. InFindings of the Association for Computational Linguistics (EMNLP), 13033–13046. ArXiv:2406.14563. He, Y.; Zeng, S.; Hu, Y.; Yang, R.; Zhang, T.; and Zhao, H

    arXiv

Show all 39 references
  1. [13]

    InAdvances in Neural Information Processing Systems (NeurIPS)

    MergeBench: A Benchmark for Merg- ing Domain-Specialized LLMs. InAdvances in Neural Information Processing Systems (NeurIPS). ArXiv:2505.10833. Hendrycks, D.; Burns, C.; Basart, S.; Zou, A.; Mazeika, M.; Song, D.; and Steinhardt, J

  2. [14]

    InInternational Conference on Learning Representations (ICLR)

    Measuring Massive Multitask Language Understanding. InInternational Conference on Learning Representations (ICLR). ArXiv:2009.03300. Hsu, C.-Y.; Tsai, Y.-L.; Lin, C.-H.; Chen, P.-Y.; Yu, C.-M.; and Huang, C.-Y

    2009 arXiv

  3. [15]

    InAdvances in Neural Information Processing Systems (NeurIPS)

    Safe LoRA: The Silver Lining of Reducing Safety Risks when Finetuning Large Language Models. InAdvances in Neural Information Processing Systems (NeurIPS). ArXiv:2405.16833. Ilharco, G.; Ribeiro, M. T.; Wortsman, M.; Schmidt, L.; Hajishirzi, H.; and Farhadi, A

    arXiv

  4. [16]

    InInternational Conference on Learning Representations (ICLR)

    Edit- ing Models with Task Arithmetic. InInternational Conference on Learning Representations (ICLR). ArXiv:2212.04089. Inan, H.; Upasani, K.; Chi, J.; Rungta, R.; Iyer, K.; Mao, Y.; Tontchev, M.; Hu, Q.; Fuller, B.; Testuggine, D.; and Khabsa, M

    arXiv

  5. [17]

    arXiv preprint arXiv:2312.06674

    Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations. arXiv preprint arXiv:2312.06674. Lermen, S.; Rogers-Smith, C.; and Ladish, J

    arXiv

  6. [18]

    Li, J.; Zhang, Z.; Zhou, S.; Li, Y.; Yu, T.; and Wang, K

    LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70B.arXiv preprint arXiv:2310.20624. Li, J.; Zhang, Z.; Zhou, S.; Li, Y.; Yu, T.; and Wang, K

    arXiv

  7. [19]

    Ma, Q.; Liu, D.; Chen, Q.; Zhang, L.; and Shao, J

    When Safe Models Merge into Danger: Exploiting Latent Vulnerabilities in LLM Fusion.arXiv preprint arXiv:2604.00627. Ma, Q.; Liu, D.; Chen, Q.; Zhang, L.; and Shao, J

  8. [20]

    InAnnual Meeting of the Association for Computational Linguistics (ACL), 21749–21767

    LED-Merging: Mitigating Safety-Utility Conflicts in Model Merging with Location-Election-Disjoint. InAnnual Meeting of the Association for Computational Linguistics (ACL), 21749–21767. ArXiv:2502.16770. Mazeika, M.; Phan, L.; Yin, X.; Zou, A.; Wang, Z.; Mu, N.; Sakhaee, E.; Li...

    arXiv

  9. [21]

    InInternational Conference on Machine Learning (ICML), 35181–35224

    HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal. InInternational Conference on Machine Learning (ICML), 35181–35224. ArXiv:2402.04249. 12 Qi, X.; Panda, A.; Lyu, K.; Ma, X.; Roy, S.; Beirami, A.; Mittal, P.; and Henderson, P

    arXiv

  10. [22]

    InInternational Conference on Learning Representations (ICLR)

    Safety Alignment Should Be Made More Than Just a Few Tokens Deep. InInternational Conference on Learning Representations (ICLR). ArXiv:2406.05946. Qi, X.; Zeng, Y.; Xie, T.; Chen, P.-Y.; Jia, R.; Mittal, P.; and Henderson, P

    arXiv

  11. [23]

    ArXiv:2310.03693

    Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! InInternational Conference on Learning Representations (ICLR). ArXiv:2310.03693. Russinovich, M.; Salem, A.; and Eldan, R

    arXiv

  12. [24]

    InUSENIX Security Symposium, 2421–2440

    Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attack. InUSENIX Security Symposium, 2421–2440. ArXiv:2404.01833. Song, M.; and Zheng, M

    arXiv

  13. [25]

    Thakkar, M.; Fournier, Q.; Riemer, M.; Chen, P.-Y.; Zouaq, A.; Das, P.; and Chandar, S

    Model Merging in the Era of Large Language Models: Methods, Applications, and Future Directions.arXiv preprint arXiv:2603.09938. Thakkar, M.; Fournier, Q.; Riemer, M.; Chen, P.-Y.; Zouaq, A.; Das, P.; and Chandar, S

  14. [26]

    InAnnual Meeting of the Association for Computational Linguistics (ACL), Short Papers, 268–277

    Combining Domain and Alignment Vectors Provides Better Knowledge-Safety Trade-offs in LLMs. InAnnual Meeting of the Association for Computational Linguistics (ACL), Short Papers, 268–277. ArXiv:2411.06824. Tramer, F.; Carlini, N.; Brendel, W.; and Madry, A

    arXiv

  15. [28]

    InInternational Conference on Machine Learning (ICML), 52588–52610

    Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications. InInternational Conference on Machine Learning (ICML), 52588–52610. ArXiv:2402.05162. Wortsman, M.; Ilharco, G.; Gadre, S. Y.; Roelofs, R.; Gontijo-Lopes, R.; Morcos, A. S.; Namkoong, H.; Far...

    arXiv

  16. [30]

    InIEEE Symposium on Security and Privacy (S&P)

    EnchTable: Unified Safety Alignment Transfer in Fine-tuned Large Language Models. InIEEE Symposium on Security and Privacy (S&P). ArXiv:2511.09880. Xu, Z.; Liu, F.; and Liu, H

  17. [31]

    InAdvances in Neural Information Processing Systems (NeurIPS)

    Bag of Tricks: Benchmarking of Jailbreak Attacks on LLMs. InAdvances in Neural Information Processing Systems (NeurIPS). ArXiv:2406.09324. Yadav, P.; Tam, D.; Choshen, L.; Raffel, C.; and Bansal, M

    arXiv

  18. [32]

    InAdvances in Neural Information Processing Systems (NeurIPS)

    TIES-Merging: Resolving Inter- ference When Merging Models. InAdvances in Neural Information Processing Systems (NeurIPS). ArXiv:2306.01708. Yang, J.; Tang, A.; Zhu, D.; Chen, Z.; Shen, L.; and Wu, F

    arXiv

  19. [33]

    InInternational Conference on Learning Representations (ICLR)

    Mitigating the Backdoor Effect for Multi-Task Model Merging via Safety-Aware Subspace. InInternational Conference on Learning Representations (ICLR). ArXiv:2410.13910. Yu, L.; Yu, B.; Yu, H.; Huang, F.; and Li, Y

    arXiv

  20. [34]

    InInternational Conference on Machine Learning (ICML)

    Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch. InInternational Conference on Machine Learning (ICML). ArXiv:2311.03099. Zeng, X.; Liang, S.; Lu, L.; Zhu, H.; Liu, E.; Dang, J.; Zhou, Y.; and Pang, S

    arXiv

  21. [35]

    13 Zhang, J.; He, Y.; Cai, K.; Zhao, H.; Suya, F.; and Tian, Y

    SafeSteer: Adap- tive Subspace Steering for Efficient Jailbreak Defense in Vision-Language Models.arXiv preprint arXiv:2509.21400. 13 Zhang, J.; He, Y.; Cai, K.; Zhao, H.; Suya, F.; and Tian, Y

  22. [36]

    Zhong, Z.; and Raghunathan, A

    RogueMerge: Robust and Unified Attacks against LLM Model Merging.arXiv preprint arXiv:2606.03344. Zhong, Z.; and Raghunathan, A

    arXiv

  23. [37]

    Zou, A.; Wang, Z.; Carlini, N.; Nasr, M.; Kolter, J

    Instruction-Following Evaluation for Large Language Models.arXiv preprint arXiv:2311.07911. Zou, A.; Wang, Z.; Carlini, N.; Nasr, M.; Kolter, J. Z.; and Fredrikson, M

    arXiv

  24. [38]

    Universal and Transferable Adversarial Attacks on Aligned Language Models.arXiv preprint arXiv:2307.15043. 14 This appendix collects material referenced from the main text: the full staticλ-grid (§S1), adaptive-attack configurations including the template set (§S2), the cross-...

    arXiv

  25. [2020]

    InAdvances in Neural Information Processing Systems (NeurIPS)

    On Adaptive Attacks to Adversarial Example Defenses. InAdvances in Neural Information Processing Systems (NeurIPS). ArXiv:2002.08347. Wei, B.; Huang, K.; Huang, Y.; Xie, T.; Qi, X.; Xia, M.; Mittal, P.; Wang, M.; and Henderson, P

    2002 arXiv

  26. [2021]

    Cobbe, K.; Kosaraju, V.; Bavarian, M.; Chen, M.; Jun, H.; Kaiser, L.; Plappert, M.; Tworek, J.; Hilton, J.; Nakano, R.; Hesse, C.; and Schulman, J

    Evaluating Large Language Models Trained on Code.arXiv preprint arXiv:2107.03374. Cobbe, K.; Kosaraju, V.; Bavarian, M.; Chen, M.; Jun, H.; Kaiser, L.; Plappert, M.; Tworek, J.; Hilton, J.; Nakano, R.; Hesse, C.; and Schulman, J

    arXiv

  27. [2022]

    InInternational Conference on Machine Learning (ICML)

    Model Soups: Averaging Weights of Multiple Fine-tuned Models Improves Accuracy Without Increasing Inference Time. InInternational Conference on Machine Learning (ICML). ArXiv:2203.05482. Wu, J.; Li, K.; Huang, Z.; Li, X.; Wang, X.; and Hong, C

    arXiv

  28. [2023]

    InAdvances in Neural Information Processing Systems (NeurIPS)

    Jailbreaking Black Box Large Language Models in Twenty Queries. InAdvances in Neural Information Processing Systems (NeurIPS). ArXiv:2310.08419. Chen, M.; et al

    arXiv

  29. [2024]

    InAdvances in Neural Information Processing Systems (NeurIPS)

    Refusal in Language Models Is Mediated by a Single Direction. InAdvances in Neural Information Processing Systems (NeurIPS). ArXiv:2406.11717. Bhardwaj, R.; Do, D. A.; and Poria, S

    arXiv

  30. [2025]

    InInternational Conference on Learning Representations (ICLR)

    Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks. InInternational Conference on Learning Representations (ICLR). ArXiv:2404.02151. Arditi, A.; Obeso, O.; Syed, A.; Paleka, D.; Panickssery, N.; Gurnee, W.; and Nanda, N

    arXiv

  31. [2026]

    Djuhera, A.; Kadhe, S

    Colluding LoRA: A Compositional Vulnerability in LLM Safety Alignment.arXiv preprint arXiv:2603.12681. Djuhera, A.; Kadhe, S. R.; Ahmed, F.; Zawad, S.; and Boche, H

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.