{"id":"4277b26c-0ef5-4193-a09b-765bc9e55a3e","arxiv_id":"2504.21307","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Attack token embeddings learned as non-negative sums of CLIP vocabulary tokens can jailbreak unlearned diffusion models, and projecting those embeddings out of the vocabulary reduces attack success while preserving image quality.","lead":"Researchers propose SubAttack, a jailbreak that represents a forbidden concept as a combination of existing text-token embeddings, and SubDefense, which projects those embeddings out to protect unlearned image models. The work offers a tool to expose and partially fix residual harmful knowledge in fine-tuned diffusion models.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The core 'implicit textual components' claim lacks causal evidence: SubAttack and SubDefense operate on the full learned subspace, and the paper never shows that the displayed tokens (e.g., 'slave', 'nip') actually drive attack success.","rationale":"The paper's strongest empirical content is credible: SubAttack is a global, transferable attack, and SubDefense reduces ASR across several attacks while maintaining FID/CLIP. The orthogonalization via deflation (Algorithm 1) is standard, and the non-negative parameterization is a reasonable route to interpretable coefficients. However, the headline conceptual contribution—that unlearned models retain concepts through implicit textual components—depends on the assumption that the top-alpha tokens displayed in Figures 2-3 are the causal carriers of the attack. The paper never tests this. Section 3.1.1 learns v_att over the full vocabulary; Eq. (4) removes the whole learned subspace. The word-cloud analysis is post-hoc. Figure 13 shows that keeping roughly the top 1500 alpha entries preserves the attack, but that set is far larger than the 'slave/nip/babes' list and does not isolate implicit tokens. Without a sparse-ablation or token-removal experiment, the interpretability claim could be an epiphenomenon of a distributed direction that happens to have large weights on semantically suggestive tokens. Because SubDefense is motivated by this interpretability, the missing causal link is load-bearing. The shared-CLIP design (Section 4.1) also makes the transfer-to-SD result (Table 5) less informative, but the causal isolation issue is the more fundamental gap. A single ablation experiment—constructing v_att from only the top-k implicit tokens—would settle it. The reader's conditional verdict already asks for this; my read does not move the verdict, hence UNCHANGED.","tokens_in":24278,"tokens_out":11992,"duration_ms":125403,"concrete_test":"Run SubAttack on ESD for 'nudity' with v_att constructed from only the top-50 alpha-weighted tokens in Fig. 3b (e.g., 'slave', 'nip', 'babes'), with all other coefficients zeroed, and compare ASR on the same 900-prompt set against the full-vocabulary v_att and a top-50 random-token control. If the sparse implicit-token ASR is far below the full attack, the word-cloud tokens are not the causal carriers.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's interpretive centerpiece—that unlearned models retain the target concept via implicit textual components such as 'slave', 'nip', and 'babes' (Sec. 4.2, Fig. 3)—rests on a post-hoc word-cloud reading of the attack embedding, not on causal evidence. In Sec. 3.1.1, v_att is a non-negative combination over the entire ~40k-token CLIP vocabulary; the defense in Eq. (4) removes the full K-dimensional subspace V_att. Nothing in the experiments isolates the displayed implicit tokens from the many other alpha-weighted components. Figure 13 shows only that keeping roughly the top 1500 alpha entries preserves the nudity attack—a set far larger than the handful of tokens highlighted as 'implicit'—and even that is a sufficiency check, not a necessity check. Consequently, the ASR drops in Tables 3, 4, and 17-20 are equally consistent with the presence of a distributed, non-interpretable concept direction that merely has large weights on semantically suggestive vocabulary items. Since the paper's claimed insight and the motivation for SubDefense both depend on the implicit tokens being the causal carriers, this missing causal link is the most load-bearing weakness.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper addresses jailbreaking and defense for unlearned text-to-image diffusion models. The authors propose SubAttack, which learns K orthogonal token embeddings in the frozen CLIP token embedding space, each constrained to be a non-negative superposition of vocabulary embeddings. Replacing the target concept word with these learned tokens regenerates unlearned concepts (nudity, Van Gogh, church, etc.) across ESD, FMN, UCE, and SPM. The paper claims the learned embeddings are interpretable in a bag-of-words sense and that word-cloud inspection reveals unlearned models retain the target concept through implicit textual associations (e.g., 'slave', 'nip', 'babes'). It further proposes SubDefense, which projects every CLIP vocabulary embedding onto the null space of the learned attack embeddings, and reports reduced ASR against SubAttack, UnlearnDiff, and CCE while preserving FID/CLIP on COCO-10k. The appendices include transfer tables, ablations on K and on number of blocked tokens, and visualizations.","tokens_in":24503,"tokens_out":7744,"duration_ms":81074,"significance":"If the claims hold, SubAttack is a useful global, transferable attack for auditing concept erasure, and SubDefense is a simple plug-in defense. Strengths: the attack evaluation spans four unlearned models and six concepts; the transfer tables and I2P defense results are extensive; and the paper explicitly attempts to close the attack-defense loop. The paper does not ship code or data, and no error bars or significance tests are reported. The conceptual contribution—that residual vulnerability is carried by human-interpretable implicit tokens—is currently not causally established, and the defense is evaluated largely against the same embeddings used to construct it. The manuscript's value would be materially higher if those gaps were closed.","major_comments":[{"comment":"The ASR comparison is asymmetric in inference budget. For each prompt-seed pair, SubAttack generates K=5 images (one per attack embedding) and counts success if any of the five contains the concept, whereas CCE generates one image from a single learned embedding and UnlearnDiff is evaluated with its optimized prompt. The headline claim that SubAttack 'surpasses CCE in many circumstances' (Section 4.2) may therefore partly reflect the fivefold generation chance rather than a genuinely stronger attack. Please report per-embedding ASR and per-image hit rate, and provide a budget-matched comparison such as allowing CCE or a single SubAttack embedding to draw multiple seeds.","section":"4.1, C.1, Tables 1–2"},{"comment":"The central interpretability claim—that unlearned models retain the target concept through implicit textual components such as 'slave', 'nip', and 'babes'—lacks causal evidence. In Eq. (2), v_att is a non-negative combination over the entire CLIP vocabulary, and SubDefense in Eq. (4) removes the full learned subspace V_att. No experiment isolates the displayed implicit tokens by ablating only those entries of alpha and measuring the resulting ASR change; Figure 13 is a sufficiency check (top ~1500 entries preserve the attack), not a necessity check for the handful of tokens highlighted in Figure 3. The ASR reductions in Tables 3, 4, and 17–20 are equally consistent with removal of a distributed, non-interpretable concept direction, so the paper's interpretive centerpiece is not currently established.","section":"4.2, Fig. 3, Eq. (2), Fig. 13"},{"comment":"The defense evaluation is partly circular: SubDefense is constructed from exactly the attack embeddings it is then evaluated against. In Table 3, for UCE/nudity, SubAttack ASR drops from 81.67% to 34.11% after 20 blocked tokens, but those are the same tokens removed by Eq. (4). The independent evidence is weaker: UCE/nudity UnlearnDiff ASR drops only from 78.22% to 73.55% (Table 11), and on ESD, CCE drops from 85.11% to 75.67% (Table 4). To support the claim that the defense 'can defend unlearned models against various attacks,' the authors should evaluate against attacks re-optimized after the defense is applied, or hold out concepts and embeddings that were not used to build the projection.","section":"3.2, Tables 3–4"},{"comment":"The 'inherited from the original SD' transfer argument is near-tautological given the fixed text encoder. The CLIP text encoder and token embedding table are frozen and identical across SD v1.4 and all victim models, so any token embedding learned in that space is by construction a valid embedding for the original SD; the high transfer ASRs in Table 5 (84–98%) may simply reflect that the learned vector is a good language-side trigger for the shared text encoder, independent of UNet unlearning. This does not by itself establish that the unlearned UNet retains a concept subspace. A control with a model whose text encoder has been modified, or a comparison of the same embeddings' ASR before and after UNet fine-tuning, is needed.","section":"4.2, Table 5, D.1"}],"minor_comments":[{"comment":"No error bars, confidence intervals, or significance tests are reported for ASR, FID, or CLIP scores. Differences of a few points, such as the Nudity UnlearnDiff row in Table 3 (73.55% vs 76.44%), should be accompanied by variance estimates across seeds or dataset subsamples.","section":"All tables"},{"comment":"The relative reduction arithmetic is inconsistent; for example, Church UnlearnDiff in Table 11 goes from 61.67% to 39.78% but is labeled '-64.34%', whereas the relative reduction is about 35.5%. Please use percentage-point changes or relative percentages consistently and label them clearly.","section":"Tables 11–12"},{"comment":"Section 4.1 has two items labeled '(iv)', with the Baselines paragraph as the second; the second should be '(v)'.","section":"4.1"},{"comment":"No public code or dataset release is mentioned. Given the described dataset (900 prompt-seed pairs per main concept, six times larger than UnlearnDiff's) and the volume of reported numbers, releasing the constructed dataset and attack/defense code would substantially aid reproducibility.","section":"Throughout"},{"comment":"The paper acknowledges in Section F.2 that interpretability relies on the predefined CLIP vocabulary and may miss nuanced retained representations; this important limitation should be stated in the main-text discussion rather than only in an appendix.","section":"F.2, J"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is at the boundary between major revision and reject. I recommend major revision rather than reject because the attack itself is a novel and likely useful contribution, and the main gaps (budget-matched ASR comparisons, causal token ablations, held-out defense evaluation, and control analyses for transfer) are in principle addressable within the scope of the paper."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nRead arXiv:2504.21307. Short version: the attack is real and worth knowing about; the defense is simple and partly effective; the paper's headline interpretability claim should be read with skepticism, because it isn't causally supported.\n\nWhat's new: SubAttack learns global token embeddings as non-negative combinations of CLIP vocabulary, then uses orthogonal deflation to get a diverse subspace of attack vectors. That's a clean idea, and it works: high ASR across ESD, FMN, UCE, SPM for nudity, Van Gogh, church, garbage truck, parachute, and tench, with strong transfer across prompts, seeds, and models. The dataset is six times larger than UnlearnDiff's, which is a real plus. SubDefense—projecting the learned attack subspace out of the token embedding table—is a neat, plug-and-play defense. It cuts SubAttack's own ASR substantially and also reduces UnlearnDiff ASR, with modest FID/CLIP cost. The comparison against RECE is favorable.\n\nSoft spots. First, no code or data release, and no error bars anywhere. Every table is a single point estimate. That's addressable, but it limits how much weight I'd put on the exact numbers. Second, the defense is partly self-referential: the attack embeddings used to build SubDefense are the same ones used to evaluate it. Independent attacks give real but weaker evidence—UnlearnDiff drops, CCE barely moves, and the paper admits CCE is still a problem. Third, the interpretability story. The word clouds ('slave', 'nip', 'babes') are descriptive, not causal. Nothing shows that those specific tokens drive attack success; the defense removes the entire learned subspace, and the sparsity ablation in Fig 13 only shows that ~1500 alpha entries suffice to preserve the attack, which is far larger than the handful of highlighted words. So the claim that unlearned models retain the concept 'through implicit textual components' is not established by the experiments.\n\nThe transfer-to-SD evidence is also weaker than it looks: CLIP text encoder is frozen and shared, so embeddings trained on any of these models live in the same input space. Showing they work on SD means the embedding is in a region the original model responds to—unsurprising, and not the same as showing what unlearning failed to remove.\n\nBottom line: this is a solid empirical contribution from authors who know the literature and are honest about limitations. The attack is a useful baseline; the defense is a useful starting point. But the causal/interp retability centerpiece needs more work before I'd teach it as fact. I'd send to peer review, with a request for code, variance, and a causal ablation that isolates the claimed implicit tokens.","headline":"SubAttack is a solid transferable embedding attack and SubDefense a useful defense, but the paper's claim that unlearned models retain concepts via specific implicit tokens is not causally established.","tokens_in":25046,"tokens_out":3731,"would_cite":true,"duration_ms":40828,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Unlearned diffusion models still tie harmful concepts to hidden words","keywords":["diffusion model unlearning","jailbreaking attack","token embedding subspace","CLIP text encoder","orthogonal projection defense","interpretability","concept erasure","transferable attack"],"falsifier":"A direct disproof would be to retrain the CLIP text encoder or replace it (e.g., with a fine-tuned text encoder) while keeping the same unlearned UNet, and then check whether the SubAttack token embeddings learned on the original CLIP space still achieve high attack success rates on the unlearned model; if the attack success drops to near the no-attack baseline, then the residual vulnerability is not inherent to the token embedding space but an artifact of the frozen encoder. Alternatively, one could remove only the top-weighted interpretable tokens identified for a concept (e.g., 'slave', 'nip' for nudity) from the vocabulary and measure whether the attack success rate falls as much as when the entire learned subspace is projected out; if the ASR does not drop, then the interpretable tokens are not the causal carriers of the residual concept.","tokens_in":1895,"feed_emoji":"","tokens_out":2216,"duration_ms":34201,"temperature":0.7,"pith_summary":"This paper claims that current fine-tuning-based unlearning methods for text-to-image diffusion models do not actually erase the target concept; they only weaken its explicit associations while leaving a large subspace of implicit associations intact. The authors show that this residual subspace can be systematically uncovered by learning a set of orthogonal token embeddings—each a non-negative combination of existing CLIP token embeddings—that reliably regenerate the harmful concept, and that these embeddings are transferable across prompts, noises, and even across different unlearned models. Because the attack embeddings are interpretable, they reveal the specific implicit words (e.g., 'slave', 'nip', 'babes' for nudity) through which the unlearned model still 'remembers' the concept. Building on this, the paper proposes a defense that projects the entire token embedding vocabulary onto the null space of the learned attack subspace, which reduces attack success rates while preserving generation quality on safe prompts.","feed_headline":"Hidden words still trigger unlearned models","feed_subtitle":"A new attack exposes residual concepts in token embeddings, and a projection defense can block them.","key_machinery":"The central mechanism is the CLIP token embedding space of the frozen text encoder. The attack learns a token embedding $v_{\\text{att}} = \\sum_i \\alpha_i v_i$ with non-negative coefficients $\\alpha_i = g_\\Theta(v_i)$ produced by an MLP, so the learned embedding is interpretable as a weighted combination of existing vocabulary tokens. Orthogonality is enforced through deflation: after learning one embedding, each vocabulary token is projected to the subspace orthogonal to it, and the next embedding is learned on the projected vocabulary, yielding a set of orthogonal attack embeddings spanning a low-dimensional subspace. The defense (SubDefense) applies the same projection idea to every token in the vocabulary: $v_{\\text{def},i} = v_i - \\mathrm{Proj}_{V_{\\text{att}}}(v_i)$, removing the learned attack subspace from all token embeddings.","core_discovery":"The central claim is that an unlearned text-to-image diffusion model has not actually forgotten the erased concept; it has merely suppressed its explicit textual associations, while the concept remains accessible through a low-dimensional subspace of implicit associations. These implicit associations can be learned as orthogonal attack token embeddings, each expressible as a non-negative linear combination of interpretable CLIP vocabulary tokens (bag-of-words style), and these embeddings achieve high attack success rates across prompts, initial noises, and unlearned models. Moreover, the same embeddings transfer back to the original Stable Diffusion model with high success, indicating that the residual vulnerability is inherited from the base model rather than created by unlearning. The paper then shows that projecting the CLIP vocabulary onto the null space of these learned attack embeddings substantially reduces the attack success rate of both their own SubAttack and other jailbreaking methods (UnlearnDiff, CCE) while maintaining generation quality.","pith_inferences":["If the residual vulnerability is truly inherited from the frozen text encoder, then improving unlearning robustness may require modifying or replacing the CLIP text encoder itself, rather than only fine-tuning the UNet; this follows from the paper's finding that attack embeddings transfer back to the original SD model.","The paper's interpretation of implicit associations (e.g., 'slave', 'nip' for nudity) is based on vocabulary tokens with high weights, but the defense removes the entire learned subspace rather than isolating these specific tokens; a testable extension would be to remove only the top-weighted interpretable tokens and see whether the attack success rate drops proportionally.","Because the attack embeddings are learned on the frozen CLIP space, the same embedding could potentially be transferred to other text-to-image models that share or approximate the CLIP text encoder, extending the attack beyond the specific unlearned SD models tested.","The sparsity analysis (top few hundred tokens carry most of the weight) suggests that a sparse, token-level defense might be possible—blocking only dozens of tokens instead of hundreds could preserve utility while still cutting the largest attack channels, a direction the paper leaves implicit."],"forward_implications":["Existing unlearning methods that only fine-tune the denoising UNet while freezing the text encoder leave the target concept accessible in the token embedding space; a defense that modifies the token embedding vocabulary directly can reduce attack success rates across multiple attack types.","Attack token embeddings learned from one unlearned model transfer to other unlearned models and back to the original Stable Diffusion model, implying that robustness evaluations should consider transfer-based attacks rather than only per-model optimization.","The interpretability of the attack embeddings provides a diagnostic tool: unlearned models with higher explicit-token retention (e.g., FMN) are more vulnerable to simple attacks, while those with more implicit associations (e.g., ESD, UCE, SPM) still fail under the subspace attack.","The defense is plug-and-play: it can be applied on top of existing unlearned models without retraining the UNet, and it preserves generation quality on safe prompts better than the RECE baseline.","Defending against CCE (which learns unconstrained token embeddings) is harder than defending against attacks tied to the vocabulary, and requires blocking more tokens at the cost of utility—pointing to a trade-off that future defenses must address."],"supporting_citations":[{"why":"Hidden language of diffusion models: supplies the bag-of-words interpretability mechanism (non-negative linear combination of token embeddings) that SubAttack builds on.","marker":"[22]"},{"why":"CCE: the baseline embedding-space attack and the method SubAttack compares against and is compared with.","marker":"[16]"},{"why":"UnlearnDiff: the baseline discrete-prompt attack and the dataset construction protocol that SubAttack's evaluation dataset and metrics follow.","marker":"[15]"},{"why":"Textual inversion: the technique of learning a new token embedding for a concept, which both CCE and SubAttack use as their attack setup.","marker":"[20]"},{"why":"Stable Diffusion v1.4: the base model from which all unlearned victim models are fine-tuned, defining the CLIP text encoder and token embedding space.","marker":"[4]"},{"why":"RECE: the only existing defense baseline for unlearned diffusion models, which SubDefense is compared against on UCE.","marker":"[21]"}],"fun_headline_variants":["Token-level jailbreak exposes unlearning blind spot","Interpretable attack tokens reveal hidden concept persistence","Defense projects away attack tokens to restore safety","Unlearning is surface-level: hidden tokens still fire","Attack tokens transfer across prompts and models"],"cache_read_input_tokens":27136,"weakest_assumption_plain":"The load-bearing premise is that the residual vulnerability of an unlearned model is fully captured in the shared CLIP token embedding space and can be excised by projecting every vocabulary vector onto the null space of a set of attack embeddings learned on that same space; this premise is not established because the text encoder is frozen and identical across all victim models, making the attack embeddings live in the same space regardless of the UNet's unlearning.","fun_headline_variants_meta":{"raw":{"variants":["Token-level jailbreak exposes unlearning blind spot","Interpretable attack tokens reveal hidden concept persistence","Defense projects away attack tokens to restore safety","Unlearning is surface-level: hidden tokens still fire","Attack tokens transfer across prompts and models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000684,"raw_usage":{"total_tokens":3101,"prompt_tokens":938,"completion_tokens":2163,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":554,"completion_tokens_details":{"reasoning_tokens":2094}},"tokens_in":554,"tokens_out":2163,"duration_ms":16358,"temperature":1.0,"reasoning_tokens":2094,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:07:12.775256+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct disproof would be to retrain the CLIP text encoder or replace it (e.g., with a fine-tuned text encoder) while keeping the same unlearned UNet, and then check whether the SubAttack token embeddings learned on the original CLIP space still achieve high attack success rates on the unlearned model; if the attack success drops to near the no-attack baseline, then the residual vulnerability is not inherent to the token embedding space but an artifact of the frozen encoder. Alternatively, one could remove only the top-weighted interpretable tokens identified for a concept (e.g., 'slave', 'nip' for nudity) from the vocabulary and measure whether the attack success rate falls as much as when the entire learned subspace is projected out; if the ASR does not drop, then the interpretable tokens are not the causal carriers of the residual concept.","supporting_citations":[{"cited_title":"The hidden language of diffusion models","cited_arxiv_id":null,"evidence_quote":"Hidden language of diffusion models: supplies the bag-of-words interpretability mechanism (non-negative linear combination of token embeddings) that SubAttack builds on."},{"cited_title":"Marshall, Niv Cohen, Govind Mittal, and Chinmay Hegde","cited_arxiv_id":null,"evidence_quote":"CCE: the baseline embedding-space attack and the method SubAttack compares against and is compared with."},{"cited_title":"To generate or not? safety-driven unlearned diffusion models are still easy to generate unsafe images","cited_arxiv_id":null,"evidence_quote":"UnlearnDiff: the baseline discrete-prompt attack and the dataset construction protocol that SubAttack's evaluation dataset and metrics follow."},{"cited_title":"InThe Eleventh International Conference on Learning Representations, 2023","cited_arxiv_id":null,"evidence_quote":"Textual inversion: the technique of learning a new token embedding for a concept, which both CCE and SubAttack use as their attack setup."}],"review_version":1}