{"id":"1449a6dd-95f9-47b3-b4da-87ac86bb8d94","arxiv_id":"2501.01125","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"DuMo erases target concepts from text-to-image models by adding a frozen-backbone skip-connection eraser with learned timestep and layer modulation, reporting the best trade-off on three concept erasure benchmarks.","lead":"This paper presents DuMo, a method for erasing unsafe concepts from text-to-image models by adding a trainable module to the U-Net's skip connections while keeping the backbone frozen, plus learned per-timestep and per-layer scaling factors. It claims better removal of nudity, cartoon characters, and artistic styles than prior approaches while preserving unrelated generation.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The SOTA claim is under-supported: the authors' own erasure method RECE appears only in Table 1 and is missing from the cartoon and style benchmarks (Tables 2-3) without explanation.","rationale":"I read the central claim as an empirical SOTA claim. The FreeU-based rationale is important for understanding the design, but even if that rationale were imperfect, the empirical tables could still support SOTA if the comparisons are fair. The least secure condition for the SOTA claim is the completeness of the baseline set. RECE is listed as a baseline and appears in only one of three tables. This is an objective, verifiable omission that directly affects the comparison. The 'without preservation loss' contradiction is real but would affect interpretation, not necessarily the numbers; the genitalia count is a metric-design concern. The RECE omission is the most concrete threat. A single experiment—running RECE on the missing benchmarks—would settle it. I therefore keep the reader's CONDITIONAL verdict but on somewhat different grounds.","tokens_in":11609,"tokens_out":8172,"duration_ms":77563,"concrete_test":"Run RECE (released code) on the cartoon and style benchmarks using the exact DuMo/SPM protocol: same 80 text templates per concept, 5 images per template, seed 2024, with LPIPSe, LPIPSu, and LPIPSda computed on the same generated sets. Compare RECE's LPIPSda to DuMo's reported values (e.g., 0.425 for single-Snoopy removal in Table 2; 0.358 for Van Gogh in Table 3). If RECE meets or exceeds these values, the SOTA claim fails; if RECE's code cannot produce results on these tasks, the paper must document that limitation explicitly.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is state-of-the-art performance on three benchmarks (Abstract; Sec. 4). For that claim to hold, the comparison must include all relevant baselines on all three tasks. The experimental setup says 'we compare ours with 10 baseline methods, including ... RECE ...' (Sec. 4), but RECE appears only in Table 1 (explicit content erasure) and is absent from Tables 2 and 3 (cartoon concept removal and artistic style erasure). No reason is given. Since RECE is a general concept-erasure method from the same group (Gong et al. 2024), its omission leaves open the possibility that DuMo is not SOTA on those tasks. If RECE is inapplicable to style/cartoon erasure, the paper should say why; if it is applicable, omitting it from the headline comparison is selective. Either way, the evidence as presented does not fully establish the SOTA claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DuMo, a two-stage concept erasure method for text-to-image diffusion models. In the first stage, an Eraser with Prior Knowledge (EPR) module is trained to modify only U-Net skip-connection features while the backbone is frozen; in the second stage, a Timestep-Layer Modulation (TLMO) process learns per-timestep and per-layer scaling factors for the EPR outputs. The method is evaluated on explicit content erasure using the I2P benchmark and Nudenet detection, on cartoon concept removal, and on artistic style erasure, where it reports the lowest number of detected nude body parts and the highest LPIPSda scores in most comparisons.","tokens_in":11671,"tokens_out":6177,"duration_ms":53769,"significance":"DuMo is an attractive plug-and-play design: it leaves the backbone parameters untouched, intervenes only on skip connections, and the authors provide code. If the reported numbers hold, the method would be a useful contribution to concept erasure because it offers a better preservation of non-target concepts while still erasing the target concept. The use of external metrics (Nudenet, CLIP Score, FID, LPIPS) on held-out prompts is a strength. However, the evidence for the headline state-of-the-art claim is weakened by the omission of the RECE baseline from two of the three benchmarks and by an ablation result that appears to contradict the claimed benefit of the TLMO module, so the current version does not fully substantiate the strongest conclusions.","major_comments":[{"comment":"The paper states that it compares with 10 baseline methods, including RECE (Gong et al. 2024), but RECE appears only in Table 1 and is absent from both the Cartoon Concept Removal and Artistic Style Erasure tables. Since RECE is a general concept-erasure method and is listed among the baselines, its omission leaves the state-of-the-art claim for those two benchmarks unsupported. The authors should include RECE in Tables 2 and 3 or explain explicitly why RECE cannot be applied to those tasks.","section":"Sec. 4 (Tables 2 and 3)"},{"comment":"Table 5 reports LPIPSda values of 0.426 for EPR alone and 0.358 for EPR+TLMO on the 'Van Gogh' concept, so adding the TLMO modulation stage decreases the composite trade-off metric by 0.068. The decrease comes from a drop in LPIPSe from 0.459 to 0.383, i.e., weaker erasure, even though LPIPSu improves from 0.033 to 0.025. This contradicts the narrative that TLMO improves the erasure-preservation balance and that the joint use of timestep and layer factors attains the optimal trade-off. The authors need to reconcile the quantitative result with the claim or revise the claim.","section":"Sec. 4.4, Table 5"},{"comment":"The 'Results of Explicit Content Erasure' paragraph says that 'our method achieves superior preservation effects without incorporating any preservation loss,' but Eq. (6) defines the TLMO training loss as L = L_era2 + lambda * L_pre. Unless the authors used lambda = 0 for the nudity experiments, the statement is inconsistent with the method description. Please clarify whether and where the preservation loss was used, and how this affects the interpretation of the explicit-content results.","section":"Sec. 4.1 and Eq. (6)"}],"minor_comments":[{"comment":"The sentence says 'we compare ours with 10 baseline methods' but only nine baseline methods are enumerated: ESD, UCE, SLD-Med, SA, CA, SDD, RECE, MACE, and SPM. Please correct the count or add the missing method.","section":"Sec. 4, first paragraph"},{"comment":"There are several typographical errors: 'noval' should be 'novel', 'briefy' should be 'briefly', 'protential' should be 'potential', 'our gold' should be 'our goal' in the Method section, and 'MOdulation' should be 'Modulation' in the module name.","section":"Throughout"},{"comment":"Eq. (1) and Fig. 2(a) do not clearly specify the internal structure of the EPR module, particularly how the copied encoder block relates to the function E_cera and the zero-convolution layers. A short architectural description would improve reproducibility.","section":"Sec. 3.1, Eq. (1)"},{"comment":"The text introduces 'Timestep-Layer MOdulation process (TLMO)' but the abstract and some later passages use 'Time-Layer MOdulation'; please unify the terminology.","section":"Sec. 3.3"},{"comment":"The definition of LPIPSe and LPIPSu is implicit; please state explicitly that LPIPSe is computed against images generated by the unmodified model for erased concepts and LPIPSu against images for unerased concepts, and specify which images are used as references.","section":"Sec. 4.2"}],"recommendation":"major_revision","confidential_remarks":"The omission of RECE from Tables 2 and 3 is particularly concerning because RECE is from the same research group and is explicitly named as a baseline in Sec. 4. The authors should be asked to provide those comparisons or a clear justification for their absence, as this is directly relevant to the paper's central state-of-the-art claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Honest take: this is a solid, well-written engineering paper. The EPR module is a straightforward ControlNet transplant to erasure, which is fine, but the real contribution is TLMO—learning per-timestep, per-layer scaling factors on the erasure module output. The ablations (Table 5) show EPR alone already preserves much better than direct fine-tuning (LPIPSu from 0.24 to 0.033), and TLMO further balances the trade-off. That is a real, if narrow, step forward.\n\nThe main tables are externally evaluated on standard benchmarks with Nudenet, CLIP, FID, and LPIPS, and the code link makes verification plausible. I'd credit the authors for using held-out prompts and not just loss curves.\n\nNow the soft spots, in order.\n\nFirst, the SOTA claim is under-supported as written. The paper says it compares against 10 baselines 'including RECE,' but RECE appears only in Table 1 (explicit content erasure). It is absent from the cartoon and style benchmarks (Tables 2-3) with no explanation. RECE is their own general-purpose erasure method from the same group. Either it is not applicable to those tasks—then say why—or it was selectively omitted. Without that comparison, calling the cartoon/style results SOTA is not fully established.\n\nSecond, there is a direct internal contradiction. In the explicit erasure results, the text says 'our method achieves superior preservation effects without incorporating any preservation loss,' but Eq. 4-6 define Lpre and the final TLMO loss includes λLpre. Maybe they mean the EPR stage has no preservation loss, while TLMO does. The sentence needs fixing.\n\nThird, no error bars or significance tests anywhere. The LPIPSda gaps are large (~0.1), so this is minor, but still worth a line.\n\nFourth, the headline nudity number is a bit cherry-picked. DuMo has the lowest total nude parts (34) but its genitalia counts (F:4, M:6) are worse than SPM's (F:0, M:5). For a safety method, that matters.\n\nThe FreeU assumption that skip connections are high-frequency only is a real design bet, and the paper's own nudity case (which requires 'putting clothes on characters') involves structural edits. That may bound how much the preservation benefit extends to explicit content.\n\nVerdict: worth refereeing. The TLMO idea is solid, the evaluation is mostly sound, and the missing RECE comparison is fixable. Send it to review, but ask the authors to add RECE to all three benchmarks, clarify the preservation-loss sentence, and report variance.\n\nWho is this for? People working on diffusion model safety and concept erasure. Reading group: maybe—the modulation idea is a nice datapoint for the FreeU-style decomposition of U-Net features.\n\nRecommendation: accept peer review, expect major-to-minor revision.","headline":"DuMo's TLMO modulation is a genuine improvement for concept erasure preservation, but the SOTA claim rests on an incomplete baseline comparison and an internal contradiction about the preservation loss.","tokens_in":12354,"tokens_out":2624,"would_cite":false,"duration_ms":23575,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"DuMo erases concepts by tuning only the U-Net skip connections with the backbone frozen, preserving non-target images.","keywords":["concept erasure","diffusion models","text-to-image generation","skip connections","U-Net","modulation","LPIPSda","I2P"],"falsifier":"On a set of concepts that require structural change (e.g., changing a person's posture or removing an object while keeping the scene), measure whether DuMo's frozen backbone leaves the structure untouched while the target concept remains; if erasing such a concept forces visible structural corruption of non-target content, the high-frequency-only assumption is shown to fail.","tokens_in":121,"feed_emoji":"🎨","tokens_out":5601,"duration_ms":105749,"temperature":0.7,"pith_summary":"This paper claims that the best way to erase an unwanted concept from a text-to-image diffusion model is to leave the U-Net backbone untouched and put all the erasing power into the skip connections, which carry the high-frequency details of the image. It proposes DuMo, a two-part network: an Eraser with PRior Knowledge (EPR) module that adds a trainable branch onto each skip connection, and a Time-Layer MOdulation (TLMO) process that learns per-layer and per-timestep scaling factors so the erasure is applied exactly where it helps. On three benchmarks—explicit content erasure, cartoon character removal, and artistic style erasure—DuMo reports the best trade-off between removing the target concept and preserving other concepts. If the claim holds, it gives model providers a way to scrub unsafe or copyrighted content without degrading the model's general generative ability.","feed_headline":"Skip-connection-only eraser sets best trade-off in concept erasure","feed_subtitle":"DuMo freezes the U-Net and tunes only skip connections to remove nudity, cartoons, and art styles.","key_machinery":"The Eraser with PRior Knowledge (EPR) module is a copy of the U-Net encoder blocks, connected to the original skip connection features through zero-initialized 1x1 convolutions (ZeroConv); the original U-Net parameters stay frozen and the original skip features are added to the EPR output. The Timestep-Layer MOdulation process (TLMO) introduces per-layer and per-timestep scalar factors $M_t^l$ that scale each EPR output, with the final skip feature computed as $x_t^l + M_t^l S_{t,\\mathrm{cera}}^l$. The modulation factors are learned with the erasing loss plus a preservation loss on empty-prompt outputs. The key mechanism is the combination: freezing the backbone preserves low-frequency structure, while the prior-knowledge skip features and learned scaling localize the erasure to the high-frequency details that carry style and concept identity.","core_discovery":"The central discovery is that concept erasure can be achieved almost entirely on the high-frequency detail path of a U-Net, not on the structural backbone. The paper shows that a plug-in EPR module, trained with the backbone frozen and the original skip features kept as prior knowledge, removes a target concept while preserving the structure of non-target objects. The subsequent TLMO analysis reveals that erasure effectiveness varies strongly by skip-connection layer and denoising timestep, so a set of learned modulation factors per layer and timestep yields the final trade-off. On the I2P nudity benchmark DuMo detects 34 nude body parts versus 41 for the next-best method, and on cartoon and style erasure it reports LPIPSda of 0.428 and 0.387 respectively, the highest among compared methods. The paper presents these results as state-of-the-art for the three tasks on Stable Diffusion v1.4.","pith_inferences":["The FreeU-based separation of high/low frequency is approximate; DuMo's success suggests that the skip-connection path is a natural 'style knob', so the same architectural intervention could be used for style transfer or attribute editing, not just erasure.","Because the modulation factors are learned per layer and timestep, their final values (e.g., zero on layer group 4 for 'Van Gogh') could be interpreted as a signature of where a concept lives in the network, potentially serving as a diagnostic for concept localization.","The method's second-stage preservation loss is applied only on empty prompts; extending it to a set of frequent non-target prompts could further improve preservation, especially for concepts that overlap with common visual categories.","The reported 34 nude parts vs 41 for the next best is on a single detector (Nudenet) at 0.6 threshold; the ranking could shift if detector or threshold changes."],"forward_implications":["Removing a concept leaves the model's general image-generation ability largely intact, because the backbone parameters never change.","Multiple concepts can be erased simultaneously by summing the outputs of several EPR modules on the skip connections.","The per-layer, per-timestep modulation factors provide a controllable way to dial the erasure strength without retraining the whole model.","The two-stage training procedure (EPR then TLMO) gives a practical recipe for extending the method to new concepts: train the eraser branch first, then calibrate its scaling."],"supporting_citations":[{"why":"Supplies the FreeU finding that backbone features carry low-frequency structure and skip connections carry high-frequency details, which motivates restricting erasure to skip connections.","marker":"(Si et al. 2024)"},{"why":"Provides the zero-initialized convolution and copied-encoder design that the EPR module builds on for stable, structure-preserving fine-tuning.","marker":"(Zhang, Rao, and Agrawala 2023)"},{"why":"Provides the erasing loss in Eq. 3 that aligns the erased concept with the empty prompt, the training signal for EPR.","marker":"(Gandikota et al. 2023)"},{"why":"Defines the evaluation protocol for cartoon and style erasure (LPIPS-based metrics and multi-concept settings) and is the SPM baseline DuMo compares against.","marker":"(Lyu et al. 2024)"},{"why":"Provides the I2P dataset of toxic prompts and the SLD baseline used for explicit-content erasure evaluation.","marker":"(Schramowski et al. 2023)"}],"fun_headline_variants":["Only skip connections: DuMo erases with minimal collateral damage","DuMo shows concept erasure lives in high frequencies","Frozen U-Net, tuned skips: DuMo's recipe for precise erasure","High-frequency erasure: DuMo's dual modulation beats baselines","Concept erasure without backbone changes: DuMo's EPR + TLMO"],"cache_read_input_tokens":14336,"weakest_assumption_plain":"The method assumes that the concept to be erased lives entirely in the high-frequency detail stream (the skip connections) and that the low-frequency backbone structure can be left untouched; if erasing a concept like nudity requires structural changes, such as putting clothes on a character, the preservation advantage weakens.","fun_headline_variants_meta":{"raw":{"variants":["Only skip connections: DuMo erases with minimal collateral damage","DuMo shows concept erasure lives in high frequencies","Frozen U-Net, tuned skips: DuMo's recipe for precise erasure","High-frequency erasure: DuMo's dual modulation beats baselines","Concept erasure without backbone changes: DuMo's EPR + TLMO"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001004,"raw_usage":{"total_tokens":4296,"prompt_tokens":1044,"completion_tokens":3252,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":660,"completion_tokens_details":{"reasoning_tokens":3158}},"tokens_in":660,"tokens_out":3252,"duration_ms":18981,"temperature":1.0,"reasoning_tokens":3158,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:35:11.217869+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a set of concepts that require structural change (e.g., changing a person's posture or removing an object while keeping the scene), measure whether DuMo's frozen backbone leaves the structure untouched while the target concept remains; if erasing such a concept forces visible structural corruption of non-target content, the high-frequency-only assumption is shown to fail.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the erasing loss in Eq. 3 that aligns the erased concept with the empty prompt, the training signal for EPR."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the evaluation protocol for cartoon and style erasure (LPIPS-based metrics and multi-concept settings) and is the SPM baseline DuMo compares against."}],"review_version":1}