Pith. sign in

REVIEW 2 major objections 6 minor 40 references

Erase the concept, keep the neighbors: covariance-aware direction protects

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

CARE replaces the raw target direction in value-space concept erasure with a covariance-aware retained-subspace direction, reducing collateral damage to non-target concepts at negligible computational cost.

T0 review reviewed 2026-07-07 challenge →

load-bearing objection CARE is a clean, correct incremental improvement on AdaVD value-space erasure; the math is sound and the preservation gains are real, but deployment gaps around anchor selection and γ tuning need referee attention. the 2 major comments →

arxiv 2607.05274 v1 pith:CLZACM3E submitted 2026-07-06 cs.CV

Erasing Without Collateral Damage: Precise Concept Removal in Diffusion Models

Parth Upman , Nishita Jain , Shreyank N Gowda This is my paper
classification cs.CV
keywords concept erasurediffusion modelscross-attentionvalue-space editingcovariance-aware projectiontraining-free interventionprior preservationgenerative AI safety
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper identifies a specific source of collateral damage in concept erasure for diffusion models: when you remove a target concept by subtracting its raw value-space direction from cross-attention activations, you also remove visual structure shared with non-target concepts (e.g., the generic face manifold shared among celebrities, or brushstroke patterns shared among painters). The authors argue this conflation of target-specific identity with shared visual structure is the root cause of prior methods degrading nearby concepts. Their solution, CARE, replaces the raw target erasure direction with a covariance-aware direction computed from a small bank of retained concept anchors. This direction is obtained by multiplying the target vector by the inverse of a shrinkage-regularized covariance matrix built from the anchor bank, which down-weights components of the target that lie in high-variance retained directions while preserving target-distinctive components. The resulting operator is closed-form, training-free, and adds only a 1.2-second offline computation over standard value-space erasure. A single shrinkage parameter controls the erase-preserve trade-off, recovering standard erasure as one limit and approaching a hard projection that leaves the retained subspace exactly invariant as the other. Experiments across instance, style, and celebrity erasure show CARE consistently improves non-target FID while maintaining or improving target CLIP-score erasure, with the strongest gains on concepts whose target-specific component is separable from the retained subspace.

Core claim

The central object is the covariance-aware erasure direction d_j = (1/M * B_j^T B_j + gamma*I)^{-1} * t_j, where t_j is the raw target concept direction and B_j is a small bank of M retained concept anchor vectors recorded at the same cross-attention token position. This direction acts as a Fisher/Mahalanobis-style discriminant: it suppresses components of the target that are shared with retained concepts (high-variance directions under the retained-anchor covariance) while preserving components that distinguish the target. The paper proves three properties: (1) the update is a minimum-disturbance projection (closest vector satisfying a prescribed inner-product constraint), (2) standard raw-

What carries the argument

Covariance-aware retained-subspace erasure direction d_j = (1/M * B_j^T B_j + gamma I)^{-1} t_j, computed via low-rank Woodbury solve over an M x M matrix, applied as a gated rank-one subtraction in cross-attention value space during diffusion sampling.

Load-bearing premise

The method assumes that a small bank of six manually chosen retained concept anchors provides a covariance estimate sufficient to separate target-specific identity from shared visual structure. The paper does not provide a principled procedure for selecting these anchors in deployment, and the ablation shows that poorly chosen anchors (especially ones too related to the target) can weaken erasure or fail to protect retained concepts. The assumption that a fixed, small, hand-p

What would settle it

If the retained-anchor covariance does not actually capture the shared structure between target and non-target concepts (e.g., because the anchors are too few, too unrelated, or too related to the target), then the covariance-aware direction will either fail to protect retained concepts or fail to erase the target. The ablation already shows this: related anchors raise target CLIP score from 17.19 to 22.88. A stronger falsifier would be a concept whose target-specific component is inseparable from the shared subspace under any reasonable anchor bank, making the erase-preserve trade-off unimpro

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Concept erasure in deployed generative models can be made more surgical without sacrificing the speed and training-free advantages of value-space intervention, making it practical for production safety pipelines that need to remove specific copyrighted characters or celebrity identities without degrading related content.
  • The shrinkage parameter gamma provides a deployment-time dial between erasure aggressiveness and prior preservation, allowing model operators to set the operating point per concept rather than per model.
  • The covariance-aware direction framework could extend to other modalities where cross-attention value editing applies, such as text-to-video or text-to-3D generation, where concept removal faces similar shared-structure entanglement.
  • The hard-projection limit's retained-subspace invariance property suggests that exact subspace protection is achievable, and the soft covariance form trades exactness for robustness to imperfect anchor selection.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The method's dependence on a manually curated anchor bank of M=6 retained concepts creates a deployment bottleneck: for each new target concept, an operator must select anchors that capture relevant shared structure without overlapping the target identity. An automated anchor-selection procedure (e.g., retrieving nearest non-target concepts in embedding space) would be needed for scalable deployme
  • The ablation showing that 'related' anchors increase target CLIP score to 22.88 versus 17.19 for 'blind disjoint' anchors suggests a fundamental tension: the most semantically informative anchors for protecting a concept are also the ones most likely to absorb target-specific signal. This implies the method works best when the anchor bank captures broad visual categories (e.g., 'general faces') ra
  • The framework implicitly assumes that concept identity in cross-attention value space decomposes additively into target-specific and shared components. If concepts are represented non-additively (e.g., through multiplicative or nonlinear interactions), the covariance-aware projection may not cleanly separate them, and the gains could diminish for deeply entangled concepts.
  • Connecting to broader model editing, the minimum-disturbance interpretation suggests that CARE-style covariance-aware directions could be applied to weight-space editing methods (like Unified Concept Editing), not just value-space intervention, potentially improving preservation in fine-tuning-based erasure as well.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper introduces CARE (Covariance-Aware Retained-subspace Erasure), a training-free concept erasure method for text-to-image diffusion models. CARE modifies the value-space erasure direction used by AdaVD: instead of erasing along the raw target value vector t_j, it computes a covariance-aware direction d_j = (1/M B_j^T B_j + gamma I)^{-1} t_j, where B_j is a small bank of retained concept anchors. This down-weights components of the target direction that lie in high-variance retained directions, reducing collateral damage to non-target concepts. The operator is closed-form, computed via a low-rank Woodbury solve, and recovers standard value-space erasure as gamma -> infinity. Propositions 1-3 establish a minimum-disturbance interpretation, the limiting case, and a hard-projection invariance property. Experiments across instance, style, and celebrity erasure show improved FID for retained concepts with competitive CLIP scores for erased concepts, at negligible additional cost over AdaVD.

Significance. The paper makes a solid contribution to the training-free concept erasure literature. The core mathematical derivation is clean and correct: the Woodbury computation (Eq. 8), the limiting cases (Propositions 2-3), and the projection-form invariance (Corollary 1) are all sound. The method is parameter-light (gamma, M, anchor composition) and the code is publicly available. The empirical gains over AdaVD on preservation (FID) are consistent and sometimes substantial, particularly in the celebrity and style settings. The framing of concept erasure as choosing the right direction to forget, rather than merely tuning erasure strength, is a useful conceptual contribution.

major comments (2)
  1. §4.7, Table 5(b): The anchor-bank composition ablation reveals severe sensitivity: 'blind disjoint' anchors give CS 17.19 while 'related' anchors give CS 22.88, a 33% relative degradation that effectively disables erasure. The paper uses 'blind disjoint' anchors in all main experiments but never specifies which concepts constitute these anchors for each target, nor provides an automated or principled procedure for selecting them. A practitioner erasing a new concept has no way to determine whether their chosen anchors are 'disjoint enough' without running the full evaluation. This is load-bearing for the central claim of generality. The paper should either (a) specify the exact anchor concepts used for each target in the main experiments, (b) provide an automated anchor-selection procedure, or (c) explicitly scope the claim to settings where disjoint anchors can be reliably identified.
  2. §4.1: The shrinkage parameter gamma is set per-category (0.5 for instances, 0.2 for styles/celebrities) but no principled selection procedure is given. Table 5(a) shows that gamma has a large effect on the erase-preserve trade-off (e.g., gamma=0.05 gives CS 29.99 for Bruce Lee, near the unedited model, while gamma=0.20 gives CS 17.19). Since the paper claims gamma 'exposes a continuous erase-preserve trade-off,' some guidance on how to select gamma for a new concept category is needed. At minimum, the paper should state whether gamma was selected on a held-out set or tuned on the evaluation concepts, and ideally provide a heuristic or cross-validation procedure.
minor comments (6)
  1. Table 1, multi-concept rows: When erasing Snoopy and Mickey jointly, Mickey CS increases from 19.93 (AdaVD) to 22.55 (CARE). When erasing all three, Mickey CS is 22.56 vs 19.73. The paper acknowledges this (§4.2) but the framing as 'not designed to maximize erasure strength at any cost' somewhat understates the regression. Consider noting explicitly that multi-target erasure can weaken removal of specific targets when the covariance metric absorbs target-specific structure.
  2. Table 3: Several AdaVD FID values for erased concepts appear very high (e.g., 138.70 for Bruce Lee, 116.94 for Marilyn Monroe). These are greyed as non-primary metrics, but it would help readers to clarify in the caption that FID for the erased concept measures distributional shift rather than preservation quality.
  3. §3.4: The gate delta(cos(t_j, v_j)) uses the raw target direction t_j, not the covariance-aware direction d_j. The paper notes this preserves compatibility with prior value-space erasure, but it would be worth briefly discussing whether using cos(d_j, v_j) for the gate was considered and why it was rejected.
  4. Table 4: The CARE row reports '+0.7%' per-image generation overhead but the total time is listed as 22 (matching AdaVD). Consider clarifying whether the 22 already includes the overhead or whether it excludes the 1.2s offline build.
  5. Figure 1: The diagram is informative but the arrow from 'd' to the value update box could be clearer about where Eq. 6 is applied. Consider adding the equation number reference in the figure.
  6. References: The citation for AdaVD [36] lists a 2025 CVPR date, but the arXiv identifier format and other details suggest it may be a 2024 preprint. Please verify.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the careful reading and positive assessment. Both major comments identify legitimate gaps in the manuscript's treatment of anchor selection and gamma tuning. We will address both in revision.

read point-by-point responses

  1. Referee: §4.7, Table 5(b): The anchor-bank composition ablation reveals severe sensitivity: 'blind disjoint' anchors give CS 17.19 while 'related' anchors give CS 22.88, a 33% relative degradation that effectively disables erasure. The paper uses 'blind disjoint' anchors in all main experiments but never specifies which concepts constitute these anchors for each target, nor provides an automated or principled procedure for selecting them. A practitioner erasing a new concept has no way to determine whether their chosen anchors are 'disjoint enough' without running the full evaluation. This is load-bearing for the central claim of generality. The paper should either (a) specify the exact anchor concepts used for each target in the main experiments, (b) provide an automated anchor-selection procedure, or (c) explicitly scope the claim to settings where disjoint anchors can be reliably identified.

    Authors: The referee is correct that the manuscript does not specify the exact anchor concepts used per target, and this is a genuine gap that weakens reproducibility and practical applicability. We will address this by implementing option (a) in full and option (b) partially. Specifically, we will add a table to the supplementary material listing every anchor concept used for each target concept in each main experiment (instances, styles, celebrities). For example, when erasing Snoopy, the blind disjoint anchors are Pikachu, Dog, and Legislator; when erasing Bruce Lee, the anchors are Anne Hathaway and Tom Cruise; when erasing Van Gogh, the anchors are Andy Warhol and Caravaggio. We will also describe the selection criterion explicitly: anchors are chosen from the same semantic category as the target (cartoon characters, art styles, identities) but are visually distinct from the target. Additionally, we will provide a simple automated heuristic for anchor selection: given a target concept, compute its value-space representation and select M anchors from a candidate pool that have the lowest cosine similarity to the target direction while remaining within the same broad category. We will include preliminary results showing that this heuristic recovers anchor sets comparable to the manually selected ones. We agree that the generality claim should be scoped, and we will add language to the conclusion explicitly noting that CARE assumes the availability of anchors that are semantically near the target but visually distinct, and that performance degrades when this assumption is violated (as Table 5(b) already demonstrates). revision: yes

  2. Referee: §4.1: The shrinkage parameter gamma is set per-category (0.5 for instances, 0.2 for styles/celebrities) but no principled selection procedure is given. Table 5(a) shows that gamma has a large effect on the erase-preserve trade-off (e.g., gamma=0.05 gives CS 29.99 for Bruce Lee, near the unedited model, while gamma=0.20 gives CS 17.19). Since the paper claims gamma 'exposes a continuous erase-preserve trade-off,' some guidance on how to select gamma for a new concept category is needed. At minimum, the paper should state whether gamma was selected on a held-out set or tuned on the evaluation concepts, and ideally provide a heuristic or cross-validation procedure.

    Authors: The referee is right that the manuscript does not explain how gamma was selected, and this is a legitimate concern. To be transparent: gamma was selected by manual inspection of the erase-preserve trade-off on the evaluation concepts themselves, not on a held-out set. We will state this explicitly in the revised manuscript. We agree that a more principled procedure is desirable. We will add the following heuristic: for a new target concept, run CARE at a small set of gamma values (e.g., {0.05, 0.1, 0.2, 0.5, 1.0}) on a small probe set (5-10 prompts for the target and 5-10 for a retained probe), compute CS for the target and FID for the retained probe, and select the smallest gamma that achieves target CS within a specified margin of the raw-erasure baseline (AdaVD). This is a lightweight procedure since each CARE configuration requires only the 1.2-second offline computation plus a small number of generations. We will include a table showing that this heuristic recovers gamma values close to the deployed ones for the three concept categories. We will also note that the per-category defaults (0.5 for instances, 0.2 for styles/celebrities) serve as reasonable starting points, and that the bank-size ablation (Table 5(b)) shows the method is not highly sensitive to M, so the main sensitivity is to gamma, which the heuristic addresses. revision: yes

Circularity Check

0 steps flagged

No circularity found: CARE's derivation is self-contained standard linear algebra with externally falsifiable benchmarks

full rationale

The paper's core derivation is self-contained. The CARE direction d_j = (1/M B_j^T B_j + γI)^{-1} t_j (Eq. 5) is derived from standard linear algebra (Woodbury identity, eigendecomposition) applied to a retained-anchor covariance matrix. The three propositions in Appendix B (minimum-disturbance interpretation, raw erasure as limiting case γ→∞, hard-projection limit γ→0) are straightforward proofs from the paper's own equations, not imported from self-citation. The shrinkage parameter γ is a free trade-off parameter (ablated in Table 5a), not a value fitted to produce a target result. The anchor bank B_j is constructed from recorded value vectors of chosen retained concepts (Eq. 3), not defined in terms of the erasure output. No prediction reduces to a fitted input by construction. The method is evaluated against external baselines (AdaVD, SPM, MACE, etc.) under a standard protocol with independent metrics (CLIP score, FID). The concerns about anchor-bank composition sensitivity (Table 5b) and per-category γ selection are correctness/generality risks, not circularity. No self-citation chain is load-bearing for the mathematical derivation. The derivation chain is clean: inputs (target direction t_j, anchor bank B_j, shrinkage γ) produce output (erasure direction d_j) via a well-defined linear-algebraic transformation that is not tautologically equivalent to its inputs.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

CARE introduces no new entities (particles, forces, dimensions, etc.). It repurposes standard mathematical tools (covariance estimation, Woodbury identity, Mahalanobis distance) for a new application. The free parameters (γ, M, anchor composition) are standard hyperparameters, not invented entities. The method's novelty lies in the combination of existing tools, not in postulating new objects.

free parameters (3)
  • γ (shrinkage parameter) = 0.5 (instance), 0.2 (style/celebrity)
    Controls the erase-preserve trade-off. Selected per-category based on empirical performance. Not fit to individual concepts but chosen post-hoc from ablation.
  • M (anchor bank size) = 6
    Number of retained concept anchors. Default chosen from ablation (Table 5b) showing stable behavior for M∈{4,6,8,10}.
  • Anchor bank composition = blind disjoint (default)
    The type of retained anchors used. Ablation shows 'blind disjoint' performs best, but the specific anchors for each erasure task are manually curated and not specified by a general procedure.
axioms (4)
  • domain assumption The cross-attention value vector t_j for a target concept contains both target-specific identity and shared visual structure with non-target concepts.
    Stated in §1 and §3.1. This is the core motivation for CARE. It is plausible but not independently verified beyond the empirical observation that AdaVD causes collateral damage.
  • domain assumption A small bank of M retained concept anchors provides a covariance estimate sufficient to separate target-specific from shared structure.
    Implicit in §3.2-3.3. The method's effectiveness depends on this, but the paper does not test sensitivity to anchor quality in deployment (only in ablation with fixed concepts).
  • domain assumption CLIP score is a valid proxy for concept erasure efficacy, and FID against the original model is a valid proxy for prior preservation.
    Stated in §4.1. Follows the AdaVD protocol [36]. These metrics are standard but known to have limitations (e.g., CLIP score may not capture nuanced visual identity).
  • standard math Standard linear algebra identities (Woodbury matrix identity, eigendecomposition of symmetric matrices) hold as stated.
    Used in §3.3, Appendix A.1, and Propositions 1-3. These are well-established mathematical results.

reviewed 2026-07-07 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Erasing Without Collateral Damage: Precise Concept Removal in Diffusion Models." pith.science (2026). https://pith.science/paper/CLZACM3E

@misc{pith2026260705274,
  author       = {Pith},
  title        = {Pith review of: Erasing Without Collateral Damage: Precise Concept Removal in Diffusion Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CLZACM3E}},
  note         = {Machine review of arXiv:2607.05274}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Training-free concept erasure is an attractive mechanism for controlling text-to-image diffusion models, but precise erasure often comes at the cost of damaging semantically related non-target concepts. Existing value-space methods remove the component of each cross-attention value along the target concept direction, implicitly treating target identity and shared visual structure as the same signal. We argue that this is the source of much of the collateral damage in prior preservation. We introduce CARE, a closed-form concept erasure operator that replaces the raw target direction with a kept-subspace-aware direction computed from a small bank of retained concept anchors. The resulting edit is applied directly in cross-attention value space, requires no model fine-tuning, and adds only a negligible offline computation. A single shrinkage parameter controls the erase-preserve trade-off. We further show that the operator admits a minimum-disturbance interpretation and, in its projection form, leaves the kept subspace invariant. Experiments under the standard concept-erasure protocol show that our method preserves non-target concepts more faithfully while maintaining competitive erasure across instance, style, and celebrity concepts. Code: https://github.com/parthupman/care

Figures

Figures reproduced from arXiv: 2607.05274 by Nishita Jain, Parth Upman, Shreyank N Gowda.

Figure 1
Figure 1. Figure 1: Overview of CARE. Given a target concept and a small bank of retained an￾chors, CARE records their value-space representations from the frozen text encoder and value projections, without editing the prompt or model weights. During sam￾pling, CARE intervenes inside the cross-attention value space of the diffusion U-Net. The retained-anchor bank defines a covariance-aware direction d = [PITH_FULL_IMAGE:figu… view at source ↗
Figure 2
Figure 2. Figure 2: Qualitative celebrity erasure. Each 3 × 3 panel shows columns as SD v1.4 original, AdaVD, and CARE, and rows as erased target, bystander 1, and bystander 2. Left: erase Bruce Lee, with Marilyn Monroe and Tom Cruise as bystanders. Centre: erase Marilyn Monroe, with Bruce Lee and Tom Cruise as bystanders. Right: erase Melania Trump, with Marilyn Monroe and Anne Hathaway as bystanders. The edited columns supp… view at source ↗
Figure 3
Figure 3. Figure 3: Qualitative style and instance erasure. Each 3 × 3 panel shows columns as SD v1.4 original, AdaVD, and CARE, and rows as erased target, bystander 1, and bystander 2. Left: erase Van Gogh style, with Picasso and Monet as bystanders. Centre: erase Picasso style, with Van Gogh and Monet as bystanders. Right: erase Snoopy, with Mickey Mouse and Pikachu as bystanders. CARE suppresses the target style or charact… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

40 extracted references · 40 canonical work pages · 3 internal anchors

  1. [1]

    In: ACM Trans- actions on Graphics

    Avrahami, O., Lischinski, D., Fried, O.: Blended latent diffusion. In: ACM Trans- actions on Graphics. vol. 42, pp. 1–11 (2023) 4

    work page 2023

  2. [2]

    Multimodal datasets: misogyny, pornography, and malignant stereotypes

    Birhane, A., Prabhu, V.U., Kahembwe, E.: Multimodal datasets: Misogyny, pornography, and malignant stereotypes. arXiv preprint arXiv:2110.01963 (2021) 1, 3

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  3. [3]

    Cambridge University Press (2004) 3

    Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press (2004) 3

    work page 2004

  4. [4]

    In: USENIX Security Symposium

    Carlini, N., Hayes, J., Nasr, M., Jagielski, M., Sehwag, V., Tramer, F., Balle, B., Ippolito, D., Wallace, E.: Extracting training data from diffusion models. In: USENIX Security Symposium. pp. 5253–5270 (2023) 1, 3

    work page 2023

  5. [5]

    In: ACM Transactions on Graphics

    Chefer, H., Alaluf, Y., Vinker, Y., Wolf, L., Cohen-Or, D.: Attend-and-excite: Attention-based semantic guidance for text-to-image diffusion models. In: ACM Transactions on Graphics. vol. 42, pp. 1–10 (2023) 4

    work page 2023

  6. [6]

    In: Ad- vances in Neural Information Processing Systems

    Dhariwal, P., Nichol, A.: Diffusion models beat gans on image synthesis. In: Ad- vances in Neural Information Processing Systems. vol. 34, pp. 8780–8794 (2021) 1, 3

    work page 2021

  7. [7]

    In: In- ternational Conference on Learning Representations (2025) 4

    Fang, J., Jiang, H., Wang, K., Ma, Y., Shi, J., Wang, X., He, X., Chua, T.S.: AlphaEdit: Null-space constrained knowledge editing for language models. In: In- ternational Conference on Learning Representations (2025) 4

    work page 2025

  8. [8]

    Annals of Eugenics7(2), 179–188 (1936) 3, 4

    Fisher, R.A.: The use of multiple measurements in taxonomic problems. Annals of Eugenics7(2), 179–188 (1936) 3, 4

    work page 1936

  9. [9]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Gandikota, R., Materzyńska, J., Fiotto-Kaufman, J., Bau, D.: Erasing concepts from diffusion models. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 2426–2436 (2023) 2, 3, 13

    work page 2023

  10. [10]

    In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision

    Gandikota, R., Orgad, H., Belinkov, Y., Materzyńska, J., Bau, D.: Unified concept editing in diffusion models. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. pp. 5111–5120 (2024) 2, 4

    work page 2024

  11. [11]

    In: Proceedings of the European Conference on Computer Vision (2024) 2, 4

    Gong, C., Chen, K., Wei, Z., Chen, J., Jiang, Y.G.: Reliable and efficient con- cept erasure of text-to-image diffusion models. In: Proceedings of the European Conference on Computer Vision (2024) 2, 4

    work page 2024

  12. [12]

    In: International Conference on Learning Representations (2023) 4

    Hertz, A., Mokady, R., Tenenbaum, J., Aberman, K., Pritch, Y., Cohen-Or, D.: Prompt-to-prompt image editing with cross-attention control. In: International Conference on Learning Representations (2023) 4

    work page 2023

  13. [13]

    In: Advances in Neural Information Processing Systems

    Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models. In: Advances in Neural Information Processing Systems. vol. 33, pp. 6840–6851 (2020) 1, 3

    work page 2020

  14. [14]

    Classifier-Free Diffusion Guidance

    Ho, J., Salimans, T.: Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598 (2022) 2, 4

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  15. [15]

    In: Proceedings of the European Conference on Computer Vision (2024) 3

    Huang, C.P., Chang, K.P., Tsai, C.T., Lai, Y.H., Yang, F.E., Wang, Y.C.F.: Re- celer: Reliable concept erasing of text-to-image diffusion models via lightweight erasers. In: Proceedings of the European Conference on Computer Vision (2024) 3

    work page 2024

  16. [16]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Kumari, N., Zhang, B., Wang, S.Y., Shechtman, E., Zhang, R., Zhu, J.Y.: Ablat- ing concepts in text-to-image diffusion models. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 22634–22645 (2023) 2, 3, 7, 8, 10, 11, 13, 22

    work page 2023

  17. [17]

    Journal of Multivariate Analysis88(2), 365–411 (2004) 3, 4

    Ledoit, O., Wolf, M.: A well-conditioned estimator for large-dimensional covariance matrices. Journal of Multivariate Analysis88(2), 365–411 (2004) 3, 4

    work page 2004

  18. [18]

    diffusion models

    Li, S., van de Weijer, J., Hu, T., Khan, F.S., Hou, Q., Wang, Y., Yang, J.: Get what you want, not what you don’t: Image content suppression for text-to-image 16 Upman et al. diffusion models. In: International Conference on Learning Representations (2024) 4

    work page 2024

  19. [19]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Lu, S., Wang, Z., Li, L., Liu, Y., Kong, A.W.K.: MACE: Mass concept erasure in diffusion models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 6430–6440 (2024) 2, 3, 7, 8, 10, 11, 13, 22

    work page 2024

  20. [20]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Lyu, M., Yang, Y., Hong, H., Chen, H., Jin, X., He, Y., Xue, H., Han, J., Ding, G.: One-dimensional adapter to rule them all: Concepts, diffusion models and erasing applications. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 7559–7568 (2024) 2, 3, 7, 8, 10, 11, 13, 22

    work page 2024

  21. [21]

    Proceedings of the National Institute of Sciences of India2(1), 49–55 (1936) 3, 4

    Mahalanobis, P.C.: On the generalised distance in statistics. Proceedings of the National Institute of Sciences of India2(1), 49–55 (1936) 3, 4

    work page 1936

  22. [22]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Mokady, R., Hertz, A., Aberman, K., Pritch, Y., Cohen-Or, D.: Null-text inver- sion for editing real images using guided diffusion models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 6038– 6047 (2023) 4

    work page 2023

  23. [23]

    In: Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition

    Mustafa, A.B., Ye, Z., Lu, Y., Pound, M.P., Gowda, S.N.: Low-effort jailbreak at- tacks against text-to-image safety filters. In: Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition. pp. 8590–8599 (2026) 1, 3

    work page 2026

  24. [24]

    In: International Conference on Machine Learning

    Nichol, A.Q., Dhariwal, P.: Improved denoising diffusion probabilistic models. In: International Conference on Machine Learning. pp. 8162–8171 (2021) 3

    work page 2021

  25. [25]

    In: International Con- ference on Learning Representations (2024) 3

    Pham, M., Marshall, K.O., Cohen, N., Mittal, G., Hegde, C.: Circumventing con- cept erasure methods for text-to-image generative models. In: International Con- ference on Learning Representations (2024) 3

    work page 2024

  26. [26]

    In: International Conference on Learning Representations (2024) 1, 3

    Podell, D., English, Z., Lacey, K., Blattmann, A., Dockhorn, T., Müller, J., Penna, J., Rombach, R.: SDXL: Improving latent diffusion models for high-resolution im- age synthesis. In: International Conference on Learning Representations (2024) 1, 3

    work page 2024

  27. [27]

    In: International Conference on Machine Learning

    Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., Sutskever, I.: Learning transferable visual models from natural language supervision. In: International Conference on Machine Learning. pp. 8748–8763 (2021) 1, 3

    work page 2021

  28. [28]

    Hierarchical Text-Conditional Image Generation with CLIP Latents

    Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., Chen, M.: Hierarchical text- conditional image generation with CLIP latents. arXiv preprint arXiv:2204.06125 (2022) 3

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  29. [29]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 10684–10695 (2022) 1, 2, 3, 4, 5

    work page 2022

  30. [30]

    In: Advances in Neural Information Processing Systems

    Saharia, C., Chan, W., Saxena, S., Li, L., Whang, J., Denton, E.L., Ghasemipour, K., Gontijo Lopes, R., Karagol Ayan, B., Salimans, T., Ho, J., Fleet, D.J., Norouzi, M.: Photorealistic text-to-image diffusion models with deep language understand- ing. In: Advances in Neural Information Processing Systems. vol. 35, pp. 36479– 36494 (2022) 3

    work page 2022

  31. [31]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Schramowski, P., Brack, M., Deiseroth, B., Kersting, K.: Safe latent diffusion: Mitigating inappropriate degeneration in diffusion models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 22522– 22531 (2023) 1, 2, 3, 4, 7, 8, 10, 11, 13, 22

    work page 2023

  32. [32]

    In: Advances in Neural Information Processing Systems

    Schuhmann, C., Beaumont, R., Vencu, R., Gordon, C., Wightman, R., Cherti, M., Coombes, T., Katta, A., Mullis, C., Wortsman, M., Schramowski, P., Kundurthy, Abbreviated paper title 17 S., Crowson, K., Schmidt, L., Kaczmarczyk, R., Jitsev, J.: LAION-5b: An open large-scale dataset for training next generation image-text models. In: Advances in Neural Inform...

    work page 2022

  33. [33]

    In: Interna- tional Conference on Learning Representations (2021) 3

    Song, Y., Sohl-Dickstein, J., Kingma, D.P., Kumar, A., Ermon, S., Poole, B.: Score- based generative modeling through stochastic differential equations. In: Interna- tional Conference on Learning Representations (2021) 3

    work page 2021

  34. [34]

    Tsai, Y.L., Hsu, C.Y., Xie, C., Lin, C.H., Chen, J.Y., Li, B., Chen, P.Y., Yu, C.M., Huang, C.Y.: Ring-a-bell! how reliable are concept removal methods for diffusion models? In: International Conference on Learning Representations (2024) 3

    work page 2024

  35. [35]

    In: Advances in Neural Information Processing Systems

    Vaswani,A.,Shazeer,N.,Parmar,N.,Uszkoreit,J.,Jones,L.,Gomez,A.N.,Kaiser, L., Polosukhin, I.: Attention is all you need. In: Advances in Neural Information Processing Systems. vol. 30 (2017) 4

    work page 2017

  36. [36]

    In: Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Wang, Y., Li, O., Mu, T., Hao, Y., Liu, K., Wang, X., He, X.: Precise, fast, and low- cost concept erasure in value space: Orthogonal complement matters. In: Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 28759–28768 (2025) 2, 4, 7, 8, 10, 11, 13, 22

    work page 2025

  37. [37]

    Woodbury, M.A.: Inverting modified matrices. Tech. Rep. Memorandum Report 42, Statistical Research Group, Princeton University (1950) 3, 4

    work page 1950

  38. [38]

    In: International Conference on Learning Representations (2025) 2, 4

    Yoon, J., Yu, S., Patil, V., Yao, H., Bansal, M.: SAFREE: Training-free and adap- tive guard for safe text-to-image and video generation. In: International Conference on Learning Representations (2025) 2, 4

    work page 2025

  39. [39]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops

    Zhang, E., Wang, K., Xu, X., Wang, Z., Shi, H.: Forget-me-not: Learning to forget in text-to-image diffusion models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. pp. 1755–1764 (2024) 2, 3

    work page 2024

  40. [40]

    In: Advances in Neural Information Pro- cessing Systems, Datasets and Benchmarks Track (2024) 3 18 Upman et al

    Zhang, Y., Fan, C., Zhang, Y., Yao, Y., Jia, J., Liu, J., Zhang, G., Liu, G., Kom- pella, R.R., Liu, X., Liu, S.: Unlearncanvas: A stylized image dataset to benchmark machine unlearning for diffusion models. In: Advances in Neural Information Pro- cessing Systems, Datasets and Benchmarks Track (2024) 3 18 Upman et al. A Additional Method Details This sect...

    work page 2024

This paper was first reviewed by glm-5.2 on July 7, 2026.