REVIEW 5 major objections 6 minor 24 references
FlowErase-OPD: Multi-Concept Erasure via Anchored On-Policy Distillation in Flow Matching Models
T0 review · 5 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read Multi-concept erasure in flow-matching text-to-image models can be done by distilling single-concept erased teachers into one LoRA student, anchored by a frozen retention teacher and reweighted by erase difficulty, beating existing…
desk verdict Solid multi-concept erasure method with a genuinely new training objective, but the flagship nudity numbers may be inflated by using the same detector in training and evaluation. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the anchored multi-teacher OPD objective with an adaptive allocation loop. On-policy distillation (OPD) treats each denoising step as a Gaussian transition, so matching teacher and student reduces to minimizing $\sum_t \tfrac{1}{2}\|\mu_\theta(x_t,t)-\mu_{\phi_k}(x_t,t)\|_2^2$ over the student's own rollouts. Anchored Multi-Teacher Distillation (AMTD) indexes teachers $k$ that are either concept erasure experts $\phi_c$ or the frozen anchor teacher $\theta_0$, and samples prompts $c_k$ accordingly, with a linear warmup on the retention batch ratio so erasure directions form before retention constraints dominate. Adaptive Retention Control (ARC) computes an EMA-smoothed unweighted erase loss $\bar{\ell}_k$ per concept, turns it into a sampling share $s_k$ and a clipped loss weight $w_k$, and adjusts the retention fraction $\rho$ by a hysteresis rule with thresholds $\ell_{\mathrm{lo}}=0.001$ and $\ell_{\mathrm{hi}}=0.004$. The mechanism's job is to make the training budget follow erasure difficulty automatically, which is what lets one LoRA module hold many erasures without destroying general generation.
What would settle it
Fix the total training budget and replace ARC with uniform sampling and fixed loss weights for the same twelve concepts, keeping AMTD's anchor teacher; if the uniform version matches or beats FlowErase-OPD on both average ASR and FID, the adaptive difficulty signal is not carrying the claimed improvement. A second check is to sweep $\ell_{\mathrm{lo}}$ and $\ell_{\mathrm{hi}}$ over one order of magnitude and record whether per-concept ASR moves by more than a few points; large swings would indicate the controller is miscalibrated.
Extended reading notes
Core claim
On the paper's own terms, the central discovery is that on-policy distillation is a workable mechanism for multi-concept erasure in flow matching models. Because each solver step of a rectified-flow generator is a Gaussian transition with fixed covariance, the per-step KL divergence between student and teacher reduces to a squared $L^2$ distance between their predicted transition means; the student rolls out under its own policy and is trained to match each teacher at every step. FlowErase-OPD instantiates one expert teacher per concept (obtained by FlowErase-RL), one frozen anchor teacher equal to the original model, and a LoRA student, then optimizes the unified AMTD objective with ARC controlling teacher sampling and loss weights. The paper's reported result is that this jointly erases all twelve tested concepts, achieves the best average attack-success rate on nudity benchmarks, keeps object and artist erasure rates near or above the single-concept teacher, and preserves FID/CLIP quality better than training-based baselines. The method also transfers to Stable Diffusion v1.4 when the teachers are ESD models, which the paper reads as evidence that the distillation framework, not the specific teacher type, is the load-bearing component.
Load-bearing premise
The load-bearing premise is that the EMA-smoothed, unweighted on-policy erase loss of a concept is a dependable and comparable measure of how hard that concept is to erase, and that the fixed thresholds and initial nudity-specific weights stay well calibrated across different concepts and prompt distributions.
Editorial extensions
If this is right
- New concepts can be added by training one additional single-concept erasure teacher and re-distilling into the same LoRA student, so multi-concept erasure becomes composable rather than a single monolithic unlearning run.
- The anchor teacher makes retention part of the objective, so the method directly targets the drift problem that makes naive multi-concept erasure degrade FID and CLIP scores.
- ARC's difficulty-aware reweighting removes the need for per-concept hyperparameter tuning: concepts that resist erasure automatically receive more sampling share, loss weight, and a lower retention fraction.
- Because FlowErase-OPD runs on student rollouts and only needs per-step transition means, it applies to transformer-based flow matching backbones such as FLUX.1 Schnell rather than only U-Net diffusion models.
- The reported robustness against MMA, Ring-a-bell, P4D, and UnlearnDiff implies that a distilled LoRA eraser can survive the common adversarial-prompt attacks used to test NSFW suppression.
Reading between the lines
- Editorial inference: The same ARC machinery could govern other multi-teacher distillation tasks with conflicting objectives, such as style transfer, watermark removal, or safety editing, whenever a per-teacher loss can be measured on the student's own rollouts.
- Editorial inference: The method decouples concept selection from combination cost: because the teachers are trained independently, the expensive per-concept step happens once, and only the cheap LoRA distillation must be rerun when the concept set changes.
- Editorial inference: The hand-picked thresholds and the nudity-specific initial weight suggest the controller is calibrated to this twelve-concept set; a rank-normalized difficulty signal would be a natural testable extension for larger concept sets.
- Editorial inference: If the anchor teacher's interference with proximal concept erasure (a limitation the paper itself states) is real, a promising extension would be concept-aware prompt sanitation or a per-concept anchoring schedule.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes FlowErase-OPD, a multi-concept erasure method for flow-matching text-to-image models. It distills multiple FlowErase-RL single-concept-erased FLUX models into one LoRA student, using an anchored retention teacher (the frozen original model) to preserve general generation and an Adaptive Retention Control (ARC) mechanism that reweights per-concept sampling shares and loss weights based on an EMA-smoothed erase loss. The paper reports experiments on nudity, object, and artist-style erasure on FLUX.1 Schnell and claims state-of-the-art results, including robustness to adversarial attacks.
Significance. If the evaluation concerns are resolved, the method would be a practical contribution: it is, to my knowledge, the first on-policy distillation framing for multi-concept erasure in flow matching models, it uses only a lightweight LoRA module, and the AMTD/ARC design gives a concrete mechanism for controlling the erasure-retention trade-off. The paper does not release code or trained models, but the experimental protocol is sufficiently detailed to be reproduced in principle. However, the current evidence base is weakened by the unverified relationship between the teacher reward model and the NudeNet auditor, the missing non-target preservation metric in the object-erasure study, and the unvalidated hand-picked ARC hyperparameters, so the central state-of-the-art claim is not yet established.
major comments (5)
- [Experiments; Appendix 'Additional details of baseline' and 'Additional details of metrics'] The nudity evaluation may be circular. The FlowErase-RL teacher for the nudity concept is trained with a reward from a pretrained NSFW detection model (Appendix, 'Additional details of baseline, FlowErase-RL'), while the evaluation in Tables 1 and 2 counts exposed body parts with the NudeNet detector (Appendix, 'Additional details of metrics'). The manuscript never names the reward model or shows that it is different from NudeNet. Because FlowErase-OPD distills exactly these teachers, the low ASR values could reflect detector-specific reward hacking rather than semantic removal of nudity. Please name the reward model, add an evaluation with a different detector (for example, Q16 or a CLIP-based nudity classifier), and report prompt-based or human checks that the concept is actually absent from the generated images.
- [Table 3 and object erasure text] The object-erasure evaluation omits the non-target preservation metric for the proposed method. The text says that ASR for non-target concepts is measured, but Table 3 leaves the ASR_k cell blank for Ours, and the following sentence states that non-target values are left out because all target objects are erased at once. Without ASR_k it is impossible to tell whether the low target ASR is achieved by selective erasure or by destroying unrelated object generation. Please report ASR_k for Ours (for example, by generating prompts for each object individually and measuring classifier accuracy on the remaining nine concepts) and include per-concept retention numbers analogous to the detail given in Table 4.
- [Contribution 2 and 'Details of implementation' in the appendix] The claim that ARC enables 'automatic rebalancing ... without per-concept hyperparameter tuning' is contradicted by the implementation details, which set a nudity-specific initial sampling weight of 4.0 and a distillation loss weight of 3.0, as well as hand-picked thresholds ell_lo=0.001, ell_hi=0.004 and bounds rho_min=0.15, rho_max=0.30. Since the adaptive mechanism is only one component, the paper should either soften the claim or provide a sensitivity analysis over these thresholds and initial weights. Without such an analysis, the main advantage of ARC over manual per-concept tuning is not established.
- [Method, Preliminaries, Eqs. (2)-(3)] The derivation of the OPD objective is not valid as stated. In Eq. (2) the KL divergence between the student and teacher Gaussian transitions is ||mu_theta - mu_phi||^2 / (2 sigma_t^2). In the deterministic ODE limit sigma_t -> 0 this quantity diverges for any nonzero mean mismatch; it does not 'simplify' to the unweighted L2 loss in Eq. (3). Please present Eq. (3) explicitly as a heuristic mean-matching objective with a chosen scaling, or give the limiting argument that justifies dropping the 1/(2 sigma_t^2) factor. As written, the mathematical grounding of the core loss is unclear.
- [Results Analysis after Table 2; Table 3] The statement that FlowErase-OPD yields 'simultaneous improvements in ASR, CLIP and FID Scores' is not supported by the reported numbers. In Table 2, Ours has the best FID but the second-best CLIP (32.19 versus 32.57 for FlowErase-RL). In Table 3, Ours has CLIP 31.25, below FlowErase-RL's 31.58, and FID 42.05, well above DVE's 23.68. Please revise the claim to describe the trade-off explicitly and, ideally, report standard deviations or multiple seeds for the main tables so that 'consistently improving' can be assessed.
minor comments (6)
- [Table 5 and surrounding text] The table header uses 'ASR(%)' but the text says the metric is the Top-3 classification accuracy of the Q16 style classifier; please align the column name with the metric definition.
- [Figure 2 caption] The caption says 'RAC dynamically adjusts...' but the method is called ARC; please correct the abbreviation.
- [Table 8] The row label 'Vanila' is a typo for 'Vanilla'.
- [Ablation on The Number of Epochs] The text says Nudity erasure 'slightly decreases' with more epochs, but the 1000- to 2000-epoch comparison in Table 9 improves from 10.96 to 10.63; please clarify whether the claimed decrease refers only to the later portion of training.
- [Ablation on The Number of Concepts] The first sentence says the experiment is performed 'within the FLUX.1-dev framework' while the rest of the paper uses FLUX.1 Schnell; please clarify which backbone was used.
- [Ablation on Different Base Modules] The text says 'we performed erasure of the Nudity concept' but also says 12 target concepts are erased with ESD to form the teachers; please reconcile these statements and specify which concepts were erased for the SD v1.4 experiment.
Circularity Check
No significant circularity: the AMTD/ARC training objectives and the external evaluations are independent of the paper's central claims, and the FlowErase-RL teachers are an input component reproduced in experiments rather than a conclusion.
full rationale
The paper's derivation chain is not circular in the definitional sense. The student LoRA is trained with an on-policy squared-L2 distillation objective (Eqs. 3, 6, and 11) against teacher models produced by FlowErase-RL; those teachers are an input component, not the paper's conclusion, and the paper also reports FlowErase-RL as a separately run baseline, so the self-citation is not the only support for the central claim. The AMTD anchor-teacher objective and the ARC adaptive weighting (Eqs. 7-10) are heuristics whose contributions are tested by ablations (Tables 8 and 9) against external metrics: NudeNet body-part counts, FID, CLIP Score, a ResNet-50 ImageNet classifier, and the UnlearnDiff style classifier. The evaluations use external benchmarks and datasets (I2P, MMA, Ring-a-Bell, P4D, UnlearnDiff, MS-COCO), and no reported metric is defined in terms of the training loss itself, so there is no fitted parameter renamed as a prediction. The only caveat is a correctness risk rather than a demonstrated circularity: the FlowErase-RL nudity reward is described only as 'a pretrained NSFW detection model' (Appendix, Additional details of baseline), while the evaluation uses 'the NudeNet Detector' (Appendix, Additional details of metrics); the paper does not establish whether these are the same detector, so if they coincide the nudity numbers would partly reflect teacher imitation of detector-specific suppression. The appended Limitation section also concedes potential adversarial vulnerabilities introduced by the anchor teacher. These concerns do not amount to a circular reduction because the paper neither defines erasure success as matching the training reward nor relies solely on the self-cited teacher method to justify its results.
Assumptions & free parameters
free parameters (11)
- rho_star (target retention ratio) =
0.25
- E_warm =
10
- alpha (EMA smoothing) =
0.9
- Delta_rho (retention step) =
0.03
- rho_min, rho_max =
0.15, 0.30
- ell_lo, ell_hi =
0.001, 0.004
- kappa
- eta
- w_min, w_max
- Nudity initial sampling weight =
4.0
- Nudity initial distillation loss weight =
3.0
assumptions (4)
- standard math Shared-covariance Gaussian transitions give a closed-form per-step KL divergence (Eq. 2).
- domain assumption Anchoring the student to the original model via COCO prompts preserves general generation.
- domain assumption FlowErase-RL teachers are expert single-concept erasers and their behavior can be combined in one LoRA student.
- ad hoc to paper The unweighted erase loss is a valid difficulty signal for resource allocation.
Cite this review
Pith. "Pith review of FlowErase-OPD: Multi-Concept Erasure via Anchored On-Policy Distillation in Flow Matching Models." pith.science (2026). https://pith.science/paper/EAT75JAC
@misc{pith2026260807620,
author = {Pith},
title = {Pith review of: FlowErase-OPD: Multi-Concept Erasure via Anchored On-Policy Distillation in Flow Matching Models},
year = {2026},
howpublished = {\url{https://pith.science/paper/EAT75JAC}},
note = {Machine review of arXiv:2608.07620}
}
read the original abstract
Recent advances in flow matching models have substantially improved the quality of text-to-image generation, but have also raised increasing safety concerns due to their potential to generate harmful or undesirable content. Existing concept erasure methods for flow matching models predominantly focus on removing individual concepts, while effectively erasing multiple concepts simultaneously remains challenging. We propose FlowErase-OPD, a framework for multi-concept erasure based on on-policy distillation (OPD). Our approach first distills multiple single-concept erased models into a unified LoRA module and introduces Anchored Multi-Teacher Distillation (AMTD), which incorporates a retention teacher to mitigate the trade-off between concept erasure and preservation of generative capabilities. To further improve the coordination of multiple erasure objectives, we develop Adaptive Retention Control (ARC), which dynamically adjusts the sampling frequency and loss weight of each erasure teacher, together with the relative contribution of erasure and retention teachers throughout training. Extensive experiments on nudity, object, and artistic-style erasure demonstrate that FlowErase-OPD consistently improves the trade-off between erasure effectiveness, image quality, and semantic alignment, achieving state-of-the-art performance across diverse multi-concept erasure settings. Furthermore, the resulting models exhibit strong robustness against adversarial attacks. These results highlight the potential of on-policy distillation as a principled framework for safe and controllable generation in flow matching models.
Figures
Figures from the paper (6 more)
Reference graph
Works this paper leans on
-
[3]
Rombach,R.;Blattmann,A.;Lorenz,D.;Esser,P.;andOm- mer, B. 2022a. High-resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, 10684– 10695. Rombach,R.;Blattmann,A.;Lorenz,D.;Esser,P.;andOm- mer, B. 2022b. Stable Diffusion 2.0. Roose,K.2022. AnAI-GeneratedPictureWonanAr...
work page 2022
-
[4]
InForty- second International Conference on Machine Learning
Eraseanything: Enabling concept erasure in rectified flow transformers. InForty- second International Conference on Machine Learning. He,K.;Zhang,X.;Ren,S.;andSun,J.2015. DeepResidual Learning for Image Recognition.2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR),770–778. Heusel,M.;Ramsauer,H.;Unterthiner,T.;Nessler,B.;Klam- bauer, G...
work page 2015
-
[7]
InProceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society,363–374
AIArtanditsImpactonArtists. InProceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society,363–374. Kim, C. S.; Min, K.; and Yang, Y
work page 2023
-
[8]
Li, Q.; Yu, J.; Jiang, K.; Wei, Y.; Xing, Z.; Li, P.; Chu, R.; Zhang, S.; Liu, Y.; and Wu, Z
R.A.C.E.: Ro- bust Adversarial Concept Erasure for Secure Text-to-Image Diffusion Model.ArXiv, abs/2405.16341. Li, Q.; Yu, J.; Jiang, K.; Wei, Y.; Xing, Z.; Li, P.; Chu, R.; Zhang, S.; Liu, Y.; and Wu, Z
-
[10]
Lin, T.-Y.; Maire, M.; Belongie, S
SafeGen: Mitigating Sexually Explicit Con- tent Generation in Text-to-Image Models.Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security. Lin, T.-Y.; Maire, M.; Belongie, S. J.; Hays, J.; Perona, P.; Ramanan, D.; Dollár, P.; and Zitnick, C. L
work page 2024
-
[12]
arXiv preprint arXiv:2209.03003
Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint arXiv:2209.03003. Lu, S.; Wang, Z.; Li, L.; Liu, Y.; and Kong, A. W.-K
-
[14]
Hierarchical text-conditional image generation with clip latents.arXiv preprint arXiv:2204.06125, 1(2):
-
[16]
L.; Ghasemipour, K.; Gontijo Lopes, R.; Karagol Ayan, B.;Salimans,T.;etal.2022
Saharia,C.;Chan,W.;Saxena,S.;Li,L.;Whang,J.;Denton, E. L.; Ghasemipour, K.; Gontijo Lopes, R.; Karagol Ayan, B.;Salimans,T.;etal.2022. Photorealistictext-to-imagedif- fusion models with deep language understanding.Advances in neural information processing systems,35:36479–36494. Salimans, T.; and Ho, J
work page 2022
Show all 24 references
-
[17]
Schramowski, P.; Brack, M.; Deiseroth, B.; and Kersting, K
Progressive distillation for fast sampling of diffusion models.arXiv preprint arXiv:2202.00512. Schramowski, P.; Brack, M.; Deiseroth, B.; and Kersting, K
-
[18]
Denoisingdiffusion implicit models.arXiv preprint arXiv:2010.02502
Song,J.;Meng,C.;andErmon,S.2020. Denoisingdiffusion implicit models.arXiv preprint arXiv:2010.02502. Song, Y.; and Dhariwal, P
2020 arXiv
-
[19]
InInternational Conference on Learning Representations, volume 2024, 15078–15097
Improved techniques for trainingconsistencymodels. InInternational Conference on Learning Representations, volume 2024, 15078–15097. Srivatsan, K.; Shamshad, F.; Naseer, M.; Patel, V. M.; and Nandakumar, K
2024
-
[20]
Sun, Y.; Zhang, Z.; Zhong, X.; Zhou, Y.; Sun, S.; Chen, B.; Xia, S.-T.; and Xu, K
STEREO: A Two-Stage Frame- work for Adversarially Robust Concept Erasing from Text- to-ImageDiffusionModels.2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 23765– 23774. Sun, Y.; Zhang, Z.; Zhong, X.; Zhou, Y.; Sun, S.; Chen, B.; Xia, S.-T.; and Xu,...
2025 arXiv
-
[21]
Wang,Y.;Li,O.;Mu,T.;Hao,Y.;Liu,K.;Wang,X.;andHe, X.2025
Ring- A-Bell! How Reliable are Concept Removal Methods for Diffusion Models?ArXiv, abs/2310.10012. Wang,Y.;Li,O.;Mu,T.;Hao,Y.;Liu,K.;Wang,X.;andHe, X.2025. Precise,fast,andlow-costconcepterasureinvalue space: Orthogonal complement matters. In2025 IEEE/CVF Conference on Compute...
2025 arXiv
-
[22]
2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 7737–7746
MMA-Diffusion: MultiModal Attack on Diffusion Models. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 7737–7746. Yin, T.; Gharbi, M.; Park, T.; Zhang, R.; Shechtman, E.; Durand,F.;andFreeman,W.T.2024. Improveddistribution matching distillation for f...
2024
-
[23]
ArXiv, abs/2602.01089
Differential Vector Erasure: Unified Training-Free Concept Erasure for Flow Matching Models. ArXiv, abs/2602.01089. Zhong,X.;Zhou,Y.;Zhang,Z.;Li,J.;Sun,Y.;Chen,B.;Xia, S.-T.; Wang, X.; and Xu, K
-
[24]
Nudity”andthemostsimilarpromptsthatdonot containthe“Nudity
Closing the safety gap: Surgical concept erasure in visual autoregressive models. arXiv preprint arXiv:2509.22400. Details of implementation Details of hyper-parameters For all concept erasure tasks, the following settings remain identical.DuetotheresourceconstraintsoftheA6000...
2023
-
[25]
Glide:To- wards photorealistic image generation and editing with text- guided diffusion models.arXiv preprint arXiv:2112.10741
Nichol, A.; Dhariwal, P.; Ramesh, A.; Shyam, P.; Mishkin, P.;McGrew,B.;Sutskever,I.;andChen,M.2021. Glide:To- wards photorealistic image generation and editing with text- guided diffusion models.arXiv preprint arXiv:2112.10741. O’Connor,R.2022. StableDiffusion1vs2-WhatYouNeed ...
2021 arXiv
-
[2017]
ArXiv, abs/1706.08500
GANs Trained by a Two Time-Scale Update Rule Converge to a Nash Equilibrium. ArXiv, abs/1706.08500. Ho, J.; Jain, A.; and Abbeel, P
-
[2020]
Ho,J.;andSalimans,T.2022
Denoising diffusion probabilistic models.Advances in neural information pro- cessing systems, 33: 6840–6851. Ho,J.;andSalimans,T.2022. Classifier-freediffusionguid- ance.arXiv preprint arXiv:2207.12598. Jiang, H. H.; Brown, L.; Cheng, J.; Khan, M.; Gupta, A.; Workman, D.; Hann...
2022 arXiv
-
[2022]
Liu, X.; Gong, C.; and Liu, Q
Flow matching for generative modeling.arXiv preprint arXiv:2210.02747. Liu, X.; Gong, C.; and Liu, Q
-
[2023]
Gandikota,R.;Orgad,H.;Belinkov,Y.;Materzyńska,J.;and Bau, D
Erasing Concepts from Diffusion Models.2023 IEEE/CVF International Conference on Computer Vision (ICCV), 2426–2436. Gandikota,R.;Orgad,H.;Belinkov,Y.;Materzyńska,J.;and Bau, D
2023
-
[2024]
https://github.com/black- forest-labs/flux
FLUX. https://github.com/black- forest-labs/flux. Chavhan,R.;Li,D.;andHospedales,T.2024.Conceptprune: Concepteditingindiffusionmodelsviaskilledneuronprun- ing.arXiv preprint arXiv:2405.19237. Chen,R.;Guo,H.;Wang,L.;Zhang,C.;Nie,W.;andLiu,A.- A
2024 arXiv
-
[2025]
Deng, J.; Dong, W.; Socher, R.; Li, L
SAeUron: Interpretable Concept Unlearning in Diffusion Models with Sparse Au- toencoders.ArXiv, abs/2501.18052. Deng, J.; Dong, W.; Socher, R.; Li, L. J.; Li, K.; and Fei- Fei, L
-
[2026]
Li, X.; Yang, Y.; Deng, J.; Yan, C.; Chen, Y.; Ji, X.; and Xu, W
DiffusionOPD: A unified perspective of on-policy distillation in diffusion models.arXiv preprint arXiv:2605.15055. Li, X.; Yang, Y.; Deng, J.; Yan, C.; Chen, Y.; Ji, X.; and Xu, W
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.