{"id":"86922277-7d51-41ab-8408-48c57ed7e953","arxiv_id":"2504.20111","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A black-box attack using one watermarked image and a proxy VAE can forge or remove latent-noise diffusion watermarks by perturbing images in the VAE's latent space.","lead":"The paper shows that watermarks embedded in the initial noise of diffusion models can be forged or removed using only a single watermarked image and no access to the model's internal denoising network. If the attack holds, it breaks a common assumption that such watermarks are hard to forge, pushing watermark designers to rethink their schemes.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Proxy-VAE transferability is the load-bearing assumption: Eq. 3 minimizes distance in an off-the-shelf VAE's latent space, but the detector uses DDIM inversion; transfer is only tested with same-family VAEs, not a genuinely different proxy.","rationale":"The paper's contribution hinges on transferring an optimization performed in a proxy VAE's encoder space to the true DDIM-inversion detector's watermark region. Section 4.2 validates only the existence of a linearly separable region in the true VAE space; it does not establish that a proxy VAE shares that geometry. The only cross-VAE experiment in Section 5 pairs SDv1.4's VAE with SDv2.0 watermarks, two models from the same family and training distribution. The Introduction's claim that a VAE trained on a similar dataset suffices is therefore broader than the evidence. This is the weakest link because, if transfer fails, the attack cannot run without access to the exact diffusion-pipeline VAE, which would weaken the black-box claim. The paper does provide independent support: a public codebase, evaluation across four watermark schemes and two diffusion models, and the Tree-Ring removal result at higher perturbation budgets. These support a conditional verdict. I agree with the Reader's identification of the proxy-transfer assumption as the central fragile premise. The removal ASR of 0% for RingID and WIND in Table 5 is an acknowledged scope limitation, not a flaw in the surviving claims, and it is already stated in the Discussion. The proposed test, swapping the proxy VAE for one trained on a different distribution, would settle whether the transfer assumption holds beyond the same-family case.","tokens_in":16191,"tokens_out":6067,"duration_ms":57827,"concrete_test":"Hold all attack parameters fixed (Tree-Ring on SDv1.4, λ=2×10^4) and replace the SDv1.4 VAE with a VAE trained on a different dataset or distribution, e.g., an ImageNet-trained autoencoder or the VAE from a face-diffusion model. Recompute the forgery ASR over the same 200 COCO images. If ASR drops materially (more than about 20 points), the proxy-transfer assumption is the load-bearing condition; if ASR stays near the reported 86.93%, the assumption is robust across proxy choice.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central mechanism depends on the assumption that minimizing distance in a proxy VAE's latent space (Eq. 3) transfers to the actual DDIM-inversion watermark detector's geometry. The paper's only direct evidence for this is the SVM linear-separability experiment in Section 4.2, which is run in the true VAE's latent space, not in a proxy's. The cross-model evaluation in Section 5 uses SDv1.4's VAE to optimize attacks on SDv2.0 watermarks; these are two Stable Diffusion VAEs from the same family and training distribution. No experiment varies the proxy VAE independently, so the Introduction's claim that 'a VAE that was trained on a similar dataset suffices' is unsupported for genuinely different proxies. If the proxy encoder's geometry does not reflect the inversion geometry of the true U-Net, the optimized perturbation will not enter the watermark region defined in Definition 1, and the single-image black-box attack fails.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes black-box forgery and removal attacks against latent-noise watermarking schemes for diffusion models. The attacker is assumed to have access to a single watermarked image and an off-the-shelf VAE (possibly not the exact VAE used by the diffusion model), but not the denoising U-Net or the secret key. The forgery attack (Eq. 3) optimizes an additive perturbation to a clean image so that its proxy-VAE latent representation moves close to the latent representation of the watermarked reference image; the removal attack (Eq. 4) similarly pushes a watermarked image's latent toward that of a plain mean-valued image. The method is evaluated on Tree-Ring, RingID, WIND, and Gaussian Shading across SDv1.4 and SDv2.0, using 200 image pairs per setting, with attack success rates defined by the schemes' p-value or bit-accuracy thresholds. The paper reports high forgery success for RingID, WIND, and Gaussian Shading, moderate-to-high success for Tree-Ring, and successful removal mainly for Tree-Ring, with an honest discussion of the difficulty of removal for multi-pattern watermarks.","tokens_in":16425,"tokens_out":5509,"duration_ms":55246,"significance":"If the results hold, this is a meaningful contribution to the security analysis of latent-noise watermarks. The paper demonstrates that a single watermarked image can suffice to forge a watermark in a black-box manner, without access to the denoising network, and it evaluates the attack across four watermarking schemes and two diffusion models. The public codebase is a strength for reproducibility, and the use of first-attempt success (rather than cherry-picked attempts) makes the evaluation protocol cleaner than in some prior work. The reported failure of removal for RingID and WIND is also a useful, honest data point. However, the significance depends on the transferability of proxy-VAE optimization to the DDIM-inversion detector geometry and on the representativeness of the reference-image selection; both assumptions are currently tested only under favorable conditions.","major_comments":[{"comment":"The claim that 'a VAE that was trained on a similar dataset suffices' is supported only by using the SDv1.4 VAE to attack watermarks embedded via SDv2.0. These are two Stable Diffusion VAEs from the same model family and effectively the same training distribution. No experiment varies the proxy VAE independently, e.g., using a VAE from a different architecture or a different training domain. Since Eq. (3) optimizes in the proxy encoder's latent geometry while detection relies on DDIM inversion with a U-Net, this transfer is the load-bearing assumption for the black-box, single-image attack. Please add experiments with a genuinely different proxy VAE (for example, a VAE from a different latent diffusion model or an autoencoder trained on a different image domain) or explicitly limit the claim to same-family proxies. Without such evidence, the Introduction's 'similar dataset suffices' assertion is not established.","section":"Section 4.3, Eq. (3); Section 5, Experimental Setup"},{"comment":"The reference watermarked images used for the forgery attack are generated from simple prompts because, as the paper states, 'the resultant images contain more visible watermark patterns/signal due to lower amounts of high-frequency information.' This selection directly biases the reported attack success rates upward relative to arbitrary watermarked images that an attacker would encounter in practice. The paper does not report forgery ASR when the reference image is generated from a complex prompt (e.g., the Gustavosta prompts used to create the main watermarked set). Please report this variant, or provide evidence that simple-prompt references are representative of real-world target images. This is important because the attack optimizes toward a single reference latent, and the size of the vulnerable region may depend on the reference image's content.","section":"Section 5, Experimental Setup"}],"minor_comments":[{"comment":"All ASR values are point estimates from 200 pairs with no confidence intervals or standard errors. Given that the paper makes comparative statements (e.g., 'almost perfect' versus '≈85%'), please add standard errors or confidence intervals to support the strength of the conclusions.","section":"Tables 1, 2, 5, 6"},{"comment":"The FID metric is reported for what appears to be a single set of attacked images; please clarify how FID is computed in this paired setting (e.g., between the original and attacked distributions) and note that FID is a distributional metric rather than a per-image distortion measure.","section":"Tables 1, 2, 5, 6"},{"comment":"The optimization runs for 15,000 iterations with a learning rate of 0.02 per image; this computational cost should be stated in the main text so that readers can judge the practicality of the attack.","section":"Appendix 11.3"},{"comment":"The removal objective uses a plain image with all pixel values equal to the mean of the watermarked image. The justification is relegated to the appendix; the main text should at least summarize the comparison with real-image guidance, since the choice is non-obvious and affects the success of the removal attack.","section":"Section 4.3, Eq. (4)"},{"comment":"There are several typographical issues, including inconsistent spacing in 'V AE' and some misspellings. The manuscript should be proofread for these presentation issues.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The stress-test concern about proxy-VAE transferability is legitimate and is reflected in major comment 1. The paper's central attack is not circular—the optimization does not use the detector or the secret key—but its claimed generality rests on an assumption that is only tested within the Stable Diffusion VAE family. The reference-image selection bias in major comment 2 is also a real evaluation concern; both are fixable with additional experiments or appropriately narrowed claims. The WIND implementation is cited via an anonymous ICLR submission URL, which may not be a stable resource; please ask the authors to provide a more permanent reference."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know this paper is worth a look. The attack setting is genuinely new: one watermarked image, no U-Net, no detector access, no auxiliary diffusion model. The authors show that in a proxy VAE's latent space, pulling a clean image toward a single watermarked image is enough to fool DDIM-inversion watermark detectors across four schemes (Tree-Ring, RingID, WIND, Gaussian Shading) and two Stable Diffusion versions. They also verify against held-out detectors with no circularity—no detector or key used during optimization. That is a solid, honest empirical contribution, and the code is public.\n\nThe SVM motivation experiment (linear separability of watermarked vs. clean latents) is a nice intuition-builder, and the paper is candid about where removal fails: RingID and WIND removal is 0% at all tested perturbation budgets, and the successful Tree-Ring removal comes at PSNR ~20 dB, which is not imperceptible. The presentation of the trade-off via λ is fine.\n\nNow the soft spots, in proportion. The stress-test concern is real: the paper claims \"a VAE trained on a similar dataset suffices,\" but the only cross-model evidence is SDv1.4's VAE attacking SDv2.0 watermarks. Those are same-family VAEs, likely nearly identical. No genuinely different proxy (e.g., a VAE from another model family or a standalone image autoencoder) is tested. So the load-bearing assumption—that proxy latent distance transfers to DDIM-inversion geometry—is plausible but only partially supported. This is not fatal, but it is under-tested, and the claim in the introduction is broader than the evidence.\n\nSecond, reference watermarked images for forgery are generated from simple prompts, which the authors themselves say have \"more visible watermark patterns/signal.\" That selection likely inflates ASR relative to watermarked images from complex, realistic prompts. This should be tested directly.\n\nThird, the evaluation is all point estimates over 200 pairs, no confidence intervals. Minor, but it would help.\n\nThe central argument—forgery works under the stated conditions—holds up. The generalizability claims (proxy VAE family, prompt difficulty) are where it gets shaky. I would send this to peer review rather than desk-reject, and expect a conditional accept with requests for a dissimilar-proxy experiment and a more realistic prompt distribution. If I were working on watermarking, I'd cite it.","headline":"A genuinely new attack setting—single-image, black-box watermark forgery without the U-Net—that mostly delivers, but the proxy-VAE transfer claim is under-tested and the removal results are weaker than the abstract implies.","tokens_in":16895,"tokens_out":2617,"would_cite":true,"duration_ms":27826,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A single watermarked image is enough to forge or erase latent-noise diffusion watermarks.","keywords":["diffusion model watermarking","latent noise watermark","watermark forgery","watermark removal","DDIM inversion","adversarial attack","black-box attack","AI image provenance"],"falsifier":"Repeat the forgery attack using a proxy VAE trained on a very different domain, such as medical or cartoon images; if the attack success rate falls to chance while the same-domain proxy stays high, the proxy-transfer premise is false and the attack is not black-box in the claimed sense. Alternatively, take a non-watermarked image whose proxy latent distance to a watermarked image is zero and check whether the detector still rejects it; if rejection persists, latent closeness does not imply watermark-region entry.","tokens_in":16073,"feed_emoji":"🖼️","tokens_out":7370,"duration_ms":65108,"temperature":0.7,"pith_summary":"Most diffusion-model watermarks hide a secret key in the initial noise, and detectors recover that key by inverting the image back to noise. This paper argues that the inversion map is many-to-one: many clean images share the same recovered initial noise, so each key corresponds to a whole region of image-latent space, not a single point. The authors show that with just one watermarked image and an off-the-shelf encoder, which need not be the model's own encoder, an attacker can nudge any unrelated image into that region and be falsely verified as watermarked, or nudge a watermarked image out of it to erase the mark. They demonstrate high forgery success against Tree-Ring, RingID, WIND, and Gaussian Shading on two Stable Diffusion versions, and effective removal for Tree-Ring. If correct, this means current initial-noise watermarking is forgeable from a single public example, which matters for attribution and misinformation defenses.","feed_headline":"One watermarked image can forge or erase latent-noise watermarks","feed_subtitle":"The attack needs only a proxy encoder and defeats four major latent-noise watermarking schemes.","key_machinery":"The load-bearing object is the watermark region, defined as all clean latents whose DDIM inversion matches the secret key below the detector's threshold. The mechanism behind it is the many-to-one inversion map: the same initial noise can denoise into many images under different prompts, but detection inverts with an empty prompt, so a key's preimage in latent space is a region rather than a point. The attack operationalizes this region using a proxy VAE encoder $E_{\\phi}$: for forgery it solves $\\min_{\\delta} \\|E_{\\phi}(x^{(c)}+\\delta)-E_{\\phi}(x^{(w)})\\|_2 + \\lambda\\|\\delta\\|_2$, and for removal it targets the mean of the watermarked image $E_{\\phi}(\\mu_{x^{(w)}})$ instead. A linear separator experiment shows the region has a usable direction in latent space, and the optimization exploits the encoder's non-smoothness to keep perturbations small.","core_discovery":"The central claim is that latent-noise watermarking schemes, where a secret key is embedded in the initial noise used by a diffusion model, contain a geometric vulnerability: because DDIM inversion with an empty prompt maps many different clean latents to the same recovered key, the set of images that verify as watermarked forms a region in the clean latent space, not isolated examples. The paper defines this watermark region and shows it is linearly separable from non-watermarked latents. It then converts this into a black-box attack: minimize the distance between the target image's encoding and a single reference watermarked image's encoding in a proxy VAE latent space, and the resulting imperceptible perturbation moves the target inside the region, so DDIM inversion recovers the key and the detector accepts it. Reversing the objective, pulling a watermarked latent toward a plain mean image, removes the mark. The attack needs only one watermarked image and no access to the denoising network or the secret key. Experiments report near-perfect forgery on RingID, WIND, and Gaussian Shading, roughly 79 to 94 percent success on Tree-Ring, and strong removal on Tree-Ring while multi-pattern watermarks resist removal.","pith_inferences":["A natural extension is that content-dependent watermarks, where the embedded message is a function of the image itself, may close the region loophole because an attacker would need to know the message-generation rule, not just one example.","The region-entry view suggests a detector-side defense: reject images whose recovered key matches but whose latent sits suspiciously close to a known reference latent, or require multiple independent inversions with different prompts or encoders before accepting the key.","Although the paper evaluates generic non-watermarked images, the forgery mechanism does not depend on image semantics; if the transfer assumption holds, targeted forgery of a specific harmful image should work with the same objective, which would be a stronger real-world concern than the reported average success rate.","The proxy-encoder assumption could be stress-tested across encoder families; if the attack transfers across architectures that never saw the detector's inversion geometry, it would suggest the region is a property of the diffusion prior itself, not of a particular VAE."],"forward_implications":["Any latent-noise watermark that relies on empty-prompt DDIM inversion inherits a region-based forgery surface; a single watermarked sample can be used to falsely attribute unrelated images to the model owner.","The attack succeeds without access to model weights, the secret key, or the denoising U-Net; a proxy VAE trained on similar imagery is enough, making the practical threat broader than prior attacks that required key or model access.","Forgery and removal asymmetries follow from watermark content: single-pattern Tree-Ring is harder to forge but easy to remove, while multi-pattern RingID, WIND, and Gaussian Shading are easy to forge but resist removal, so the choice of key structure determines which attack is practical.","Reported image-quality metrics at high $\\lambda$ show the successful perturbations are subtle, meaning detection cannot rely on visible distortion as a simple defense.","The results imply that distortion-free watermarking does not guarantee robustness against adversarial manipulation; robustness must be tested against region-based attacks, not just transformations or cropping."],"supporting_citations":[{"why":"Supplies the Tree-Ring watermarking scheme, the main target for forgery and removal.","marker":"[35]"},{"why":"Supplies RingID, a multi-key watermarking scheme tested for both attacks.","marker":"[5]"},{"why":"Supplies WIND, a two-stage noise watermarking scheme tested for both attacks.","marker":"[2]"},{"why":"Supplies Gaussian Shading, a distribution-preserving watermarking scheme tested for both attacks.","marker":"[39]"},{"why":"Provides the DDIM sampling and inversion formulas that define the many-to-one mapping geometry.","marker":"[31]"},{"why":"Supplies the null-text DDIM inversion procedure used by watermark detectors and by the attack's motivation.","marker":"[23]"},{"why":"Supplies Stable Diffusion and its VAE encoder, the latent space in which the attack optimizes perturbations.","marker":"[27]"},{"why":"Provides the COCO validation set used as the source of non-watermarked images for forgery tests.","marker":"[19]"}],"fun_headline_variants":["Single image breaks latent-noise diffusion watermarks","One photo forges or erases diffusion watermarks","Latent watermark cracked with a single input","One sample compromises four diffusion watermark schemes","Forge and erase watermark from one image: attack unveiled"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that minimizing distance in the proxy VAE's latent space transfers to the detector's DDIM-inversion geometry, so images that look close in the proxy space also invert to similar initial-noise patterns.","fun_headline_variants_meta":{"raw":{"variants":["Single image breaks latent-noise diffusion watermarks","One photo forges or erases diffusion watermarks","Latent watermark cracked with a single input","One sample compromises four diffusion watermark schemes","Forge and erase watermark from one image: attack unveiled"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000219,"raw_usage":{"total_tokens":1470,"prompt_tokens":1000,"completion_tokens":470,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":616,"completion_tokens_details":{"reasoning_tokens":400}},"tokens_in":616,"tokens_out":470,"duration_ms":5418,"temperature":1.0,"reasoning_tokens":400,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:54:27.405156+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Repeat the forgery attack using a proxy VAE trained on a very different domain, such as medical or cartoon images; if the attack success rate falls to chance while the same-domain proxy stays high, the proxy-transfer premise is false and the attack is not black-box in the claimed sense. Alternatively, take a non-watermarked image whose proxy latent distance to a watermarked image is zero and check whether the detector still rejects it; if rejection persists, latent closeness does not imply watermark-region entry.","supporting_citations":[{"cited_title":"Ringid: Rethinking tree-ring watermarking for enhanced multi-key identification","cited_arxiv_id":null,"evidence_quote":"Supplies RingID, a multi-key watermarking scheme tested for both attacks."},{"cited_title":"Gaussian shading: Prov- able performance-lossless image watermarking for diffusion models","cited_arxiv_id":null,"evidence_quote":"Supplies Gaussian Shading, a distribution-preserving watermarking scheme tested for both attacks."},{"cited_title":"Microsoft coco: Common objects in context","cited_arxiv_id":null,"evidence_quote":"Provides the COCO validation set used as the source of non-watermarked images for forgery tests."}],"review_version":1}