{"id":"8265af21-3023-40a8-925d-2b425f96146c","arxiv_id":"2412.07274","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A diffusion model's conditional and unconditional scores can be combined to generate transferable adversarial perturbations for segmentation without a segmentation victim model.","lead":"This paper proposes an adversarial attack on image segmentation that uses a generative image model's scores instead of a pre-trained segmentation model to craft perturbations. If it works, attackers could fool segmentation systems without access to the target model or a surrogate segmentation network.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The attack's update direction is taken from a diffusion score evaluated at an unspecified timestep, so the central derivation that Eq. (10) minimizes log p(y|x) is not actually established.","rationale":"The strongest claim is that adversarial samples for segmentation can be generated solely from diffusion-model scores, and Eqs. (7)-(10) are the theoretical bridge that makes this claim non-heuristic. The reader correctly identifies the missing timestep as the most fragile premise. I agree: the diffusion score is trained as a function of (x_t, t), and the paper never states which t is used in Eq. (10), nor does it justify that the pseudo-adversarial image xadv_m lives on the Gaussian-perturbed manifold on which the score estimator is valid. This is load-bearing because it affects even the strong binary-segmentation results, not only the acknowledged multi-class weakness. The paper's own Sec. IV-B.3 admits the noise-addition step is an emulation of the diffusion forward process, which confirms the distribution mismatch is real rather than hypothetical. The empirical COD results and the query-based comparison provide useful evidence that the method works as an attack heuristic, and the transferability tables are informative; the issue is not that the experiments are fabricated, but that the theoretical derivation overclaims what Eq. (10) computes. The authors' self-stated limitations about multi-class performance and the implicit correlation between classification error and score estimation are scope constraints; they do not replace the need to resolve the timestep/distribution gap. My recommended action is to keep the reader's CONDITIONAL verdict, with the condition explicitly extended to require the authors to specify the noise level used during attack and to demonstrate that the score update remains valid (or is a deliberate heuristic) when xadv_m is not a Gaussian-perturbed sample.","tokens_in":18522,"tokens_out":7040,"duration_ms":77765,"concrete_test":"Obtain or implement the attack and instrument every sθ call to record the timestep t used. Then run the COD attack with ω=90, mmax=30, and δ=8/255 under four ablations: (a) t=0 for all steps; (b) t=m with α_m as in Eq. (10); (c) t sampled uniformly from {1,...,1000} per step; and (d) before scoring, add Gaussian noise of variance matching t to xadv_m instead of the δ_m update in Eq. (11). Compare the Table I/IV metrics (M, CC, Sα, Eξ). If all variants give similar attack success, the unspecified timestep is not load-bearing; if t=0 or random t collapses the attack, or if Gaussian noise changes results substantially, the paper must state the noise level and show xadv_m lies on the diffusion training manifold for Eq. (10) to be the claimed gradient.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central derivation (Eqs. 4-8) concerns gradients of the clean-image conditional likelihood log p(y|x). But the algorithm never computes that gradient directly. Eq. (10) evaluates sθ on the pseudo-adversarial image xadv_m, and Eqs. (15)-(16) train sθ as a function of (x_t, t) over a 1000-step noise schedule. Nowhere in Eq. (10), Eq. (11), or Algorithm 1 is the timestep t supplied to sθ stated; saying that α_m denotes the noise schedule as in Eq. (9) is not enough, because the score is only meaningful at a specified noise level. Moreover, xadv_m is updated by adding the adversarial direction δ_m, not by Gaussian noise, so even with a t chosen, xadv_m is not a sample from the diffusion marginal p_t used in training; the model is evaluated off-manifold. The ablation in Sec. IV-B.3 shows the authors know they are emulating noise, but an emulation does not establish the score identity. Consequently the claimed equivalence between the generative-score update and minimizing log p(y|xadv) rests on an unstated and untested assumption about the noise level and the distribution of xadv_m.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes an adversarial attack method for segmentation that replaces the conventional victim segmentation model with a conditional diffusion model. The authors use Bayes' rule to express the gradient of the conditional segmentation log-likelihood log p(y|x) as the difference between a conditional and an unconditional image-generation score, and then construct an iterative attack that evaluates these scores on pseudo-adversarial images. Experiments on camouflaged object detection (COD) and semantic segmentation compare the method with transfer-based and query-based black-box attacks, and an optional querying mechanism is introduced.","tokens_in":18774,"tokens_out":7623,"duration_ms":81596,"significance":"If the theoretical connection is made rigorous, the paper offers a novel and potentially influential perspective: generative models can act as surrogate victims for segmentation attacks, avoiding the need for a pre-trained segmentation network. The COD results in Tables I and II are strong, showing competitive or better transferability than established transfer-based attacks, and the optional query strategy in Table IV achieves competitive performance with very few queries. This is a creative and practically motivated contribution. However, the semantic segmentation results are weak, and the core derivation does not currently connect to the algorithm due to an unspecified noise-level/timestep in the score evaluation.","major_comments":[{"comment":"The attack procedure never specifies the diffusion timestep t at which the score network s_theta is evaluated. In Eq. (10), s_theta(x^adv_m | y) and s_theta(x^adv_m) are treated as gradients of the data log-likelihood, but the network was trained as a function of (x_t, t) over a 1000-step noise schedule. Saying that alpha_m denotes the noise schedule in Eq. (9) is insufficient because the score is only meaningful at a specified noise level. Moreover, even if a t were supplied, the update in Eq. (11) injects an adversarial-sign perturbation delta_m rather than Gaussian noise, so x^adv_m is not a sample from the diffusion marginal p_t used in training. The ablation in Section IV-B3 explicitly calls this an \"emulation\" of noise addition, which is a heuristic, not a derivation. This gap is load-bearing for the central claim that the update minimizes log p(y|x^adv); the authors should either state the timestep and justify the off-manifold evaluation, or explicitly present Eqs. (8)-(12) as a heuristic motivated by the score identity.","section":"Section III-C, Eq. (10)-(11), Algorithm 1"},{"comment":"The attack requires the ground-truth segmentation mask y as a conditioning input at attack time (see Eq. (10), s_theta(x^adv_m | y), and the algorithm input \"Image x and corresponding ground truth y\"). The paper never discusses how y is obtained in a black-box scenario. In standard threat models for segmentation, the attacker does not have pixel-level ground-truth annotations for test images. This substantially narrows the \"without task-specific victim model\" claim: the method still assumes access to task annotations, which is a significant practical restriction. The authors should discuss this assumption and, if possible, evaluate a variant that uses an estimated y or operates unconditionally.","section":"Section III-C, Algorithm 1"},{"comment":"The semantic segmentation results do not support the unqualified claim of \"effective adversarial attacks with good transferability.\" With either surrogate (PSPR50 or DL3Mob), the proposed method is consistently among the weakest transfer attacks; for example, with PSPR50 as surrogate and PSPR50 as victim, Ours achieves mIoU 0.520 and ACC 0.848, whereas RAP achieves mIoU 0.371, DI2-FGSM achieves 0.356, and PGD achieves 0.159. The same pattern holds for cross-model transfers. The paper acknowledges this as a \"multi-peak problem\" in Section IV-A.1, but the abstract and conclusion still claim effectiveness for segmentation in general. Since the paper is specifically about segmentation attacks, this scope limitation should be stated prominently in the abstract and conclusion rather than as a passing remark.","section":"Section IV-A, Table III"}],"minor_comments":[{"comment":"The notation for the noise schedule is ambiguous: Eq. (9) defines a per-step transition with alpha_t, while the training losses in Eqs. (15)-(16) use sqrt(1 - \\bar{alpha}_t). It is not clear whether alpha_m in Eqs. (10)-(11) is the per-step coefficient or the cumulative product, nor how m is mapped to the 1000-step schedule. Please clarify.","section":"Eq. (9)-(11), Eq. (15)-(16)"},{"comment":"The weighting hyperparameter omega is set to 90 empirically and described as canceling the gap between the actual and estimated scores, but no derivation or measurement of that gap is given. Table V shows the results are fairly stable to omega, so this is acceptable as an empirical choice; however, it should be presented as an experimentally tuned scale factor rather than as part of the theoretical derivation.","section":"Section III-B, Eq. (8)"},{"comment":"Several cells in Table IV appear to have formatting errors, such as \"10.7.155\" in the Bandit row and the query/time columns. Please reformat the table for readability.","section":"Table IV"},{"comment":"The phrase \"for the first time\" is used in Sections I and II-C, but diffusion-based attacks already exist (e.g., [23], [56], [57]). Please clarify the specific novel contribution relative to these works and avoid a blanket \"first time\" claim.","section":"Related Work, Section II-C"},{"comment":"Reference [33] appears to be misattributed: the sentence in Section II-A says \"Arnab et al. [33] introduced a generalized perturbation,\" but [33] is the universal perturbation paper by Metzen et al. Please correct the citation.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The main technical gap is the missing timestep in the score evaluation and the off-manifold evaluation of x^adv_m; this is likely addressable with a clarified protocol, but it is central to the paper's theoretical claim. The ground-truth-y assumption is also a substantial practical concern that should be discussed. The paper's empirical evaluation on COD is strong and supports the heuristic value of the approach even if the derivation needs reframing."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, the novel idea: replacing the segmentation victim model with a difference of conditional and unconditional diffusion scores (CFG-style) is new in the attack literature. The authors don't pretend the underlying identity is deep—it's Bayes' rule—but the construction is original. The COD experiments are strong: without any segmentation surrogate, their transfer attacks beat strong baselines across five backbones, and the query variant does it with 100 queries. The data isolation between diffusion training set and victim training set rules out the easiest circularity concern. This is a conference-level contribution if the central step holds up.\n\nNow the soft spots. The stress-test is right: the derivation in Eqs. (4)–(8) concerns the clean-image conditional likelihood log p(y|x+δ), but the algorithm never computes that gradient. Eq. (10) plugs xadv_m into sθ without specifying a diffusion timestep t, and xadv_m is not a sample from any training-time marginal p_t. Saying 'α_m denotes the noise schedule' is not enough, because the index m is never tied to the trained timestep, and the update direction δ_m is not Gaussian noise. The authors' own ablation (Sec. IV-B.3) calls it an 'emulation,' which is honest but does not provide the missing justification. The paper needs either a proper derivation of why this update approximates −∇ log p(y|xadv), or an explicit statement that this is a heuristic with empirical support.\n\nSecond, the semantic segmentation results are considerably weaker than the abstract and conclusion suggest. Table III shows the method often underperforms PGD and SegPGD on multi-class transfer; the authors acknowledge this in Sec. IV-A.1, but the framing 'demonstrated effectiveness in multi-class tasks' overstates it. A revision should state more carefully what the method actually achieves.\n\nThird, the method requires the ground-truth mask y at attack time. That is worth saying plainly, since it limits the threat model relative to fully black-box settings. Not fatal, but should be discussed.\n\nMy take: this deserves a serious referee. The idea is novel and the COD results are a genuine empirical advance. The timestep/off-manifold question is the load-bearing issue; if the authors can close that gap, this is a solid paper. If not, it is a useful empirical paper that needs repositioning. Either way, send it to review with a request for clarification and additional multi-class experiments, plus code and the exact inference-time timestep.","headline":"A genuinely novel construction for segmentation attacks without a segmentation victim model, with a real gap in the score-evaluation step that needs closing before the central claim is established.","tokens_in":19257,"tokens_out":3942,"would_cite":false,"duration_ms":37334,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Diffusion scores replace the segmentation victim model in attacks","keywords":["adversarial attack","semantic segmentation","diffusion model","score estimation","transferability","camouflaged object detection","black-box attack","generative victim model"],"falsifier":"Run the paper's attack with the diffusion timestep explicitly fixed to each value from 0 to 1000 and measure transfer attack success on a fixed victim set; if no fixed timestep reproduces the reported effectiveness, the claim is not pinned to a well-defined score evaluation. Alternatively, estimate $\\nabla \\log p(y|x^{\\mathrm{adv}})$ by finite differences on a real segmentation model and compare its cosine similarity with the paper's score difference; near-zero similarity would falsify the derivation.","tokens_in":18334,"feed_emoji":"🎯","tokens_out":4833,"duration_ms":47861,"temperature":0.7,"pith_summary":"For segmentation attacks, the paper's central claim is that you do not need a segmentation victim model at all: the score of a conditional diffusion model, combined with its unconditional score, gives the gradient direction that lowers the likelihood of the true segmentation under an attacked image. Since the conditional segmentation score can be written as the difference between a conditional and an unconditional image-generation score, a generative model trained for image generation can serve as the victim model for adversarial attack generation. This removes the usual prerequisite of a trained, gradient-accessible segmentation network. The paper demonstrates the attack on camouflaged-object detection and semantic segmentation, with transfer across different backbones and an optional query mode that uses very few queries. A sympathetic reader would care because it reframes adversarial attacks as a problem of estimating data-distribution gradients rather than exploiting a particular model's loss surface.","feed_headline":"Diffusion scores replace the segmentation victim model in attacks","feed_subtitle":"A conditional diffusion model's score difference crafts transferable adversarial perturbations without any segmentation network.","key_machinery":"The load-bearing object is Eq. (8), $s(y|x^{\\mathrm{adv}}) = \\omega\\big(s_\\theta(x^{\\mathrm{adv}}|y) - s_\\theta(x^{\\mathrm{adv}})\\big)$, the weighted difference of conditional and unconditional diffusion scores, mirroring classifier-free guidance. The diffusion model is trained as a noise estimator, so the negative of its output approximates the gradient of log data density; the unconditional branch is obtained by setting the conditioning variable to empty during training. Eq. (10) then turns that score difference into a per-step perturbation, and Eq. (11) adds noise to the pseudo-adversarial sample so the diffusion score is evaluated on inputs that resemble its training distribution. The same machinery supplies both the transfer attack and the optional query refinement, and $\\omega$ absorbs the gap between the true score and the estimated score.","core_discovery":"The paper's core discovery is that the log-likelihood of a label map $y$ given a perturbed image $x^{\\mathrm{adv}}$ decomposes by Bayes' rule, so minimizing $\\log p(y|x^{\\mathrm{adv}})$ is equivalent to minimizing $\\log p(x^{\\mathrm{adv}},y) - \\log p(x^{\\mathrm{adv}})$. Taking gradients, the conditional segmentation score equals a conditional image-generation score minus an unconditional image-generation score. Approximating those scores with a jointly trained conditional/unconditional diffusion model, and weighing their difference by a hyper-parameter $\\omega$ as in Eq. (8), yields a step direction that the paper uses exactly where PGD would use the victim segmentation model's loss gradient. The resulting attack is image-dependent and sample-dependent, transfers across architectures because it follows density rather than a surrogate's decision boundary, and can optionally query the victim to select the best accumulated perturbation. The paper states this as the first derivation of a victim model for segmentation from image-generation principles.","pith_inferences":["If the score difference is truly a density direction, the method should extend to conditional generation tasks beyond segmentation, such as depth estimation or edge detection, whenever a conditional diffusion model can be trained on paired data.","The undefined diffusion timestep at attack time means the reported attack is really one member of a family indexed by $t$; fixing $t$ explicitly could yield a tuning knob for transferability versus perceptibility that the paper leaves implicit.","The $\\omega$ parameter behaves like a guidance scale, so attack strength and transferability may trade off against each other in a way that mirrors classifier-free guidance rather than being an arbitrary robustness fix.","A direct comparison on the same victim models between the score-based direction and the surrogate gradient would separate how much of the transfer gain comes from density descent versus from the diffusion model's implicit smoothing."],"forward_implications":["Segmentation attacks can be launched without training or querying any segmentation surrogate; the same conditional diffusion model serves every target architecture.","Because the perturbation follows the data-distribution gradient, transfer across different backbones (ViT, ResNet, Swin, VGG, PVTv2) is a direct consequence rather than a side effect.","The attack degrades gracefully when querying is allowed: with a small number of queries it selects the best accumulated step, matching or beating query-based attacks that need thousands of queries.","The same recipe transfers to other dense prediction tasks with binary maps, as shown by experiments on salient object detection."],"supporting_citations":[{"why":"Establishes that the learned score approximates the gradient of the log data distribution, the premise behind Eqs. (6) and (7).","marker":"[21]"},{"why":"Supplies the Gaussian noising schedule and noise-estimator training used in Eqs. (9), (15), and (16).","marker":"[20]"},{"why":"Provides the weighted combination of conditional and unconditional scores that Eq. (8) adapts for segmentation.","marker":"[58]"},{"why":"Defines the iterative PGD update that the paper replaces with the score-based step in Eqs. (10)-(12).","marker":"[4]"},{"why":"Defines SegPGD, the segmentation-specific white-box attack baseline the paper compares against.","marker":"[1]"},{"why":"Provides the diffusion model architecture and training setup the paper adopts for the score network.","marker":"[61]"}],"fun_headline_variants":["Diffusion scores craft attacks without segmentation nets","No segmentation model needed: diffusion scores attack","Generative victim: diffusion score difference for attacks","Adversarial attacks via diffusion score gap","Victim model derived from image generation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The diffusion model's score, evaluated on an attacked image without specifying which noise level the model was trained for, still points in the direction that genuinely lowers the likelihood of the true label.","fun_headline_variants_meta":{"raw":{"variants":["Diffusion scores craft attacks without segmentation nets","No segmentation model needed: diffusion scores attack","Generative victim: diffusion score difference for attacks","Adversarial attacks via diffusion score gap","Victim model derived from image generation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000155,"raw_usage":{"total_tokens":1173,"prompt_tokens":865,"completion_tokens":308,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":481,"completion_tokens_details":{"reasoning_tokens":242}},"tokens_in":481,"tokens_out":308,"duration_ms":59312,"temperature":1.0,"reasoning_tokens":242,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T18:55:37.306143+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the paper's attack with the diffusion timestep explicitly fixed to each value from 0 to 1000 and measure transfer attack success on a fixed victim set; if no fixed timestep reproduces the reported effectiveness, the claim is not pinned to a well-defined score evaluation. Alternatively, estimate $\\nabla \\log p(y|x^{\\mathrm{adv}})$ by finite differences on a real segmentation model and compare its cosine similarity with the paper's score difference; near-zero similarity would falsify the derivation.","supporting_citations":[{"cited_title":"Generative modeling by estimating gradients of the data distribution,","cited_arxiv_id":null,"evidence_quote":"Establishes that the learned score approximates the gradient of the log data distribution, the premise behind Eqs. (6) and (7)."},{"cited_title":"Denoising diffusion probabilistic models,","cited_arxiv_id":null,"evidence_quote":"Supplies the Gaussian noising schedule and noise-estimator training used in Eqs. (9), (15), and (16)."},{"cited_title":"Classifier-free diffusion guidance,","cited_arxiv_id":null,"evidence_quote":"Provides the weighted combination of conditional and unconditional scores that Eq. (8) adapts for segmentation."}],"review_version":1}