{"id":"fa7a5424-464f-499e-8be8-26be69d87840","arxiv_id":"2412.01339","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"NegToMe pushes each generated image token away from its closest matching token in a reference image during reverse diffusion, improving diversity and reducing copyright similarity without retraining.","lead":"This paper introduces NegToMe, a training-free method that steers diffusion image generators away from a reference image by pushing matching visual features apart during generation. It reports better output diversity and reduced similarity to copyrighted characters while adding under four percent inference time.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The abstract's 34.57% copyright-similarity reduction is not derivable from Table 1, undermining the paper's headline quantitative claim.","rationale":"The reader's verdict is CONDITIONAL, and I agree with that conclusion, but for a different primary reason. The reader's weakest_assumption concerns the reliability of cosine-similarity token matching in noisy diffusion features; that is a reasonable methodological risk but is partly supported by prior work on diffusion-feature correspondence and is not the most decisive issue. The load-bearing problem I identify is that the abstract's headline 34.57% copyright-similarity reduction cannot be recovered from the paper's only quantitative table (Table 1). This is a factual, checkable discrepancy: the reported DreamSim values imply at most a 16.7% relative reduction for the combined method and only 8.2% for NegToMe alone. Because this number is the paper's most prominent quantitative claim for a core application, its unsupported status should block unconditional acceptance. The recommended verdict remains CONDITIONAL pending either a corrected claim or the missing derivation, so I do not change the reader's verdict.","tokens_in":11494,"tokens_out":6108,"duration_ms":55219,"concrete_test":"Reproduce the Section 4.2 evaluation using the released code and the same 50-character prompts, and explicitly compute the metric used for the '34.57%' claim. If Table 1's max-DreamSim values are reproduced, verify whether (0.766-0.638)/0.766 or any other reported combination gives 34.57%; if not, ask the authors for the exact formula and table row. If the number cannot be recovered, the abstract's copyright-reduction claim must be corrected.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central advertised result is that NegToMe 'reduces visual similarity to copyrighted content by 34.57%.' The only quantitative copyright table is Table 1, which reports max DreamSim scores: base = 0.766, NegPrompt alone = 0.684, NegToMe alone = 0.703, and both = 0.638. The relative reduction from base to the combined method is (0.766-0.638)/0.766 = 16.7%; NegToMe alone gives only (0.766-0.703)/0.766 = 8.2%. No other row or metric in Section 4.2 is reported that would produce 34.57%. Thus the abstract's most prominent quantitative claim is unsupported by the evidence shown, and the discrepancy is large enough to change how the contribution is assessed. This is separate from the broader concern about cosine-similarity matching reliability, which is plausible but untested; the concrete numeric mismatch is directly checkable and, if real, requires either a missing definition or an amended abstract.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces NegToMe, a training-free module for adversarial guidance in text-to-image diffusion models. At each transformer block, the module matches each generated (source) token to its most similar token in a reference image by cosine similarity, applies a threshold, and linearly extrapolates the source token away from the matched target token (Eq. 4). The same mechanism is used for several applications by changing the reference: other images in the batch for diversity, copyrighted RAG images for copyright mitigation, blurry images for aesthetic improvement, and interpolation/extrapolation of object features. Quantitative evaluations on SDXL and Flux report improvements in diversity metrics and a 34.57% reduction in visual similarity to copyrighted characters, with under 4% inference overhead, plus a human study.","tokens_in":11693,"tokens_out":6800,"duration_ms":59700,"significance":"The contribution is conceptually simple and potentially useful: it provides a training-free, architecture-agnostic way to use image features as adversarial guidance, and it demonstrates complementarity with negative prompting. The paper makes code available, tests on two architectures, includes a human study, and reports a diverse set of applications. If the quantitative claims hold after fixing the issues below, the method is a solid practical contribution; the current version, however, has two load-bearing problems in the copyright claim: the headline 34.57% figure is not derivable from Table 1, and the RAG evaluation protocol in Sec. 4.2 excludes the reference used for NegToMe from the similarity computation, biasing the comparison.","major_comments":[{"comment":"The abstract and Sec. 4.2 state that NegToMe 'reduces visual similarity to copyrighted content by 34.57%,' but no result in the paper supports this number. Table 1 reports maximum DreamSim values of 0.766 (base), 0.684 (NegPrompt), 0.703 (NegToMe alone), and 0.638 (NegPrompt + NegToMe). The relative reductions are (0.766−0.638)/0.766 = 16.7% for the combined method and (0.766−0.703)/0.766 = 8.2% for NegToMe alone. Since no other row, metric, or appendix table is reported, the 34.57% figure either needs a precise definition or the abstract and Sec. 4.2 need to be amended to match the evidence.","section":"Abstract and Sec. 4.2 (Table 1)"},{"comment":"The copyright evaluation protocol is not apples-to-apples across methods. The text says that for each prompt NegToMe is applied 'using the best-matching RAG asset (asset with highest Dreamsim score),' and that the reported maximum DreamSim score is computed 'excluding the reference used for NegToMe.' Thus the base and NegPrompt rows take the maximum over all RAG assets, while the NegToMe rows take the maximum over the same set minus the most similar asset. This mechanically lowers the NegToMe maxima and inflates the apparent reduction. Please recompute all rows over an identical held-out RAG set (e.g., include the reference in the maximum for every method, or use a fixed evaluation subset disjoint from the reference selection).","section":"Sec. 4.2, Table 1"},{"comment":"Reproducibility of the main experiments is incomplete: the paper defines alpha_t as a time-dependent coefficient and a threshold tau, but it does not report the values or schedule used in Fig. 4 or Table 1. Fig. 12 shows only qualitative variation with alpha. Please specify the default schedule and threshold, and provide a quantitative sensitivity analysis or ablation for these hyperparameters.","section":"Sec. 3, Eq. (4); Sec. 5, Fig. 12"},{"comment":"The quantitative claims rest on point estimates without uncertainty quantification. Fig. 4 plots diversity and quality curves with no error bars, and Table 1 reports single DreamSim/quality numbers with no confidence intervals or significance tests. Given that the headline DreamSim difference between base and combined mitigation is 0.066, the reader cannot assess whether the reported improvements are robust. Please add error bars or significance measures, and state the number of samples per point.","section":"Fig. 4 and Table 1"}],"minor_comments":[{"comment":"The caption of Fig. 11 contains typos ('V ariation', 'wtih'), and reference [47] still contains the placeholder '[insert date]'.","section":"Fig. 11 and References"},{"comment":"The masked-guidance bias term log(M_ref + eps) is not fully specified: the paper does not state how M_ref is resized/interpolated to the token length N, and the additive bias for masked-out tokens is about -13.8 for eps=1e-6, which is a large and seemingly arbitrary shift that should be justified.","section":"Sec. 3, Eq. (5)"},{"comment":"The table caption 'Dreamsim ↓' does not indicate that the reported values are maxima over RAG assets and over seeds; align the caption with the metric definition in the text so that readers can interpret the numbers correctly.","section":"Sec. 4.2, Table 1"}],"recommendation":"major_revision","confidential_remarks":"The paper is timely and the method is simple, but the editorial decision should hinge on whether the authors can reconcile the 34.57% claim and rerun the copyright evaluation with a fair protocol. I would not recommend rejection if these are corrected; I would recommend a major revision and re-review."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea here is simple and new enough to matter: instead of using a text negative prompt, match each generated token to its closest token in a reference image and linearly extrapolate it away (Eq. 4). That is not the same as token merging for speed or text-based negative guidance. Applying this between attention and MLP in transformer blocks, with thresholded argmax matching and a mask bias for partial references, is a clean mechanism. The paper also earns credit for testing on SDXL and Flux, for the human study, and for the <4% overhead claim. The diversity results in Figure 4 look consistent: DreamSim goes down, entropy/IS go up, FID mostly holds. The prompt-rewriting comparison in Figure 6 is fair and shows the method helps even on top of expensive rewriting.\n\nThe soft spots are not fatal, but one is concrete and headline-adjacent. The abstract says NegToMe reduces visual similarity to copyrighted content by 34.57%. Table 1 shows base max DreamSim 0.766, combined NegPrompt+NegToMe 0.638: that is a 16.7% relative reduction. NegToMe alone is 8.2%. I could not find any other metric or row in Section 4.2 that yields 34.57%. So either there is a missing definition or the abstract overstates the result. That needs fixing before publication. Also missing: error bars or significance tests on any of the quantitative claims, and the exact schedule for alpha_t and threshold tau. The method's sensitivity to these is shown qualitatively only. The assumption that cosine similarity on noisy diffusion features gives reliable token correspondences at every timestep is plausible but asserted rather than tested; it deserves an ablation or at least a paragraph.\n\nI disagree with any reading that this is a takedown. The mechanism is plausible, the diversity claims are largely supported, and the copyright-mitigation story could be good once the number is reconciled. The citation pattern is solid and the related work is honestly positioned. This paper deserves a serious referee. The right outcome is likely major revision: amend the abstract, report error bars, and provide full hyperparameters and code. If those reproduce the tables, I'd accept.","headline":"NegToMe is a genuinely new, training-free token-level adversarial guidance trick with solid diversity results, but the headline 34.57% copyright reduction is not backed by the paper's own Table 1.","tokens_in":12242,"tokens_out":2007,"would_cite":false,"duration_ms":20067,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Diffusion outputs can be steered away from reference images by negative token merging, a training-free module.","keywords":["negative token merging","adversarial guidance","diffusion models","output diversity","copyright mitigation","training-free","image-based guidance","token merging"],"falsifier":"Run NegToMe with a reference image that has no semantic content, such as pure Gaussian noise, keeping the threshold fixed. If diversity or copyright-similarity metrics still shift as much as with a real reference image, the effect does not depend on semantic token correspondence; if the shift vanishes, the semantic-matching assumption is supported.","tokens_in":11307,"feed_emoji":"🎨","tokens_out":4939,"duration_ms":39878,"temperature":0.7,"pith_summary":"The paper proposes Negative Token Merging (NegToMe), a training-free module inserted into diffusion transformer blocks that steers generation away from a reference image by pushing each output token away from its best-matching reference token. It claims that this image-based adversarial guidance complements text-based negative prompts and works where negative prompts fail, such as models that do not natively support them. The two demonstrated applications are increased output diversity when other batch images are used as references and reduced visual similarity to copyrighted characters when copyrighted reference images are used. The authors report that NegToMe lowers DreamSim similarity to copyrighted content by 34.57% and adds under 4% inference overhead on SDXL and Flux.","feed_headline":"Push diffusion outputs apart with negative token merging","feed_subtitle":"Training-free NegToMe pushes output features away from reference images to boost diversity and cut copyright similarity.","key_machinery":"The load-bearing object is the cross-image token correspondence computed from noisy intermediate diffusion features. Given attention-block outputs $O_{\\mathrm{src}}$ and reference tokens $O_{\\mathrm{ref}}$, NegToMe normalizes them, forms the cosine-similarity matrix $S = \\tilde{O}_{\\mathrm{src}} \\cdot \\tilde{O}_{\\mathrm{ref}}^T$, takes the argmax match for each source token, masks out matches below threshold $\\tau$, and applies linear extrapolation $O_{\\mathrm{merge}} = (1+\\alpha_t) O_{\\mathrm{src}} - \\alpha_t O_{\\mathrm{target}}$. This operation is inserted between the attention and MLP branches of each transformer block; a mask bias $\\log(\\tilde{M}_{\\mathrm{ref}}+\\epsilon)$ restricts guidance to chosen regions. The whole procedure is about ten lines of code and requires no training.","core_discovery":"NegToMe's central claim is that adversarial guidance can be performed directly in visual feature space, without retraining, by applying a linear extrapolation at every transformer block during reverse diffusion: $O_{\\mathrm{merge}} = (1+\\alpha_t) O_{\\mathrm{src}} - \\alpha_t O_{\\mathrm{target}}$, where each source token is matched to its cosine-most-similar target token in the reference image and only tokens above a similarity threshold are moved. The paper argues that this pushes the generated image away from the reference in a semantically meaningful way, and that changing the reference changes the application: batch references increase diversity, copyrighted character references reduce similarity to those characters, blurry references improve aesthetics, and masked references allow targeting specific parts. The authors claim the mechanism is model-agnostic, including MM-DiT architectures like Flux that do not natively support negative prompts.","pith_inferences":["A testable extension would be applying the same negative token merging to video diffusion, where references from other frames could increase temporal diversity; the paper does not discuss temporal consistency.","The method's dependence on the threshold $\\tau$ and the schedule of $\\alpha_t$ is not characterized in the paper; a sensitivity analysis across these hyperparameters would tell whether the reported gains are robust or require careful tuning.","The diversity improvements are measured with DreamSim and entropy scores; a direct perceptual study comparing paired images with and without NegToMe at matched inference budgets would clarify how much of the improvement is noticeable to humans.","One implicit consequence is that the reference image choice acts as a control knob for generation, which could be used for targeted interventions such as reducing specific demographic skews in a batch without retraining."],"forward_implications":["If NegToMe works as claimed, diffusion models can be steered adversarially without text, which matters for models like Flux that lack native negative-prompt support.","Using other batch images as references offers a training-free route to increase demographic and visual diversity without the inference cost of prompt rewriting.","Copyright mitigation can be combined with negative prompts, since the paper reports the best similarity reduction when both are used together.","The same module can be repurposed for style exclusion, aesthetic improvement, and object feature interpolation or extrapolation by changing the reference image.","Because the mechanism is architecture-agnostic, it could be applied to future transformer-based diffusion backbones without finetuning."],"supporting_citations":[{"why":"Defines classifier-free guidance and the negative-prompt baseline that NegToMe complements and is compared against.","marker":"[21]"},{"why":"Provides the copyright-character generation problem and the negative-prompt baseline that NegToMe improves on.","marker":"[17]"},{"why":"Shows intermediate diffusion features carry semantic correspondence, the premise for token matching.","marker":"[45]"},{"why":"Supports that stable diffusion intermediate features are useful for zero-shot semantic correspondence.","marker":"[52]"},{"why":"DreamSim is the metric used to measure output diversity and similarity to copyrighted characters.","marker":"[14]"},{"why":"SDXL is the primary text-to-image backbone on which quantitative results are reported.","marker":"[33]"},{"why":"Flux demonstrates extension to MM-DiT models that lack native negative-prompt support.","marker":"[6]"},{"why":"HQ-SAM supplies the masks used for masked negative token merging in copyright mitigation.","marker":"[23]"}],"fun_headline_variants":["Training-free image guidance steers diffusion outputs apart","NegToMe: push diffusion features apart for diversity and copyright","Visual adversarial guidance without text or training","Image-based steering boosts diversity, cuts copyright similarity","No training, no text: image-guided diffusion editing"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that cosine similarity between noisy intermediate diffusion features reliably identifies semantically matching tokens at every timestep, and that linearly extrapolating those matched tokens apart moves the final image in the intended direction.","fun_headline_variants_meta":{"raw":{"variants":["Training-free image guidance steers diffusion outputs apart","NegToMe: push diffusion features apart for diversity and copyright","Visual adversarial guidance without text or training","Image-based steering boosts diversity, cuts copyright similarity","No training, no text: image-guided diffusion editing"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000176,"raw_usage":{"total_tokens":1296,"prompt_tokens":960,"completion_tokens":336,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":576,"completion_tokens_details":{"reasoning_tokens":263}},"tokens_in":576,"tokens_out":336,"duration_ms":3492,"temperature":1.0,"reasoning_tokens":263,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T04:27:00.072301+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run NegToMe with a reference image that has no semantic content, such as pure Gaussian noise, keeping the threshold fixed. If diversity or copyright-similarity metrics still shift as much as with a real reference image, the effect does not depend on semantic token correspondence; if the shift vanishes, the semantic-matching assumption is supported.","supporting_citations":[{"cited_title":"A tale of two features: Stable diffusion complements dino for zero-shot semantic correspondence","cited_arxiv_id":null,"evidence_quote":"Supports that stable diffusion intermediate features are useful for zero-shot semantic correspondence."},{"cited_title":"Scaling recti- fied flow transformers for high-resolution image synthe- sis","cited_arxiv_id":null,"evidence_quote":"Flux demonstrates extension to MM-DiT models that lack native negative-prompt support."},{"cited_title":"Segment anything in high quality","cited_arxiv_id":null,"evidence_quote":"HQ-SAM supplies the masks used for masked negative token merging in copyright mitigation."}],"review_version":1}