{"id":"a9518322-0e0e-4b6b-8402-acb1ee18ede7","arxiv_id":"2501.14046","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"An LLM plus open-vocabulary detector locates instances in a generated image, and cross-attention and feature guidance reposition one instance without masks or retraining.","lead":"This paper proposes a pipeline that uses an LLM and an open-vocabulary detector to find objects in an AI-generated image, then applies diffusion guidance to move one chosen object while preserving the others. The goal is instance-level image control without masks, bounding boxes, or fine-tuning, using only the text prompt and a user-selected shift.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Paper's §4.1 admits the moved object's appearance changes completely, while §3.2.2's preservation term is applied only to non-manipulated objects, directly contradicting the claimed appearance-preserving instance-level manipulation.","rationale":"The reader correctly identified the token-mapping ambiguity in Eq. 5 as a significant implementation gap, but the paper's own text contains a direct, more load-bearing contradiction. Section 3.2.2 states the preservation term is computed only for non-manipulated objects, and Section 4.1 explicitly states the moved object's appearance changes completely. This is not a speculative risk; it is an admission that the central promise of appearance-preserving instance editing is not met by the described mechanism. Even if the token mapping were fully specified, the method would still regenerate the moved object rather than edit it, because the position guidance operates solely on cross-attention maps, which the paper itself says encode only location and shape. The token mapping issue affects whether the correct spatial region is targeted; the appearance-preservation issue affects whether the manipulation is instance-level at all. I therefore consider the appearance contradiction to be the single most load-bearing concern. That said, the paper still offers a plausible pipeline for instance-selective position manipulation without masks or fine-tuning, and the qualitative figures show partial success, so the appropriate verdict remains CONDITIONAL rather than REJECT. The reader's verdict is unchanged, though my justification differs; hence UNCHANGED. The proposed LPIPS/CLIP crop-similarity check is a concrete, low-cost way to settle the degree of appearance drift.","tokens_in":9147,"tokens_out":6986,"duration_ms":61450,"concrete_test":"Run the released code (or ask authors for execution traces) on the four Figure 2 prompts, crop the manipulated object's bounding box in the original and edited images using the same OWLv2 detection, and compute a perceptual similarity (LPIPS or CLIP image similarity) between the two crops. If the similarity is low (e.g., LPIPS > 0.3), the §4.1 admission is confirmed and the appearance-preservation claim is falsified; if high, the §4.1 statement must be reconciled with the quantitative result.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim, as stated in the introduction and Fig. 2 caption, is that the pipeline enables precise instance-level manipulation 'while ensuring the preservation of original appearances.' However, Section 3.2.2 defines the preservation loss (Eq. 6) only for 'the rest of the objects that are not manipulated'; the manipulated object is never feature-preserved. Section 4.1 then admits: 'the appearance of the moved object changes completely because the position term in Eq. 5 uses cross-attention maps that contain only general location and shape information, not appearances.' This is not a minor limitation; it directly contradicts the claimed outcome. The position guidance in Eq. 5 reshapes the cross-attention map to shift a token's spatial focus, but cross-attention maps do not carry instance-specific appearance, so the moved object is effectively regenerated at the target location rather than edited. Consequently, the method cannot be said to edit a specific instance if that instance's identity is not preserved; at best it generates a new instance of the same category at a new position. This undermines the comparison to DragonDiffusion and the headline claim of 'precise manipulations at the instance level.' The token-mapping ambiguity identified by the reader is real but secondary: even with a perfectly resolved token, the appearance of the moved object would still not be preserved under the described mechanism.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a training-free pipeline for instance-level object repositioning in diffusion-generated images. It combines an LLM and an open-vocabulary detector (Gemma-7b, OWLv2) to locate objects mentioned in the prompt, then applies guidance terms based on SDXL cross-attention maps and U-Net intermediate activations to move a selected instance. The total guidance (Eq. 7) combines a preservation term for non-manipulated objects (Eq. 6) and a position term for the manipulated object (Eq. 5). The paper reports qualitative comparisons with Self-Guidance and DragonDiffusion on four examples involving monkeys and a motorcycle.","tokens_in":27,"tokens_out":3359,"duration_ms":81767,"significance":"If the claimed capabilities were fully demonstrated, the pipeline would provide a useful zero-shot alternative to mask-based and fine-tuning-based editing, and the integration of LLM-parsed prompts with detector bounding boxes is a sensible direction for instance-level control. The paper also ships a public code link and states that only pretrained models are used, which are strengths. However, the central claim of precise instance-level manipulation with preserved appearances is currently undermined by the method's own admitted limitations, the absence of quantitative evaluation, and an unspecified token-mapping component.","major_comments":[{"comment":"The central claim is contradicted by the paper's own results. The abstract and introduction state that the pipeline enables 'precise manipulations at the instance level' and 'ensuring the preservation of original appearances,' but §4.1 explicitly admits that 'the appearance of the moved object changes completely because the position term in Eq. 5 uses cross-attention maps that contain only general location and shape information, not appearances.' Moreover, Eq. 6 defines the preservation loss only for 'the rest of the objects that are not manipulated,' so the moved object is never feature-preserved. This means the method regenerates a category-level object at the target location rather than editing the specific detected instance, which invalidates the comparison to DragonDiffusion and the headline claim of instance-level manipulation as stated.","section":"§4.1 (with Abstract and §1)"},{"comment":"The position guidance requires 'the cross-attention map corresponding to token k' for the manipulated object, but the paper does not specify how an LLM-extracted object (e.g., 'green motorcycle' or 'monkey id=0') is mapped to a specific token in SDXL's two text encoders. For multi-token phrases or repeated nouns, the mapping is nontrivial; without a defined and validated mapping, the position term may steer attention in the wrong spatial region or affect the wrong object. This is load-bearing because instance-level precision is the entire contribution.","section":"§3.1 and §3.2.1, Eq. (5)"},{"comment":"The paper's claim of 'precise manipulation' rests on four qualitative examples, one of which is explicitly admitted to fail: the motorcycle move 'resulted in no displacement.' The text refers to 'additional examples and quantitative evaluation' in supplementary materials, but no such materials accompany this submission. For a precision-oriented editing method, a quantitative evaluation is necessary, at minimum measuring bounding-box displacement accuracy and appearance preservation (e.g., LPIPS or identity similarity) for the moved object and the background.","section":"§4.1 and Fig. 2"},{"comment":"The paper acknowledges that 'the weights require tuning, as a combination that works well for one object may not yield satisfactory results for another.' Since the guidance weights w0 and w1 in Eq. 7, the scales s and v in Eq. 4, and the U-Net layer choices are free parameters selected per manipulation, the method is not parameter-free or fully automatic as the abstract's 'without fine-tuning or auxiliary information' might imply. The authors should either provide a fixed parameter-selection protocol or a sensitivity analysis showing that results are robust over a reasonable range of hyperparameters.","section":"§4.1 and §5"}],"minor_comments":[{"comment":"In Eq. (5), the function is written as gposition(o), but the formula uses a cross-attention map Ak; the index k should be defined or the summation over attention heads/maps should be made explicit.","section":"§3.2.1, Eq. (5)"},{"comment":"Eq. (7) refers to gmanipulation(ok) but the earlier definitions only introduce gposition(o); this notation should be clarified.","section":"§3.2.3, Eq. (7)"},{"comment":"The captions state 'Coordinates shift is represented by (x, y)' but do not specify the units or normalization; the reader should be told whether the shift is in pixels, latent-space coordinates, or normalized image coordinates.","section":"Fig. 2 and Fig. 3 captions"},{"comment":"The sentence 'Additional examples and quantitative evaluation can be found in the supplementary materials' is not verifiable in the current submission, since no supplementary material is provided; this should either be included or removed.","section":"§4.1"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is unusually honest about its limitations, which is commendable, but the admitted failure to preserve the moved object's appearance directly contradicts the abstract and introduction. The paper could be salvageable by reframing the claim as 'instance-localized category-level repositioning' and by adding the missing quantitative evaluation and token-mapping specification. The editors may also wish to verify whether the supplementary materials referenced in §4.1 exist, as they are not included in the current version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper combines three known ingredients — SLD's LLM+detector object localization, Self-Guidance's cross-attention position term, and DragonDiffusion-style activation preservation — into a training-free pipeline for moving one detected instance in a generated image. That integration is genuinely new, and the figures show it can selectively shift one monkey among several without masks. Give credit: the method is clearly described, the equations are simple, the limitations are admitted rather than hidden, and the promise of code is good practice. The citation pattern is appropriate, with due credit to the borrowed components and no obvious self-citation inflation.\n\nWhere it falls short: the headline claim that this is 'precise instance-level manipulation' with 'preservation of original appearances' is contradicted by the paper's own Section 4.1. The moved object's appearance changes completely, and Eq. 6 preserves only the non-manipulated objects. So what is actually delivered is a new instance of the same category at the target location, not an edit of the original instance. That is a real gap between the abstract and the result. The evaluation is also thin: four qualitative examples, one of which fails (the motorcycle does not move). The promised quantitative evaluation is deferred to a supplement we do not have. Token mapping from detection boxes to SDXL cross-attention tokens is unspecified, which matters for phrases like 'green motorcycle'. And the weights need per-manipulation tuning, so 'precise' is over-claimed.\n\nThis is not a fatal flaw because the authors disclose all of it. The core mechanism is plausible and the failure modes are well understood. What needs to change is the framing and the evidence: weaken the claim to 'instance repositioning that regenerates the moved object while preserving the rest of the scene', specify the token mapping, and include a real quantitative comparison (detection IoU, appearance similarity of preserved objects, etc.).\n\nVerdict: I would send this to peer review. It deserves referee time as a serious training-free editing method with known limitations, and the review process can force the honest reframing. I would not cite it in its current form. Take it to reading group only if you want to discuss the gap between claims and disclosed limitations.","headline":"Training-free instance repositioning built from known parts, but the headline appearance-preservation claim is contradicted by the paper's own text.","tokens_in":9940,"tokens_out":2317,"would_cite":false,"duration_ms":20639,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims a training-free pipeline that repositions a single detected instance in a generated image using LLM parsing, open-vocabulary detection, and cross-attention guidance.","keywords":["instance-level image editing","diffusion models","cross-attention maps","LLM-guided editing","open-vocabulary detection","diffusion guidance","Stable Diffusion XL"],"falsifier":"Take a prompt with several instances of an object (e.g., 'three monkeys') and a prompt with a multi-word object ('green motorcycle'); request a horizontal shift of the rightmost monkey and of the motorcycle, then measure whether the detected bounding box actually moves by the requested fraction and whether the moved instance's appearance is preserved. The paper already reports that the motorcycle did not move, so a systematic evaluation across multi-token objects of varying sizes would decide whether the instance-level control claim holds.","tokens_in":8908,"feed_emoji":"🎯","tokens_out":4738,"duration_ms":37004,"temperature":0.7,"pith_summary":"The paper tries to establish that instance-level image manipulation—moving one object in a generated image while leaving other objects untouched—can be done without fine-tuning the diffusion model and without user-supplied masks or bounding boxes. It combines a large language model that reads the prompt and lists the objects, an open-vocabulary detector that finds those objects in the already generated image, and a guidance term that acts on cross-attention maps to shift the selected object while a second term preserves other objects' appearances via intermediate U-Net activations. If the method works as described, a user could select a particular monkey among several in a scene and have it move sideways with the rest of the picture intact. The paper reports qualitative examples and acknowledges failure cases for large objects and hyperparameter sensitivity.","feed_headline":"Move one monkey among three without masks or fine-tuning","feed_subtitle":"An LLM and open-vocabulary detector find each prompt object; cross-attention guidance then shifts only the chosen instance.","key_machinery":"The pipeline's load-bearing components are: (1) an LLM that extracts objects and attributes from the prompt, (2) an open-vocabulary detector that localizes those objects on the generated image with bounding boxes, and (3) guidance terms applied during sampling—a position term $g_{\\mathrm{position}}(o)$ that suppresses cross-attention mass at the original box $M_{\\mathrm{orig}}$ and encourages it at the shifted box $M_{\\mathrm{target}}$, and a preservation term $g_{\\mathrm{preserve}}(o)$ that penalizes differences between original and edited intermediate activations $\\Psi_{\\mathrm{orig}}$ and $\\Psi_{\\mathrm{target}}$ over the other objects. The total guidance is combined with classifier-free guidance as in Eq. 4.","core_discovery":"The central discovery is that the attention maps inside a pretrained diffusion U-Net, when targeted at a token whose object has been localized by an LLM-plus-detector stage, can be used as an energy function to drag that object to a new position while a separate preservation term on intermediate activations keeps all other detected instances visually consistent. This gives instance-level control that neither object-type editing via cross-attention maps (which moves every occurrence of a word) nor mask-requiring methods provide, and it does so with no training and no extra user input beyond a desired shift.","pith_inferences":["If token-to-instance alignment is made robust for multi-token phrases (e.g., using per-token aggregation or resolved noun phrases), the same pipeline could generalize to attribute-based edits like color or size without masks.","The position guidance only uses the cross-attention map of one token; combining it with cross-attention refinement could fix the failure on large objects that the paper reports.","Because the preservation term runs on all detected non-manipulated objects, the method's quality degrades gracefully with detection mistakes: if the detector misses an object or mislabels it, that object is neither preserved nor steered.","A quantitative benchmark measuring how far the moved instance's bounding box actually shifts versus the requested shift would let practitioners compare this approach with mask-based methods on equal footing; the paper itself relies on qualitative comparison."],"forward_implications":["Users can select one instance among several of the same object type (e.g., the rightmost of three monkeys) and move it without moving the others.","No fine-tuning or masks means the pipeline runs entirely on pretrained components and can be applied to any prompt the detector can parse.","Cross-attention-only preservation is insufficient; intermediate activations are necessary to keep appearance, so the method's design point is the combination of the two terms.","The method can be extended to other manipulations beyond position because the same guidance machinery applies to any attribute encoded by attention or features, though the paper implements only position.","Large-object displacements may fail as the paper reports, so instance-level control has practical size limits."],"supporting_citations":[{"why":"Supplies the LLM object-extraction and open-vocabulary detection pipeline that the method adopts.","marker":"[37]"},{"why":"Provides the open-vocabulary detector used to localize prompt objects on the generated image.","marker":"[24]"},{"why":"Shows cross-attention maps encode position and shape and is the object-type-level guidance baseline.","marker":"[10]"},{"why":"Is the mask-requiring instance-level baseline that the method compares against.","marker":"[25]"},{"why":"Demonstrates that intermediate diffusion features preserve appearance, justifying the preservation term.","marker":"[35]"},{"why":"Provides the universal guidance formulation used to integrate the position and preservation energy terms.","marker":"[4]"},{"why":"Is the Stable Diffusion XL model the pipeline is tested on.","marker":"[26]"}],"fun_headline_variants":["Drag a single object in a generated image using LLM-guided attention","Move one instance among many: no masks, no fine-tuning","LLM pinpoints target, diffusion U-Net attention slides it","Instance-level image editing via cross-attention energy","Shift one object while others stay put, no training needed"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that each detected object can be mapped to one specific text token whose cross-attention map cleanly covers that object; for phrases like 'green motorcycle' this mapping is not explained, and if it fails, the guidance will move attention in the wrong place or leave the object unmoved.","fun_headline_variants_meta":{"raw":{"variants":["Drag a single object in a generated image using LLM-guided attention","Move one instance among many: no masks, no fine-tuning","LLM pinpoints target, diffusion U-Net attention slides it","Instance-level image editing via cross-attention energy","Shift one object while others stay put, no training needed"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000227,"raw_usage":{"total_tokens":1409,"prompt_tokens":822,"completion_tokens":587,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":438,"completion_tokens_details":{"reasoning_tokens":503}},"tokens_in":438,"tokens_out":587,"duration_ms":5230,"temperature":1.0,"reasoning_tokens":503,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T15:25:14.871938+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a prompt with several instances of an object (e.g., 'three monkeys') and a prompt with a multi-word object ('green motorcycle'); request a horizontal shift of the rightmost monkey and of the motorcycle, then measure whether the detected bounding box actually moves by the requested fraction and whether the moved instance's appearance is preserved. The paper already reports that the motorcycle did not move, so a systematic evaluation across multi-token objects of varying sizes would decide whether the instance-level control claim holds.","supporting_citations":[{"cited_title":"Scaling open-vocabulary object detection","cited_arxiv_id":null,"evidence_quote":"Provides the open-vocabulary detector used to localize prompt objects on the generated image."},{"cited_title":"Diffu- sion Self-Guidance for controllable image generation","cited_arxiv_id":null,"evidence_quote":"Shows cross-attention maps encode position and shape and is the object-type-level guidance baseline."},{"cited_title":"DragonDiffu- sion: Enabling drag-style manipulation on diffusion models","cited_arxiv_id":null,"evidence_quote":"Is the mask-requiring instance-level baseline that the method compares against."},{"cited_title":"Plug-and-play diffusion features for text-driven image-to-image translation","cited_arxiv_id":null,"evidence_quote":"Demonstrates that intermediate diffusion features preserve appearance, justifying the preservation term."},{"cited_title":"Universal guidance for diffusion models","cited_arxiv_id":null,"evidence_quote":"Provides the universal guidance formulation used to integrate the position and preservation energy terms."},{"cited_title":"SDXL: Improving latent diffusion models for high-resolution image synthesis","cited_arxiv_id":null,"evidence_quote":"Is the Stable Diffusion XL model the pipeline is tested on."}],"review_version":1}