{"id":"19e88649-d27f-442b-b81e-a2e293604e09","arxiv_id":"2411.10232","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A training-free method changes an object's color by aligning cross-attention value matrices with a reference image during early denoising steps, plus a new COLORBENCH benchmark.","lead":"This paper introduces a training-free way to change an object's color in a photo by borrowing color statistics from a reference image inside a Stable Diffusion model. The authors also introduce a color-editing benchmark, COLORBENCH, and report better structure and background preservation than six text-guided editing baselines.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim hinges on Value-matrix AdaIN (Eq. 3–4) transferring only color; the paper's own ablations indicate that Value alignment also transfers structural information, so the method's success is primarily carried by auxiliary masking/self-attention constraints rather than the claimed…","rationale":"The paper reports a concrete, reproducible-sounding pipeline (AdaIN on cross-attention Value matrices, self-attention replacement, latent blending, background preservation) and quantitative experiments on generated and real data. I agree with the reader that the conditional verdict is appropriate for a training-free editing method with a new benchmark, but I find a more specific and load-bearing gap than the reader's weakest_assumption as phrased (which focuses on the Value/geometry separation and is in fact the right general area). The reader's concern is the same underlying issue, but my concrete articulation is sharper: the paper's own ablation Table 4 and Table 2 show that the Value-alignment is not color-specific in practice—it progressively destroys structure as the alignment window grows. That is not just a missing proof; it is internal evidence that the mechanism smuggles structure through the Value path, and that the reported success depends on auxiliary modules (self-attention replacement, masking, early stopping). I therefore mark agreement as 'partial'—the reader identified the mechanism assumption but did not extract the load-bearing internal evidence from the ablations that turns the assumption into a testable risk. The proposed swap experiment would settle whether the Value channel is color-specific or appearance-transferring, and would hence decide whether the central claim's mechanism is correct or whether the method should be re-described as a masked appearance-transfer pipeline with auxiliary structural constraints. This does not change the verdict (CONDITIONAL), but it sharpens the condition that must be satisfied before the mechanism claim is accepted.","tokens_in":18086,"tokens_out":1903,"duration_ms":16871,"concrete_test":"Run a swap experiment that isolates the Value-alignment mechanism from the auxiliary constraints: take a fixed source image, a fixed object mask, and fix the source self-attention maps and latent-blending ratio at the paper's defaults (0.1), but remove the self-attention replacement and background preservation. Then (a) align Value matrices from a reference image that is identical to the source object except for color (e.g., the same Pikachu render recolored), versus (b) align Value matrices from a reference image of a different object of similar color (e.g., a red car). If DS/SSIM and LPIPS-object in case (b) degrade far more than in case (a), the Value alignment is transferring appearance beyond color, contradicting the color-specific mechanism in Section 4.3. If case (b) matches case (a), the mechanism is color-specific.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's central assertion is that aligning the cross-attention Value matrices of the target with those of a reference color image via AdaIN (Eq. 3–4) transfers color while preserving object structure. For that assertion to hold, the Value matrices must encode color that is separable from geometry/structure. The paper's own evidence contradicts this: Table 2 shows that replacing self-attention maps (which are claimed to carry structure) is required to prevent structural breakage, and Table 4 shows that extending Value alignment from 20% to 50% of timesteps degrades DS from 0.957 to 0.876 and SSIM from 0.830 to 0.715, while alignment over all timesteps collapses DS to 0.622. These results show the Value matrices carry substantial structure and that the stability of the method relies on limiting the Value-alignment window and on auxiliary self-attention replacement and background masking—not on a clean appearance/geometry separation. The method works, per the reported experiments, but the mechanism is not established as color-specific; the claim in Section 4.1 is therefore an over-interpretation of the visualizations in Figs. 4–7, which show cross-attention maps but do not isolate the Value subspace. If the Value matrices load-bear structure, then any edit that matches reference Value statistics will also transfer non-color appearance (texture, pose, identity), and the method's generalization to new objects or reference images that differ more strongly in geometry cannot be predicted.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper targets the task of object color editing in diffusion-based image generation and editing. It analyzes cross-attention layers in Stable Diffusion and proposes ColorEdit, a training-free method that recolors an object by aligning the Value matrices of cross-attention in the U-Net decoder with those extracted from a reference color image via AdaIN during the early denoising stage, while replacing self-attention maps from the source to preserve structure and blending a masked reference latent for stronger color transfer. The authors also introduce COLORBENCH, a real-image color-change benchmark, and compare against six text-guided editing baselines with automatic and human metrics.","tokens_in":18397,"tokens_out":7852,"duration_ms":79200,"significance":"The paper addresses a well-defined, understudied editing task and offers a simple training-free baseline plus a benchmark, both of which are potentially useful to the community. The ablation study is systematic and shows that each component contributes, and the human evaluation provides complementary evidence. However, the central mechanistic claim about Value matrices being color-specific is not established by the paper's own results, and the quantitative evaluation has methodological weaknesses that require revision.","major_comments":[{"comment":"The load-bearing mechanistic claim is that Value matrices in cross-attention carry color/appearance information separable from geometry, so that AdaIN alignment transfers color without transferring structure. The paper's own evidence undercuts this claim: Table 2 shows that self-attention map replacement is necessary for structure preservation, and Table 4 shows that extending Value alignment from 20% to 50% of timesteps drops DS from 0.957 to 0.876 and SSIM from 0.830 to 0.715, while all-timestep alignment collapses DS to 0.622. These results indicate that the Value subspace also encodes identity and structure, and that the stability of the method is carried by the early-only alignment window and the auxiliary structural constraints. Please soften the mechanistic interpretation in Section 4.1 or add a positive control that isolates the Value subspace, for example by ablating alignment on a low-rank projection of V, to show what information is actually transferred.","section":"Section 4.1 and Section 4.3, Eq. (3)-(4), Tables 2 and 4"},{"comment":"The default hyperparameters (alignment window tau, latent blending ratio R, and background-preservation count) are tuned on the same generated dataset used for the main results in Table 1: Table 4 tunes the alignment window and Table 5 tunes the blending ratio on that set. There is no validation split and no error bars or significance tests, so the comparison against all baselines uses settings selected to maximize the proposed method's scores. Please report results on a held-out split or with fixed defaults, and include multi-seed variance or significance tests for the main metrics.","section":"Section 5, Tables 4 and 5"},{"comment":"The algorithm specification is ambiguous in two places. First, the loop in Algorithms 1 and 2 iterates 'for i = 1 to n', but n is never defined or set in the paper. Second, the background-preservation condition in line 14 is 'if t < T - N', while the text says preservation is applied only in the final few steps; with T=50 and N=5, this condition is true for almost the entire denoising trajectory, so either the inequality is reversed or N has a different meaning that must be stated. These ambiguities prevent faithful reproduction of the method.","section":"Section 4.4 and Algorithms 1-2"},{"comment":"The evaluation metrics for the color-change goal are not fully adequate. L1Hue is computed as a linear distance on the HSV hue channel, but hue is circular and is undefined for black, gray, and white, which are included in the seven target colors. On COLORBENCH, LPIPS on the object area is used as a color-change score, but LPIPS is a perceptual content metric rather than a color metric, so a larger value can reflect unintended structure or texture change rather than successful recoloring. In addition, the real-image benchmark sets all backgrounds to white, so the background-preservation results on this benchmark may not transfer to natural scenes. Please add a proper masked-object color metric and report results on more realistic background distributions.","section":"Section 8.3 and Table 1 (COLORBENCH)"}],"minor_comments":[{"comment":"There is a typo in 'For symthesised images'; it should be 'synthesized'.","section":"Section 8.1"},{"comment":"The phrase 'keeping preview turn editing results' is unclear and should be rephrased.","section":"Figure 10 caption"},{"comment":"The notation for the projection functions fQ, fK, and fV is missing the argument parentheses in Eq. (2); the intended mathematical expression should be made explicit.","section":"Section 4.2 and Eq. (2)"},{"comment":"The paper states that for the generated dataset the best SAM mask was 'manually selected'; please clarify how this selection was performed and whether the same selected masks are used for all compared methods.","section":"Section 5.2"},{"comment":"It is unclear whether the generated dataset is also part of COLORBENCH or only the real-image set; please define the benchmark scope precisely.","section":"Section 5.2 and Section 8.2"},{"comment":"The text says R=0.15 gives better human-perceived color change while R=0.1 is used in the main quantitative results; please state the exact default used in Table 1 and justify the choice.","section":"Section 5 and Section 8.5"},{"comment":"No code or implementation details such as the PyTorch hook setup are released; providing the code or a detailed configuration would substantially improve reproducibility.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The stress-test concern is partly borne out: the paper's own Tables 2 and 4 show that structural preservation depends on the short alignment window and on self-attention replacement, not on a clean color/geometry separation. I regard this as a framing and evidence problem that can be fixed by revision, not grounds for rejection. The more urgent revision points are the tuned evaluation settings and the incomplete algorithm specification."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the method is a genuine training-free recolor recipe and COLORBENCH is a useful new artifact, but the paper's own ablations undercut the claim that Value-matrix alignment is a clean appearance/geometry separation. The mechanism story needs to be reframed before the generalization claims are believable.\n\nWhat's new: using AdaIN over reference cross-attention Value matrices to transfer object color, while keeping self-attention maps from the source, is not in P2P, PnP, FreePromptEditing, or MasaCtrl. Those methods operate on attention maps or keys from text; this one uses a reference image. The benchmark is also new: 2,842 source-target pairs (white background for real images) across 100 subjects, and the comparison set is broad. The ablations are honest enough to show each component contributes, and the human evaluation lines up with the table numbers.\n\nWhere it's soft: the central claim in Section 4.1 is an over-interpretation. Table 4 is the tell. Extending Value alignment from 20% to 50% to all timesteps drops DS from 0.957 to 0.876 to 0.622 and SSIM from 0.830 to 0.715 to 0.622, and Table 2 shows removing self-attention replacement drops DS to 0.811. So Value matrices carry a lot of structure and identity, not just color. The paper acknowledges that in passing (\"may also introduce additional information\"), but then continues to call the alignment a \"color attribute alignment.\" In practice the method works because Value AdaIN injects color while self-attention replacement and the mask-based latent blending hold the object together. That is a valid recipe, but it is not the clean story the abstract and Section 4.1 tell. On top of that, none of the quantitative comparisons have error bars or significance tests; the alignment interval and blending ratio are tuned on the evaluation set (Tables 4-5); and the real-image benchmark has white backgrounds, which makes background preservation easier than a general real-scene claim would require. No code or data are released. None of these are fatal, but they mean the headline margins should be read as promising, not established.\n\nWho this is for: people doing diffusion-based editing, or anyone who wants a first color-change benchmark. It deserves a serious referee, not a desk reject. I would ask the authors to soften the mechanism claim, add a hold-out protocol or significance testing, and release code and data. My own verdict is skeptical-but-interested: the editing recipe probably works, but the explanation needs another round of evidence.","headline":"A real training-free recolor recipe plus a useful first benchmark, but the paper's own ablations undercut its clean 'Value matrices carry color, not geometry' story.","tokens_in":18927,"tokens_out":3520,"would_cite":true,"duration_ms":36892,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A training-free method recolors objects in diffusion-generated and real images by aligning cross-attention Value matrices with a reference color image during early denoising.","keywords":["training-free image editing","cross-attention Value alignment","AdaIN","diffusion model color editing","COLORBENCH","image-guided color change","null-text inversion","object-level editing"],"falsifier":"Take a fixed source object and align its cross-attention Value matrices with a reference image of a different object of unrelated shape but the same target color, keeping all other steps identical; if the output object's structure distorts significantly even in the early-window setting, then Value matrices are not color-separable and the central claim fails. A more focused variant would swap only the Value matrices of the third CrossAttnUpBlock while leaving the other up-blocks untouched, to test whether the claimed locus of color information is correct.","tokens_in":17870,"feed_emoji":"🎨","tokens_out":4413,"duration_ms":42595,"temperature":0.7,"pith_summary":"The paper tries to establish that an object's color can be edited without training or fine-tuning by aligning the Value matrices of a text-to-image diffusion model's cross-attention layers with those of a reference color image during early denoising. This matters because text-guided editing often fails to change color reliably, due to attention leakage and attribute collision between the object and a color word in the prompt. An image reference avoids introducing that color word, so the failure modes disappear. The paper also contributes a benchmark, COLORBENCH, for evaluating color-change methods on generated and real images.","feed_headline":"Recolor any object without training the diffusion model","feed_subtitle":"Aligning cross-attention Value matrices with a reference image changes color while preserving shape and background.","key_machinery":"The load-bearing mechanism is color attribute alignment via AdaIN on cross-attention Value matrices: $V^* = \\operatorname{AdaIN}(V^T, V^c)$, which rescales the target Value distribution to match the mean and standard deviation of the reference Value distribution. This alignment runs only in the first, second, and third CrossAttnUpBlocks of the U-Net decoder during the early denoising steps $t \\in [0.8T, T]$, when shape and contour are being fixed. The supporting components are self-attention map replacement from the source image, latent blending of the reference image's noise at a ratio near 0.1, and background restoration in the final steps; these guards keep object structure and background intact. The early-window timing is what separates clean color transfer from structural breakage, since the paper's ablation shows that aligning Value matrices across all steps severely degrades structure.","core_discovery":"The central discovery is that stable object-level color editing can be achieved by substituting the cross-attention Value matrices of the target image with Value matrices extracted from a reference color image, rather than by amplifying the Key or Value of a color term in the prompt. The paper argues that shape, contour, and texture are established in the early denoising stage of the U-Net decoder, specifically in the first, second, and third CrossAttnUpBlocks, and that Value alignment applied only during that early window transfers color without destroying structure. Because no color term is added to the text prompt, cross-attention leakage and attribute collision are eliminated. The paper further builds COLORBENCH and reports that the method outperforms text-guided baselines in preserving object structure and background while matching them on color change.","pith_inferences":["If Value alignment truly isolates color, the same principle could plausibly extend to transferring other appearance attributes, such as lighting or material gloss, by aligning Value statistics in the same U-Net decoder blocks; the paper does not test that generalization.","An independent test of the core assumption could align Values from a reference image of a different object with unrelated shape; if structure remains intact, color information is genuinely separable, whereas if it distorts, the method depends on coarse geometric similarity between source and reference objects.","COLORBENCH may become a standard evaluation set for future color-change methods, even though the paper only evaluates it on the proposed method and text-guided baselines.","The method inherits segmentation quality from the mask-generation model used, so thin or small objects are likely to be under-covered; this is consistent with the paper's reported difficulty with small objects."],"forward_implications":["Because the Value matrices of a reference color image need to be extracted only once, the same reference can recolor many different source images with no additional inversion cost.","The method extends to real images by first running null-text inversion, so it is not limited to synthesized inputs.","Avoiding a color word in the prompt sidesteps cross-attention leakage and attribute collision, which are the specific failure mechanisms the paper identifies in text-guided editing.","The ablation evidence implies that early-stage Value alignment is both sufficient and necessary for color transfer, with full-process alignment harming object identity.","The paper itself notes that smaller objects are harder to recolor and that multi-object color changes must be performed in a multi-turn way."],"supporting_citations":[{"why":"Supplies the AdaIN normalization operation used to transfer reference color statistics into the target Value matrices.","marker":"[17]"},{"why":"Establishes that cross-attention maps carry object layout and semantics; its color-editing failures motivate the Value-alignment approach.","marker":"[13]"},{"why":"Shows that self-attention maps preserve spatial structure, the basis for the paper's self-attention map replacement.","marker":"[42]"},{"why":"Analyzes cross- and self-attention in Stable Diffusion and identifies attribute collision, the failure mode the paper aims to bypass.","marker":"[24]"},{"why":"Provides null-text inversion, used both to invert real images and to extract Value matrices from the reference color image.","marker":"[27]"},{"why":"Supplies the segmentation model used to generate precise object masks instead of thresholded cross-attention masks.","marker":"[21]"},{"why":"Provides the text-conditioned latent diffusion model that serves as the backbone for all experiments.","marker":"[35]"}],"fun_headline_variants":["Swap object colors without training diffusion models","Train-free recoloring via cross-attention alignment","Recolor objects by aligning Value matrices","Color editing without text prompts or fine-tuning","Diffusion color change via value alignment"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that the Value matrices of cross-attention in the U-Net decoder carry color and appearance information that can be transferred without also transferring shape, pose, or texture; if those matrices also encode object identity, the alignment step would alter or break the object.","fun_headline_variants_meta":{"raw":{"variants":["Swap object colors without training diffusion models","Train-free recoloring via cross-attention alignment","Recolor objects by aligning Value matrices","Color editing without text prompts or fine-tuning","Diffusion color change via value alignment"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000141,"raw_usage":{"total_tokens":1153,"prompt_tokens":919,"completion_tokens":234,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":535,"completion_tokens_details":{"reasoning_tokens":170}},"tokens_in":535,"tokens_out":234,"duration_ms":2946,"temperature":1.0,"reasoning_tokens":170,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T19:49:12.177109+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a fixed source object and align its cross-attention Value matrices with a reference image of a different object of unrelated shape but the same target color, keeping all other steps identical; if the output object's structure distorts significantly even in the early-window setting, then Value matrices are not color-separable and the central claim fails. A more focused variant would swap only the Value matrices of the third CrossAttnUpBlock while leaving the other up-blocks untouched, to test whether the claimed locus of color information is correct.","supporting_citations":[{"cited_title":"Arbitrary style transfer in real-time with adaptive instance normalization","cited_arxiv_id":null,"evidence_quote":"Supplies the AdaIN normalization operation used to transfer reference color statistics into the target Value matrices."},{"cited_title":"Plug-and-play diffusion features for text-driven image-to-image translation","cited_arxiv_id":null,"evidence_quote":"Shows that self-attention maps preserve spatial structure, the basis for the paper's self-attention map replacement."},{"cited_title":"Towards understanding cross and self-attention in stable diffusion for text-guided image editing","cited_arxiv_id":null,"evidence_quote":"Analyzes cross- and self-attention in Stable Diffusion and identifies attribute collision, the failure mode the paper aims to bypass."},{"cited_title":"Null-text inversion for editing real im- ages using guided diffusion models","cited_arxiv_id":null,"evidence_quote":"Provides null-text inversion, used both to invert real images and to extract Value matrices from the reference color image."},{"cited_title":"Segment any- thing","cited_arxiv_id":null,"evidence_quote":"Supplies the segmentation model used to generate precise object masks instead of thresholded cross-attention masks."}],"review_version":1}