{"id":"8224337d-dc10-4966-b02f-5ff4dfe8bdc3","arxiv_id":"2501.09826","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"PIXELS performs exemplar-based image editing at inference time with a progressive latent mixing mask that provides region-wise strength control, needs no training, and accepts any number of exemplars.","lead":"PIXELS is a new image editing method that lets users change specific parts of a photo using example images, without retraining the AI model. It gives pixel-level control over how strongly each region is edited and can combine many example images at once.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported quantitative gains may be an artifact of the evaluation protocol: PIXELS receives non-binary edit maps while all baselines are restricted to binarized maps, so the Table 1 comparison does not isolate the progressive mechanism.","rationale":"I read the paper as an inference-time editing method whose central claims are (1) per-pixel strength control via a non-binary edit map and (2) state-of-the-art quality over exemplar-editing baselines. Claim (2) is the empirically load-bearing part of the abstract, and it is exactly where the evaluation is asymmetric. The reader's conditional verdict focuses on the flawed Lemma 1 proof and the latent-alignment assumption; those are real issues, but the lemma is not used by the algorithm itself, and the alignment assumption is standard in latent-diffusion editing. The evaluation confound is more consequential because if PIXELS needs the soft map to win, the comparison does not demonstrate that the progressive mechanism is superior. I therefore propose a rerun with binarized maps for PIXELS and, at minimum, uncertainty estimates. The recommended verdict remains conditional: the core method may still work and the paper is worth publishing after the benchmark is made fair and the lemma proof is corrected. Because the reader already recommended conditional, the final verdict is unchanged, but the primary reason shifts to the empirical confound.","tokens_in":819,"tokens_out":1069,"duration_ms":141628,"concrete_test":"Reproduce Table 1 under an equal-information protocol: run PIXELS with the same binary edit map used for the baselines (binarized at the same threshold), and compare PIXELS(binary) against PBE, IP-Adapter, and MimicBrush(binary) on FID and CLIP-I. Also report the standard error or confidence intervals over the 3000 sampled pairs. If PIXELS(binary) no longer leads on both metrics, the reported gains are attributable to the soft-map advantage rather than to the progressive mechanism.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim is that PIXELS bests four baselines (Table 1a: FID 5.412 vs 7.915, CLIP-I 91.71 vs 84.95). The evaluation section states: \"Since we are the first method to allow non-binary edit maps, we test other methods with the binarized version of the map.\" This makes the benchmark asymmetric: baselines must edit the whole binary-masked region at uniform strength, while PIXELS is given a soft edit map with graded transitions. A soft map can improve FID by leaving more of the image close to the source distribution, and it can improve CLIP-I by restricting the strongest edit to a sub-region that better matches the exemplar. The observed superiority may therefore reflect the extra information in the soft map rather than the progressive editing algorithm itself. The user study is subject to the same confound if PIXELS outputs were generated from soft maps and baseline outputs from binary maps. This is a correctness risk in the headline empirical claim, independent of whether the algorithm works as described.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"PIXELS proposes an inference-only method for exemplar-based image editing with off-the-shelf latent diffusion models. It uses a non-binary edit map to control per-pixel denoising strength by progressively mixing the source latent with the denoised output through a shifting mask. The authors claim granular spatial control, support for multiple exemplars and text guidance, and improved FID/CLIP-I over four baselines, plus a user study.","tokens_in":16235,"tokens_out":7707,"duration_ms":84519,"significance":"If the method works as described, it is a useful and lightweight contribution: no training, compatibility with existing SDXL/Kandinsky/SD2.1, arbitrary number of exemplars, user-controllable progressive edits, and released code. However, the paper's headline quantitative advantage is currently confounded by the asymmetric evaluation, and the theoretical support (Lemma 1) is flawed. The central algorithmic idea is still plausible and worth publishing after revision.","major_comments":[{"comment":"The derivation of Lemma 1 is not sound as written. Eq. (5) claims an upper bound on the expectation of a squared norm by bounding each summand separately, but this omits the cross terms 2E⟨a,b⟩ that appear when expanding ||a+b+c||²; no argument is given that these cross terms vanish. In addition, the setup states that a K-Lipschitz decoder implies ∥z_surgical−z(0)∥ ≤ K∥x_surgical−x(0)∥, but Lipschitz continuity of the decoder gives the opposite bound on image distance. Finally, Eq. (10) places the deterministic expectation E∥z(0)−z(t_ds)∥² inside a probability statement; for a fixed z_surgical this quantity is not random, so the claim P(...) ≥ 1−p is not a well-posed tail bound. Since Lemma 1 is invoked to motivate the denoising-strength ablation (§Ablation, Fig. 6), these issues are load-bearing.","section":"Appendix §1, Eq. (5) and Eq. (10)"},{"comment":"The quantitative comparison is asymmetric because PIXELS receives a soft edit map while all baselines receive its binarization, as the paper states: 'Since we are the first method to allow non-binary edit maps, we test other methods with the binarized version of the map.' A soft map allows PIXELS to leave part of the edited region close to the source, which can improve FID, and to concentrate exemplar influence in a subregion, which can improve CLIP-I; the reported gains therefore do not isolate the progressive editing mechanism. The user study (§User Study) has the same confound if PIXELS outputs were generated from soft maps and baseline outputs from binary maps. Please report a matched comparison (PIXELS on binarized maps, or baselines given soft maps where possible) and make the user-study inputs symmetric.","section":"Results, Table 1"},{"comment":"There is an inconsistency between the prose and the algorithm. The text says the mask selects 'all points lower than the current threshold determined by normalized timestep count,' but line 14 of Algorithm 1 uses maskt ← µd > (T−t)/T, i.e., points above the threshold. The caption of Fig. 3 additionally says darker regions are copied from the residue earlier, while the code, with a mask that shrinks as t → 0, keeps high-µd pixels in the source branch longer. Please clarify the intended polarity and correct the inconsistency; this determines which pixels are edited more strongly and is central to the controllability claim.","section":"Method, Algorithm 1 line 14"},{"comment":"The per-pixel control relies on the claim, attributed to (Levin and Fried 2023), that the latent encoders of SDXL and Kandinsky encode pixels to the same relative positions. This assumption is not validated for the specific models and resolutions used here. A small positional-encoding experiment (e.g., encoding a synthetic grid or dot pattern and measuring alignment) would make the granular-control claim reproducible and would rule out misalignment of the downsampled edit map.","section":"Method, Local Strength Control Using Edit Maps"}],"minor_comments":[{"comment":"The appendix numbering skips from Section 1.4 to Section 1.6; renumber the subsections.","section":"Appendix §1"},{"comment":"The phrase 'for a fixed Markov chain of tds steps' is confusing; it should refer to the total number of steps T or to a continuous time interval [0, T], not to the denoising strength variable t_ds.","section":"Method, Background and Setup"},{"comment":"The description of the quantitative evaluation is under-specified: please state how the edit maps in the 'database' are generated (e.g., blur and erosion parameters), whether they are normalized to [0,1], and how they are downsampled for PIXELS versus binarized for baselines.","section":"Results, Evaluation on Exemplar-Driven Editing"},{"comment":"The memory overhead claim of 0.04% should be accompanied by the measurement protocol (peak GPU memory, batch size, resolution, and whether the base model is SDXL img2img) to be reproducible.","section":"Appendix §6"}],"recommendation":"major_revision","confidential_remarks":"The technical issues are substantive but addressable: the evaluation asymmetry can be fixed with a matched comparison, the Algorithm 1 polarity can be clarified, and Lemma 1 can be corrected or de-emphasized. I see no reason to suspect the core inference-time algorithm is non-functional. The main risk is that the headline quantitative gains may shrink once baselines receive the same edit-map information, so the revision should be judged on the new matched experiments."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Know this: PIXELS is a clean inference-time algorithm for exemplar-based editing with per-pixel strength control, and it is the first to use non-binary edit maps with off-the-shelf latent diffusion. The core shifting-mask trick is simple and believable. But the paper's headline numbers are weakened by an asymmetric comparison, and the proof of Lemma 1 is not correct. Neither kills the contribution; both need fixing.\n\nWhat's new: extending Differential Diffusion's per-pixel strength idea to exemplar guidance, with a progressive masking schedule that gradually exposes regions to denoising based on the edit map, and supporting multiple exemplars in one pass. The algorithm is specified clearly (Algorithms 1-2), it is training-free, and the reported overhead (0.04% memory, ~3.9% time) is plausible. The intuition that the shifting mask mimics the diffusion training distribution by not feeding holes is reasonable. That is real value.\n\nSoft spots, in proportion. First, the Lemma 1 bound: the triangle inequality step in Eq. (5) drops the cross terms, the probability statement wraps an expectation inside a probability in a way that is not meaningful, and the bound is not used anywhere in the method. It is decorative; remove it or replace it with a correct statement. Second, the empirical comparison: the paper says baselines get binarized edit maps while PIXELS gets the soft map, so the FID/CLIP-I gap (5.412 vs 7.915; 91.71 vs 84.95) likely reflects the extra information in the soft map, not the progressive mechanism per se. This is a real confound. The user study has the same problem if PIXELS outputs came from soft maps and baselines from binary maps. It is not fatal—the qualitative results still look good—but the \"notable improvement\" claim should be softened until the comparison is matched, e.g., by also running PIXELS with a binarized map or feeding soft maps to baselines in a fair way. Also, there are no error bars or significance tests on the quantitative table, and the Fig. 6 ablation that increases tds until a realism threshold is met is mildly circular but not a major issue.\n\nWho this is for: anyone working on diffusion-based editing or composing objects into scenes. It deserves a serious referee; the core algorithm is worth engaging, and the reproducibility assets (linked code, off-the-shelf SDXL) make it easy to check. Recommendation: send to review, but insist the authors fix the lemma and re-run the comparison with a matched edit-map setting.","headline":"PIXELS has a clean, useful inference-time idea for per-pixel exemplar editing, but the flawed Lemma 1 and an asymmetric quantitative comparison mean the paper needs revision, not rejection.","tokens_in":16796,"tokens_out":1525,"would_cite":true,"duration_ms":14923,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"PIXELS claims exemplar-driven image editing with per-pixel strength control can be done purely at inference time on off-the-shelf diffusion models, reporting FID 5.412 and CLIP-I 91.71 without any retraining.","keywords":["exemplar-based image editing","latent diffusion models","inference-time editing","non-binary edit maps","progressive editing","multimodal prompts","zero-training editing"],"falsifier":"Take a source image and a point edit map that marks a single latent cell, run PIXELS, then shift the map by one latent cell and run it again; if the output change does not move with the map, the pixel-to-latent alignment assumption is broken. Alternatively, measure the empirical expected squared latent displacement over many source–exemplar pairs at several denoising strengths and check whether it violates the Lemma 1 bound more often than the claimed probability allows.","tokens_in":15847,"feed_emoji":"🎨","tokens_out":10899,"duration_ms":98437,"temperature":0.7,"pith_summary":"The paper claims that high-quality, region-specific image edits guided by reference photos can be done entirely during inference, with no retraining or fine-tuning, by exploiting the latent space of existing text-to-image diffusion models. The key mechanism is a non-binary edit map—a grayscale mask that assigns each pixel a desired strength of change—and as denoising proceeds, a shifting mask gradually lets darker-map regions be re-generated from exemplar-influenced latents while lighter regions stay closer to the source. This gives users control previous exemplar-editing methods lack: gradual transitions, selective object changes, and any number of reference images in one pass. On ImageNet-based evaluation it reports FID 5.412 and CLIP-I 91.71, ahead of four trained baselines, and a user study ranks it best on adherence and realism. If correct, the approach makes fine-grained exemplar editing a property of any compatible open diffusion model rather than a separately trained system.","feed_headline":"No retraining needed: PIXELS steers image edits pixel by pixel","feed_subtitle":"Set how much each region changes, blend multiple reference photos, and keep text control—all inside the existing model.","key_machinery":"The load-bearing object is the surgical latent $z^t_{\\mathrm{mix}} = z^t_1 \\odot \\mathrm{mask}_t + z^{t+1}_{\\mathrm{mix}} \\odot (1-\\mathrm{mask}_t)$, a per-timestep latent blend selected by the thresholded edit map. The load-bearing identity is Lemma 1, which bounds $\\mathbb{E}[\\|z(0)-z_{\\mathrm{surgical}}(t_{\\mathrm{ds}})\\|^2]$ above by $\\sigma^4(t_{\\mathrm{ds}})B + \\sigma^2(t_{\\mathrm{ds}})(k+2\\sqrt{-k\\log p}-2\\log p)$ with probability at least $1-p$, tying denoising strength to how far the edit may travel from the source latents. The linear thresholding of the edit map turns that scalar knob into spatially varying control.","core_discovery":"PIXELS discovers that a pre-trained text-to-image diffusion model can perform exemplar-based editing with no training if the edit is staged through the latent space. Starting from a surgical latent—the source and exemplar encodings blended by the edit map, then noised—the algorithm runs reverse diffusion while a linearly shifting mask controls how much of each spatial region is re-copied from the noisy source latent versus the model's own previous denoising output. Regions marked darker in the edit map enter denoising earlier and therefore deviate more from the source, yielding gradual, realistic transitions instead of binary copy-paste seams. Lemma 1 makes the control principled: it bounds the expected squared distance between the surgical latent and the final realistic latent by $\\sigma^4(t_{\\mathrm{ds}})B+\\sigma^2(t_{\\mathrm{ds}})(k+2\\sqrt{-k\\log p}-2\\log p)$, so raising the denoising strength permits the larger latent traversal needed when source and exemplar are far apart. The paper reports FID 5.412 and CLIP-I 91.71 on ImageNet-based evaluation, beating Versatile Diffusion, Paint-By-Example, IP-Adapter, and MimicBrush, and winning user rankings on adherence and realism.","pith_inferences":["Extension: The shifting-mask schedule is just a linear threshold; replacing it with a content-aware schedule, for instance one that reads semantic segmentation of the edit map, could bias the adherence–realism tradeoff differently in different regions without changing the algorithm.","Extension: Since the method only rewires inference, any diffusion model whose latent encoder preserves spatial correspondence—including video or depth-conditioned models—could inherit the same per-region control, subject to the same alignment assumption.","Extension: The paper's own distance–strength correlation suggests an autotuning rule: measure the latent distance between source and exemplar before editing and set the maximum denoising strength from that distance, turning Lemma 1 into an automatic realism guide rather than a manual knob.","Extension: The stated bottleneck is manual edit-map creation; coupling PIXELS with an automatic map generator from text, segmentation, or depth would make per-pixel control a drop-in feature of interactive editors."],"forward_implications":["With a non-binary edit map, users can give different pixels different edit strengths in one pass, enabling gradual spatial transitions such as a forest-to-beach change.","An arbitrary number of exemplars can be introduced in a single inference pass by nesting the blend operation, or iteratively by feeding the previous output as the new source.","Because the base text-to-image model is untouched, text prompts remain available alongside exemplars for attribute and style edits.","Following Lemma 1, users must raise the maximum denoising strength as the source and exemplar latents grow farther apart, accepting lower exemplar fidelity in exchange for realism.","The overhead over stock SDXL is about 7 MB of memory and roughly 3.86% more inference time, so the control comes cheaply."],"supporting_citations":[{"why":"Supplies the off-the-shelf SDXL model whose inference loop is modified for progressive editing.","marker":"(Podell et al. 2023)"},{"why":"Provides the latent diffusion formulation—encoder, decoder, and reverse denoising—that the surgical-latent procedure relies on.","marker":"(Rombach et al. 2022)"},{"why":"Introduces per-pixel strength editing and the observation that latent encoders preserve relative pixel positions.","marker":"(Levin and Fried 2023)"},{"why":"Supplies the noised-latent perturbation step that starts reverse diffusion from the surgical latent.","marker":"(Song, Meng, and Ermon 2020)"},{"why":"Gives the bounded score assumption used in the Lemma 1 proof.","marker":"(Yang et al. 2023b)"},{"why":"Provides the chi-square upper-tail bound that controls the stochastic term of Lemma 1.","marker":"(Laurent and Massart 2000)"},{"why":"Versatile Diffusion, a trained baseline in the quantitative and user-study comparisons.","marker":"(Xu et al. 2023)"},{"why":"Paint-By-Example, a trained exemplar-editing baseline the method is compared against.","marker":"(Yang et al. 2023a)"},{"why":"IP-Adapter, an image-prompt baseline in the comparison set.","marker":"(Ye et al. 2023)"},{"why":"MimicBrush, the strongest trained baseline, used to judge realism and copy-paste artifacts.","marker":"(Chen et al. 2024a)"}],"fun_headline_variants":["Pixel-perfect edits: PIXELS uses latent surgery without retraining","No training needed: PIXELS edits images with pixel-level control","PIXELS: edit any region, blend refs, no fine-tuning","Latent surgery gives pixel-level image editing without retraining","Off-the-shelf diffusion models edit images by latent surgery, no training"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method works only if the source and exemplar encoders map pixels to the same relative latent positions, so the downsampled edit map lines up with the latent regions it is supposed to control; if that spatial correspondence is not exact, per-pixel strength control edits the wrong places.","fun_headline_variants_meta":{"raw":{"variants":["Pixel-perfect edits: PIXELS uses latent surgery without retraining","No training needed: PIXELS edits images with pixel-level control","PIXELS: edit any region, blend refs, no fine-tuning","Latent surgery gives pixel-level image editing without retraining","Off-the-shelf diffusion models edit images by latent surgery, no training"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000821,"raw_usage":{"total_tokens":3669,"prompt_tokens":1097,"completion_tokens":2572,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":713,"completion_tokens_details":{"reasoning_tokens":2479}},"tokens_in":713,"tokens_out":2572,"duration_ms":15817,"temperature":1.0,"reasoning_tokens":2479,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T19:37:31.069500+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a source image and a point edit map that marks a single latent cell, run PIXELS, then shift the map by one latent cell and run it again; if the output change does not move with the map, the pixel-to-latent alignment assumption is broken. Alternatively, measure the empirical expected squared latent displacement over many source–exemplar pairs at several denoising strengths and check whether it violates the Lemma 1 bound more often than the claimed probability allows.","supporting_citations":[],"review_version":1}