{"id":"ad6e6b02-72a0-4545-8443-e6812a9f92d5","arxiv_id":"2505.08889","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Exact diffusion inversion plus prompt tuning lets users edit intrinsic image channels precisely while preserving identity and automatically resolving lighting effects.","lead":"This paper presents a method to edit photos by manipulating intrinsic image properties (such as surface color, roughness, and lighting) instead of pixels, then having a diffusion model re-render the photo. It claims to preserve the original image's identity while enabling precise edits like relighting, object insertion and removal, and material changes without retraining.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper's own Fig. 10 and Section 5 show the SD VAE latent loses high-frequency detail irrecoverably, so the pixel-precise identity claim is bounded by an external component; the main text never quantifies this ceiling against its reported edit errors.","rationale":"The reader's weakest assumption identifies the SD VAE as the main source of identity shift, and the paper itself confirms this in Section 5 with Fig. 10. This is the right soft spot because the paper's core contributions—exact inversion and prompt optimization—operate entirely in latent space and therefore cannot compensate for irrecoverable VAE compression loss. The concern is load-bearing for the strongest claims of 'pixel precision' and 'identity preservation': if the VAE ceiling is high relative to the edit tolerance, no amount of inversion or prompt tuning can achieve the claimed precision. The paper partially addresses this by comparing D(E(I)) with D(D^{-1}(I)), but it does not quantify the no-edit pipeline error on the same datasets used for quantitative evaluation, nor does it clearly state which latent representation is used in the reported experiments. The proposed test would separate the VAE ceiling from the method's contribution: if the no-edit error is close to the edit error, the method is already at the VAE limit and the central claim should be qualified; if the edit error is much larger, the inversion and prompt optimization still have a measurable identity gap. The paper's ablations and qualitative results support the usefulness of the method, and the CONDITIONAL verdict remains appropriate; this stress-test does not change that verdict, so the output is UNCHANGED.","tokens_in":23650,"tokens_out":8304,"duration_ms":84616,"concrete_test":"On the 12-pair real-world object-removal dataset of Fig. 14, add a no-edit baseline: run the full pipeline with edited channels equal to the original channels, using the same latent initialization as the paper's experiments, and report whole-image and mask PSNR/LPIPS against the ground-truth 'before' image. Separately re-run the same experiment with z0 obtained by decoder inversion D^{-1}(I) rather than the SD encoder E(I), if the current experiments use E(I). If the no-edit reconstruction error is comparable to the reported edit error, the remaining identity shift is dominated by the VAE ceiling and the 'pixel precision' claim should be qualified; if the edit error substantially exceeds the no-edit error, the inversion and prompt optimization still have a measurable identity gap that needs explanation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim of pixel-precise editing with identity preservation is bounded by the SD 2.1 VAE latent used by RGB↔X. Section 5 states that 'the vast majority of identity shifts are caused by the latent-space encoding of the base Stable Diffusion (SD) 2.1 model', and Fig. 10 measures the loss: D(E(I)) loses high-frequency detail (average L1 values around 0.014, 0.024, and 0.019 in the shown examples), and even decoder inversion D(D^{-1}(I)) is described as 'not perfect—an indication of irrecoverable compression information loss.' Since the pipeline operates on a latent z0, exact DDIM inversion (Eq. 5) and prompt optimization (Eq. 8) can at best recover that latent; they cannot restore pixels discarded by the VAE. The identity-preservation ceiling is therefore set by an external component, not by the proposed inversion and transfer machinery. The main text does not explicitly state whether the quantitative experiments use encoder latents z0 = E(I) or decoder-inverted latents z0 = D^{-1}(I), and it does not report a no-edit reconstruction error on the quantitative datasets. Without this decomposition, it is unclear how much of the reported edit error is already present before any edit is made, and how much of the identity shift the method actually removes.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces IntrinsicEdit, a training-free editing framework that operates in the intrinsic-image latent space of the RGB↔X diffusion pipeline. The method couples exact DDIM inversion of the X→RGB renderer with two prompt-embedding optimizations: prompt tuning, which absorbs image-specific identity information before inversion, and channel-to-prompt transfer, which encodes dropped intrinsic channels into the prompt so that only a small selected subset of channels needs to be edited. This is applied to material (color, normal, roughness) editing, object removal and insertion, and relighting, with qualitative comparisons to RGB↔X and several specialized baselines, plus small quantitative evaluations on synthetic material-editing pairs and real-world object-removal pairs. The paper also includes ablations of the inversion method and prompt-optimization components.","tokens_in":23914,"tokens_out":5402,"duration_ms":54530,"significance":"This is a timely and well-motivated extension of the RGB↔X framework, addressing two known deficiencies—identity preservation and intrinsic-channel entanglement—using only inference-time optimization, without fine-tuning or data collection. The qualitative results across diverse editing tasks are compelling, and the method is presented honestly, including a discussion section that attributes residual identity shifts to the base Stable Diffusion VAE. The paper's strengths include the clear formulation of the prompt-tuning and channel-transfer losses, the systematic ablations, and the explicit acknowledgment of the external VAE ceiling. However, the quantitative evaluation is thin (n=10, 4, and 12), and the VAE ceiling is not separated from the method's own edit error, so the central identity-preservation claim is not yet fully quantified.","major_comments":[{"comment":"The paper explicitly states that \"the vast majority of identity shifts are caused by the latent-space encoding of the base Stable Diffusion (SD) 2.1 model\" and shows in Fig. 10 that both D(E(I)) and even D(D^{-1}(I)) lose high-frequency detail, yet it never quantifies this ceiling on the datasets used for the reported PSNR/LPIPS numbers. In particular, the paper does not state whether the quantitative experiments in §4.2 use z0 = E(I) or z0 = D^{-1}(I), and it does not report a no-edit reconstruction error (e.g., D(E(I)) or D(D^{-1}(I)) against the input) for the same 10, 4, and 12 pairs. Without this decomposition, the reported edit errors conflate the external VAE compression loss with the errors introduced by the inversion, prompt optimization, and editing steps. This is load-bearing for the central claim of pixel-precise, identity-preserving editing. I request that the authors (i) state explicitly how z0 is computed in all experiments, (ii) report no-edit reconstruction PSNR/LPIPS on the quantitative datasets, and (iii) if possible, express edit errors relative to that reconstruction ceiling.","section":"§5, Fig. 10, §4.2"},{"comment":"The quantitative evaluation uses only 10 before/after pairs for texture editing, 4 pairs for roughness editing, and 12 pairs for real-world object removal, reporting only mean PSNR and LPIPS with no variance, confidence intervals, or significance tests. With n=4, a single outlier can determine the ranking, and the paper's claim of \"state-of-the-art performance\" across tasks is not statistically supported. I ask the authors to report per-pair results (or scatter plots), provide standard deviations or confidence intervals, and run a simple significance test (e.g., paired bootstrap or Wilcoxon) for the comparisons where the method is claimed to be best. If the datasets cannot be expanded, the claims in the abstract and conclusion should be tempered to reflect the small scale.","section":"§4.2, Fig. 13, Fig. 14"},{"comment":"The relighting evaluation is qualitative only, and the target irradiance maps are generated by a bootstrapping procedure using the same X→RGB model that is then evaluated: the authors sample X→RGB with a prompt and all conditions except irradiance, decompose the result with RGB→X, and use the extracted irradiance as the edit target. This makes the relighting demonstration partly self-confirming, since the model is asked to re-render a lighting condition it produced itself. The paper should either add a quantitative relighting comparison with an independent ground truth (e.g., multi-illumination captured data) or explicitly state in the limitations that relighting results are validated only for plausibility and are generated with the same model used for evaluation.","section":"§4.1, Relighting"}],"minor_comments":[{"comment":"The caption says \"Naive DIM inversion\" but this appears to be a typo for \"Naive DDIM inversion.\"","section":"Fig. 8"},{"comment":"The channel-to-prompt transfer loss optimizes the prompt to imitate the network's own behavior with the full set of channels; it may be helpful to state explicitly that the transferred prompt is therefore optimal for this particular X→RGB renderer and that physical plausibility is inherited from the model's prior rather than from any independent physical supervision.","section":"§3.3, Eq. (7)"},{"comment":"The hyperparameters λ (in [0.1,10] and [1,10] for roughness) and ω (=1.5, or 6 for roughness) are given without any sensitivity analysis; a small ablation or at least a statement of how these values were chosen would help reproducibility.","section":"§4, Implementation details"},{"comment":"There is a typo: \"identity shifts\" appears as \"identify shifts\" in the first sentence of the \"Identity preservation\" subsection.","section":"§5"},{"comment":"The roughness row in the channel table contains a confusing \"✔ ✔ / ✘ ✦ ✘\" entry; this should be reformatted for readability.","section":"Table in §4, Channel organization"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This paper does one genuinely useful thing: it makes the RGB↔X decompose-edit-recompose pipeline actually usable by tackling identity drift and channel entanglement with exact DDIM inversion plus prompt optimization. The combination is new, the ablations are convincing, and the qualitative results across material edits, object removal/insertion, and relighting are strong. I trust the core machinery.\n\nWhere it earns credit: inference-time only, no fine-tuning, built on public baselines. The channel-to-prompt transfer idea—dropping entangled channels and absorbing them into the prompt embedding—is clever and well motivated by the ghosting artifacts shown when kept channels conflict. Ablations in Figs. 8–9 clearly isolate each component's contribution. The synthetic material-editing and real object-removal evaluations use ground truths independent of the method, and the numbers favor it meaningfully.\n\nSoft spots: the quantitative evidence is thin. Ten and four before/after pairs for material editing, twelve for object removal, mean PSNR/LPIPS only, no error bars or significance tests. That is enough to support a \"state-of-the-art\" claim only because the margins are usually large, but it wouldn't survive a stricter statistical look. The relighting evaluation is partly self-referential: the target irradiance is generated by the same X→RGB model (OutCast is used once), so it tests consistency more than correctness. More importantly, the paper never tells you which latent they start from in the quantitative runs—z0 = E(I) or z0 = D^{-1}(I)—and does not report the no-edit reconstruction error. Their own Section 5 and Fig. 10 show the SD VAE loses high-frequency detail irrecoverably. Without that decomposition, you cannot tell how much of the reported edit error is inherited from the VAE before any edit is made. That is a real hole in the evaluation, though not in the method itself.\n\nMinor: the hyperparameter sweep is coarse (λ in 0.1–10, ω fixed at 1.5), and the 75s pre-edit for 512² is slow, but they disclose it.\n\nWho it's for: anyone working in diffusion-based image editing or intrinsic decomposition. The pipeline is reproducible from public baselines and will likely be a baseline for future work. I'd bring it to a reading group and would cite it.\n\nRecommendation: send it to peer review. The core idea is sound, the writing is honest about limitations, and the missing pieces are evaluation rigor, not fatal flaws. A good referee should push for reporting reconstruction error separately from edit error, specifying the latent choice, and adding variance or more data to the quant tables.","headline":"Solid inference-time fix for RGB↔X editing, but the quantitative evaluation is thinner than the claims and the VAE ceiling is never quantified.","tokens_in":24461,"tokens_out":3243,"would_cite":true,"duration_ms":28896,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A diffusion-based editing workflow that operates on albedo, normals, roughness, and irradiance instead of pixels or prompts can perform material edits, object insertion and removal, and relighting with identity preservation.","keywords":["image editing","intrinsic decomposition","diffusion models","identity preservation","relighting","object insertion and removal","diffusion inversion","material editing"],"falsifier":"Run the full no-edit pipeline on a high-frequency texture (hair, fur, fabric) and compare the reconstruction to the input and to the base VAE's encode–decode reconstruction; if the identity shift is not clearly smaller than the encode–decode baseline, the central claim is falsified. A second check: edit only the albedo in a scene with a conflicting normal channel and see whether the output ghosts; the paper's Figure 11 shows this is exactly where channel-to-prompt transfer must prevent artifacts.","tokens_in":23449,"feed_emoji":"🎨","tokens_out":7990,"duration_ms":76471,"temperature":0.7,"pith_summary":"The paper claims that diffusion-based image editing becomes both precise and versatile if the edit is performed not on pixels or text prompts but on the intrinsic channels—albedo (surface color), normal (surface orientation), roughness (glossiness), and irradiance (incoming light)—that a decomposition network estimates from a single photo. To make this practical, the authors add three inference-time components to a decompose–recompose diffusion pipeline: exact inversion of the recomposition model, optimization of the text embedding so identity information lives in the prompt instead of the noise, and transfer of unedited, entangled channels into that prompt so only the channel being edited needs per-pixel manipulation. If the method works as claimed, one untrained workflow can recolor or re-texture surfaces, insert or remove objects together with their shadows and reflections, and relight indoor scenes, all while keeping the rest of the image recognizably the same. The paper's own results identify the compression loss of the underlying latent autoencoder as the main remaining source of identity shift.","feed_headline":"Pixel-precise image editing moves into intrinsic-image space","feed_subtitle":"A single workflow recolors, re-textures, inserts, removes, and relights scenes while preserving identity.","key_machinery":"The load-bearing object is the X→RGB latent diffusion model, treated as a neural renderer that maps an initial noise, a set of intrinsic channel latents, and a text embedding to an image latent. The argument is carried by three mechanisms: exact DDIM inversion, which finds the $z_T$ that reproduces the input under fixed conditions; prompt tuning with the loss $\\mathcal{L}_{\\mathrm{tune}} = \\mathbb{E}_{t,\\boldsymbol{\\varepsilon}} \\|\\boldsymbol{\\varepsilon} - \\boldsymbol{\\varepsilon}_\\theta(z_t, t, c_i, c_p)\\|^2$, which absorbs residual identity into $c_p$; and channel-to-prompt transfer with a loss $\\mathcal{L}_{\\mathrm{transfer}}$, which matches prompt-conditioned predictions to the full-channel prediction so dropped channels (null conditions $\\emptyset_i$) are still represented. The key property they exploit is that the recomposer was trained with channel dropout, so it accepts any subset of valid and null intrinsic conditions; this is what makes it possible to edit one channel while the prompt silently carries the others.","core_discovery":"On its own terms, the central claim is that the intrinsic-image decompose–recompose idea, which was previously blocked by random identity shifts and by the need to update all channels in sync, becomes a usable editor once the recomposer is anchored. The anchor is exact DDIM inversion: treating the recomposition model as a neural renderer $z_0 = \\mathrm{X\\to RGB}(z_T, c_i, c_p)$, the method solves for the initial noise $z_T$ that reproduces the input latent $z_0$ under the given intrinsic conditions and a tuned prompt. Because such inversion tends to bake image-specific detail into the noise, a prompt-tuning step first moves that detail into the text embedding $c_p$; because the intrinsic channels are entangled, a channel-to-prompt transfer step encodes the dropped channels into $c_p$ as well. The result is a single, deterministic noise anchor plus a prompt that carries all non-edited information, so the user can edit only the albedo (for color, texture, and object removal), albedo with normal (for insertion), roughness, or irradiance (for relighting), and the recomposer resolves shadows, reflections, and indirect lighting automatically.","pith_inferences":["Editorial inference: if the VAE compression loss is the true identity bottleneck, replacing the base autoencoder with a higher-fidelity or invertible encoder should improve every downstream task; the paper does not test this, but it is directly measurable with the same no-edit reconstruction comparison.","Editorial inference: the channel-to-prompt transfer recipe is generic—any generative model with dropout-trained conditioning could borrow it to disentangle entangled condition spaces (e.g., depth, normal, segmentation) without retraining.","Editorial inference: the paper's relighting bootstrapping amounts to a light-transfer operator: decompose a scene, sample a lit version, decompose that, and reuse its irradiance. This suggests a copy-lighting-from-one-photo-to-another tool that the paper only demonstrates implicitly.","Editorial inference: because the user-facing effort is now a single-channel inpainting or editing step, a dedicated intrinsic-channel inpainting model would make object removal fully automatic and remove the manual mask requirement."],"forward_implications":["A single untrained pipeline covers material color, texture, normal, and roughness edits, object insertion and removal, and full-scene relighting, so an artist no longer needs a different specialized model per task.","Editing albedo alone removes an object together with its cast shadow and reflection, and editing roughness or normal alone propagates the change to reflections and lighting; global illumination is resolved by the recomposer rather than hand-painted.","Relighting reduces to supplying a new irradiance channel, whether generated by prompting the recomposer, by a shading model such as OutCast, or by decomposing another image, while scene identity stays anchored by the inverted noise.","The pipeline is deterministic after decomposition, so edits can be composed sequentially on the same noise anchor (remove, insert, re-texture, relight) with repeatable results.","Performance is bounded by the underlying RGB↔X models: the method targets indoor scenes and inherits their weak spots (people, garments, metals, transparent objects, imperfect decompositions), which the paper documents as limitations."],"supporting_citations":[{"why":"Supplies the RGB→X decomposition and X→RGB recomposition models, the intrinsic-channel latent space, and the channel-dropout training that makes prompt transfer possible.","marker":"[Zeng et al. 2024a]"},{"why":"Supplies the exact DDIM inversion used to anchor the X→RGB recomposer to the input image.","marker":"[Hong et al. 2024]"},{"why":"Defines the DDIM sampling recurrence that the inversion inverts, and the naive DDIM inversion baseline used in ablations.","marker":"[Song et al. 2021]"},{"why":"Provides the edit-friendly DDPM inversion baseline that the ablation shows over-bakes image information into the noise.","marker":"[Huberman-Spiegelglas et al. 2024]"},{"why":"Establishes the null-text inversion pivot-tuning paradigm that motivates the prompt-optimization step.","marker":"[Mokady et al. 2023]"},{"why":"Defines the latent diffusion framework and VAE whose compression loss the paper identifies as the main identity-shift bottleneck.","marker":"[Rombach et al. 2022]"},{"why":"Provides the base Stable Diffusion 2.1 VAE, which the paper identifies as the dominant source of identity shift and compression loss.","marker":"[AI 2022]"}],"fun_headline_variants":["Intrinsic space enables pixel-precise edits across tasks","Exact inversion anchors identity in intrinsic-space editing","One editor: albedo, texture, insert, remove, relight","Pixel-precise editing without fine-tuning or extra data"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the compressed latent representation of the input image retains enough detail for the inverted noise and tuned prompt to reconstruct identity; the paper's own Figure 10 shows that plain encode–decode already loses high-frequency detail, so if that compression loss exceeds the edit tolerance the identity-preservation claim fails regardless of inversion.","fun_headline_variants_meta":{"raw":{"variants":["Intrinsic space enables pixel-precise edits across tasks","Exact inversion anchors identity in intrinsic-space editing","One editor: albedo, texture, insert, remove, relight","Pixel-precise editing without fine-tuning or extra data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001003,"raw_usage":{"total_tokens":4229,"prompt_tokens":919,"completion_tokens":3310,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":535,"completion_tokens_details":{"reasoning_tokens":3242}},"tokens_in":535,"tokens_out":3310,"duration_ms":24056,"temperature":1.0,"reasoning_tokens":3242,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:45:23.124533+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the full no-edit pipeline on a high-frequency texture (hair, fur, fabric) and compare the reconstruction to the input and to the base VAE's encode–decode reconstruction; if the identity shift is not clearly smaller than the encode–decode baseline, the central claim is falsified. A second check: edit only the albedo in a scene with a conflicting normal channel and see whether the output ghosts; the paper's Figure 11 shows this is exactly where channel-to-prompt transfer must prevent artifacts.","supporting_citations":[],"review_version":1}