{"id":"b01a8073-1287-4868-903f-9ae3e7bfe17b","arxiv_id":"2412.15050","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"One diffusion network jointly learns the forward and inverse rendering distributions, improving intrinsic decomposition and material editing via a cycle-consistent constraint.","lead":"A unified diffusion model performs both rendering (from material and lighting attributes to an image) and inverse rendering (from an image back to those attributes) in one framework with a cycle-consistent re-rendering loss. If the results hold, this could replace per-object optimization for material editing and relighting tasks.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Equation 5's cycle-consistent loss is written with ground-truth attributes C as the conditioning input, so the claimed gradient path through the inverse-rendering branch may not exist and the core ablation could reflect a different objective.","rationale":"The reader's weakest-assumption identification matches the most load-bearing concern in the paper. The central claim in the abstract, introduction, and Section 3.2 is that the cycle-consistent constraint mitigates inherent inverse-rendering ambiguity, and the ablation results (Tables 2 and 3) attribute large improvements to this constraint. Yet Eq. 5, the only formal statement of that constraint, appears to condition the second pass on the original attributes C rather than on the model's predicted inverse attributes. If C is ground truth, the loss cannot backpropagate through the inverse branch, so the stated mechanism is absent. This is an internal inconsistency, not a disagreement with external consensus, and it is concrete: one can trace the implementation or compute the gradient. The paper provides no code, dataset, or formal verification, and the open-source release is only promised, so the uncertainty cannot currently be resolved. The concern does not by itself prove the method fails; it establishes that the central contribution is unverified as described. That supports keeping the reader's CONDITIONAL verdict rather than moving to ACCEPT or REJECT: clarification and code inspection could settle it either way. I therefore recommend UNCHANGED relative to the reader's verdict, while emphasizing that the cycle-loss equation is the decisive point to check.","tokens_in":12105,"tokens_out":7431,"duration_ms":53530,"concrete_test":"Inspect the released training code (or, if unavailable, ask the authors for one training step's pseudocode) and trace the forward/backward pass for the cycle term. Specifically, check whether the conditioning input to the second denoising pass is the predicted inverse attributes C-hat obtained from an inverse pass on x0, with gradients enabled, or the ground-truth attributes C. A decisive analytical companion check: compute the gradient of Eq. 5 with respect to the inverse-branch parameters; if it is identically zero when C is ground truth, the loss cannot optimize inverse rendering, so the Table 2/3 'w/o constrain' drops must come from a different implemented objective. Alternatively, re-run the ablation with a literal cycle loss L = E[||x0 - Renderer_one_step(InverseRenderer(x0))||^2]; if the reported albedo PSNR gain disappears, then Eq.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.2's 'Cycle-Consistent Constrain' is the paper's central mechanism for mitigating inverse-rendering ambiguity, and Tables 2-3 credit it with roughly 2 dB PSNR on albedo plus large relighting gains. However, Eq. 5, L = E[||x0 - xhat0(xhat_rgb, t, C)||^2], conditions the second denoising pass on the original attribute set C, not on the model's predicted inverse attributes. If C is the ground-truth attribute set, no gradient flows from this loss into the inverse-rendering branch; the term becomes just another forward-rendering reconstruction from the correct attributes, so the sentence 'we use the model's predicted inverse results to perform an additional cycle of rendering' cannot be realized by the equation as written. The symbol xhat_rgb is also ambiguous: it is called a 'noisy input' in the same sentence, yet in a true cycle it should be the re-rendered output produced from predicted attributes. This is not a cosmetic typo: the entire benefit of the cycle-consistent constraint and the 'Ours w/o constrain' ablation hinge on the second pass actually consuming the predicted attributes C-hat with gradients flowing. If the implemented loss is Eq. 5 literally, the reported improvements cannot be attributed to cycle consistency. The promised code and dataset have not been released, so the ambiguity cannot be resolved from the manuscript, and the main quantitative claim lacks a verified mechanism.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Uni-Renderer, a unified dual-stream diffusion framework that performs both rendering (attributes to RGB) and inverse rendering (RGB to attributes) in a single model. The method uses two pre-trained diffusion branches conditioned on each other through a dual-stream architecture, with timestep scheduling that forces one branch clean so that the two conditional distributions q(x0|y0) and q(y0|x0) are learned. A cycle-consistent constraint is introduced to improve inverse rendering by re-rendering the model's own predictions. The authors train on a large synthetic dataset rendered from Objaverse assets and report quantitative and qualitative improvements over several baselines, with ablations showing that both the unified training and the cycle constraint contribute to the reported gains.","tokens_in":12368,"tokens_out":4064,"duration_ms":29157,"significance":"If the central mechanism is correctly specified, the paper offers a plausible and practically relevant contribution: a single diffusion model that approximates the rendering equation in both directions, with a consistency constraint that targets the ambiguity of inverse rendering. The reported quantitative gains are substantial, and the ablation structure directly supports the unified-training claim. The main weakness is that the cycle-consistent loss, which is the paper's core mechanism for reducing ambiguity, is written in a way that appears inconsistent with the text, and the implementation details needed to verify the gradient path are missing. Because the code and dataset are not yet available, this ambiguity cannot currently be resolved from the manuscript. The paper should be credited for the scale of the dataset effort and for evaluating both forward and inverse rendering, but the central mechanism needs a precise and verifiable specification.","major_comments":[{"comment":"The cycle-consistent loss is written as L = E[||x0 - xhat0(xhat_rgb, t, C)||^2], but the surrounding text says the cycle uses 'the model's predicted inverse results to perform an additional cycle of rendering.' If C denotes the same ground-truth attribute set defined in Section 3, then the second denoising pass is conditioned on the original attributes, and no gradient flows from this loss into the inverse-rendering branch; the loss reduces to another forward-rendering reconstruction from the correct attributes. The symbol xhat_rgb is also ambiguous: it is called a 'noisy input' rather than a re-rendered image produced from predicted attributes, and the subsequent sentence uses xhat0(xt, t, C), inconsistent with the equation. This is not a cosmetic issue: the 'Ours w/o constrain' ablation in Tables 2 and 3 credits the cycle constraint with large gains, so the exact form of the loss and the gradient path determine whether the reported improvements can be attributed to cycle consistency. Please rewrite Eq. (5) with unambiguous definitions of C_hat and xhat_rgb, specify whether gradients pass through both branches, and clarify whether the predicted attributes are produced by a one-step x0 prediction or by full denoising.","section":"3.2, Eq. (5)"},{"comment":"The training procedure for the cycle-consistent constraint is underspecified. The pseudo-code in Section 6.3 only describes how timesteps are sampled; it does not state when the cycle loss is computed, how many forward passes are required, whether the re-rendered output is detached or differentiated, or what the target x0 in Eq. (5) is (the original clean image, presumably). Since the core claim is that the cycle constraint mitigates ambiguity, this omission is load-bearing. Please provide a complete training algorithm, including the exact inputs to the second denoising pass and the loss computation.","section":"3.2 and Supplementary Algorithm 1"},{"comment":"The ablations are not described with enough precision to interpret the reported improvements. 'Ours w/o unified' is said to be trained without the unified framework, but the text does not specify how the two separate models are trained or whether they share any components. 'Ours w/o constrain' is the same ablation as 'Ours w/o re-render' in Table 1, but the terminology differs between tables. Please state exactly which components are removed in each ablation and, for the unified ablation, how the two-stream coupling is disabled while keeping the timestep scheduling and data identical.","section":"Section 4.3 and Tables 2-3"}],"minor_comments":[{"comment":"The phrase 'cycle-consistent constrain' is used throughout; the standard term is 'constraint'.","section":"Abstract and Section 3.2"},{"comment":"Figure 2 does not illustrate the cycle-consistent re-rendering pass, even though it is a central contribution. A diagram showing the second pass and where the loss is applied would help readers understand the mechanism.","section":"Figure 2"},{"comment":"Reference [44] has the author names reversed ('Chen Xi, Peng Sida, ...' instead of 'Xi Chen, Sida Peng, ...') and is missing the title; reference [16] lacks the conference or venue information.","section":"References"},{"comment":"The roughness and metallic MSE results are reported for only two optimization-based baselines because the data-driven baselines do not estimate these properties, but the text should state this explicitly in the main paper rather than only in the supplementary.","section":"Section 4.2.2"},{"comment":"The algorithm returns 'timesteps' but the loop variable and comments are inconsistent ('all t' vs 'timesteps'), which makes the pseudocode harder to follow; please align notation.","section":"Section 6.3, Algorithm 1"}],"recommendation":"major_revision","confidential_remarks":"The central issue is verifiability. The paper's main mechanism, the cycle-consistent constraint, is described in words but the published equation appears to be incompatible with the description, and no code or dataset is available to resolve the discrepancy. I am recommending major revision rather than rejection because the specification could be corrected and the experiments re-verified; however, if the implemented loss is exactly what Eq. (5) writes, the reported improvements from the cycle constraint would not support the paper's central claim. The authors should be asked to provide the corrected equation, a precise training algorithm, and ideally release the code prior to final acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: this paper has a genuinely interesting idea — unifying forward and inverse rendering in one diffusion model with cross-conditioning — and the ablations show consistent gains, but the central cycle-consistency loss, Equation 5, is written in a way that doesn't match the text, and without code or data the core mechanism can't be verified.\n\nWhat's new: the dual-stream diffusion setup, where two pre-trained models are connected via zero-convolution and trained with UniDiffuser-style timestep sampling, is a reasonable extension of RGB2X. The dataset effort is real: 200K Objaverse objects with randomized metallic/roughness/lighting variations at 1024 resolution. That alone is useful. On the quantitative side, the method beats several baselines on albedo, normal, and relighting metrics, and the ablations show clear drops when the unified training or the cycle term are removed. So the framework does something.\n\nThe soft spot is Equation 5. The text says the model's predicted inverse results are re-rendered and the loss is computed against the original rendering, with gradients flowing back to the inverse branch. But Eq. 5 conditions the second denoising pass on the original attribute set C, not on the predicted attributes C-hat. If that's the actual loss, no gradient reaches the inverse branch — the term is just another forward-render reconstruction from ground-truth attributes. The symbol xhat_rgb is also ambiguous: in a true cycle it should be the re-rendered output, not a noisy input. This isn't a nitpick; the ablation \"w/o constrain\" attributes roughly 2 dB PSNR to this term, and the mechanism as written cannot produce that. The paper promises open-sourced code but nothing has been released, so the ambiguity can't be resolved from the manuscript.\n\nMinor issues: no error bars or significance tests on the quantitative comparisons, and the test set is small (100 objects). These are standard complaints, not fatal.\n\nOverall: the idea is worth pursuing, and the authors clearly put in serious engineering work, but the central contribution is unverifiable as presented. I'd send it to review, but ask for a corrected formulation of the cycle loss, the actual training objective, and the code/data. Without that, I wouldn't cite it as evidence.","headline":"A promising unified rendering/inverse-rendering framework whose central cycle-consistency loss is written in a way that doesn't do what the text claims, and the code needed to resolve the ambiguity isn't available.","tokens_in":12921,"tokens_out":3535,"would_cite":false,"duration_ms":26606,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"One diffusion model performs rendering and inverse rendering together, and the two tasks help each other.","keywords":["unified diffusion","inverse rendering","rendering","intrinsic decomposition","cycle consistency","material editing","relighting","dual stream"],"falsifier":"Reimplement the model with the cycle loss exactly as written in Equation 5 (conditioning on original C and noisy inverse output) and compare to the described implementation on the validation split used in Table 2; if the reported 2 dB albedo gain disappears when the loss is implemented as written, the core contribution rests on a specification error.","tokens_in":11916,"feed_emoji":"🎨","tokens_out":3748,"duration_ms":20767,"temperature":0.7,"pith_summary":"The paper claims that rendering and inverse rendering can be tamed as two conditional generation tasks in a single dual-stream diffusion model, rather than as separate pipelines. The forward direction turns intrinsic attributes (albedo, roughness, metallic, normals, lighting) into an RGB image; the inverse direction decomposes a single RGB image back into those attributes. The key move is a cycle-consistent constraint that re-renders the model's own inverse predictions and uses the reconstruction error as an extra training signal, which the authors argue reduces the intrinsic ambiguity of inverse rendering. If true, a single model could replace costly path tracing for material editing and relighting while improving the quality of single-image intrinsic decomposition.","feed_headline":"One diffusion model now does rendering and inverse rendering","feed_subtitle":"Cycle-consistent training lets one model split a photo into materials and lighting, then re-render it.","key_machinery":"The engine is a dual-stream latent diffusion with two timesteps: during a rendering iteration the clean attributes are fed at timestep 0 while the RGB latent is noised to a random timestep; during inverse rendering the roles flip. The two streams cross-communicate through zero-convolution connections at mid-block features, and the cycle-consistent loss re-renders the model's own inverse output to compute an extra regression target.","core_discovery":"The central discovery is that jointly training the two conditional distributions with a shared dual-stream architecture and a cycle-consistent re-rendering loss improves both tasks, with inverse rendering reaching albedo PSNR 23.20, SSIM 0.9182, LPIPS 0.0532, beating the strongest baseline (IntrinsicAnything) by 0.5 PSNR and 0.0036 SSIM. Ablations show that removing the unified training drops albedo PSNR by roughly 4.6 dB, and removing the cycle-consistent constraint drops it by about 2 dB, supporting the claim that the two processes facilitate each other.","pith_inferences":["The improvement from 'unified' training might partly come from weight sharing acting as a multi-task regularizer; a test would be to compare against an ensemble of two independently trained models with the same total compute.","The cycle-consistent constraint could be viewed as a form of self-distillation; one could test whether a simpler reconstruction loss on the inverse output alone (auto-encoding) captures part of the gain.","The synthetic-to-real domain gap the authors acknowledge suggests the next testable step: fine-tuning on a small set of real images with a cycle loss should materially close the gap they observe in real-world inversing."],"forward_implications":["One trained model can both edit materials from a real photo (inverse render, tweak attributes, re-render) and synthesize images directly from attributes, removing the per-scene optimization and path-tracing cost of traditional rendering.","Cycle consistency acts as a self-supervised regularizer that reduces the ill-posedness of inverse rendering, so single-image decomposition improves without needing multi-view or ground-truth materials.","The unified model yields relighting and normal estimation that beat dedicated optimization-based baselines (NvdiffRec, GaussianShader) on the synthetic test set.","The dual-stream timestep scheme from UniDiffuser transfers from text-image to image-attribute pairs, giving a recipe for unifying other forward/inverse modality pairs."],"supporting_citations":[{"why":"Supplies the two-timestep unified diffusion formulation that the dual-stream design adapts.","marker":"[1]"},{"why":"Closest prior work with separate forward/inverse diffusion models; the baseline whose lack of cross-connection motivates the cycle constraint.","marker":"[47]"},{"why":"Strongest data-driven inverse-rendering baseline; the method exceeds it on albedo metrics.","marker":"[44]"},{"why":"Image-editing baseline for rendering/material-editing comparisons.","marker":"[3]"},{"why":"Perceptual material-editing baseline for rendering comparisons.","marker":"[41]"},{"why":"Optimization-based inverse rendering baseline for albedo/normal/lighting comparisons.","marker":"[31]"},{"why":"Source of the 200K synthetic assets used to build the training dataset.","marker":"[6]"}],"fun_headline_variants":["Uni-Renderer: one diffusion model for rendering and its inverse","Dual-stream diffusion unifies rendering and inverse rendering","Cycle-consistent diffusion does rendering and inverse rendering","One diffusion model, two-way rendering with cycle consistency","Uni-Renderer joins rendering and inverse rendering in one model"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire benefit of the cycle-consistent constraint rests on that loss actually re-rendering the model's own inverse predictions through a second diffusion pass, but the equation as printed appears to condition on the original attributes and a noisy inverse output instead.","fun_headline_variants_meta":{"raw":{"variants":["Uni-Renderer: one diffusion model for rendering and its inverse","Dual-stream diffusion unifies rendering and inverse rendering","Cycle-consistent diffusion does rendering and inverse rendering","One diffusion model, two-way rendering with cycle consistency","Uni-Renderer joins rendering and inverse rendering in one model"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000336,"raw_usage":{"total_tokens":1828,"prompt_tokens":881,"completion_tokens":947,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":497,"completion_tokens_details":{"reasoning_tokens":865}},"tokens_in":497,"tokens_out":947,"duration_ms":7604,"temperature":1.0,"reasoning_tokens":865,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T11:39:37.195116+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Reimplement the model with the cycle loss exactly as written in Equation 5 (conditioning on original C and noisy inverse output) and compare to the described implementation on the validation split used in Table 2; if the reported 2 dB albedo gain disappears when the loss is implemented as written, the core contribution rests on a specification error.","supporting_citations":[{"cited_title":"One transformer fits all distributions in multi-modal diffusion at scale, 2023","cited_arxiv_id":null,"evidence_quote":"Supplies the two-timestep unified diffusion formulation that the dual-stream design adapts."},{"cited_title":"Rgb ↔x: Image decomposition and synthesis using material- and lighting-aware diffusion models","cited_arxiv_id":null,"evidence_quote":"Closest prior work with separate forward/inverse diffusion models; the baseline whose lack of cross-connection motivates the cycle constraint."},{"cited_title":"Daniel Subias and Manuel Lagunas","cited_arxiv_id":null,"evidence_quote":"Perceptual material-editing baseline for rendering comparisons."},{"cited_title":"Extracting Triangular 3D Models, Materials, and Lighting From Images","cited_arxiv_id":null,"evidence_quote":"Optimization-based inverse rendering baseline for albedo/normal/lighting comparisons."},{"cited_title":"Objaverse: A universe of annotated 3d objects","cited_arxiv_id":null,"evidence_quote":"Source of the 200K synthetic assets used to build the training dataset."}],"review_version":1}