{"id":"fba70394-627b-4eb6-8d6d-5f4bb02b9735","arxiv_id":"2412.12083","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A diffusion model that decomposes arbitrary numbers of images of the same object, under varying lighting, into albedo, normal, metallic, and roughness maps with multi-view consistency.","lead":"IDArb is a diffusion-based model that takes one or more photographs of an object, under different lighting, and estimates its surface normals, base color, metalness, and roughness in a consistent way. It also introduces a large synthetic dataset of multi-view renderings to train such models.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 1's SOTA claim may rest on evaluating objects from the same synthetic distribution used for training; the paper never states a held-out split for the 441-object evaluation set, so the central quantitative claim is not yet supported.","rationale":"The reader's rationale correctly lists 'no stated split' and 'no code or data release' as weaknesses but chooses synthetic-material generalization as the weakest assumption. I think the missing held-out split is more load-bearing because it directly undermines Table 1, the only quantitative support for the central SOTA claim. Without a split statement, the reported margins over IID, RGB↔X, IntrinsicAnything, and GeoWizard could reflect familiarity with the exact evaluation objects rather than generalization. This is not an accusation of misconduct; the absence of an explicit split is enough to require verification. The proposed check is feasible because the authors control the rendering pipeline and can release object IDs or a held-out split. If the split is clean, the paper's systems contribution and downstream demonstrations stand and the conditional verdict remains appropriate; if not, the quantitative claims need to be re-derived. I therefore recommend UNVERDICTED until the evaluation protocol is disclosed, rather than accepting the current numbers at face value.","tokens_in":14554,"tokens_out":4399,"duration_ms":39332,"concrete_test":"","verdict_should_be":"UNVERDICTED","load_bearing_attack":"","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes IDArb, a diffusion-based model for intrinsic decomposition (albedo, normal, metallic, roughness) that takes an arbitrary number of input RGB images captured under varying, unconstrained illumination and outputs multi-view consistent intrinsic maps. The method builds on Stable Diffusion by adding cross-view and cross-component attention modules, an illumination-augmented training strategy, and a view-adapted training scheme that alternates between single-view and 3-view inputs. The authors also introduce ARB-Objaverse, a synthetic dataset of 5.7M multi-view renderings with lighting variation, and evaluate IDArb on synthetic benchmarks, real-world images, MIT-Intrinsic, Stanford-ORB, photometric stereo, and optimization-based inverse rendering. The main quantitative claim is that IDArb outperforms state-of-the-art single-image and multi-view methods on all intrinsic decomposition metrics.","tokens_in":14563,"tokens_out":4137,"duration_ms":37496,"significance":"If the quantitative claims hold, IDArb would be a notable advance: it is a feed-forward model that handles a variable number of views, integrates cross-view and cross-component reasoning, and enables downstream relighting, photometric stereo, and 3D reconstruction without per-scene optimization. The ARB-Objaverse dataset is a large-scale resource that could benefit the inverse rendering community. The authors are also transparent about limitations, such as oversimplification of spatially varying materials and O(N^2) attention complexity. However, the significance is currently tempered by evaluation gaps: the primary synthetic benchmark appears to draw objects from the same distribution as training, multi-view consistency is claimed but not quantified, and one real-world benchmark shows mixed results. These issues do not invalidate the approach but must be addressed before the central performance claims can be accepted.","major_comments":[{"comment":"The main quantitative result, Table 1, is potentially circular. The evaluation set is described as '441 objects from Arb-Objaverse and G-Objaverse', but these are also the sources of the training data (Section 3.2). The paper does not state whether the 441 objects are disjoint from the training objects, how the split was made, or whether any object-level separation was enforced. Since the model is trained on 68k+ Objaverse objects, random sampling without a held-out split could easily include training objects, which would make the SOTA gains in Table 1 self-referential. Please specify the exact object-level train/eval split and, ideally, report results on an independent synthetic benchmark or a manually verified subset with no overlap.","section":"Section 4.1 (Evaluation Dataset) and Table 1"},{"comment":"The paper repeatedly claims 'multi-view consistent estimation' and 'multi-view consistency' as a central contribution, but no quantitative consistency metric is ever reported. The ablations in Figure 7 and Tables 4-7 measure per-view accuracy as a function of the number of views, not whether the outputs agree across views (e.g., pairwise albedo/normal differences on overlapping surfaces, reprojection error, or 3D consistency). Without such a metric, the multi-view consistency claim is supported only by qualitative figures. Please add a numerical consistency evaluation, such as average pairwise error between predictions for the same surface point seen from different views.","section":"Section 3.3 and Section 4.2"},{"comment":"The real-world benchmark results are mixed and appear to contradict the unqualified claim of superiority in Section 4.2. On MIT-Intrinsic, IntrinsicAnything achieves higher SSIM (0.896) than IDArb (0.876), while IDArb achieves better PSNR and LPIPS. The paper's statement that IDArb 'outperforms' IntrinsicAnything on real data is therefore not universally supported by the reported metrics. Please report all three metrics in the main text, discuss the SSIM gap, and either temper the claim or provide analysis explaining why PSNR/LPIPS improvements should be weighted more heavily.","section":"Appendix D, Table 8 and Section 4.2"}],"minor_comments":[{"comment":"Specify how many of the 441 evaluation objects came from Arb-Objaverse versus G-Objaverse, and confirm that the four viewpoints per object are distinct from any views used during training.","section":"Section 4.1"},{"comment":"The sentence 'We analysis the effects of the number of viewpoints and lighting conditions' contains a grammatical error; it should read 'We analyze...'.","section":"Section 4.3"},{"comment":"The claim that increasing lighting conditions 'generally improves' performance is not monotonic in the tables; for example, in Table 6 metallic error at #V=2 increases from 0.061 (#L=1) to 0.068 (#L=2). Please either show statistical significance or qualify the claim as a trend rather than a uniform improvement.","section":"Figure 7 and Tables 4-7"},{"comment":"The paper states that cross-view attention is 'proposed' here but also says the module is 'adopted' from Wonder3D in Section 1. Please clarify exactly which components are adapted from Wonder3D and which are novel to IDArb, to avoid ambiguity about the contribution.","section":"Section 3.3 (Architecture)"},{"comment":"There is a typo in the caption: 'NVDiffecMC' should be 'NVDiffRecMC'.","section":"Figure 9 caption"}],"recommendation":"major_revision","confidential_remarks":"The paper is well within the scope of the journal and addresses a timely problem. The main concern is evaluation integrity: the quantitative SOTA claim rests on a synthetic benchmark that may overlap the training set, and the multi-view consistency claim is never measured. These are fixable with additional experiments, but they are load-bearing for the paper's principal claims. I would also urge the authors to make the ARB-Objaverse dataset and evaluation split publicly available if they expect the results to be reproducible."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is a competent systems contribution, but the headline SOTA numbers in Table 1 rest on evaluation data drawn from the same object pool used for training, so the quantitative claim is not yet supported as stated. The stress-test note is right: the paper says it evaluates on 441 objects sampled from Arb-Objaverse and G-Objaverse, and those are exactly the sources used for training, with no held-out split mentioned. That is a real problem for the central claim, and it is addressable.\n\nWhat is genuinely new here is the task formulation: a feed-forward diffusion model that takes one or many views under varying illumination and outputs albedo, normal, metallic, and roughness. The Arb-Objaverse dataset, 5.7M rendered images with multi-illumination and multi-view intrinsics, is a potentially valuable resource even if not yet released. The illumination-augmented, view-adapted training strategy is sensible, and the ablations show it matters. The attention modules are borrowed from Wonder3D and GeoWizard, but adapting them to intrinsic decomposition is reasonable engineering, not a flaw.\n\nThe paper is honest about its limits. The Limitations section explicitly says most synthetic data use global metallic and roughness values, so the model may oversimplify spatially varying real-world materials. That is a fair self-assessment and matches what I see in the failure cases. The real-world qualitative results are plausible, and the Stanford-ORB numbers look strong, though the MIT-Intrinsic comparison is mixed: SSIM is lower than IntrinsicAnything, PSNR is higher, which suggests some metric sensitivity.\n\nOther soft spots: multi-view consistency is claimed but never measured numerically, and there is no code or data release, so independent verification is impossible right now. The \"arbitrary number of views\" claim is honest in spirit but the model is trained only on 1 or 3 views; inference works for 2, 4, 8, 12, which is demonstrated, so that is fine.\n\nWho is this for? Researchers working on inverse rendering, intrinsic decomposition, and material estimation. It deserves a serious referee despite the evaluation flaw. My recommendation is to send it to peer review with a request for a clear train/test split on the synthetic benchmark, a quantitative multi-view consistency metric, and a commitment to release data and code. The core idea is promising and the dataset alone could be a contribution, but the numbers as presented should not be taken at face value.","headline":"Solid systems paper with a promising dataset and training recipe, but the headline SOTA numbers rest on an unstated evaluation split from the training distribution.","tokens_in":15078,"tokens_out":2196,"would_cite":false,"duration_ms":20678,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper's model, IDArb, decomposes any number of images taken under different lights into consistent albedo, normal, metallic, and roughness maps in a single feed-forward pass.","keywords":["intrinsic decomposition","inverse rendering","diffusion model","multi-view consistency","material estimation","relighting","photometric stereo","BRDF"],"falsifier":"Run the trained model on a collection of real objects with measured per-pixel metallic and roughness maps that vary across the surface, such as corroded bronze, and compare error against synthetic objects with global values; a large error gap confined to spatially varying materials would show that the synthetic-data assumption fails.","tokens_in":14345,"feed_emoji":"🎨","tokens_out":11189,"duration_ms":86857,"temperature":0.7,"pith_summary":"IDArb claims that a single feed-forward diffusion model can decompose any number of input images of an object, captured under different and unknown lighting, into mutually consistent intrinsic maps: albedo, surface normal, metallic, and roughness. The load-bearing idea is that multi-view consistency and lighting disentanglement can be learned from data rather than enforced by optimization, so the same model handles one image or many images without knowing camera poses. If the claim holds, inverse rendering becomes a fast one-shot operation for relighting, material editing, photometric stereo, and 3D reconstruction, replacing hours of per-scene optimization and avoiding the inconsistent results of running single-image models repeatedly. The paper reports that the model outperforms existing single-image intrinsic decomposition and normal-estimation methods on every reported metric, and that adding views and lighting variations improves accuracy, especially for metallic and roughness.","feed_headline":"One diffusion model decomposes any number of views into materials","feed_subtitle":"One model pass turns arbitrary photos into albedo, normal, and material maps for relighting and 3D reconstruction.","key_machinery":"The mechanism that carries the argument is a cross-view, cross-component attention block added to the U-Net of a pretrained latent diffusion model. Within the attention operation, keys and values are gathered from all input views and from all three intrinsic triplets simultaneously, so each denoising step can exchange information about geometry, albedo, metal, and roughness across viewpoints. This is supported by an illumination-augmented, view-adaptive training schedule: images are sampled from different viewpoints and lighting conditions, and the model alternates between single-image and multi-image batches so that it retains material priors while learning multi-view fusion. A zero-terminal-SNR noise schedule shifted toward high noise levels is used because the structure of metallic and roughness maps differs from natural images.","core_discovery":"The central discovery, on the paper's own terms, is that intrinsic decomposition can be posed as a conditional diffusion-generation problem over an arbitrary number of views: sample $X_{1:N} \\sim p(X_{1:N} \\mid I_{1:N})$ from an arbitrary number of RGB images captured under unconstrained illumination. The intrinsic components are divided into three triplets (albedo, normal, and metallic with roughness), each denoised as a separate latent channel grouped with the others, and the diffusion U-Net is modified so attention runs both across views and across components. This lets the model jointly reason about shape, material, and lighting, which the authors say removes the ambiguity that makes a black pixel either dark albedo or shadow. Trained on a newly built 5.7-million-image synthetic dataset with multiple HDR environment lights and point lights per viewpoint, and with a training schedule that alternates one-view and multi-view inputs, the model claims to decompose from 1 to 12 viewpoints with consistent outputs and to improve photometric stereo and optimization-based inverse rendering as a prior.","pith_inferences":["Beyond the paper, the same architecture could be retrained on datasets with per-pixel spatially varying metal and roughness; the paper's own limitation suggests this is the next test for real-world generality.","Beyond the paper, the $O(N^2)$ cost of concatenating all views in cross-view attention implies a practical ceiling around a dozen or so views; sparse or hierarchical attention would be needed for dense capture.","Beyond the paper, multi-illumination training suggests a controllable experiment: train two models, one with varied lights and one with a single light, and compare metallic and roughness accuracy under cast shadows, which would isolate the contribution of photometric cues.","Beyond the paper, because the model is view-count agnostic, video frames of a rotating object under moving lights could be decomposed frame-by-frame with cross-view attention providing temporal consistency; the paper does not test this."],"forward_implications":["With a trained IDArb, a user can feed one image or many images under unknown lights and receive albedo, normal, metallic, and roughness maps that agree across views, enabling direct material editing and relighting.","Adding viewpoints and lighting conditions improves decomposition quality, especially for metallic and roughness, with gains diminishing beyond roughly eight viewpoints.","The model can act as a prior for optimization-based inverse rendering by supplying pseudo-material labels, reducing the problem of lighting baked into albedo and improving relighting quality.","The same unmodified model handles photometric stereo from one-light-at-a-time images and outperforms a dedicated photometric-stereo method in the paper's experiments.","Because training uses only object-centric synthetic data, real-world generalization is shown qualitatively, with known failures on outdoor scenes, text, and fine material detail."],"supporting_citations":[{"why":"Defines the IID single-image intrinsic-diffusion baseline that Table 1 must beat on albedo, metallic, and roughness.","marker":"Kocsis et al. (2024)"},{"why":"Defines the RGB↔X material-aware diffusion baseline and the RGB-to-X / X-to-RGB formulation the paper extends and re-implements.","marker":"Zeng et al. (2024)"},{"why":"Defines the IntrinsicAnything albedo baseline used for comparison on synthetic, real, and MIT-Intrinsic data.","marker":"Chen et al. (2024)"},{"why":"Defines the GeoWizard normal-estimation baseline and contributes the cross-domain attention idea the paper adapts.","marker":"Fu et al. (2024)"},{"why":"Supplies the cross-view, cross-domain attention module that IDArb repurposes for intrinsic-component fusion.","marker":"Long et al. (2023)"},{"why":"Supplies the pretrained latent diffusion prior whose U-Net is finetuned for the intrinsic modalities.","marker":"Rombach et al. (2021)"},{"why":"Supplies the object 3D-model collection from which the ARB-Objaverse training dataset is built.","marker":"Deitke et al. (2022)"},{"why":"Provides the optimization-based inverse renderer used in the pseudo-label prior experiments and downstream comparison.","marker":"Hasselgren et al. (2022)"},{"why":"Supplies the zero-terminal-SNR noise schedule that the paper uses to improve metallic and roughness prediction.","marker":"Lin et al. (2024)"},{"why":"Motivates shifting the diffusion noise scheduler toward high noise levels for non-RGB intrinsic domains.","marker":"Shi et al. (2023)"}],"fun_headline_variants":["Any number of views: diffusion splits albedo, normals, and materials","One diffusion pass turns arbitrary photos into material and normal maps","Decompose 1–12 views into albedo, normals, and materials via diffusion","Arbitrary view counts: diffusion decouples shape, material, and light","Diffusion-based intrinsic decomposition for any view count, any lighting"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole approach rests on synthetic objects with mostly uniform metallic and roughness values being representative enough of real-world materials, so a model trained on them can estimate spatially varying materials on real objects.","fun_headline_variants_meta":{"raw":{"variants":["Any number of views: diffusion splits albedo, normals, and materials","One diffusion pass turns arbitrary photos into material and normal maps","Decompose 1–12 views into albedo, normals, and materials via diffusion","Arbitrary view counts: diffusion decouples shape, material, and light","Diffusion-based intrinsic decomposition for any view count, any lighting"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001348,"raw_usage":{"total_tokens":5497,"prompt_tokens":991,"completion_tokens":4506,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":607,"completion_tokens_details":{"reasoning_tokens":4407}},"tokens_in":607,"tokens_out":4506,"duration_ms":31037,"temperature":1.0,"reasoning_tokens":4407,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T14:17:03.789655+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the trained model on a collection of real objects with measured per-pixel metallic and roughness maps that vary across the surface, such as corroded bronze, and compare error against synthetic objects with global values; a large error gap confined to spatially varying materials would show that the synthetic-data assumption fails.","supporting_citations":[{"cited_title":"Input image and ground truth have been contrast-adjusted for better visibility","cited_arxiv_id":null,"evidence_quote":"Defines the IntrinsicAnything albedo baseline used for comparison on synthetic, real, and MIT-Intrinsic data."}],"review_version":1}