Pith. sign in

REVIEW 3 major objections 5 minor 64 references

DNF-Intrinsic: Deterministic Noise-Free Diffusion for Indoor Inverse Rendering

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper shows that for diffusion-based inverse rendering, starting the flow trajectory from the source image rather than Gaussian noise yields deterministic one-step prediction of five indoor intrinsic properties and, on InteriorVerse…

desk verdict Strong empirical paper with a genuinely new image-to-intrinsic flow matching idea; the reconstruction loss does not actually integrate lighting, so the physical faithfulness claim is overstated. read the letter →

arxiv 2507.03924 v2 pith:W6E6OK6V submitted 2025-07-05 cs.CV

Rongjia Zheng , Qing Zhang , Chengjiang Long , Wei-Shi Zheng This is my paper
classification cs.CV
keywords inverserenderingintrinsicimagedecompositiondiffusionmodelsflowmatchingindoorsceneunderstandingmaterialestimationsingle-imagereconstructiongenerativerenderer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Single-image inverse rendering means recovering a scene's material and geometry from one photo, an ill-posed problem that recent diffusion-based methods have approached by starting from Gaussian noise and denoising toward the desired intrinsic map. This paper argues that the noise-to-intrinsic route throws away exactly the structure and appearance information the task needs, and replaces it with a deterministic image-to-intrinsic mapping: the latent code of the source photo is the starting point, and flow matching carries it along a short straight path to each of five intrinsic properties. A generative renderer, trained with a score-distillation reconstruction loss, keeps the predictions physically faithful to the input image. On the synthetic InteriorVerse benchmark the method reports albedo PSNR 21.95 while the best prior baseline reaches 18.84, and it beats all five compared methods on all five predicted properties; it also improves on real-world IIW, NYUv2, and DIODE. The practical point is that a single pre-trained diffusion model, lightly fine-tuned, can deliver holistic inverse rendering in about 0.1 seconds.

What carries the argument

The central object is a deterministic flow-matching ODE between the latent code $Z$ of the source image and each target latent intrinsic $Z_i$, with interpolation $Z_t = Z(1-t) + Z_i t$ and a network $\mu_{\theta}$ trained to predict the velocity $Z_i - Z$. A text prompt per intrinsic property selects which of the five trajectories to follow, and LoRA, a parameter-efficient low-rank fine-tuning method, adapts a pre-trained diffusion transformer so only a small parameter set is trained. The other half is a generative renderer: a ControlNet fine-tuned to map the five predicted intrinsic maps back to the input image, whose score-distillation reconstruction loss provides the constraint; the Gaussian noise in that renderer represents unknown lighting, and sampling many noises is argued to integrate over all possible lighting.

What would settle it

Take a fixed predicted set of the five intrinsics, zero out one channel (for example, replace albedo with a gray map), and render with the trained generative renderer under many noise seeds; if the output images barely change, the reconstruction loss is not actually measuring whether the intrinsics explain the image, which would undermine the lighting-integration argument.

Watch

Extended reading notes

Core claim

The paper's central claim is that for diffusion-based inverse rendering, the mapping should be image-to-intrinsic rather than image-conditioned noise-to-intrinsic. Starting the flow trajectory at the VAE-encoded latent of the input image preserves the structure and appearance cues that inverse rendering depends on, and using flow matching instead of DDPM gives a straight trajectory that needs only ten sampling steps and already shows strong results with one. The same model serves all five properties by using a text prompt as a trigger that selects the direction of the trajectory. A ControlNet-based generative renderer, supervised to turn the five intrinsic maps back into the source image with Gaussian noise standing in for unknown lighting, adds a score-distillation reconstruction loss that removes shadow residuals and pushes the intrinsics toward lighting independence. The paper reports that this design outperforms prior CNN and diffusion baselines on all evaluated properties on InteriorVerse and on real-world IIW, NYUv2, and DIODE.

Load-bearing premise

The load-bearing premise is that a renderer trained without any lighting input, with random noise standing in for unknown lighting, gives a trustworthy signal about whether the predicted surface properties truly explain the image.

Editorial extensions

If this is right

  • The paper's reported single-step inference (0.1 seconds) makes holistic inverse rendering practical for interactive editing and on-device use.
  • Starting from the image latent removes the run-to-run variance caused by random noise initialization in prior diffusion methods.
  • A text prompt selects the target intrinsic, so the same network can be extended to new properties by adding prompts and training data rather than new branches.
  • The reconstruction loss removes lighting artifacts such as shadow residuals in albedo, which the paper shows is needed for faithful relighting and material editing.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same image-to-intrinsic flow recipe could generalize to other deterministic image-to-image tasks with diffusion priors, such as video-depth or semantic segmentation, wherever input structure matters as much as output fidelity.
  • One testable extension is to stop treating the renderer's noise as anonymous: optimize the noise code jointly with the intrinsics to recover an explicit lighting estimate, which the paper only does as a post hoc application.
  • Because the reconstruction gradient comes from score distillation, part of the reported gain may reflect renderer-specific consistency rather than physical correctness; a controlled check is to ablate the renderer's sensitivity to each intrinsic channel.
  • The real-world results come from a model trained only on synthetic indoor data, so a natural next test is whether the same recipe transfers to other synthetic datasets or pseudo-labeled real data without retuning.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes DNF-Intrinsic, a single-image indoor inverse rendering method that fine-tunes a pre-trained Stable Diffusion 3 Diffusion Transformer with LoRA to learn deterministic image-to-intrinsic mappings via flow matching, instead of the usual noise-to-intrinsic diffusion paradigm. The model jointly predicts albedo, metallic, roughness, normal, and depth from an RGB image, using text prompts to select the intrinsic trajectory. A generative renderer (ControlNet on Stable Diffusion 2) is introduced, together with a reconstruction loss based on score distillation sampling, to encourage the predicted intrinsics to be physically faithful to the input image. Experiments on the InteriorVerse synthetic dataset and on IIW, NYUv2, and DIODE real-world datasets show consistent improvements over existing inverse rendering methods, with faster inference and fewer trainable parameters.

Significance. If the benchmark results hold, the deterministic flow-matching formulation is a genuine advance for diffusion-based inverse rendering: it avoids noise-to-intrinsic sampling, achieves state-of-the-art accuracy on InteriorVerse across all five predicted properties, and reduces inference cost to roughly 10 steps or even a single step. The paper provides a thorough ablation study (Table 3), comparisons with recent baselines, and code release. However, the secondary claim that the reconstruction loss enforces physical faithfulness is not adequately supported; the lighting-integration argument in Sec. 3.3 is questionable. The deterministic mapping contribution appears sound and could stand independently, but the physical-faithfulness claim needs revision or stronger evidence.

major comments (3)
  1. [Sec. 3.3, Eq. (10)] The reconstruction loss in Eq. (10) does not implement the stated lighting integration. In Eq. (8), x_t = sqrt(alpha_t) Z + sqrt(1-alpha_t) epsilon is the standard forward noising of the input latent Z, and the generative renderer R is trained in Eq. (9) to predict epsilon from x_t, t, and the intrinsic condition. Since x_t already contains nearly all information about the input image at small t, R can denoise almost independently of the predicted intrinsics, and the SDS gradient carries little physical-consistency signal. At large t the target lighting is destroyed, so sampling different epsilon realizations averages over noise, not over lighting. The statement in Sec. 3.3 that sampling various lighting 'is functionally equivalent to integrating all the possible lighting' is therefore unsupported: no lighting variable appears in R's inputs, and epsilon_t does not parameterize illumination. This concern is load-bearing for the 'physically faithful' contribution; please provide a formal justification or an experiment that directly measures whether L_rec improves agreement with a physics-based renderer.
  2. [Eq. (9) vs Eq. (10); Appendix B] There is a train/test condition mismatch in the reconstruction loss. The generative renderer is trained with ground-truth intrinsics as the ControlNet condition in Eq. (9), but L_rec in Eq. (10) uses the predicted intrinsics µ_theta(I), which are initially inaccurate and hence out-of-distribution for R. The SDS gradient may then reflect renderer artifacts rather than physical consistency. Please either fine-tune R on predicted intrinsics (or a distribution of corrupted intrinsics) or provide an analysis showing that the gradient remains informative. Appendix B's pseudo-code additionally diffuses the pixel-space image x rather than the latent Z used in Eq. (10), so the exact computation of L_rec should be clarified.
  3. [Table 3, rows 8-9] The ablation for L_rec shows only a 0.92 dB PSNR improvement (21.03 to 21.95) and a 0.02 LPIPS improvement; the qualitative benefit is demonstrated on a single example. Given the theoretical concerns above, this small gain does not by itself establish that L_rec provides physical-consistency supervision, as opposed to a denoising shortcut that slightly improves albedo. Please report additional evidence, such as re-rendering error on held-out scenes, invariance of predictions to lighting changes, or a comparison against a physics-based renderer, to support the claim that the predicted intrinsics are 'physically faithful'.
minor comments (5)
  1. [Sec. 3.5] The optimizer name 'AdmaW' should read 'AdamW'.
  2. [Figure 2 and Sec. 3.4] The Figure 2 caption says 'one step sampling' while the inference procedure uses K=10 steps; please make the terminology consistent.
  3. [Appendix B, Figure 12] The pseudo-code computes z_t = alpha_t * x + sigma_t * eps on the pixel-space image x, whereas Eq. (10) uses the latent code Z; please clarify whether the reconstruction loss is applied in latent or pixel space.
  4. [Appendix D and E] The appendix contains repeated figure captions and duplicated text blocks, and the main text refers to both 'Figure 10' and 'Figure 23' for the applications; please clean up the formatting.
  5. [Sec. 3.3] The SDS implementation should specify whether classifier-free guidance is used and, if so, the guidance scale and how the renderer's epsilon prediction is converted into a gradient for the intrinsic network.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reported benchmark results are held-out external evaluations, and the auxiliary reconstruction loss does not make any prediction reduce to its own inputs by construction.

full rationale

The paper's central claims are benchmark results on the InteriorVerse test set and on external real-world datasets (IIW, NYUv2, DIODE), with ground-truth metrics that are independent of the method's training losses. The deterministic image-to-intrinsic flow matching (Eqs. 5-7) is supervised by ground-truth intrinsics and evaluated on held-out data, so those predictions are not fitted inputs renamed as predictions. The only component that could look self-referential is the generative-renderer reconstruction loss L_rec (Eq. 10), which uses the authors' own ControlNet renderer and treats Gaussian noise as a proxy for unknown lighting (Sec. 3.3 Discussion and Appendix Figure 12). However, this is an auxiliary training signal rather than the source of the headline numbers; the ablation (Table 3) shows it contributes only 0.92 dB PSNR on albedo, and the main results do not reduce to L_rec by construction. Even if the 'noise integrates all possible lighting' assertion were empirically unsupported, that would be a correctness or robustness concern, not circularity: no derived quantity is defined in terms of the target metric, and no fitted parameter is later reported as an independent prediction. Citations are standard external works (flow matching, DreamFusion, ControlNet); the only overlapping-author reference ([60]) appears in a list of prior IID methods and is not load-bearing. The derivation chain is therefore self-contained against external benchmarks, with no exhibitable circular step.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method relies on pre-trained models (SD3 VAE/DiT, SD2 ControlNet) and on the assumption that linear interpolation in latent space and a lighting-agnostic generative renderer provide valid training signal. Hyperparameters (LoRA rank, steps, learning rate) are chosen by hand but are not fitted to the test benchmarks.

free parameters (4)
  • LoRA rank = 64
    Set in Sec 3.5; chosen by hand, affects capacity of the fine-tuned DiT.
  • Number of sampling steps K = 10
    Set in Sec 3.4 as '10 in all our experiments'; trade-off between speed and accuracy.
  • Learning rate / weight decay = 1e-4 / 1e-2
    AdamW hyperparameters from Sec 3.5; standard but hand-selected.
  • Training epochs = 100
    Sec 3.5; chosen to match prior work [28,61].
assumptions (4)
  • domain assumption Pre-trained diffusion prior is preserved under noise-free image-to-target fine-tuning
    The method is predicated on SD3's prior helping after fine-tuning with image-to-intrinsic flow matching; asserted in Sec 3.2 and supported by citations [2,42] and the from-scratch ablations in Table 3.
  • domain assumption Linear interpolation in VAE latent space defines a meaningful flow trajectory between image and intrinsic latents
    Eq 5 constructs Z_t as a straight-line interpolation; the flow velocity (Zi-Z) is learned with Eq 7. This is standard rectified flow, but the semantic validity in SD3 latent space is assumed.
  • domain assumption SDS loss through a lighting-agnostic ControlNet renderer yields physically correct, lighting-independent intrinsics
    Sec 3.3: sampling different noises is claimed to be 'functionally equivalent to integrating all the possible lighting,' but the renderer is not a physics-based renderer and this equivalence is not proven.
  • domain assumption VAE encoder/decoder from SD3 preserves the information needed for intrinsic prediction
    The method operates entirely in SD3 latent space; distortions introduced by the VAE could cap intrinsic accuracy, though experiments suggest it is sufficient.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DNF-Intrinsic: Deterministic Noise-Free Diffusion for Indoor Inverse Rendering." pith.science (2026). https://pith.science/paper/W6E6OK6V

@misc{pith2026250703924,
  author       = {Pith},
  title        = {Pith review of: DNF-Intrinsic: Deterministic Noise-Free Diffusion for Indoor Inverse Rendering},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/W6E6OK6V}},
  note         = {Machine review of arXiv:2507.03924}
}
read the original abstract

Recent methods have shown that pre-trained diffusion models can be fine-tuned to enable generative inverse rendering by learning image-conditioned noise-to-intrinsic mapping. Despite their remarkable progress, they struggle to robustly produce high-quality results as the noise-to-intrinsic paradigm essentially utilizes noisy images with deteriorated structure and appearance for intrinsic prediction, while it is common knowledge that structure and appearance information in an image are crucial for inverse rendering. To address this issue, we present DNF-Intrinsic, a robust yet efficient inverse rendering approach fine-tuned from a pre-trained diffusion model, where we propose to take the source image rather than Gaussian noise as input to directly predict deterministic intrinsic properties via flow matching. Moreover, we design a generative renderer to constrain that the predicted intrinsic properties are physically faithful to the source image. Experiments on both synthetic and real-world datasets show that our method clearly outperforms existing state-of-the-art methods.

Figures

Figures reproduced from arXiv: 2507.03924 by the authors.

Figure 1
Figure 1. Performance of our method on indoor inverse rendering. By fine-tuning a pre-trained diffusion model on InteriorVerse [61], a synthetic indoor scene dataset, we achieve state-of-the-art indoor inverse rendering that can robustly recover albedo, metallic, roughness, normal, and depth from a single RGB indoor image. Taking albedo estimation as an example, as shown on the right, our method not only obtains significantly… view at source ↗
Figure 2
Figure 2. Overview of our method. Given a single indoor image, we aim to recover albedo, metallic, roughness, normal, and depth by learning deterministic image-to-intrinsic mappings triggered by text prompts. During training, instead of starting from noise as done in previous diffusion-based methods, we employ a pre-trained VAE encoder E to initialize the latent code Z of the input image as the starting point of the flow traj… view at source ↗
Figure 3
Figure 3. Qualitative comparison of inverse rendering on the synthetic InteriorVerse dataset [61]. Although both our method and IndoorIR [61] enable metallic estimation, we here do not compare on metallic as InverseIndoor [32] cannot predict it. Please see the supplementary material for more results and comparison on metallic estimation. and straight trajectory, enabling prediction with only a small number of sampling steps. … view at source ↗
Figures from the paper (18 more)
Figure 5
Figure 5. Figure 5: Qualitative comparison of normal estimation on real-world images. Method IIW NYUv2 DIODE WHDR↓ AE↓ AMRE↓ AE↓ AMRE↓ InverseIndoor [32] 21.99 38.51 0.28 44.70 0.20 IndoorIR [61] 22.90 34.38 0.23 36.03 0.19 IntrinsicAnything [13] 27.75 - - - - IntrinsicDiff [28] 22.02 - -…
Figure 6
Figure 6. Figure 6: Qualitative comparison of depth estimation on real-world images. (a) Real-world input (b) SD3+Noise+Flow+LoRA (c) SD3+Image+DDPM+LoRA (d) UNet from scratch (e) DiT from scratch (f) UNet from SD2 (g) Ours with ControlNet (h) Fine-tuning all parameters (i) Ours without L…
Figure 7
Figure 7. Figure 7: Qualitative ablation study on real-world albedo estimation. our method produces albedo predictions that are more faith￾ful to the input images. Visual comparisons on normal and depth estimation are given in Figures 5 and 6, respectively. As can be seen, despite that ou…
Figure 8
Figure 8. Figure 8: Quantitative comparison of albedo estimation using varying diffusion steps on the InteriorVerse dataset. In com￾parison to recent diffusion-based inverse rendering methods, e.g., IntrinsicDiff [28], IntrinsicAnything [13], and RGBX [58], our method achieves better albe…
Figure 9
Figure 9. Figure 9: Qualitative comparison of albedo estimation using different number of diffusion steps. Effect of Diffusion Transformer (DiT) [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]
Figure 11
Figure 11. Figure 11: , our generative renderer enables various potential applications, including uncontrollable relighting, material editing, and object removal. Specifically, given the scene’s intrinsic properties, our generative renderer can generate images with lighting conditions diff…
Figure 12
Figure 12. Figure 12: Pseudo code for the SDS-based reconstruction loss via the generative renderer that defines a differentiable mapping from [PITH_FULL_IMAGE:figures/full_fig_p013_12.png]
Figure 13
Figure 13. Figure 13: More qualitative comparison of albedo estimation on the synthetic InteriorVerse dataset [61] [PITH_FULL_IMAGE:figures/full_fig_p014_13.png]
Figure 17
Figure 17. Figure 17: More qualitative comparison of metallic estimation on the synthetic InteriorVerse dataset [61]. Input IndoorIR [61] RGBX [58] Ours GT [PITH_FULL_IMAGE:figures/full_fig_p015_17.png]
Figure 19
Figure 19. Figure 19: More qualitative comparison of normal estimation on the synthetic InteriorVerse dataset [61] [PITH_FULL_IMAGE:figures/full_fig_p016_19.png]
Figure 20
Figure 20. Figure 20: More qualitative comparison of depth estimation on the synthetic InteriorVerse dataset [61] [PITH_FULL_IMAGE:figures/full_fig_p017_20.png]
Figure 21
Figure 21. Figure 21: More qualitative comparison of albedo estimation on real-world images [PITH_FULL_IMAGE:figures/full_fig_p018_21.png]
Figure 25
Figure 25. Figure 25: More qualitative comparison of metallic estimation on real-world images. Input IndoorIR IntrinsicDiff [28] RGBX [58] Ours [PITH_FULL_IMAGE:figures/full_fig_p019_25.png]
Figure 20
Figure 20. Figure 20: Figure 26 [PITH_FULL_IMAGE:figures/full_fig_p019_20.png]
Figure 27
Figure 27. Figure 27: More qualitative comparison of normal estimation on real-world images [PITH_FULL_IMAGE:figures/full_fig_p020_27.png]
Figure 28
Figure 28. Figure 28: More qualitative comparison of depth estimation on real-world images [PITH_FULL_IMAGE:figures/full_fig_p021_28.png]
Figure 19
Figure 19. Figure 19: More results of virtual object insertion. Input Material editing Lighting editing: result 1 Lighting editing: result 2 [PITH_FULL_IMAGE:figures/full_fig_p022_19.png]
Figure 20
Figure 20. Figure 20: More results of material and lighting editing. 17 [PITH_FULL_IMAGE:figures/full_fig_p022_20.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

64 extracted references · 51 canonical work pages

  1. [1]

    Rethinking inductive biases for surface normal estimation

    Gwangbin Bae and Andrew J Davison. Rethinking inductive biases for surface normal estimation. InCVPR, 2024. 2, 5, 7

    work page 2024

  2. [2]

    Cold diffusion: Inverting arbitrary im- age transforms without noise.NeurIPS, 36, 2023

    Arpit Bansal, Eitan Borgnia, Hong-Min Chu, Jie Li, Hamid Kazemi, Furong Huang, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Cold diffusion: Inverting arbitrary im- age transforms without noise.NeurIPS, 36, 2023. 2

    work page 2023

  3. [3]

    Shape, albedo, and illumination from a single image of an unknown object

    Jonathan T Barron and Jitendra Malik. Shape, albedo, and illumination from a single image of an unknown object. In CVPR, pages 334–341, 2012. 2

    work page 2012

  4. [4]

    Shape, illumination, and reflectance from shading.IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(8):1670–1687, 2014

    Jonathan T Barron and Jitendra Malik. Shape, illumination, and reflectance from shading.IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(8):1670–1687, 2014. 2, 3

    work page 2014

  5. [5]

    Re- covering intrinsic scene characteristics.Comput

    Harry Barrow, J Tenenbaum, A Hanson, and E Riseman. Re- covering intrinsic scene characteristics.Comput. vis. syst, 2 (3-26):2, 1978. 1

    1978

  6. [6]

    Joint learn- ing of intrinsic images and semantic segmentation

    Anil S Baslamisli, Thomas T Groenestege, Partha Das, Hoang-An Le, Sezer Karaoglu, and Theo Gevers. Joint learn- ing of intrinsic images and semantic segmentation. InECCV, pages 286–302, 2018. 1

    work page 2018

  7. [7]

    Intrinsic images in the wild.ACM Transactions on Graphics (TOG), 33(4): 1–12, 2014

    Sean Bell, Kavita Bala, and Noah Snavely. Intrinsic images in the wild.ACM Transactions on Graphics (TOG), 33(4): 1–12, 2014. 2, 5

    work page 2014

  8. [8]

    Nerd: Neural reflectance decomposition from image collections

    Mark Boss, Raphael Braun, Varun Jampani, Jonathan T Bar- ron, Ce Liu, and Hendrik Lensch. Nerd: Neural reflectance decomposition from image collections. InICCV, pages 12684–12694, 2021. 2

    work page 2021

Show all 64 references
  1. [9]

    Intrinsic image decomposi- tion via ordinal shading.ACM Transactions on Graphics, 43 (1):1–24, 2023

    Chris Careaga and Ya ˘gız Aksoy. Intrinsic image decomposi- tion via ordinal shading.ACM Transactions on Graphics, 43 (1):1–24, 2023. 2

    2023

  2. [10]

    Colorful diffuse intrinsic image decomposition in the wild.ACM Transactions on Graphics (TOG), 43(6):1–12, 2024

    Chris Careaga and Ya ˘gız Aksoy. Colorful diffuse intrinsic image decomposition in the wild.ACM Transactions on Graphics (TOG), 43(6):1–12, 2024

    2024

  3. [11]

    Intrinsic harmonization for illumination-aware image com- positing

    Chris Careaga, S Mahdi H Miangoleh, and Ya ˘gız Aksoy. Intrinsic harmonization for illumination-aware image com- positing. InSIGGRAPH Asia 2023 Conference Papers, 2023. 2

    2023

  4. [12]

    Shape from shading for robotic ma- nipulation

    Arkadeep Narayan Chaudhury, Leonid Keselman, and Christopher G Atkeson. Shape from shading for robotic ma- nipulation. InWACV, pages 8389–8398, 2024. 1

    2024

  5. [13]

    Intrinsicanything: Learning diffusion priors for inverse rendering under unknown illumi- nation

    Xi Chen, Sida Peng, Dongchen Yang, Yuan Liu, Bowen Pan, Chengfei Lv, and Xiaowei Zhou. Intrinsicanything: Learning diffusion priors for inverse rendering under unknown illumi- nation. InECCV, 2024. 1, 2, 3, 5, 6, 8

    2024

  6. [14]

    Structure from duplicates: Neu- ral inverse graphics from a pile of objects.NeurIPS, 36,

    Tianhang Cheng, Wei Chiu Ma, Kaiyu Guan, Antonio Tor- ralba, and Shenlong Wang. Structure from duplicates: Neu- ral inverse graphics from a pile of objects.NeurIPS, 36,

  7. [15]

    Differentiable point-based inverse rendering

    Hoon-Gyu Chung, Seokjun Choi, and Seung-Hwan Baek. Differentiable point-based inverse rendering. InCVPR, pages 4399–4409, 2024. 2

    2024

  8. [16]

    Pie-net: Pho- tometric invariant edge guided network for intrinsic image decomposition

    Partha Das, Sezer Karaoglu, and Theo Gevers. Pie-net: Pho- tometric invariant edge guided network for intrinsic image decomposition. InCVPR, 2022. 2

    2022

  9. [17]

    Idtransformer: Transformer for intrinsic image decom- position

    Partha Das, Maxime Gevers, Sezer Karaoglu, and Theo Gev- ers. Idtransformer: Transformer for intrinsic image decom- position. InICCV, pages 816–825, 2023. 2

    2023

  10. [18]

    Generative models: What do they know? do they know things? let’s find out!arXiv preprint arXiv:2311.17137, 2023

    Xiaodan Du, Nicholas Kolkin, Greg Shakhnarovich, and Anand Bhattad. Generative models: What do they know? do they know things? let’s find out!arXiv preprint arXiv:2311.17137, 2023. 2

    2023 arXiv

  11. [19]

    Diffusion reflectance map: Single-image stochastic inverse rendering of illumination and reflectance

    Yuto Enyo and Ko Nishino. Diffusion reflectance map: Single-image stochastic inverse rendering of illumination and reflectance. InCVPR, pages 11873–11883, 2024. 2

    2024

  12. [20]

    Scaling recti- fied flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M ¨uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling recti- fied flow transformers for high-resolution image synthesis. InICML, 2024. 4, 5

    2024

  13. [21]

    Revisiting deep intrinsic image decompositions

    Qingnan Fan, Jiaolong Yang, Gang Hua, Baoquan Chen, and David Wipf. Revisiting deep intrinsic image decompositions. InCVPR, pages 8944–8952, 2018. 3

    2018

  14. [22]

    Intrinsic images by entropy minimization

    Graham D Finlayson, Mark S Drew, and Cheng Lu. Intrinsic images by entropy minimization. InECCV, 2004. 2

    2004

  15. [23]

    Geowiz- ard: Unleashing the diffusion priors for 3d geometry estima- tion from a single image

    Xiao Fu, Wei Yin, Mu Hu, Kaixuan Wang, Yuexin Ma, Ping Tan, Shaojie Shen, Dahua Lin, and Xiaoxiao Long. Geowiz- ard: Unleashing the diffusion priors for 3d geometry estima- tion from a single image. InECCV, 2024. 2, 3

    2024

  16. [24]

    Deep parametric indoor lighting estimation

    Marc-Andr ´e Gardner, Yannick Hold-Geoffroy, Kalyan Sunkavalli, Christian Gagn ´e, and Jean-Franc ¸ois Lalonde. Deep parametric indoor lighting estimation. InICCV, pages 7175–7183, 2019. 2

    2019

  17. [25]

    Denoising diffu- sion probabilistic models.NeurIPS, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models.NeurIPS, 2020. 3

    2020

  18. [26]

    Lora: Low-rank adaptation of large language models.arXiv preprint arXiv:2106.09685, 2021

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models.arXiv preprint arXiv:2106.09685, 2021. 4

    2021 arXiv

  19. [27]

    Repurpos- ing diffusion-based image generators for monocular depth estimation

    Bingxin Ke, Anton Obukhov, Shengyu Huang, Nando Met- zger, Rodrigo Caye Daudt, and Konrad Schindler. Repurpos- ing diffusion-based image generators for monocular depth estimation. InCVPR, pages 9492–9502, 2024. 2, 3, 5, 7

    2024

  20. [28]

    In- trinsic image diffusion for single-view material estimation

    Peter Kocsis, Vincent Sitzmann, and Matthias Nießner. In- trinsic image diffusion for single-view material estimation. InCVPR, 2024. 1, 2, 3, 5, 6, 8, 12

    2024

  21. [29]

    Shading annotations in the wild

    Balazs Kovacs, Sean Bell, Noah Snavely, and Kavita Bala. Shading annotations in the wild. InCVPR, 2017. 2

    2017

  22. [30]

    Deep- light: Learning illumination for unconstrained mobile mixed reality

    Chloe LeGendre, Wan-Chun Ma, Graham Fyffe, John Flynn, Laurent Charbonnel, Jay Busch, and Paul Debevec. Deep- light: Learning illumination for unconstrained mobile mixed reality. InCVPR, pages 5918–5928, 2019. 2

    2019

  23. [31]

    In- verse rendering of translucent objects using physical and neural renderers

    Chenhao Li, Trung Thanh Ngo, and Hajime Nagahara. In- verse rendering of translucent objects using physical and neural renderers. InCVPR, pages 12510–12520, 2023. 2

    2023

  24. [32]

    Inverse ren- dering for complex indoor scenes: Shape, spatially-varying lighting and svbrdf from a single image

    Zhengqin Li, Mohammad Shafiei, Ravi Ramamoorthi, Kalyan Sunkavalli, and Manmohan Chandraker. Inverse ren- dering for complex indoor scenes: Shape, spatially-varying lighting and svbrdf from a single image. InCVPR, pages 2475–2484, 2020. 1, 2, 4, 5, 6

    2020

  25. [33]

    Openrooms: An end-to-end open framework for photorealistic indoor scene datasets.arXiv preprint arXiv:2007.12868, 2020

    Zhengqin Li, Ting-Wei Yu, Shen Sang, Sarah Wang, Meng Song, Yuhan Liu, Yu-Ying Yeh, Rui Zhu, Nitesh Gun- davarapu, Jia Shi, et al. Openrooms: An end-to-end open framework for photorealistic indoor scene datasets.arXiv preprint arXiv:2007.12868, 2020. 2

    2007 arXiv

  26. [34]

    Physically-based editing of indoor scene lighting from a single image

    Zhengqin Li, Jia Shi, Sai Bi, Rui Zhu, Kalyan Sunkavalli, Miloˇs Haˇsan, Zexiang Xu, Ravi Ramamoorthi, and Manmo- han Chandraker. Physically-based editing of indoor scene lighting from a single image. InECCV, pages 555–572. Springer, 2022. 2

    2022

  27. [35]

    Phyir: Physics-based inverse rendering for panoramic indoor images

    Zhen Li, Lingli Wang, Xiang Huang, Cihui Pan, and Jiaqi Yang. Phyir: Physics-based inverse rendering for panoramic indoor images. InCVPR, pages 12713–12723, 2022. 2, 6

    2022

  28. [36]

    Multi-view inverse rendering for large-scale real- world indoor scenes

    Zhen Li, Lingli Wang, Mofang Cheng, Cihui Pan, and Ji- aqi Yang. Multi-view inverse rendering for large-scale real- world indoor scenes. InCVPR, 2023. 2

    2023

  29. [37]

    Photorealistic object insertion with diffusion-guided inverse rendering.arXiv preprint arXiv:2408.09702, 2024

    Ruofan Liang, Zan Gojcic, Merlin Nimier-David, David Acuna, Nandita Vijaykumar, Sanja Fidler, and Zian Wang. Photorealistic object insertion with diffusion-guided inverse rendering.arXiv preprint arXiv:2408.09702, 2024. 1

    2024 arXiv

  30. [38]

    Flow matching for generative mod- eling.arXiv preprint arXiv:2210.02747, 2022

    Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximil- ian Nickel, and Matt Le. Flow matching for generative mod- eling.arXiv preprint arXiv:2210.02747, 2022. 3

    2022 arXiv

  31. [39]

    Flow straight and fast: Learning to generate and transfer data with rectified flow.arXiv preprint arXiv:2209.03003, 2022

    Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow.arXiv preprint arXiv:2209.03003, 2022. 3

    2022 arXiv

  32. [40]

    Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017

    I Loshchilov. Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017. 5

    2017 arXiv

  33. [41]

    Intrinsicdiffusion: Joint in- trinsic layers from latent diffusion models

    Jundan Luo, Duygu Ceylan, Jae Shin Yoon, Nanxuan Zhao, Julien Philip, Anna Fr ¨uhst¨uck, Wenbin Li, Christian Richardt, and Tuanfeng Wang. Intrinsicdiffusion: Joint in- trinsic layers from latent diffusion models. InACM SIG- GRAPH 2024 Conference Papers, 2024. 2, 3, 6

    2024

  34. [42]

    Fine-tuning image-conditional diffusion models is easier than you think

    Gonzalo Martin Garcia, Karim Abou Zeid, Christian Schmidt, Daan de Geus, Alexander Hermans, and Bastian Leibe. Fine-tuning image-conditional diffusion models is easier than you think. InWACV, 2025. 2

    2025

  35. [43]

    Extracting triangular 3d models, materials, and lighting from images

    Jacob Munkberg, Jon Hasselgren, Tianchang Shen, Jun Gao, Wenzheng Chen, Alex Evans, Thomas M¨uller, and Sanja Fi- dler. Extracting triangular 3d models, materials, and lighting from images. InCVPR, pages 8280–8290, 2022. 2

    2022

  36. [44]

    Inverse neural rendering for explainable multi-object tracking.arXiv preprint arXiv:2404.12359, 2024

    Julian Ost, Tanushree Banerjee, Mario Bijelic, and Felix Heide. Inverse neural rendering for explainable multi-object tracking.arXiv preprint arXiv:2404.12359, 2024. 1

    2024 arXiv

  37. [45]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. InICCV, pages 4195–4205, 2023. 4

    2023

  38. [46]

    Dreamfusion: Text-to-3d using 2d diffusion.arXiv preprint arXiv:2209.14988, 2022

    Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Milden- hall. Dreamfusion: Text-to-3d using 2d diffusion.arXiv preprint arXiv:2209.14988, 2022. 5, 12

    2022 arXiv

  39. [47]

    Hypersim: A photorealistic syn- thetic dataset for holistic indoor scene understanding

    Mike Roberts, Jason Ramapuram, Anurag Ranjan, Atulit Kumar, Miguel Angel Bautista, Nathan Paczan, Russ Webb, and Joshua M Susskind. Hypersim: A photorealistic syn- thetic dataset for holistic indoor scene understanding. In ICCV, pages 10912–10922, 2021. 2

    2021

  40. [48]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. InCVPR, pages 10684– 10695, 2022. 4

    2022

  41. [49]

    Neural inverse rendering of an indoor scene from a single image

    Soumyadip Sengupta, Jinwei Gu, Kihwan Kim, Guilin Liu, David W Jacobs, and Jan Kautz. Neural inverse rendering of an indoor scene from a single image. InICCV, pages 8598– 8607, 2019. 2, 6

    2019

  42. [50]

    Indoor segmentation and support inference from rgbd images

    Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. Indoor segmentation and support inference from rgbd images. InECCV, pages 746–760. Springer, 2012. 5

    2012

  43. [51]

    Diode: A dense indoor and outdoor depth dataset.arXiv preprint arXiv:1908.00463, 2019

    Igor Vasiljevic, Nick Kolkin, Shanyi Zhang, Ruotian Luo, Haochen Wang, Falcon Z Dai, Andrea F Daniele, Moham- madreza Mostajabi, Steven Basart, Matthew R Walter, et al. Diode: A dense indoor and outdoor depth dataset.arXiv preprint arXiv:1908.00463, 2019. 2, 5

    1908 arXiv

  44. [52]

    Eclipse: Disam- biguating illumination and materials using unintended shad- ows

    Dor Verbin, Ben Mildenhall, Peter Hedman, Jonathan T Bar- ron, Todd Zickler, and Pratul P Srinivasan. Eclipse: Disam- biguating illumination and materials using unintended shad- ows. InCVPR, pages 77–86, 2024. 2

    2024

  45. [53]

    Learning indoor inverse rendering with 3d spatially-varying lighting

    Zian Wang, Jonah Philion, Sanja Fidler, and Jan Kautz. Learning indoor inverse rendering with 3d spatially-varying lighting. InICCV, pages 12538–12547, 2021. 2, 6

    2021

  46. [54]

    Depth anything: Unleashing the power of large-scale unlabeled data

    Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything: Unleashing the power of large-scale unlabeled data. InCVPR, 2024. 2

    2024

  47. [55]

    Depth any- thing v2.arXiv:2406.09414, 2024

    Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiao- gang Xu, Jiashi Feng, and Hengshuang Zhao. Depth any- thing v2.arXiv:2406.09414, 2024. 2, 7

    2024 arXiv

  48. [56]

    Stablenormal: Reducing diffusion variance for stable and sharp normal.ACM Transactions on Graphics (TOG), 2024

    Chongjie Ye, Lingteng Qiu, Xiaodong Gu, Qi Zuo, Yushuang Wu, Zilong Dong, Liefeng Bo, Yuliang Xiu, and Xiaoguang Han. Stablenormal: Reducing diffusion variance for stable and sharp normal.ACM Transactions on Graphics (TOG), 2024. 3

    2024

  49. [57]

    Photoscene: Photorealistic mate- rial and lighting transfer for indoor scenes

    Yu-Ying Yeh, Zhengqin Li, Yannick Hold-Geoffroy, Rui Zhu, Zexiang Xu, Milo ˇs Ha ˇsan, Kalyan Sunkavalli, and Manmohan Chandraker. Photoscene: Photorealistic mate- rial and lighting transfer for indoor scenes. InCVPR, pages 18562–18571, 2022. 1

    2022

  50. [58]

    RGB↔X: Image decomposition and synthesis using material-and lighting-aware diffusion models

    Zheng Zeng, Valentin Deschaintre, Iliyan Georgiev, Yannick Hold-Geoffroy, Yiwei Hu, Fujun Luan, Ling-Qi Yan, and Miloˇs Haˇsan. RGB↔X: Image decomposition and synthesis using material-and lighting-aware diffusion models. InACM SIGGRAPH 2024 Conference Papers, pages 1–11, 2024....

    2024

  51. [59]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In ICCV, pages 3836–3847, 2023. 4

    2023

  52. [60]

    Unsupervised intrinsic image decompo- sition using internal self-similarity cues.IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(12):9669– 9686, 2021

    Qing Zhang, Jin Zhou, Lei Zhu, Wei Sun, Chunxia Xiao, and Wei-Shi Zheng. Unsupervised intrinsic image decompo- sition using internal self-similarity cues.IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(12):9669– 9686, 2021. 2

    2021

  53. [61]

    Learning-based inverse rendering of complex indoor scenes with differentiable monte carlo raytracing

    Jingsen Zhu, Fujun Luan, Yuchi Huo, Zihao Lin, Zhihua Zhong, Dianbing Xi, Rui Wang, Hujun Bao, Jiaxiang Zheng, and Rui Tang. Learning-based inverse rendering of complex indoor scenes with differentiable monte carlo raytracing. In SIGGRAPH Asia 2022 Conference Papers, pages 1–8...

    2022

  54. [62]

    Irisformer: Dense vision transform- ers for single-image inverse rendering in indoor scenes

    Rui Zhu, Zhengqin Li, Janarbek Matai, Fatih Porikli, and Manmohan Chandraker. Irisformer: Dense vision transform- ers for single-image inverse rendering in indoor scenes. In CVPR, pages 2822–2831, 2022. 2, 6 A. Details of Generative Renderer Our generative renderer aims to tak...

    2022

  55. [63]

    Figure 14.More qualitative comparison of metallic estimation on the synthetic InteriorVerse dataset [61]

    More qualitative comparison of roughness estimation on the synthetic InteriorVerse dataset [61]. Figure 14.More qualitative comparison of metallic estimation on the synthetic InteriorVerse dataset [61]. Input IndoorIR [61] RGBX [58] Ours GT Figure 17. More qualitative comparis...

  56. [64]

    Figure 15.More qualitative comparison of roughness estimation on the synthetic InteriorVerse dataset [61]

    More qualitative comparison of roughness estimation on the synthetic InteriorVerse dataset [61]. Figure 15.More qualitative comparison of roughness estimation on the synthetic InteriorVerse dataset [61]. Input InverseIndoor [32] IndoorIR [61] DSINE [1] StableNormal [56] RGBX [...

    2025

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.