Pith. sign in

REVIEW 3 major objections 5 minor 60 references

Real-Time Global Illumination Decomposition of Videos

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims the first real-time method for decomposing a monocular color video into reflectance, direct illumination, and multiple indirect illumination layers, using sparse base-color priors.

desk verdict A genuinely useful real-time video intrinsics-plus-interreflection system, honestly limited by the palette-sparsity assumption that the paper itself openly concedes. read the letter →

arxiv 1908.01961 v2 pith:ZIJ2QEH7 submitted 2019-08-06 cs.CV

Abhimitra Meka , Mohammad Shafiei , Michael Zollhoefer , Christian Richardt , Christian Theobalt This is my paper · ORCID
classification cs.CV
keywords illuminationdecompositiondirectandindirectintrinsicvideoglobalsparse-denseoptimizationbasecolorestimationreal-timeeditinginter-reflectionrecoloring
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims to be the first to take an ordinary monocular color video and, in real time, split every frame into separate layers: the surface reflectance, the light arriving directly from light sources, and several indirect layers that capture color spill from one object onto another. The point of the split is editing: if you recolor an object, the method can also update the colored light it casts onto neighboring surfaces, keeping the edit globally consistent. Earlier methods required controlled multiplexed lighting or offline processing of a single image. The proposed method runs on regular videos at 30 Hz or faster after a short one-time initialization, and the paper reports better decomposition quality than prior intrinsic-video and illumination-decomposition approaches.

What carries the argument

The central object is the base-color palette {b_k} together with the per-pixel illumination layers T_k; the identity carrying the argument is the linear transport model I(x) = R(x) ⊙ (T_0(x) + ∑_{k=1}^{K} b_k T_k(x)), combined with ℓ_1 sparsity on indirect-layer activation and a soft-Retinex monochromaticity weight. The palette restricts the reflectance search space, the sparsity prior forces interreflections to be explained by few neighboring objects, and a sparse-dense splitting optimizer alternates Gauss-Newton steps on the large sparse layer variables with a small dense solve for base-color updates, enabling real-time GPU performance.

What would settle it

Render a synthetic video with known ground-truth reflectance, direct, and indirect layers in which one pixel receives indirect light from many differently colored surfaces at comparable strengths, violating the per-pixel sparsity assumption, and measure the error between recovered and ground-truth indirect layers; if the error is large, the decomposition fails exactly where its load-bearing sparsity premise is false.

Watch

Extended reading notes

Core claim

The central claim is that the appearance of every pixel can be written as the scene reflectance times one grayscale direct-illumination layer plus a sparse set of indirect layers, each scaled by one of a small set of jointly estimated base colors: I(x) = R(x) ⊙ ∑_{k=0}^{K} b_k T_k(x), with b_0 white for direct light. The paper shows that several sparsity priors—sparse per-pixel activation of indirect layers, sparse reflectance gradients, piecewise-smooth illumination, and non-negativity of light transport—resolve the otherwise ill-posed inverse problem, and that the base colors can be initialized by chromaticity clustering and refined by a small dense solve. The result is a temporally coherent decomposition of a video into interpretable layers that supports real-time inter-reflection-consistent recoloring, color-spill suppression, and color keying.

Load-bearing premise

Everything rests on the assumption that each pixel's appearance is well explained by its reflectance times a sparse combination of a small fixed set of base colors, so scenes whose interreflections are not sparse in that palette, or where new colors appear mid-video, cannot be decomposed correctly.

Editorial extensions

If this is right

  • Ordinary monocular video can be decomposed into reflectance, direct, and indirect illumination layers at real-time frame rates after a one-time initialization of about three seconds.
  • Recoloring an object by changing its base color also updates the colored light it casts onto other surfaces, making inter-reflection-consistent recoloring of live video possible.
  • Color-spill suppression and green-screen keying can be performed by removing or modifying specific indirect illumination layers rather than by global color filtering.
  • The spatiotemporal reflectance consistency prior keeps the decomposition temporally stable for smooth camera motion, avoiding per-frame flicker in most sequences.
  • The method extends intrinsic image decomposition from two layers (reflectance and shading) to a physically more interpretable set of direct and indirect light-transport layers.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension would replace the hand-designed clustering and occasional user clicks with a learned predictor trained on rendered interreflection data, which could handle highly textured scenes with many base colors.
  • The fixed palette assumption implies a testable boundary: a video in which an object of a new color enters mid-sequence will fail to model that object's interreflections, suggesting an online palette-update extension.
  • Because the model stops at the first bounce, higher-order interreflections would likely be folded into the direct layer or the reflectance; scenes with strong secondary bounces could reveal this error signature.
  • The sparse-dense optimization strategy may transfer to other inverse problems with a few global parameters coupled to many per-pixel unknowns, such as joint depth and albedo estimation.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper presents a variational method for decomposing a monocular RGB video into a reflectance layer, a grayscale direct-illumination layer, and K colored indirect-illumination layers associated with a set of base colors. The decomposition is driven by the model I = R ⊙ Σ b_k T_k, where the base colors are initialized from a chromaticity clustering of the first frame, optionally corrected by user clicks, and refined in a low-dimensional subproblem. The optimization alternates between a sparse high-dimensional Gauss-Newton solve for the per-pixel layers and a dense solve for the base-color updates, implemented on the GPU. The paper reports roughly 14 ms per 640×512 frame after a one-time initialization, shows an ablation study on one synthetic ground-truth sequence, and demonstrates qualitative comparisons and editing applications such as inter-reflection-consistent recoloring and color-spill suppression.

Significance. If the decomposition is correct, the paper provides a meaningful advance: it is, to my knowledge, the first real-time method to separate direct and indirect illumination in monocular video, and the runtime claim is concrete and plausible given the sparse-dense solver design. The quantitative evaluation uses an external Cycles-rendered ground-truth scene (Fig. 5) rather than an objective that embeds the method's own output, and the ablation study isolates the contributions of base-color refinement, the soft-Retinex monochromaticity term, and the spatiotemporal reflectance prior. The central risk is that the palette-sparsity model in Eq. (2) is validated on only one synthetic scene, while the paper's own limitations (Section 9) show that the decomposition is known to fail on textured, many-color scenes and on scenes with entering objects; this gap between the claim and the evidence is the main reason the paper needs revision.

major comments (3)
  1. [Section 8.1, Fig. 5] The quantitative support for the central quality claim rests on a single synthetic sequence (SyntheticRoom). The paper states an average LMSE of 0.002 but does not specify whether the metric is computed on the reflectance layer, the illumination layers, or the reconstructed image, and no per-layer errors are reported. Because the abstract and Section 8 claim improvements over Bonneel et al., Meka et al., and Carroll et al. in quality, the evaluation should include multiple ground-truth sequences with varied palette complexity and texture density, and report per-layer metrics; otherwise the qualitative comparisons on real scenes cannot support the claimed general superiority.
  2. [Section 4, Eq. (2); Section 9] The entire inverse problem is resolved by assuming that every pixel's radiance is a product of reflectance and a sparse linear combination of a small fixed palette of base colors (Eq. 2), with the L1 sparsity prior in Eq. (11) selecting the active indirect layers. Section 9 concedes that textured scenes require many base colors and become 'even more under-constrained', and Fig. 22 shows an incorrect reflectance/illumination split on the Cart sequence when the palette is large. Since this assumption is load-bearing for the decomposition, the paper needs to either narrow its claim (e.g., to scenes with a sparse color palette) or provide a quantitative characterization of the failure boundary; as written, the abstract's claim about 'regular videos' is broader than the evidence.
  3. [Section 4, Section 9] The method fixes base colors after the first frame and assumes no new objects or materials enter the video. Section 9 explicitly states that an object with an unseen color cannot be modeled once the palette is exhausted, and that the one-time initial clustering may miss objects appearing later. This restriction is not reflected in the title or abstract; the claimed 'real-time decomposition of videos' should be qualified, and the paper should state how the one-time initialization depends on the first frame being representative of the whole sequence.
minor comments (5)
  1. [Section 6.3, Eq. (14)] The chromaticity operator C(·) is used in the regularizer but is not explicitly defined in Section 6.3; please define it or refer back to Section 5.1.
  2. [Section 8.2] The WHDR value of 27.2% on Intrinsic Images in the Wild is reported without a corresponding comparison table or details on how the baseline WHDR numbers were obtained; please add the comparison or refer to a supplement.
  3. [Section 8.1] The legend of Fig. 5 reports errors for variants with different energy terms removed, but the text does not state the number of Gauss-Newton iterations or the convergence criterion used; please report the optimization settings.
  4. [Section 9] The reference to the Box2 sequence 'in the supplementary video (at 03:16)' is an awkward and brittle pointer; please state the limitation in the text itself.
  5. [Section 8, Runtime Performance] The one-time initialization time (2 s for base color refinement and 1 s for misclustering correction) should be mentioned where the 'real-time' claim is first made, so that readers understand the claim applies to steady-state decomposition after initialization.

Circularity Check

1 steps flagged · score 2.0 of 10

Minor self-consistency in the sparsity-based misclustering correction; the overall decomposition claim is independently validated against external ground truth.

  1. self definitional [Section 5.2.2, 'Reflectance Correction'; Eq. (11) in Section 6.2]
    "For each base color, we measure the sparsity obtained over the region using the illumination sparsity term to be introduced in Equation 11. The base color that provides the sparsest solution of the decomposition is then used as the corrected reflectance."

    The 'correct' reflectance is selected as the base color that minimizes the same sparsity objective E_i-sparsity (Eq. 11) that the final decomposition energy imposes on the indirect layers. Since the full energy (Eqs. 3-13) already penalizes non-sparse T_k, the selection rule ensures by construction that the corrected decomposition is sparser in the chosen region; the later claim that sparsity yields 'sparser and more realistic indirect illumination layers' (Fig. 15) is therefore partly an artifact of the selection criterion. The loop is not the sole basis of the paper's claims, because the method is additionally evaluated against external Cycles-rendered ground truth (Fig. 5) and the Cornell box (Fig. 16), so the overall finding is only minor self-consistency, not a forced prediction.

full rationale

The central factorization I(x)=R(x)⊙Σ_{k=0}^K b_k T_k(x) (Eq. 2) is a variational model, not a prediction smuggled from its inputs: the data term (Eq. 4) is balanced by the priors, and the output is judged against ground-truth reflectance and illumination from Cycles-rendered SyntheticRoom (Fig. 5, average LMSE 0.002) and the Cornell box (Fig. 16). Base-color refinement (Eq. 14) is a fit to the input, but the paper does not present the resulting decomposition error as a forecast; the external GT is the evidence. The spatiotemporal reflectance consistency prior is imported from the authors' Meka et al. [42], but it is a regularizer with ablation support, not a uniqueness theorem, so the self-citation is not load-bearing. Section 9's concessions that textured scenes require many base colors and that unseen colors cannot be modeled are stated limitations of the palette-sparsity assumption, not evidence that the derivation is equivalent to its input. The only noteworthy self-consistency is the sparsity-based misclustering correction, which selects the base color that minimizes the same sparsity energy later used to produce the decomposition; this step is mildly circular in isolation but is not the basis of the central claim.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

No new physical entities are introduced. The indirect illumination layers and base color palette are output representations and optimization variables, not new particles, forces, or causal mechanisms. The central claim rests on six domain assumptions and six hand-set parameter groups; the strongest constraint is sparsity in a fixed base-color palette, whose failure modes the paper itself documents.

free parameters (6)
  • K: maximum number of base colors = 10 (upper bound, user-specified; merged to typically 4 to 7)
    Controls the dimension of the decomposition in Eq. 2; all results use 10.
  • Cluster merge chromaticity threshold = 0.2
    Clusters closer than this are merged in Section 5.1; chosen by hand.
  • Sparsity norm exponent p = 1
    Used in the reflectance gradient sparsity term in Eq. 7; all experiments set p = 1.
  • Energy weights = lambda_clustering=200, lambda_r-sparsity=20, lambda_i-sparsity=3, lambda_smoothness=3, lambda_non-neg=1000…
    Fixed set of hand-tuned weights listed in Section 8 under Parameters.
  • Soft-Retinex scale = 50
    Exponent in Eq. 10 controlling how sharply monochromaticity is weighted near chromaticity edges.
  • Non-negativity epsilon = 0.002
    Stabilizer in Eq. 17 for re-weighted least squares.
assumptions (6)
  • domain assumption Scene surfaces are Lambertian (diffuse) with RGB albedo
    Section 4 first bullet; specular highlights are acknowledged as a failure mode in Section 9.
  • domain assumption All light sources emit white light
    Section 4 second bullet; Section 9 admits a yellowish illuminant is incorrectly assigned to reflectance.
  • domain assumption Indirect illumination is dominated by the first inter-reflection bounce
    Section 4 third bullet; higher bounces are ignored.
  • domain assumption Camera motion is smooth with large overlap and no new objects or materials enter after the first frame
    Section 4 final bullets; Section 9 says entering objects break the method.
  • domain assumption A small set of K base colors plus sparse layer activations can represent global illumination (Eq. 2, Eq. 11)
    Core representation; failure on complex palettes is reported in Figure 22 and Section 9.
  • domain assumption The non-convex energy has acceptable local optima reachable by IRLS and Gauss-Newton
    No convergence guarantee is given for Eq. 18; the paper relies on empirical convergence.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Real-Time Global Illumination Decomposition of Videos." pith.science (2026). https://pith.science/paper/ZIJ2QEH7

@misc{pith2026190801961,
  author       = {Pith},
  title        = {Pith review of: Real-Time Global Illumination Decomposition of Videos},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZIJ2QEH7}},
  note         = {Machine review of arXiv:1908.01961}
}
read the original abstract

We propose the first approach for the decomposition of a monocular color video into direct and indirect illumination components in real time. We retrieve, in separate layers, the contribution made to the scene appearance by the scene reflectance, the light sources and the reflections from various coherent scene regions to one another. Existing techniques that invert global light transport require image capture under multiplexed controlled lighting, or only enable the decomposition of a single image at slow off-line frame rates. In contrast, our approach works for regular videos and produces temporally coherent decomposition layers at real-time frame rates. At the core of our approach are several sparsity priors that enable the estimation of the per-pixel direct and indirect illumination layers based on a small set of jointly estimated base reflectance colors. The resulting variational decomposition problem uses a new formulation based on sparse and dense sets of non-linear equations that we solve efficiently using a novel alternating data-parallel optimization strategy. We evaluate our approach qualitatively and quantitatively, and show improvements over the state of the art in this field, in both quality and runtime. In addition, we demonstrate various real-time appearance editing applications for videos with consistent illumination.

Figures

Figures reproduced from arXiv: 1908.01961 by the authors.

Figure 1
Figure 1. We propose the first approach for the real-time decomposition of a video into direct and indirect illumination components. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Given a monocular color video as input, our approach estimates the light transport decomposition at real-time frame rates. At [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Example of misclustering correction (Section 5.2). The green color spill of the background causes misclustered regions in the shadow of the box and the towel (top right). We generate tracking masks (bottom left) using a few clicks for correcting the misclustered regions (bottom right). of the decomposition is then used as the corrected reflectance. The intuition behind such a sparsity prior is that using the correct… view at source ↗
Figures from the paper (19 more)
Figure 4
Figure 4. Figure 4: Here we show the improvement obtained by the base color refinement in our approach. We start from the clustered reflectance [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]
Figure 5
Figure 5. Figure 5: We quantitatively analyze our method on the [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: Our decomposition of the ChitChat sequence. We accurately decompose the color spill from the blue shirt and the red cup. Note that the reflectance is devoid of both color spills [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: Our decomposition of the Umbrella sequence. The complex color spill from the umbrella is mixed with the spill from the forest on the face and the jacket. Our method is able to decompose the colors accurately. Note that the reflectance is free from either of the two col…
Figure 8
Figure 8. Figure 8: Our decomposition of the Girl2 sequence. Even in challenging scenes, where the color palette is not well defined and thus clustering is difficult, our approach is able to estimate a plausible decomposition along with various indirect illumination layers. Note the stron…
Figure 9
Figure 9. Figure 9: Our decomposition of the Droid sequence. Note the clean reflectance map and clearly separated color casts in the indirect illumination layers. that the illumination layers instantly adapt to changes in the scene. This can best be seen in the supplemental video. Such a …
Figure 10
Figure 10. Figure 10: Our approach enables live recoloring of scene surfaces in a photo-realistic and globally consistent manner. Here, we recolor the [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]
Figure 11
Figure 11. Figure 11: Our illumination decomposition applied to two samples from the ‘Intrinsic Images in the Wild’ dataset [ [PITH_FULL_IMAGE:figures/full_fig_p017_11.png]
Figure 12
Figure 12. Figure 12: Results on the Box sequence, with and without our novel sparsity-based misclustering correction. Regions with strong inter-reflections (shadow of the box) are often misclustered in the reflectance image. This causes indirect illumination to wrongly influence the refle…
Figure 13
Figure 13. Figure 13: Comparison of our illumination decomposition to the approaches of Meka et al [PITH_FULL_IMAGE:figures/full_fig_p019_13.png]
Figure 14
Figure 14. Figure 14: Evaluation of the soft-color-Retinex weight of our monochromatic illumination term on the [PITH_FULL_IMAGE:figures/full_fig_p019_14.png]
Figure 15
Figure 15. Figure 15: Comparison of our illumination decomposition result on the [PITH_FULL_IMAGE:figures/full_fig_p020_15.png]
Figure 16
Figure 16. Figure 16: We compare our illumination decomposition qualitatively to the ground truth on the synthetic [PITH_FULL_IMAGE:figures/full_fig_p020_16.png]
Figure 17
Figure 17. Figure 17: Comparison to Carroll et al. [16] on the Paper sequence. Note that their illumination image retains a lot of color in the colored paper regions, which results in a direct illumination layer that is not uniform across the table and the paper. Our base colors ensure tha…
Figure 18
Figure 18. Figure 18: Ablation study and qualitative comparisons on the [PITH_FULL_IMAGE:figures/full_fig_p022_18.png]
Figure 19
Figure 19. Figure 19: Comparison of recoloring results to Meka et al [PITH_FULL_IMAGE:figures/full_fig_p022_19.png]
Figure 20
Figure 20. Figure 20: Recoloring result on the Cup sequence. Apart from the prominent color spills on the table cloth, even subtle inter-reflections on the green pot and the glass are captured well by our approach. 8.4 Interactive Live Applications We demonstrate several live video applica…
Figure 21
Figure 21. Figure 21: Comparison to recoloring software. The commercial software cannot remove the color spill for a particular object, such as [PITH_FULL_IMAGE:figures/full_fig_p024_21.png]
Figure 22
Figure 22. Figure 22: Comparison to state-of-the-art techniques on the [PITH_FULL_IMAGE:figures/full_fig_p025_22.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

60 extracted references · 31 canonical work pages

  1. [1]

    Yağız Aksoy, Tunç Ozan Aydın, Marc Pollefeys, and Aljoša Smolić. 2016. Interactive high-quality green-screen keying via color unmixing. ACM Trans. Graph. 35, 5 (August 2016), 152:1–12. https://doi.org/10.1145/2907940

    work page doi:10.1145/2907940 2016

  2. [2]

    Yağız Aksoy, Tunç Ozan Aydın, Aljoša Smolić, and Marc Pollefeys. 2017. Unmixing-Based Soft Color Segmentation for Image Manipulation. ACM Trans. Graph. 36, 2 (March 2017), 19:1–19. https://doi.org/10.1145/3002176

    work page doi:10.1145/3002176 2017

  3. [3]

    Anna Alperovich and Bastian Goldluecke. 2017. A Variational Model for Intrinsic Light Field Decomposition. In ACCV. 66–82. https://doi.org/10. 1007/978-3-319-54187-7_5

    work page 2017

  4. [4]

    Dejan Azinović, Tzu-Mao Li, Anton Kaplanyan, and Matthias Nießner. 2019. Inverse Path Tracing for Joint Material and Lighting Estimation. In CVPR. 2442–2451. https://doi.org/10.1109/CVPR.2019.00255

    arXiv 2019

  5. [5]

    Francis Bach, Rodolphe Jenatton, Julien Mairal, and Guillaume Obozinski. 2012. Optimization with Sparsity-Inducing Penalties. Foundations and Trends in Machine Learning 4, 1 (2012), 1–106. https://doi.org/10.1561/2200000015

    doi:10.1561/2200000015 2012

  6. [6]

    Barron and Jitendra Malik

    Jonathan T. Barron and Jitendra Malik. 2015. Shape, Illumination, and Reflectance from Shading. IEEE Trans. Pattern Anal. Mach. Intell. 37, 8 (August 2015), 1670–1687. https://doi.org/10.1109/TPAMI.2014.2377712

    arXiv 2015

  7. [7]

    Barrow and Jay M

    Harry G. Barrow and Jay M. Tenenbaum. 1978. Recovering intrinsic scene characteristics from images. In Computer Vision Systems. 3–26

    work page 1978

  8. [8]

    Shida Beigpour and Joost van de Weijer. 2011. Object recoloring based on intrinsic image estimation. In ICCV. 327–334. https://doi.org/10.1109/ ICCV.2011.6126259 26 Real-Time Global Illumination Decomposition of Videos

    arXiv 2011

Show all 60 references
  1. [9]

    Sean Bell, Kavita Bala, and Noah Snavely. 2014. Intrinsic Images in the Wild. ACM Trans. Graph. 33, 4 (July 2014), 159:1–12. https://doi.org/10.1145/ 2601097.2601206

    2014

  2. [10]

    Sai Bi, Xiaoguang Han, and Yizhou Yu. 2015. An𝐿1 Image Transform for Edge-Preserving Smoothing and Scene-Level Intrinsic Decomposition. ACM Trans. Graph. 34, 4 (July 2015), 78:1–12. https://doi.org/10.1145/2766946

    2015 doi

  3. [11]

    Sai Bi, Nima Khademi Kalantari, and Ravi Ramamoorthi. 2018. Deep Hybrid Real and Synthetic Training for Intrinsic Decomposition. InEurographics Symposium on Rendering – Experimental Ideas & Implementations

    2018

  4. [12]

    Nicolas Bonneel, Balazs Kovacs, Sylvain Paris, and Kavita Bala. 2017. Intrinsic Decompositions for Image Editing. Comput. Graph. Forum 36, 2 (May 2017), 593–609. https://doi.org/10.1111/cgf.13149

    2017 doi

  5. [14]

    Nicolas Bonneel, James Tompkin, Kalyan Sunkavalli, Deqing Sun, Sylvain Paris, and Hanspeter Pfister. 2015. Blind Video Temporal Consistency. ACM Trans. Graph. 34, 6 (November 2015), 196:1–9. https://doi.org/10.1145/2816795.2818107

    2015

  6. [15]

    Adrien Bousseau, Sylvain Paris, and Frédo Durand. 2009. User-Assisted Intrinsic Images. ACM Trans. Graph. 28, 5 (December 2009), 130:1–10. https://doi.org/10.1145/1618452.1618476

    2009

  7. [16]

    Robert Carroll, Ravi Ramamoorthi, and Maneesh Agrawala. 2011. Illumination decomposition for material recoloring with consistent interreflections. ACM Trans. Graph. 30, 4 (July 2011), 43:1–10. https://doi.org/10.1145/2010324.1964938

    2011

  8. [17]

    Fisher, III

    Jason Chang, Randi Cabezas, and John W. Fisher, III. 2014. Bayesian Nonparametric Intrinsic Image Decomposition. In ECCV, Vol. 8692. 704–719. https://doi.org/10.1007/978-3-319-10593-2_46

    2014 doi

  9. [18]

    Zachary DeVito, Michael Mara, Michael Zollhöfer, Gilbert Bernstein, Jonathan Ragan-Kelley, Christian Theobalt, Pat Hanrahan, Matthew Fisher, and Matthias Nießner. 2017. Opt: A Domain Specific Language for Non-Linear Least Squares Optimization in Graphics and Imaging. ACM Trans...

    2017 doi

  10. [19]

    Shouhong Ding, Bin Sheng, Xiaonan Hou, Zhifeng Xie, and Lizhuang Ma. 2017. Intrinsic Image Decomposition Using Multi-Scale Measurements and Sparsity. Comput. Graph. Forum 36, 6 (2017), 251–261. https://doi.org/10.1111/cgf.12874

    2017 doi

  11. [20]

    Bo Dong, Yue Dong, Xin Tong, and Pieter Peers. 2015. Measurement-based Editing of Diffuse Albedo with Consistent Interreflections. ACM Trans. Graph. 34, 4 (July 2015), 112:1–11. https://doi.org/10.1145/2766979

    2015 doi

  12. [22]

    Jean-Dominique Favreau, Florent Lafarge, and Adrien Bousseau. 2017. Photo2ClipArt: Image Abstraction and Vectorization Using Layered Linear Gradients. ACM Trans. Graph. 36, 6 (November 2017), 180:1–11. https://doi.org/10.1145/3130800.3130888

    2017

  13. [23]

    Echevarria, Wen Zhang, Hongzhi Wu, Kun Zhou, and Diego Gutierrez

    Elena Garces, Jose I. Echevarria, Wen Zhang, Hongzhi Wu, Kun Zhou, and Diego Gutierrez. 2017. Intrinsic Light Field Images. Comput. Graph. Forum 36, 8 (December 2017), 589–599. https://doi.org/10.1111/cgf.13154

    2017 doi

  14. [24]

    Elena Garces, Adolfo Muñoz, Jorge Lopez-Moreno, and Diego Gutierrez. 2012. Intrinsic Images by Clustering. Comput. Graph. Forum 31, 4 (2012), 1415–1424. https://doi.org/10.1111/j.1467-8659.2012.03137.x

    2012

  15. [25]

    Stamatios Georgoulis, Konstantinos Rematas, Tobias Ritschel, Efstratios Gavves, Mario Fritz, Luc Van Gool, and Tinne Tuytelaars. 2018. Reflectance and Natural Illumination from Single-Material Specular Objects Using Deep Learning. IEEE Trans. Pattern Anal. Mach. Intell. 40, 8 ...

    2018

  16. [26]

    Johnson, Edward H

    Roger Grosse, Micah K. Johnson, Edward H. Adelson, and William T. Freeman. 2009. Ground truth dataset and baseline evaluations for intrinsic image algorithms. In ICCV. 2335–2342. https://doi.org/10.1109/ICCV.2009.5459428

    2009

  17. [27]

    Kaiwen Guo, Feng Xu, Tao Yu, Xiaoyang Liu, Qionghai Dai, and Yebin Liu. 2017. Real-time Geometry, Albedo and Motion Reconstruction Using a Single RGBD Camera. ACM Trans. Graph. 36, 3 (June 2017), 32:1–13. https://doi.org/10.1145/3083722

    2017 doi

  18. [28]

    Holland and Roy E

    Paul W. Holland and Roy E. Welsch. 1977. Robust regression using iteratively reweighted least-squares. Communications in Statistics – Theory and Methods 6, 9 (September 1977), 813–827. https://doi.org/10.1080/03610927708827533

    1977 doi

  19. [29]

    Carlo Innamorati, Tobias Ritschel, Tim Weyrich, and Niloy J. Mitra. 2017. Decomposing Single Images for Layered Photo Retouching. Comput. Graph. Forum 36, 4 (July 2017), 15–25. https://doi.org/10.1111/cgf.13220

    2017 doi

  20. [30]

    Kulkarni, Ilker Yildirim, and Joshua B

    Michael Janner, Jiajun Wu, Tejas D. Kulkarni, Ilker Yildirim, and Joshua B. Tenenbaum. 2017. Self-Supervised Intrinsic Image Decomposition. In NIPS. http://rin.csail.mit.edu/

    2017

  21. [31]

    James T. Kajiya. 1986. The Rendering Equation. Computer Graphics (Proceedings of SIGGRAPH) 20, 4 (August 1986), 143–150. https://doi.org/10. 1145/15886.15902

    1986

  22. [32]

    Seungryong Kim, Kihong Park, Kwanghoon Sohn, and Stephen Lin. 2016. Unified Depth Prediction and Intrinsic Image Decomposition from a Single Image via Joint Convolutional Neural Fields. In ECCV. 143–159. https://doi.org/10.1007/978-3-319-46484-8_9

    2016 doi

  23. [33]

    Gehler, and Michael J

    Naejin Kong, Peter V. Gehler, and Michael J. Black. 2014. Intrinsic Video. In ECCV. 360–375. https://doi.org/10.1007/978-3-319-10605-2_24

    2014 doi

  24. [34]

    Balazs Kovacs, Sean Bell, Noah Snavely, and Kavita Bala. 2017. Shading Annotations in the Wild. In CVPR. 850–859. https://doi.org/10.1109/CVPR. 2017.97

    2017 doi

  25. [35]

    Pierre-Yves Laffont, Adrien Bousseau, Sylvain Paris, Frédo Durand, and George Drettakis. 2012. Coherent Intrinsic Images from Photo Collections. ACM Trans. Graph. 31, 6 (November 2012), 202:1–11. https://doi.org/10.1145/2366145.2366221 27 Meka, A., Shafiei, M., et al

    2012

  26. [36]

    Xiao Li, Yue Dong, Pieter Peers, and Xin Tong. 2017. Modeling Surface Appearance from a Single Photograph Using Self-augmented Convolutional Neural Networks. ACM Trans. Graph. 36, 4 (July 2017), 45:1–11. https://doi.org/10.1145/3072959.3073641

    2017

  27. [38]

    Sharon Lin, Matthew Fisher, Angela Dai, and Pat Hanrahan. 2017. LayerBuilder: Layer Decomposition for Interactive Image and Video Color Editing. (2017). arXiv:1701.03754

    2017 arXiv

  28. [39]

    Guilin Liu, Duygu Ceylan, Ersin Yumer, Jimei Yang, and Jyh-Ming Lien. 2017. Material Editing Using a Physically Based Rendering Network. In ICCV. 2280–2288. https://doi.org/10.1109/ICCV.2017.248

    2017 doi

  29. [40]

    Stephen Lombardi and Ko Nishino. 2016. Reflectance and Illumination Recovery in the Wild. IEEE Trans. Pattern Anal. Mach. Intell. 38, 1 (January 2016), 129–141. https://doi.org/10.1109/TPAMI.2015.2430318

    2016

  30. [41]

    Marschner and Donald P

    Stephen R. Marschner and Donald P. Greenberg. 1997. Inverse lighting for photography. In Proceedings of the IS&T Color Imaging Conference . 262–265

    1997

  31. [42]

    Abhimitra Meka, Michael Zollhöfer, Christian Richardt, and Christian Theobalt. 2016. Live Intrinsic Video. ACM Trans. Graph. 35, 4 (July 2016), 109:1–14. https://doi.org/10.1145/2897824.2925907

    2016

  32. [44]

    Takuya Narihira, Michael Maire, and Stella X. Yu. 2015. Direct Intrinsics: Learning Albedo-Shading Decomposition by Convolutional Regression. In ICCV. https://doi.org/10.1109/ICCV.2015.342

    2015 doi

  33. [45]

    Nayar, Gurunandan Krishnan, Michael D

    Shree K. Nayar, Gurunandan Krishnan, Michael D. Grossberg, and Ramesh Raskar. 2006. Fast separation of direct and global components of a scene using high frequency illumination. ACM Trans. Graph. 25, 3 (July 2006), 935–944. https://doi.org/10.1145/1141911.1141977

    2006 arXiv

  34. [46]

    Thomas Nestmeyer and Peter V. Gehler. 2017. Reflectance Adaptive Filtering Improves Intrinsic Image Estimation. In CVPR. 1771–1780. https: //doi.org/10.1109/CVPR.2017.192

    2017 doi

  35. [47]

    Kutulakos

    Matthew O’Toole, John Mather, and Kiriakos N. Kutulakos. 2016. 3D Shape and Indirect Appearance by Structured Light Transport. IEEE Trans. Pattern Anal. Mach. Intell. 38, 7 (July 2016), 1298–1312. https://doi.org/10.1109/TPAMI.2016.2545662

    2016

  36. [48]

    Gustavo Patow and Xavier Pueyo. 2003. A Survey of Inverse Rendering Problems. Comput. Graph. Forum 22, 4 (2003), 663–687. https://doi.org/10. 1111/j.1467-8659.2003.00716.x

    2003

  37. [49]

    Ravi Ramamoorthi and Pat Hanrahan. 2001. A signal-processing framework for inverse rendering. In SIGGRAPH. 117–128. https://doi.org/10.1145/ 383259.383271

    2001

  38. [50]

    Peiran Ren, Yue Dong, Stephen Lin, Xin Tong, and Baining Guo. 2015. Image Based Relighting Using Neural Networks. ACM Trans. Graph. 34, 4 (July 2015), 111:1–12. https://doi.org/10.1145/2766899

    2015 doi

  39. [51]

    Christian Richardt, Jorge Lopez-Moreno, Adrien Bousseau, Maneesh Agrawala, and George Drettakis. 2014. Vectorising Bitmaps into Semi- Transparent Gradient Layers. Comput. Graph. Forum 33, 4 (June 2014), 11–19. https://doi.org/10.1111/cgf.12408

    2014 doi

  40. [52]

    Christopher Schultz and Thorsten Hermes. 2006. Digital Keying Methods. TZI-Bericht 40. Technologie-Zentrum Informatik, Bremen University. http://www.tzi.de/fileadmin/resources/publikationen/tzi_berichte/TZI-Bericht-Nr._40.pdf

    2006

  41. [53]

    Seitz, Yasuyuki Matsushita, and Kiriakos N

    Steven M. Seitz, Yasuyuki Matsushita, and Kiriakos N. Kutulakos. 2005. A theory of inverse light transport. In ICCV, Vol. 2. 1440–1447. https: //doi.org/10.1109/ICCV.2005.25

    2005 doi

  42. [54]

    Jianbing Shen, Xing Yan, Lin Chen, Hanqiu Sun, and Xuelong Li. 2014. Re-texturing by intrinsic video. Information Sciences 281 (October 2014), 726–735. https://doi.org/10.1016/j.ins.2014.02.134

    2014 doi

  43. [55]

    Jianbing Shen, Xiaoshan Yang, Yunde Jia, and Xuelong Li. 2011. Intrinsic images using optimization. In CVPR. 3481–3487. https://doi.org/10.1109/ CVPR.2011.5995507

    2011

  44. [56]

    Jian Shi, Yue Dong, Hao Su, and Stella X. Yu. 2017. Learning Non-Lambertian Object Intrinsics across ShapeNet Categories. In CVPR. 5844–5853. https://doi.org/10.1109/CVPR.2017.619

    2017 doi

  45. [57]

    Jianchao Tan, Jose Echevarria, and Yotam Gingold. 2018. Efficient Palette-based Decomposition and Recoloring of Images via RGBXY-space Geometry. ACM Trans. Graph. 37, 6 (November 2018), 262:1–10. https://doi.org/10.1145/3272127.3275054

    2018

  46. [58]

    Jianchao Tan, Jyh-Ming Lien, and Yotam Gingold. 2016. Decomposing Images into Layers via RGB-space Geometry. ACM Trans. Graph. 36, 1 (November 2016), 7:1–14. https://doi.org/10.1145/2988229

    2016 doi

  47. [59]

    Chenglei Wu, Michael Zollhöfer, Matthias Nießner, Marc Stamminger, Shahram Izadi, and Christian Theobalt. 2014. Real-time Shading-based Refinement for Consumer Depth Cameras. ACM Trans. Graph. 33, 6 (November 2014), 200:1–10. https://doi.org/10.1145/2661229.2661232

    2014

  48. [60]

    Hongzhi Wu, Zhaotian Wang, and Kun Zhou. 2016. Simultaneous Localization and Appearance Estimation with a Consumer RGB-D Camera. IEEE Trans. Vis. Comput. Graph. 22, 8 (August 2016), 2012–2023. https://doi.org/10.1109/TVCG.2015.2498617

    2016

  49. [62]

    Yizhou Yu, Paul Debevec, Jitendra Malik, and Tim Hawkins. 1999. Inverse global illumination: recovering reflectance models of real scenes from photographs. In SIGGRAPH. 215–224. https://doi.org/10.1145/311535.311559

    1999

  50. [63]

    Tinghui Zhou, Philipp Krähenbühl, and Alyosha Efros. 2015. Learning Data-driven Reflectance Priors for Intrinsic Image Decomposition. In ICCV. 3469–3477. https://doi.org/10.1109/ICCV.2015.396 28 Real-Time Global Illumination Decomposition of Videos

    2015 doi

  51. [64]

    Michael Zollhöfer, Matthias Nießner, Shahram Izadi, Christoph Rhemann, Christopher Zach, Matthew Fisher, Chenglei Wu, Andrew Fitzgibbon, Charles Loop, Christian Theobalt, and Marc Stamminger. 2014. Real-time Non-rigid Reconstruction Using an RGB-D Camera. ACM Trans. Graph. 33,...

    2014

  52. [65]

    Daniel Zoran, Phillip Isola, Dilip Krishnan, and William T. Freeman. 2015. Learning Ordinal Relationships for Mid-Level Vision. In ICCV. 388–396. https://doi.org/10.1109/ICCV.2015.52 29

    2015 doi

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.