DeferredGS: Decoupled and Editable Gaussian Splatting with Deferred Shading
read the original abstract
Reconstructing and editing 3D objects and scenes both play crucial roles in computer graphics and computer vision. Neural radiance fields (NeRFs) can achieve realistic reconstruction and editing results but suffer from inefficiency in rendering. Gaussian splatting significantly accelerates rendering by rasterizing Gaussian ellipsoids. However, Gaussian splatting utilizes a single Spherical Harmonic (SH) function to model both texture and lighting, limiting independent editing capabilities of these components. Recently, attempts have been made to decouple texture and lighting with the Gaussian splatting representation but may fail to produce plausible geometry and decomposition results on reflective scenes. Additionally, the forward shading technique they employ introduces noticeable blending artifacts during relighting, as the geometry attributes of Gaussians are optimized under the original illumination and may not be suitable for novel lighting conditions. To address these issues, we introduce DeferredGS, a method for decoupling and editing the Gaussian splatting representation using deferred shading. To achieve successful decoupling, we model the illumination with a learnable environment map and define additional attributes such as texture parameters and normal direction on Gaussians, where the normal is distilled from a jointly trained signed distance function. More importantly, we apply deferred shading, resulting in more realistic relighting effects compared to previous methods. Both qualitative and quantitative experiments demonstrate the superior performance of DeferredGS in novel view synthesis and editing tasks.
This paper has not been read by Pith yet.
Forward citations
Cited by 7 Pith papers
-
PTIR-GS: Path-Traced Inverse Rendering with Global Illumination in 3D Gaussian Fields
A splatting-free path-traced inverse rendering method for 3D Gaussian fields that defines a path-space interaction model enabling unbiased Monte-Carlo rendering and optimization under the full rendering equation with ...
-
PTIR-GS: Path-Traced Inverse Rendering with Global Illumination in 3D Gaussian Fields
A splatting-free path-traced inverse rendering framework for 3D Gaussian fields that supports optimization under the full rendering equation with ray-traced visibility and multi-bounce light transport via a path-space...
-
LumiMotion: Improving Gaussian Relighting with Scene Dynamics
LumiMotion improves albedo estimation and scene relighting in dynamic scenes by leveraging motion to separate lighting effects from surface appearance in a dynamic 2D Gaussian Splatting representation.
-
PTIR-GS: Path-Traced Inverse Rendering with Global Illumination in 3D Gaussian Fields
PTIR-GS develops a splatting-free path-traced inverse rendering method for 3D Gaussian fields to achieve consistent optimization with global illumination and multi-bounce light transport.
-
RT-Splatting: Joint Reflection-Transmission Modeling with Gaussian Splatting
RT-Splatting adds a disentangled occupancy-opacity factorization and specular-aware gradient gating to 3D Gaussian Splatting, enabling joint high-fidelity reflection and transmission in real-time novel view synthesis.
-
Ouroboros: Single-step Diffusion Models for Cycle-consistent Forward and Inverse Rendering
Ouroboros uses two single-step diffusion models with cycle consistency for forward and inverse rendering, extending intrinsic decomposition to indoor/outdoor scenes with faster inference than multi-step methods.
-
MaterialClusterGS: Palette-Based Material Decomposition and Physically-Based Relighting with 2D Gaussian Splatting
A palette-based framework decomposes 2D Gaussian Splatting scenes into shared BRDF prototypes via a spatial material field for coherent editing and relighting under physical rendering.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.