Pith. sign in

REVIEW 4 major objections 4 minor 74 references

AuraFusion360: Augmented Unseen Region Alignment for Reference-based 360{\deg} Unbounded Scene Inpainting

T0 review · 4 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Reference-based 360° scene inpainting can be view-consistent if unseen regions are found by depth warping and diffusion depth is aligned only around them.

desk verdict A credible engineering integration with two real novelties and a useful new dataset, but the geometric-accuracy headline is not yet backed by direct evidence. read the letter →

arxiv 2502.05176 v3 pith:WW72IT4L submitted 2025-02-07 cs.CV

Chung-Ho Wu , Yang-Jung Chen , Ying-Huan Chen , Jie-Ying Lee , Bo-Hsu Ke , Chun-Wei Tuan Mu , Yi-Chuan Huang , Chin-Yang Lin
show 3 more authors
Min-Hung Chen Yen-Yu Lin Yu-Lun Liu
This is my paper · ORCID
classification cs.CV
keywords 360-degreesceneinpaintingGaussianSplattingobjectremovalreference-baseddepth-awareunseenmaskadaptiveguideddepthdiffusionSDEdit360-USIDdataset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Removing an object from a 360° scene leaves holes that no camera ever photographed; filling them convincingly from every viewpoint is what 3D scene inpainting has struggled to do. This paper argues that the decisive move is to find those unseen regions geometrically and to anchor the generative prior's hallucinated content to the scene's real depth. AuraFusion360 does this with three steps: depth-warped unseen masks, Adaptive Guided Depth Diffusion (AGDD), which aligns a monocular depth prior to the scene only in a band around the hole, and SDEdit-based refinement that uses structured noise from DDIM inversion to preserve the reference view. The authors introduce the 360-USID dataset with ground truth and report the best average PSNR (17.661) and LPIPS (0.388) among the compared methods. If the claim holds, 360° object removal becomes practical for VR, architectural visualization, and AR editing.

What carries the argument

Adaptive Guided Depth Diffusion (AGDD) is the central mechanism: a latent diffusion depth estimator whose denoising step reweights the predicted noise with the gradient of an adaptive loss, $L_{\text{adaptive}}$, computed between the pre-decoded depth $D_{t-1}$ and the rendered incomplete depth $D^{\text{incomplete}}$ on pixels in the band around the unseen region. A Huber-like threshold $\delta$ prevents far-away background pixels from dominating. The result is zero-shot aligned depth for the reference view, from which the inpainted reference RGB is unprojected into 3D to initialize Gaussians. Around this sit the depth-warping unseen-mask generator and the SDEdit stage, which replaces random noise by DDIM inversion so that the inpainted RGB keeps the reference view's structure.

What would settle it

Run AuraFusion360 on a scene with ground-truth depth inside the removed object's footprint (e.g., a LiDAR or structured-light scan taken after removal), compare the depth of the unprojected unseen-region Gaussians to ground truth, and render a novel view: if the mean absolute depth error there substantially exceeds the error in visible regions, the interior depth is hallucinated rather than aligned, producing floaters.

Watch

Extended reading notes

Core claim

The paper's central claim is that a reference-based 360° inpainting pipeline can achieve multi-view coherence and geometric accuracy if the unseen region is known precisely and if the depth used to unproject the reference image is aligned to the scene near that region. The method first renders the scene after object removal, warps the incomplete depth from every other view into each target view, thresholds the aggregated occlusions, and uses the contour as a SAM2 prompt to obtain an unseen mask. It then runs a latent diffusion depth estimator, but at each denoising step it nudges the predicted noise so that the decoded depth matches the rendered incomplete depth inside a band around the unseen region, with a thresholded loss that keeps distant pixels from dominating. This yields an aligned reference depth, and unprojected pixels from the inpainted reference view become new Gaussians. The rendered result is inverted with DDIM, denoised conditioned on the reference view, and used as supervision for fine-tuning only those new Gaussians. The paper reports top average PSNR and LPIPS on its own 360-USID dataset and shows visually sharper novel views than the compared baselines.

Load-bearing premise

The whole geometric claim rests on the depth that the diffusion prior produces inside the unseen region: AGDD only aligns depth in a band around the hole, never inside it, so if the interior depth is wrong the unprojected Gaussians land at incorrect 3D locations and render as floaters.

Editorial extensions

If this is right

  • A captured 360° scene can be edited by deleting an object and rerendering from any camera path, because the unseen masks and aligned depth are produced for every view rather than for a single forward-facing camera.
  • The depth prior works zero-shot: no per-scene fine-tuning of the depth network is needed, so the method transfers to new captures as long as camera poses and a reference image are available.
  • Novel views stay consistent because all inpainted content is first unprojected from a single reference view; later SDEdit refinement is anchored to that same reference.
  • The 360-USID benchmark provides the first quantitative, ground-truth comparison for reference-based 360° inpainting, so future work can be measured instead of shown qualitatively.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the band-alignment idea could be lifted into other depth-completion models; the test would be replacing global scale-shift alignment with AGDD's adaptive loss and measuring depth error.
  • Beyond the paper: structured-noise SDEdit (DDIM inversion instead of random noise) is a general recipe for keeping one reference view dominant in diffusion editing, potentially useful for video or multi-image harmonization.
  • Beyond the paper: depth-warped unseen masks could double as capture-planning signals, since they show exactly which parts of a scene remain occluded from all current cameras.
  • Beyond the paper: the paper's geometric claim is only as strong as the depth inside the unseen region, so a decisive follow-up is to compare AGDD's interior depth with ground-truth scans.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes AuraFusion360, a reference-based 3D inpainting method for 360-degree unbounded scenes represented by 2D Gaussian Splatting. The pipeline consists of three main stages: depth-aware unseen mask generation via depth warping with SAM2, Adaptive Guided Depth Diffusion (AGDD) to align a monocular depth estimate with the rendered incomplete depth and then unproject the inpainted reference image into the unseen region, and SDEdit-based detail enhancement with DDIM Inversion to produce multi-view consistent RGB guidance for Gaussian fine-tuning. The authors also introduce the 360-USID dataset with seven real scenes (180-200 training views and ~30 test views each) and report PSNR and LPIPS comparisons against several existing methods, claiming state-of-the-art performance with best average PSNR 17.661 and LPIPS 0.388. The paper includes ablations, qualitative comparisons, failure cases, and a supplementary limitations section.

Significance. If the reported results hold, AuraFusion360 would be a useful contribution to 360-degree unbounded scene inpainting, and 360-USID could serve as a benchmark for future work. The AGDD idea of aligning depth only in a band around the unseen region is a sensible way to reduce over-alignment, and the use of DDIM inversion to produce structured noise for SDEdit is a reasonable enhancement. The paper also provides an explicit limitation analysis and acknowledges failure cases in Fig. 14 and Sec. G. However, the evidence supporting the central claim of geometric accuracy is indirect, the evaluation is based on only seven scenes with no error bars or significance tests, and several baselines were adapted by the authors, which limits the strength of the current claims.

major comments (4)
  1. [Sec. 3.2, Eqs. (6)-(8); Sec. 5.1, Table 3] The geometric accuracy claim is load-bearing but not directly supported. The adaptive loss Ladaptive is applied only on M_guide = B \ U, explicitly excluding the unseen mask U; therefore the depth inside U, which is used to unproject the reference RGB into 3D, is never constrained by any scene geometry. The subsequent fine-tuning (Eq. 12) uses only photometric losses (L1, SSIM, LPIPS) and contains no depth term. Table 3 reports MAD of AGDD depth versus pseudo-GT depth 'in object mask areas' for test views, but this is not the same as measuring the final rendered depth inside unseen masks, and the pseudo-GT itself is obtained from a masked 2DGS trained on post-removal images rather than true ground-truth depth. I recommend adding a geometric evaluation of the final rendered depth inside unseen masks on held-out novel views, and optionally adding a depth-consistency term during fine-tuning with an ablation to show it is not needed.
  2. [Sec. 5.1, Table 1; Sec. 3.4] The quantitative evidence is thin for the strength of the claims. The benchmark contains only seven scenes, and the reported averages have no error bars, confidence intervals, or significance tests. The PSNR advantage over SPIn-NeRF is 0.93 dB on average, while per-scene PSNR spreads are much larger (e.g., Sunflower is about 12 dB higher than Cookie), so the average gap may not be significant. In addition, key hyperparameters (N=8, s=0.85, lambda_SSIM, lambda_LPIPS) are selected on the same dataset. I recommend reporting per-scene variance, significance tests, and a sensitivity analysis for the main hyperparameters to show the results are not overfit to 360-USID.
  3. [Sec. 5.2; Sec. F.4-F.6] The fairness of the baselines is not fully established. SPIn-NeRF is re-implemented by the authors on the 2DGS codebase with a LaMa-based inpainting substitution, and several baselines are given the same reference image and object masks as the proposed method. This could favor AuraFusion360 if the adapted baselines are not tuned to the same degree or if their official pipelines were more competitive. I recommend releasing code and checkpoints for all adapted baselines, or using official implementations with recommended settings, and clearly documenting exactly which inputs (reference image, masks, depth) each method received.
  4. [Sec. 4; Sec. 5.1] The dataset construction may bias the evaluation toward the proposed method. Test views are selected by training a masked-3DGS model and choosing frames with high PSNR outside object regions, which could filter out challenging viewpoints for reconstruction-based methods. Also, one test view is captured from the same tripod position as the reference image, which may favor reference-based methods in the average. Please report results separately for the reference-position view and for other novel views, and discuss how the frame-selection criterion affects the ranking of methods.
minor comments (4)
  1. [Sec. 3.2, Eq. (5); Sec. 3.3, Eq. (12)] There are notation inconsistencies: 'Ladpative' in Eq. (5) should be 'Ladaptive', and Eq. (12) uses 'LLPIPS' where the intended symbol is 'lambda_LPIPS' (and similarly 'LLPIPS = 0.5' in Sec. 3.4).
  2. [Sec. 4 vs. Sec. 5.1] The scene statistics are inconsistent: Section 4 describes five outdoor and two indoor scenes, while Section 5.1 states '3 indoor, 4 outdoor'. Please correct the counts.
  3. [Table 3; supplementary text] There are typos: 'InFuion' should be 'InFusion', and '360-UISD' in the supplementary should be '360-USID'.
  4. [Figure 2 and Figure 8 captions] The method name is spelled inconsistently as 'SPin-NeRF' in Figure 2 caption while the text and references use 'SPIn-NeRF'. Please unify the spelling.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the pipeline is self-contained and evaluated against held-out ground truth; the geometric-accuracy weakness is an unconstrained depth prior in unseen regions, not a tautological reduction.

full rationale

AuraFusion360's derivation chain is not circular. The unseen mask (Eqs. 1-3) is produced by depth warping plus SAM2; AGDD (Eqs. 4-8) aligns Marigold depth to the rendered incomplete depth only in the band B\U; and the unprojected Gaussians are fine-tuned with photometric losses (Eq. 12) against SDEdit/DDIM-refined guidance, with final numbers reported against held-out ground-truth novel views (Tab. 1). No step defines its output in terms of the quantity it is said to predict: the adaptive loss explicitly leaves the unseen region U unconstrained by Dincomplete, so the depth inside U comes from the diffusion prior rather than from a fitted copy of the input; this is an acknowledged limitation (Sec. G and Fig. 14) and a correctness risk for the geometric-accuracy claim, not a circular reduction. Tab. 3, which uses pseudo-GT depth from a masked 2DGS and measures object-mask areas rather than final unseen-region depth, is likewise weak evidence but not a tautology. The paper cites prior work with overlapping authorship (e.g., CorrFill [26] for reference-view selection and several related-work items), but none is load-bearing: no uniqueness theorem is invoked, and no central claim is justified solely by a self-citation. The SDEdit fine-tuning target is derived from the model's own initial render, making it a self-distillation signal, yet success is still measured against external GT, so the claim is not forced by construction. A minor benchmark concern is that the reference image is captured post-removal and a test view is captured from the same tripod position, which may favor reference-based methods, but this is an evaluation-design issue rather than circularity. Overall, the method is an engineered composition of external priors and per-scene optimization; there is no Eq. X = Eq. Y by construction and no fitted parameter renamed as a prediction.

Assumptions & free parameters 8 free parameters · 6 assumptions · 0 invented entities

The paper introduces no new physical or mathematical entities. The free parameters are standard hyperparameters set by hand or by a small ablation, plus the diffusion and segmentation priors it imports from prior work. The central load-bearing assumption is that AGDD's unconstrained depth inside unseen regions is geometrically correct.

free parameters (8)
  • Theta (theta) = 0.6
    Aggregation threshold for unseen mask contour, set in Sec. 3.1/3.4.
  • Tau (tau) = 0.6
    Gaussian masked-attribute threshold for object removal, set in Supplementary Sec. B.
  • N (AGDD iterations) = 8
    Number of noise update iterations per denoising step in AGDD, set in Sec. 3.4.
  • s (SDEdit strength) = 0.85
    Noise strength for DDIM-inversion-guided denoising; chosen from ablation in Table 2.
  • lambda_SSIM = 0.8
    Weight on SSIM loss during Gaussian fine-tuning, set in Sec. 3.4.
  • lambda_LPIPS = 0.5
    Weight on LPIPS loss during fine-tuning, set in Sec. 3.4.
  • Delta (delta) = not specified
    Threshold in Huber-like adaptive loss (Eq. 8); no value is given in the paper.
  • Fine-tuning iterations = 10000
    Number of Gaussian fine-tuning iterations, set in Sec. 3.4.
assumptions (6)
  • domain assumption Gaussian Splatting rendering with 2DGS provides accurate enough depth for warping and initialization
    The entire pipeline relies on rendered incomplete depth from a masked 2DGS model (Sec. 3.1, 3.2). If this depth is noisy, the unseen mask and AGDD alignment degrade.
  • domain assumption Marigold's diffusion depth prior can be steered by the adaptive loss to produce aligned metric depth
    AGDD takes Marigold's latent diffusion and modifies noise estimates with L_adaptive; this assumes the prior is flexible enough to align to the incomplete depth in the guided band.
  • domain assumption LeftRefill or the 2D diffusion inpainting model, conditioned on the reference view, yields multi-view consistent RGB details
    SDEdit-based guidance (Sec. 3.3) assumes the inpainting model's output is coherent across viewpoints once conditioned on the reference.
  • domain assumption SAM2 produces accurate unseen masks when prompted with depth-warped bounding boxes
    The final unseen mask U_n is SAM2's output on the contour box (Eq. 3), so an incorrect SAM2 prediction directly affects the inpainted region.
  • domain assumption COLMAP/hloc poses and intrinsics are correct and shared between training and test views
    Depth warping and unprojection use these poses; errors propagate into unseen region detection and Gaussian placement (Sec. 4).
  • domain assumption The 360-USID evaluation protocol (object-mask-only metrics, single reference view) is a fair measure of inpainting quality
    Metrics are computed only within object masks (Sec. 5.1), which may not capture global consistency or artifacts outside the mask.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AuraFusion360: Augmented Unseen Region Alignment for Reference-based 360{\deg} Unbounded Scene Inpainting." pith.science (2026). https://pith.science/paper/WW72IT4L

@misc{pith2026250205176,
  author       = {Pith},
  title        = {Pith review of: AuraFusion360: Augmented Unseen Region Alignment for Reference-based 360\deg Unbounded Scene Inpainting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WW72IT4L}},
  note         = {Machine review of arXiv:2502.05176}
}
read the original abstract

Three-dimensional scene inpainting is crucial for applications from virtual reality to architectural visualization, yet existing methods struggle with view consistency and geometric accuracy in 360{\deg} unbounded scenes. We present AuraFusion360, a novel reference-based method that enables high-quality object removal and hole filling in 3D scenes represented by Gaussian Splatting. Our approach introduces (1) depth-aware unseen mask generation for accurate occlusion identification, (2) Adaptive Guided Depth Diffusion, a zero-shot method for accurate initial point placement without requiring additional training, and (3) SDEdit-based detail enhancement for multi-view coherence. We also introduce 360-USID, the first comprehensive dataset for 360{\deg} unbounded scene inpainting with ground truth. Extensive experiments demonstrate that AuraFusion360 significantly outperforms existing methods, achieving superior perceptual quality while maintaining geometric accuracy across dramatic viewpoint changes.

Figures

Figures reproduced from arXiv: 2502.05176 by the authors.

Figure 1
Figure 1. Overview of our reference-based 360° unbounded scene inpainting method. Given input images with camera parameters, object masks, and a reference image, our AuraFusion360 approach generates an object-masked Gaussian Splatting representation. This representation can then render novel views of the inpainted scene, effectively removing the masked objects while maintaining consistency with the reference image. Abstract T… view at source ↗
Figure 2
Figure 2. Comparison with different 3D inpainting approaches. Existing methods such as SPin-NeRF [36] and GScream [61], de￾signed for forward-facing scenes, perform poorly in 360° scenarios. Reference-based methods like Infusion [29] struggle with accurate depth projection, causing fine-tuning artifacts. Gaussian Group￾ing [67] frequently misidentifies unseen regions, reducing inpaint￾ing quality. Our AuraFusion360 achieves p… view at source ↗
Figure 3
Figure 3. Overview of our method. Our approach takes multi-view RGB images and corresponding object masks as input and outputs a Gaussian representation with the masked objects removed. The pipeline consists of three main stages: (a) Depth-Aware Unseen Masks Generation to identify truly occluded areas, referred to as the “unseen region”, (b) Depth-Aligned Gaussian Initialization on Reference View to fill unseen regions with i… view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: Overview of the Unseen Mask Generation Process using Depth Warping. To obtain the unseen mask for view n, we calculate the pixel correspondences between the view n and all other views i by using the rendered incomplete depth D incomplete n . For each view i, the remova…
Figure 5
Figure 5. Figure 5: Overview of Adaptive Guided Depth Diffusion (AGDD). The framework takes image latent, incomplete depth, and unseen mask as inputs to generate aligned depth estimates. (a) The guided region is identified by dilating the unseen mask and subtracting the original mask. (b)…
Figure 6
Figure 6. Figure 6: Overview of the 360-USID dataset. Sample images from each scene, including five outdoor scenes (Carton, Cone, Newcone, Skateboard, Plant) and two indoor scenes (Cookie, Sunflower). (Bottom right) The table shows statistics for each scene, including the number of traini…
Figure 7
Figure 7. Figure 7: Illustration of the data capture process for the 360- USID dataset. (a) Capturing training views: Multiple images are taken around the object in the scene. (b) Capturing the reference view: A camera is mounted on a tripod to capture a fixed reference view (with an obje…
Figure 8
Figure 8. Figure 8: Visual Comparison on our 360-USID dataset. We compare our method against state-of-the-art approaches including Gaussian Grouping [67], 2DGS + LeftRefill, and Infusion [29]. While Gaussian Grouping struggles with misidentifying unseen regions, leading to floating artifa…
Figure 10
Figure 10. Figure 10: Compared Unseen Mask w/ Gaussian Grouping. Gaussian Grouping [67] uses a video tracker [8] and the “black blurry hole” prompt for DEVA [8] to track the unseen region. How￾ever, this can result in tracking errors, affecting inpainting. In con￾trast, our geometry-based …
Figure 9
Figure 9. Figure 9: Visual comparison of unseen mask generation method. Our method enables SAM2 [45] to generate more accurate pre￾dictions for each view without the need for manually provided prompts, as the bounding box prompts are automatically generated through depth warping. provided…
Figure 11
Figure 11. Figure 11: Compared to other depth completion methods. The depth completion model in Infusion [29] (a) performs better at depth alignment compared to traditional methods (b) and (c), but it produces noisy depth in unseen regions. Similarly, (d) Guided Depth Diffusion [70] strugg…
Figure 12
Figure 12. Figure 12: Intermediate Results of Depth Warping for Unseen Region Detection. This figure illustrates the intermediate results generated during the depth warping process. (a) and (b) show the RGB image and the corresponding removal region at view n, re￾spectively. (c) displays t…
Figure 13
Figure 13. Figure 13: Ablation Study on Removal Region Definition. Com￾parison of (a) object masks vs. (b) depth difference for defining removal regions. Object masks fail to capture geometric changes, leading to less accurate unseen masks. Depth difference better preserves scene structure…
Figure 15
Figure 15. Figure 15: Visual Comparison on our 360-USID dataset. compute a scale- and shift-invariant depth loss. F.5. Gscream [61] We follow the original GScream [61] pipeline as a baseline for comparison. We provide the same reference image and training view object masks as our method to…
Figure 16
Figure 16. Figure 16: Visual Comparison on Other-360 dataset. floaters in the final inpainted outputs. This limitation is simi￾larly observed across all compared methods, underscoring a valuable direction for future research and improvement. 15 [PITH_FULL_IMAGE:figures/full_fig_p015_16.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

74 extracted references · 56 canonical work pages

  1. [1]

    Patchmatch: A randomized correspondence algorithm for structural image editing

    Connelly Barnes, Eli Shechtman, Adam Finkelstein, and Dan B Goldman. Patchmatch: A randomized correspondence algorithm for structural image editing. ACM TOG, 2009. 2

    work page 2009

  2. [2]

    Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-Brualla, and Pratul P

    Jonathan T. Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-Brualla, and Pratul P. Srinivasan. Mip-NeRF: A multiscale representation for anti-aliasing neu- ral radiance fields. In ICCV, 2021. 2

    work page 2021

  3. [3]

    Mip-nerf 360: Unbounded anti-aliased neural radiance fields

    Jonathan T Barron, Ben Mildenhall, Dor Verbin, Pratul P Srinivasan, and Peter Hedman. Mip-nerf 360: Unbounded anti-aliased neural radiance fields. In CVPR, 2022. 6, 12, 14

    work page 2022

  4. [4]

    Image inpainting, 2000

    M Bertalmio. Image inpainting, 2000. 2

    work page 2000

  5. [5]

    Leftrefill: Filling right canvas based on left reference through generalized text-to-image diffusion model

    Chenjie Cao, Yunuo Cai, Qiaole Dong, Yikai Wang, and Yan- wei Fu. Leftrefill: Filling right canvas based on left reference through generalized text-to-image diffusion model. In CVPR,

    work page

  6. [6]

    Gaussianeditor: Swift and controllable 3d editing with gaussian splatting

    Yiwen Chen, Zilong Chen, Chi Zhang, Feng Wang, Xiaofeng Yang, Yikai Wang, Zhongang Cai, Lei Yang, Huaping Liu, and Guosheng Lin. Gaussianeditor: Swift and controllable 3d editing with gaussian splatting. In CVPR, 2024. 2

    work page 2024

  7. [7]

    Improving robustness for joint optimization of camera pose and decom- posed low-rank tensorial radiance fields

    Bo-Yu Cheng, Wei-Chen Chiu, and Yu-Lun Liu. Improving robustness for joint optimization of camera pose and decom- posed low-rank tensorial radiance fields. In AAAI, 2024. 2

    work page 2024

  8. [8]

    Tracking anything with de- coupled video segmentation

    Ho Kei Cheng, Seoung Wug Oh, Brian Price, Alexander Schwing, and Joon-Young Lee. Tracking anything with de- coupled video segmentation. In ICCV, 2023. 2, 8, 14

    work page 2023

Show all 74 references
  1. [9]

    Re- gion filling and object removal by exemplar-based image inpainting

    Antonio Criminisi, Patrick P´erez, and Kentaro Toyama. Re- gion filling and object removal by exemplar-based image inpainting. IEEE TIP, 2004. 2

    2004

  2. [10]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. In NeurIPS, 2021. 2

    2021

  3. [11]

    Spec- tromotion: Dynamic 3d reconstruction of specular scenes

    Cheng-De Fan, Chen-Wei Chang, Yi-Ruei Liu, Jie-Ying Lee, Jiun-Long Huang, Yu-Chee Tseng, and Yu-Lun Liu. Spec- tromotion: Dynamic 3d reconstruction of specular scenes. In CVPR, 2025. 2

    2025

  4. [12]

    Garbin, Marek Kowalski, Matthew Johnson, Jamie Shotton, and Julien Valentin

    Stephan J. Garbin, Marek Kowalski, Matthew Johnson, Jamie Shotton, and Julien Valentin. FastNeRF: High-fidelity neural rendering at 200FPS. In ICCV, 2021. 2

    2021

  5. [13]

    Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N. Dauphin. Convolutional sequence to sequence learning. In ICML, 2017. 2

    2017

  6. [14]

    Efros, Aleksander Holynski, and Angjoo Kanazawa

    Ayaan Haque, Matthew Tancik, Alexei A. Efros, Aleksander Holynski, and Angjoo Kanazawa. Instruct-nerf2nerf: Editing 3d scenes with instructions. In ICCV, 2023. 2, 6

    2023

  7. [15]

    Mitra, and Tobias Ritschel

    Philipp Henzler, Niloy J. Mitra, and Tobias Ritschel. Escaping Plato’s cave: 3D shape from adversarial rendering. In ICCV,

  8. [16]

    Denoising diffu- sion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models. In NeurIPS, 2020. 2

    2020

  9. [17]

    2d gaussian splatting for geometrically accu- rate radiance fields

    Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, and Shenghua Gao. 2d gaussian splatting for geometrically accu- rate radiance fields. In ACM SIGGRAPH 2024 Conference Papers, 2024. 1, 2, 5, 7, 12, 13

    2024

  10. [18]

    A comprehensive review of past and present image inpainting methods

    Jireh Jam, Connah Kendrick, Kevin Walker, Vincent Drouard, Jison Gee-Sern Hsu, and Moi Hoon Yap. A comprehensive review of past and present image inpainting methods. CVIU, 203:103147, 2021. 2

    2021

  11. [19]

    Repurpos- ing diffusion-based image generators for monocular depth estimation

    Bingxin Ke, Anton Obukhov, Shengyu Huang, Nando Met- zger, Rodrigo Caye Daudt, and Konrad Schindler. Repurpos- ing diffusion-based image generators for monocular depth estimation. In CVPR, 2024. 4, 5, 14

    2024

  12. [20]

    3d gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM TOG, 2023. 1, 2

    2023

  13. [21]

    Diffusion models for image restoration and enhancement–a comprehensive sur- vey

    Xin Li, Yulin Ren, Xin Jin, Cuiling Lan, Xingrui Wang, Wen- jun Zeng, Xinchao Wang, and Zhibo Chen. Diffusion models for image restoration and enhancement–a comprehensive sur- vey. arXiv preprint arXiv:2308.09388, 2023. 2

    2023

  14. [22]

    Taming latent diffusion model for neural radiance field inpainting

    Chieh Hubert Lin, Changil Kim, Jia-Bin Huang, Qinbo Li, Chih-Yao Ma, Johannes Kopf, Ming-Hsuan Yang, and Hung- Yu Tseng. Taming latent diffusion model for neural radiance field inpainting. In ECCV, 2024. 3

    2024

  15. [23]

    Frugalnerf: Fast conver- gence for few-shot novel view synthesis without learned pri- ors

    Chin-Yang Lin, Chung-Ho Wu, Chang-Han Yeh, Shih-Han Yen, Cheng Sun, and Yu-Lun Liu. Frugalnerf: Fast conver- gence for few-shot novel view synthesis without learned pri- ors. In CVPR, 2025. 2

    2025

  16. [24]

    Image inpainting for irregular holes using partial convolutions

    Guilin Liu, Fitsum A Reda, Kevin J Shih, Ting-Chun Wang, Andrew Tao, and Bryan Catanzaro. Image inpainting for irregular holes using partial convolutions. In ECCV, 2018. 2

    2018

  17. [25]

    NeRF-In: Free-form NeRF inpainting with RGB-D priors

    Hao-Kang Liu, I-Chao Shen, and Bing-Yu Chen. NeRF-In: Free-form NeRF inpainting with RGB-D priors. In arXiv,

  18. [26]

    Corrfill: Enhancing faithfulness in reference-based inpainting with correspondence guidance in diffusion models

    Kuan-Hung Liu, Cheng-Kun Yang, Min-Hung Chen, Yu-Lun Liu, and Yen-Yu Lin. Corrfill: Enhancing faithfulness in reference-based inpainting with correspondence guidance in diffusion models. In WACV, 2025. 4

    2025

  19. [27]

    Neural sparse voxel fields

    Lingjie Liu, Jiatao Gu, Kyaw Zaw Lin, Tat-Seng Chua, and Christian Theobalt. Neural sparse voxel fields. In NeurIPS,

  20. [28]

    Robust dynamic radiance fields

    Yu-Lun Liu, Chen Gao, Andreas Meuleman, Hung-Yu Tseng, Ayush Saraf, Changil Kim, Yung-Yu Chuang, Johannes Kopf, and Jia-Bin Huang. Robust dynamic radiance fields. InCVPR,

  21. [29]

    Infusion: Inpainting 3d gaussians via learning depth comple- tion from diffusion prior

    Zhiheng Liu, Hao Ouyang, Qiuyu Wang, Ka Leong Cheng, Jie Xiao, Kai Zhu, Nan Xue, Yu Liu, Yujun Shen, and Yang Cao. Infusion: Inpainting 3d gaussians via learning depth comple- tion from diffusion prior. arXiv preprint arXiv:2404.11613,

    arXiv

  22. [30]

    Scaffold-gs: Structured 3d gaussians for view-adaptive rendering

    Tao Lu, Mulin Yu, Linning Xu, Yuanbo Xiangli, Limin Wang, Dahua Lin, and Bo Dai. Scaffold-gs: Structured 3d gaussians for view-adaptive rendering. In CVPR, 2024. 2, 3

    2024

  23. [31]

    Dynamic 3d gaussians: Tracking by persistent dynamic view synthesis

    Jonathon Luiten, Georgios Kopanas, Bastian Leibe, and Deva Ramanan. Dynamic 3d gaussians: Tracking by persistent dynamic view synthesis. In 3DV, 2024. 2 9

    2024

  24. [32]

    SDEdit: Guided image synthesis and editing with stochastic differential equations

    Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon. SDEdit: Guided image synthesis and editing with stochastic differential equations. In ICLR, 2022. 2, 5, 14

    2022

  25. [33]

    Progressively optimized local radiance fields for robust view synthesis

    Andreas Meuleman, Yu-Lun Liu, Chen Gao, Jia-Bin Huang, Changil Kim, Min H Kim, and Johannes Kopf. Progressively optimized local radiance fields for robust view synthesis. In CVPR, 2023. 2

    2023

  26. [34]

    Srinivasan, Matthew Tancik, Jonathan T

    Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthe- sis. In ECCV, 2020. 1, 2, 6

    2020

  27. [35]

    Brubaker, Jonathan Kelly, Alex Levinshtein, Konstantinos G

    Ashkan Mirzaei, Tristan Aumentado-Armstrong, Marcus A. Brubaker, Jonathan Kelly, Alex Levinshtein, Konstantinos G. Derpanis, and Igor Gilitschenski. Reference-guided control- lable inpainting of neural radiance fields. In ICCV, 2023. 2, 3, 7

    2023

  28. [36]

    Derpanis, Jonathan Kelly, Marcus A

    Ashkan Mirzaei, Tristan Aumentado-Armstrong, Konstanti- nos G. Derpanis, Jonathan Kelly, Marcus A. Brubaker, Igor Gilitschenski, and Alex Levinshtein. SPIn-NeRF: Multiview segmentation and perceptual inpainting with neural radiance fields. In CVPR, 2023. 2, 3, 7, 13

    2023

  29. [37]

    Reffusion: Reference adapted diffusion models for 3d scene inpainting

    Ashkan Mirzaei, Riccardo De Lutio, Seung Wook Kim, David Acuna, Jonathan Kelly, Sanja Fidler, Igor Gilitschenski, and Zan Gojcic. Reffusion: Reference adapted diffusion models for 3d scene inpainting. arXiv preprint arXiv:2404.10765,

    arXiv

  30. [38]

    Negative-prompt inversion: Fast image inversion for editing with text-guided diffusion models

    Daiki Miyake, Akihiro Iohara, Yu Saito, and Toshiyuki Tanaka. Negative-prompt inversion: Fast image inversion for editing with text-guided diffusion models. arXiv preprint arXiv:2305.16807, 2023. 2

    2023 arXiv

  31. [39]

    Null-text inversion for editing real images using guided diffusion models

    Ron Mokady, Amir Hertz, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Null-text inversion for editing real images using guided diffusion models. In CVPR, 2023. 2

    2023

  32. [40]

    Context encoders: Feature learning by inpainting

    Deepak Pathak, Philipp Kr ¨ahenb¨uhl, Jeff Donahue, Trevor Darrell, and Alexei Efros. Context encoders: Feature learning by inpainting. In CVPR, 2016. 2

    2016

  33. [41]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M¨uller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023. 2

    2023 arXiv

  34. [42]

    Inpaint3d: 3d scene content generation using 2d inpainting diffusion

    Kira Prabhu, Jane Wu, Lynn Tsai, Peter Hedman, Dan B Goldman, Ben Poole, and Michael Broxton. Inpaint3d: 3d scene content generation using 2d inpainting diffusion. arXiv preprint arXiv:2312.03869, 2023. 2

    2023 arXiv

  35. [43]

    Langsplat: 3d language gaussian splatting

    Minghan Qin, Wanhua Li, Jiawei Zhou, Haoqian Wang, and Hanspeter Pfister. Langsplat: 3d language gaussian splatting. In CVPR, 2024. 2

    2024

  36. [44]

    Language-driven physics-based scene synthesis and editing via feature splatting

    Ri-Zhao Qiu, Ge Yang, Weijia Zeng, and Xiaolong Wang. Language-driven physics-based scene synthesis and editing via feature splatting. In ECCV, 2024. 2

    2024

  37. [45]

    Sam 2: Segment anything in images and videos

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman R¨adle, Chloe Rolland, Laura Gustafson, Eric Mintun, Junting Pan, Kalyan Vasudev Alwala, Nicolas Carion, Chao-Yuan Wu, Ross Girshick, Piotr Doll´ar, and Christoph Feichte...

  38. [46]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In CVPR, 2022. 2

    2022

  39. [47]

    From coarse to fine: Robust hierarchical localization at large scale

    Paul-Edouard Sarlin, Cesar Cadena, Roland Siegwart, and Marcin Dymczyk. From coarse to fine: Robust hierarchical localization at large scale. In CVPR, 2019. 6

    2019

  40. [48]

    SuperGlue: Learning feature match- ing with graph neural networks

    Paul-Edouard Sarlin, Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich. SuperGlue: Learning feature match- ing with graph neural networks. In CVPR, 2020. 6

    2020

  41. [49]

    Structure-from-motion revisited

    Johannes Lutz Sch ¨onberger and Jan-Michael Frahm. Structure-from-motion revisited. In CVPR, 2016. 6

    2016

  42. [50]

    Pixelwise view selection for unstruc- tured multi-view stereo

    Johannes Lutz Sch¨onberger, Enliang Zheng, Marc Pollefeys, and Jan-Michael Frahm. Pixelwise view selection for unstruc- tured multi-view stereo. In ECCV, 2016. 6

    2016

  43. [51]

    Nerf-in: Free-form nerf inpainting with rgb-d priors

    I-Chao Shen, Hao-Kang Liu, and Bing-Yu Chen. Nerf-in: Free-form nerf inpainting with rgb-d priors. Computer Graph- ics and Applications (CG&A), 2024. 3

    2024

  44. [52]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In ICLR, 2021. 2, 5

    2021

  45. [53]

    Boostmvsnerfs: Boosting mvs-based nerfs to generalizable view synthesis in large-scale scenes

    Chih-Hai Su, Chih-Yao Hu, Shr-Ruei Tsai, Jie-Ying Lee, Chin-Yang Lin, and Yu-Lun Liu. Boostmvsnerfs: Boosting mvs-based nerfs to generalizable view synthesis in large-scale scenes. In ACM SIGGRAPH 2024 Conference Papers, 2024. 2

    2024

  46. [54]

    Resolution-robust large mask inpainting with Fourier convolutions

    Roman Suvorov, Elizaveta Logacheva, Anton Mashikhin, Anastasia Remizova, Arsenii Ashukha, Aleksei Silvestrov, Naejin Kong, Harshith Goka, Kiwoong Park, and Victor Lem- pitsky. Resolution-robust large mask inpainting with Fourier convolutions. In WACV, pages 2149–2159, 2022. 2,...

    2022

  47. [55]

    Realfill: Reference-driven generation for authentic image completion

    Luming Tang, Nataniel Ruiz, Qinghao Chu, Yuanzhen Li, Aleksander Holynski, David E Jacobs, Bharath Hariharan, Yael Pritch, Neal Wadhwa, Kfir Aberman, et al. Realfill: Reference-driven generation for authentic image completion. ACM TOG, 2024. 2

    2024

  48. [56]

    Efros, and Ji- tendra Malik

    Shubham Tulsiani, Tinghui Zhou, Alexei A. Efros, and Ji- tendra Malik. Multi-view supervision for single-view recon- struction via differentiable ray consistency. In CVPR, 2017. 2

    2017

  49. [57]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurIPS, 2017. 2

    2017

  50. [58]

    Nerf-art: Text-driven neural radiance fields stylization

    Can Wang, Ruixiang Jiang, Menglei Chai, Mingming He, Dongdong Chen, and Jing Liao. Nerf-art: Text-driven neural radiance fields stylization. IEEE TVCG, 2023. 2

    2023

  51. [59]

    Inpaintnerf360: Text-guided 3d inpainting on unbounded neural radiance fields

    Dongqing Wang, Tong Zhang, Alaa Abboud, and Sabine S¨usstrunk. Inpaintnerf360: Text-guided 3d inpainting on unbounded neural radiance fields. In CVPR, 2024. 3, 7

    2024

  52. [60]

    Barron, Ricardo Martin- Brualla, Noah Snavely, and Thomas Funkhouser

    Qianqian Wang, Zhicheng Wang, Kyle Genova, Pratul Srini- vasan, Howard Zhou, Jonathan T. Barron, Ricardo Martin- Brualla, Noah Snavely, and Thomas Funkhouser. IBRNet: Learning multi-view image-based rendering. In CVPR, 2021. 2

    2021

  53. [61]

    Gscream: Learning 3d geometry and feature consistent gaus- sian splatting for object removal

    Yuxin Wang, Qianyi Wu, Guofeng Zhang, and Dan Xu. Gscream: Learning 3d geometry and feature consistent gaus- sian splatting for object removal. In ECCV, 2024. 2, 3, 7, 14 10

    2024

  54. [62]

    Nerfiller: Completing scenes via generative 3d inpainting

    Ethan Weber, Aleksander Holynski, Varun Jampani, Saurabh Saxena, Noah Snavely, Abhishek Kar, and Angjoo Kanazawa. Nerfiller: Completing scenes via generative 3d inpainting. In CVPR, 2024. 3

    2024

  55. [63]

    Removing objects from neural radiance fields

    Silvan Weder, Guillermo Garcia-Hernando, Aron Monszpart, Marc Pollefeys, Gabriel Brostow, Michael Firman, and Sara Vicente. Removing objects from neural radiance fields. In CVPR, 2023. 3

    2023

  56. [64]

    4d gaussian splatting for real-time dynamic scene rendering

    Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xinggang Wang. 4d gaussian splatting for real-time dynamic scene rendering. In CVPR, 2024. 2

    2024

  57. [65]

    Learn- ing object-compositional neural radiance field for editable scene rendering

    Bangbang Yang, Yinda Zhang, Yinghao Xu, Yijin Li, Han Zhou, Hujun Bao, Guofeng Zhang, and Zhaopeng Cui. Learn- ing object-compositional neural radiance field for editable scene rendering. In ICCV, 2021. 2

    2021

  58. [66]

    Deformable 3d gaussians for high- fidelity monocular dynamic scene reconstruction

    Ziyi Yang, Xinyu Gao, Wen Zhou, Shaohui Jiao, Yuqing Zhang, and Xiaogang Jin. Deformable 3d gaussians for high- fidelity monocular dynamic scene reconstruction. In CVPR,

  59. [67]

    Gaus- sian grouping: Segment and edit anything in 3d scenes

    Mingqiao Ye, Martin Danelljan, Fisher Yu, and Lei Ke. Gaus- sian grouping: Segment and edit anything in 3d scenes. In ECCV, 2024. 2, 3, 7, 8, 12, 14

    2024

  60. [68]

    Or- nerf: Object removing from 3d scenes guided by multiview segmentation with neural radiance fields

    Youtan Yin, Zhoujie Fu, Fan Yang, and Guosheng Lin. Or- nerf: Object removing from 3d scenes guided by multiview segmentation with neural radiance fields. arXiv preprint arXiv:2305.10503, 2023. 3

    2023 arXiv

  61. [69]

    pixelNeRF: Neural radiance fields from one or few images

    Alex Yu, Vickie Ye, Matthew Tancik, and Angjoo Kanazawa. pixelNeRF: Neural radiance fields from one or few images. In CVPR, 2021. 2

    2021

  62. [70]

    Freeman, and Jiajun Wu

    Hong-Xing Yu, Haoyi Duan, Charles Herrmann, William T. Freeman, and Jiajun Wu. Wonderworld: Interactive 3d scene generation from a single image. In CVPR, 2025. 4, 8, 12

    2025

  63. [71]

    Generative image inpainting with contex- tual attention

    Jiahui Yu, Zhe Lin, Jimei Yang, Xiaohui Shen, Xin Lu, and Thomas S Huang. Generative image inpainting with contex- tual attention. In CVPR, 2018. 2

    2018

  64. [72]

    Free-form image inpainting with gated convolution

    Jiahui Yu, Zhe Lin, Jimei Yang, Xiaohui Shen, Xin Lu, and Thomas S Huang. Free-form image inpainting with gated convolution. In ICCV, 2019. 2

    2019

  65. [73]

    Nerf++: Analyzing and improving neural radiance fields

    Kai Zhang, Gernot Riegler, Noah Snavely, and Vladlen Koltun. Nerf++: Analyzing and improving neural radiance fields. arXiv preprint arXiv:2010.07492, 2020. 2

    2010 arXiv

  66. [74]

    (b) Removal region 𝑅

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In CVPR, 2018. 5, 7 11 A. Overview This supplementary material provides additional details and results to support the main manu...

    2018

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.