REVIEW 4 major objections 7 minor 1 cited by
Improving Novel view synthesis of 360$^\circ$ Scenes in Extremely Sparse Views by Jointly Training Hemisphere Sampled Synthetic Images
T0 review · 4 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Adding 50 synthetic upper-hemisphere training views to a four-view 3DGS pipeline improves 360-degree novel view synthesis, with 0.98 dB PSNR and 0.08 SSIM gains over InstantSplat.
desk verdict A plausible pipeline for sparse 360 NVS with a clean ablation story, but the headline gain over InstantSplat is measured on too narrow a benchmark to be trusted as-is. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The mechanism is the upper-hemisphere view sampler defined by Eq. 1, which divides the hemisphere into $L$ elevation levels, places cameras according to Fibonacci-sequence counts, parameterizes each pose by radius, azimuth, and elevation, and orients every camera toward the scene center. This sampler is what converts a single sparse point cloud into 50 extra training views spread out in 3D space. The second carrying component is the distance-aware weight $\lambda(\pi_s^j)$ in Eq. 3, which lets synthetic views nearer to reference images contribute more to the joint loss of Eq. 2; the third is the retrained DiffBIR enhancer, which suppresses artifacts in point-cloud-rendered images and is trained with contextual loss to tolerate pose misalignment.
What would settle it
A controlled comparison would settle the central claim: train the same pipeline with the same 50 synthetic views sampled from the lower hemisphere, the full sphere, and the camera trajectory, holding enhancement and weighting fixed. If lower-hemisphere or full-sphere sampling matches or beats the upper-hemisphere strategy on the same nine scenes, the paper's geometric explanation is wrong. A second check is to run the method on a scene where DUSt3R is known to omit a large background region (e.g., the sky or a ceiling); the synthetic views should visibly fail in those regions, and if the gain persists despite that, the proposed mechanism is not the source of the improvement.
Extended reading notes
Core claim
The paper's central claim is that, in a four-view 360-degree setting, the main failure mode of 3D Gaussian Splatting is missing angular coverage rather than missing image count: training only on the reference views overfits them. To fix this, the method densely samples 50 camera poses across five elevation levels of the upper hemisphere, orients them toward the scene center, and renders each sampled pose from the DUSt3R point cloud (Eq. 1). Those point-cloud renders are sent through a retrained diffusion-based enhancer, and the cleaned synthetic images enter the 3DGS training loss together with the reference images, with a distance-aware weight that favors synthetic views close to the reference cameras (Eqs. 2 and 3). The measured result is that this joint training beats the InstantSplat baseline by 0.98 dB PSNR, 0.08 SSIM, and 0.01 LPIPS, and the ablations attribute the gain to both the upper-hemisphere sampling strategy and the enhancement step.
Load-bearing premise
The load-bearing premise is that DUSt3R, applied to only four images, produces a point cloud and camera poses accurate enough that images rendered from synthetic upper-hemisphere poses are faithful to the actual scene after diffusion enhancement; the paper's own results concede that large background areas remain unrepresented in the point cloud, producing empty regions, so this premise is only partially satisfied.
Editorial extensions
If this is right
- Synthetic views sampled in 3D space around the upper hemisphere outperform views sampled along the camera trajectory by 0.18 dB PSNR, so spatial coverage of the hemisphere is an effective knob for sparse-view 3DGS.
- Cleaning the point-cloud renders with the retrained diffusion enhancer adds 0.25 dB PSNR on top of the sampling gain, meaning artifact removal in synthetic training data is itself a source of improvement.
- The full pipeline needs only four reference images and no ground-truth poses, so it offers a practical route to sparse 360-degree capture for VR/AR and scene monitoring.
- The LPIPS gain over the baseline is only 0.01 while PSNR and SSIM gains are larger, which means the perceived-quality improvement is more modest than the pixel-level one.
Reading between the lines
- If the coverage hypothesis is the real driver, then the benefit should grow with the completeness of the DUSt3R point cloud; the paper's own empty-region artifacts suggest the gain is currently capped by missing background geometry.
- The upper-hemisphere choice is presumably tuned to scenes with a dominant floor and centered objects; a direct test would be to repeat the same sampling on the full sphere or the lower hemisphere on scenes with prominent ceilings or sky.
- The recipe used to build the enhancement training set (rendering from DUSt3R point clouds, pairing with reference images, retraining with contextual loss) could be transferred to other scene categories, such as street-level or indoor navigation, without altering the 3DGS training loop.
- Because all reported numbers sit near 14 dB PSNR, the method is best understood as a strong initialization for extremely sparse reconstruction rather than as a substitute for dense capture.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a framework for novel view synthesis (NVS) of 360-degree scenes from extremely sparse inputs (four views). It uses DUSt3R to estimate camera poses and generate a dense point cloud, densely samples additional camera poses in the upper hemisphere, renders synthetic images from the point cloud, enhances these images with a retrained DiffBIR model, and finally trains a 3D Gaussian Splatting (3DGS) model jointly on the reference and enhanced synthetic images. Experiments on Mip-NeRF 360 and Tanks & Temples scenes report improvements over InstantSplat, COGS, and DiffusioNeRF, with the central claim being an average improvement of 0.98 dB PSNR, 0.08 SSIM, and 0.01 LPIPS over InstantSplat. The paper also provides ablations isolating the contribution of the synthetic views, perceptual loss, distance-aware weighting, and image enhancement.
Significance. If the reported improvements hold under an unbiased evaluation, the method would be a practical contribution to sparse-view 360-degree NVS: it is simple, builds on open-source components, and the source code is released. The ablation study is a strength, as it demonstrates that each component contributes positively. The evaluation uses standard datasets with held-out test images, which is appropriate for comparing against baselines. However, the current evidence is weakened by the selection protocol: the same nine scenes are used for both hyperparameter tuning and final reporting, the training views are manually chosen for 360-degree coverage, and no error bars are provided. The claimed LPIPS gain is not consistent across scenes, with several scenes showing equal or worse LPIPS than InstantSplat. Additionally, because the synthetic views are rendered from the same DUSt3R point cloud that is derived from the reference views, they do not introduce truly external scene information; the method's success depends on the accuracy of that point cloud, which the paper itself notes is incomplete in large background areas.
major comments (4)
- [Table 1, Sec. 3.5] The central claim of an average 0.01 LPIPS improvement over InstantSplat is not supported by the per-scene numbers: on Garden (0.61 vs 0.60), Bicycle (0.64 vs 0.63), and Stump (0.63 vs 0.63), the proposed method is worse or equal, so the average gain is within rounding error and the improvement is not consistent.
- [Sec. 3.1, Sec. 3.3, Sec. 3.5] The evaluation protocol does not separate model selection from assessment: the hyperparameters (perceptual losses λP=0.5/0.1, elevation cap τ=0.8, L=5 levels, N=50 synthetic views, 5 DDIM steps, distance-weight normalization Dmax) are fixed using the same nine scenes on which the final numbers are reported, and the four reference views are manually selected for 360-degree coverage with no random subsets or variance estimates. A held-out validation set with multiple view selections and reported mean/std is needed before the improvement can be attributed to the method rather than to tuning and selection.
- [Sec. 3.5, Eq. (2)] The paper admits that 'large background areas remain unrepresented in the point cloud, leading to some empty regions'; since the synthetic views in Eq. (2) are rendered from this incomplete point cloud, the added supervision can inject incorrect geometry in those regions. The manuscript provides no analysis of how the results depend on point-cloud coverage or on the number and placement of synthetic views, which is a load-bearing unknown for the claimed benefit.
- [Sec. 2.3, Eq. (1)] The upper-hemisphere sampling strategy assumes that a scene has a well-defined center and that cameras point directly at it; the method is only evaluated on scenes selected for a centered object (Sec. 3.1), so it is unclear whether the gains extend to general 360-degree scenes without a centered object, and the title's claim of improving '360° scenes' is broader than the evaluated scope.
minor comments (7)
- [Sec. 2.1] There are typos: 'spare view' should be 'sparse view' and 'efficent' should be 'efficient'.
- [Sec. 3.3] 'imprically' should be 'empirically'.
- [Sec. 3.5] 'apperance' should be 'appearance'.
- [Eq. (1)] The Fibonacci sequence F(Tl) is used to determine the number of cameras per elevation level, but Tl is not defined and the mapping from Fibonacci numbers to camera counts is unclear.
- [Eq. (3)] Dmax is not explicitly defined; state whether it is the maximum pairwise distance among reference camera poses or the maximum distance of any sampled pose to a reference pose.
- [Sec. 3.4, references] The perceptual loss LP is said to follow [17] (Simonyan's VGG paper), but LPIPS is the metric from [27]; cite [27] where the perceptual loss is introduced.
- [Table 1] The caption states 'best performance values are highlighted', but no highlighting or bolding appears in the table text.
Circularity Check
No significant circularity: the synthetic-view training signal is derived from DUSt3R point clouds and external multi-view datasets, and Table 1 is measured on held-out test views.
full rationale
This paper's derivation chain is self-contained and does not reduce to its inputs. The pipeline is: DUSt3R estimates camera poses and a dense point cloud from the four reference images (Sec. 2.2); upper-hemisphere poses are sampled by the geometric formula in Eq. (1); point-cloud renders are enhanced by a DiffBIR model retrained on 52,552 pairs from WildRGB-D, MVImgNet, and DL3DV-10K (Sec. 2.4); and 3DGS is trained with the loss in Eqs. (2)-(4), which compares reference images and enhanced synthetic images to 3DGS renders. None of these equations defines the predicted output in terms of itself, and the enhancement model is trained on external datasets, not on the Mip-NeRF 360 or Tanks&Temples evaluation scenes. The quantitative claim in Table 1 compares against InstantSplat, COGS, and DiffusioNeRF using held-out test views registered with iComma, not the synthetic views used in training. The single self-citation, [3], supports only the introductory point that large scenes require reacquisition when changed and is not load-bearing for the method or the evaluation. The admitted empty-region limitation in Sec. 3.5 and the absence of error bars or a separate validation set are experimental robustness concerns, but they are not circularity under the criteria used here.
Assumptions & free parameters
free parameters (5)
- Number of synthetic views N =
50
- Elevation levels L =
5
- Maximum elevation ratio tau =
0.8
- Perceptual loss weight lambda_P for synthetic images =
0.1
- Distance-weight normalization Dmax =
Maximum distance among reference camera poses
assumptions (5)
- domain assumption DUSt3R provides sufficiently accurate camera poses and a dense point cloud for 360-degree scenes from four input views.
- domain assumption The test scenes are suitable for the centered-object setup, and the four selected views cover the 360-degree range.
- domain assumption The retrained DiffBIR enhancement removes point-cloud rendering artifacts without introducing new content into the images.
- ad hoc to paper The distance-aware weight lambda(pi) from Eq. 3 is a valid weighting for synthetic view loss.
- domain assumption 3DGS trained for 1000 iterations with the combined reference and synthetic images converges to a useful scene representation.
Cite this review
Pith. "Pith review of Improving Novel view synthesis of 360$^\circ$ Scenes in Extremely Sparse Views by Jointly Training Hemisphere Sampled Synthetic Images." pith.science (2026). https://pith.science/paper/2SNHC4KH
@misc{pith2026250519264,
author = {Pith},
title = {Pith review of: Improving Novel view synthesis of 360$^\circ$ Scenes in Extremely Sparse Views by Jointly Training Hemisphere Sampled Synthetic Images},
year = {2026},
howpublished = {\url{https://pith.science/paper/2SNHC4KH}},
note = {Machine review of arXiv:2505.19264}
}
abstract
Novel view synthesis in 360$^\circ$ scenes from extremely sparse input views is essential for applications like virtual reality and augmented reality. This paper presents a novel framework for novel view synthesis in extremely sparse-view cases. As typical structure-from-motion methods are unable to estimate camera poses in extremely sparse-view cases, we apply DUSt3R to estimate camera poses and generate a dense point cloud. Using the poses of estimated cameras, we densely sample additional views from the upper hemisphere space of the scenes, from which we render synthetic images together with the point cloud. Training 3D Gaussian Splatting model on a combination of reference images from sparse views and densely sampled synthetic images allows a larger scene coverage in 3D space, addressing the overfitting challenge due to the limited input in sparse-view cases. Retraining a diffusion-based image enhancement model on our created dataset, we further improve the quality of the point-cloud-rendered images by removing artifacts. We compare our framework with benchmark methods in cases of only four input views, demonstrating significant improvement in novel view synthesis under extremely sparse-view conditions for 360$^\circ$ scenes.
Forward citations
Cited by 1 Pith paper
-
Sparse-View 3D Reconstruction: Recent Advances and Open Challenges
A comprehensive survey that organizes sparse-view 3D reconstruction methods into geometry-based, NeRF, 3DGS, and diffusion-based categories, with benchmarks and open challenges.
Reference graph
Works this paper leans on
-
[1]
INTRODUCTION Novel view synthesis (NVS) is a key challenge in com- puter vision, focusing on rendering images from previ- ously unobserved viewpoints. Recent advancements, such as Neural Radiance Fields (NeRF) [12] and 3D Gaussian Splatting (3DGS) [6], have demonstrated remarkable ca- pabilities in generating novel views using dense captured images of a s...
arXiv 2025
-
[2]
METHOD 2.1. Preliminary 3DGS [6] is an explicit representation for modeling scenes with parameterized 3D Gaussians. A 3D Gaussian is param- eterized by G = {µ, q, s, o, c, sh}, where µ is spatial mean, q is rotation quaternion, s is scaling vector, o is opacity, c is a view-dependent color, and sh is spherical harmonic (SH) coefficients. A 3D scene is rep...
-
[3]
EXPERIMENTS 3.1. Dataset The two real-world benchmarking datasets featuring large- scale scenes: Mip-NeRF 360 [1], and Tanks&Temples ( 360◦ scenes) [7] are used to evaluate our method. Specifically, we evaluate our method in four-view cases, i.e., M = 4. To re- construct a 360◦ scene, it must have overlap between sampled images, even in extremely spare vi...
-
[4]
The framework uses reference and synthetic images to train a 3DGS model
CONCLUSION AND FUTURE WORK We propose a framework to improve the performance of 3DGS for 360 ◦ scenes with extremely sparse views. The framework uses reference and synthetic images to train a 3DGS model. Those synthetic images are created with cam- era poses sampled in the upper hemisphere space of scenes and DUSt3R-created point clouds. Moreover, we remo...
-
[5]
ACKNOWLEDGMENT This research received partial funding from the Flemish Gov- ernment under the ‘Onderzoeksprogramma Artificiele Intelli- gentie (AI) Vlaanderen’ programme
-
[6]
J. T. Barron, B. Mildenhall, D. Verbin, P. P. Srinivasan, and P. Hedman. Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields. In IEEE Conference on Computer Vision and Pattern Recognition, pages 5470–5479, 2022
work page 2022
- [7]
-
[8]
G. Chen, M. Vlaminck, W. Philips, and H. Q. Luong. Multi-View 3D Reconstruction for Construction Site Monitoring. In Conference on VISAPP, 2024
work page 2024
Show all 32 references
-
[9]
Z. Fan, W. Cong, K. Wen, K. Wang, J. Zhang, X. Ding, D. Xu, B. Ivanovic, M. Pavone, G. Pavlakos, et al. InstantSplat: Unbounded Sparse-view Pose-free Gaussian Splatting in 40 Seconds. arXiv preprint arXiv:2403.20309, 2024
2024 arXiv
-
[10]
Jiang, Y
K. Jiang, Y . Fu, M. Varma T, Y . Belhe, X. Wang, H. Su, and R. Ra- mamoorthi. A Construct-Optimize Approach to Sparse View Syn- thesis without Camera Pose. In ACM SIGGRAPH 2024 Conference Papers, pages 1–11, 2024
2024
-
[11]
Kerbl, G
B. Kerbl, G. Kopanas, T. Leimk ¨uhler, and G. Drettakis. 3D Gaussian Splatting for Real-Time Radiance Field Rendering.ACM Transactions on Graphics, 42(4):139–1, 2023
2023
-
[12]
Knapitsch, J
A. Knapitsch, J. Park, Q.-Y . Zhou, and V . Koltun. Tanks and temples: benchmarking large-scale scene reconstruction. ACM Transactions on Graphics, 36(4):1–13, 2017
2017
-
[13]
Liang, J
J. Liang, J. Cao, G. Sun, K. Zhang, L. Van Gool, and R. Timofte. SwinIR: Image Restoration Using Swin Transformer. In IEEE Inter- national Conference on Computer Vision, pages 1833–1844, 2021
2021
-
[14]
X. Lin, J. He, Z. Chen, Z. Lyu, B. Dai, F. Yu, W. Ouyang, Y . Qiao, and C. Dong. DiffBIR: Toward Blind Image Restoration with Generative Diffusion Prior. arXiv preprint arXiv:2308.15070, 2023
2023 arXiv
-
[15]
L. Ling, Y . Sheng, Z. Tu, W. Zhao, C. Xin, K. Wan, L. Yu, Q. Guo, Z. Yu, Y . Lu, et al. DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Vision. In IEEE Conference on Computer Vision and Pattern Recognition, pages 22160–22169, 2024
2024
-
[16]
Mechrez, I
R. Mechrez, I. Talmi, and L. Zelnik-Manor. The Contextual Loss for Image Transformation with Non-Aligned Data. In IEEE Conference on European Conference on Computer Vision, pages 768–783, 2018
2018
-
[17]
Mildenhall, P
B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoor- thi, and R. Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1):99–106, 2021
2021
-
[18]
S. Paul, C. Wewer, B. Schiele, and J. E. Lenssen. Sp2360: Sparse- view 360◦ Scene Reconstruction using Cascaded 2D Diffusion Priors. In ECCV 2024 Workshop on Wild 3D: 3D Modeling, Reconstruction, and Generation in the Wild , 2024
2024
-
[19]
Poole, A
B. Poole, A. Jain, J. T. Barron, and B. Mildenhall. DreamFusion: Text- to-3D using 2D Diffusion. In International Conference on Learning Representations, 2023
2023
-
[20]
Rombach, A
R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer. High-Resolution Image Synthesis With Latent Diffusion Models. In IEEE Conference on Computer Vision and Pattern Recognition, pages 10684–10695, 2022
2022
-
[21]
J. L. Schonberger and J.-M. Frahm. Structure-From-Motion Revis- ited. In IEEE Conference on Computer Vision and Pattern Recogni- tion, pages 4104–4113, 2016
2016
-
[22]
Simonyan
K. Simonyan. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014
2014 arXiv
-
[23]
Y . Sun, X. Wang, Y . Zhang, J. Zhang, C. Jiang, Y . Guo, and F. Wang. iComMa: Inverting 3D Gaussian Splatting for Camera Pose Estima- tion via Comparing and Matching. arXiv preprint arXiv:2312.09031, 2023
2023 arXiv
-
[24]
S. Wang, V . Leroy, Y . Cabon, B. Chidlovskii, and J. Revaud. DUSt3R: Geometric 3D Vision Made Easy. In IEEE Conference on Computer Vision and Pattern Recognition, pages 20697–20709, 2024
2024
-
[25]
Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600–612, 2004
2004
-
[26]
R. Wu, B. Mildenhall, P. Henzler, K. Park, R. Gao, D. Watson, P. P. Srinivasan, D. Verbin, J. T. Barron, B. Poole, et al. ReconFusion: 3D Reconstruction with Diffusion Priors. In IEEE Conference on Com- puter Vision and Pattern Recognition, pages 21551–21561, 2024
2024
-
[27]
Wynn and D
J. Wynn and D. Turmukhambetov. DiffusioNeRF: Regularizing Neu- ral Radiance Fields With Denoising Diffusion Models. In IEEE Con- ference on Computer Vision and Pattern Recognition , pages 4180– 4189, 2023
2023
-
[28]
H. Xia, Y . Fu, S. Liu, and X. Wang. RGBD Objects in the Wild: Scaling Real-World 3D Object Learning from RGB-D Videos. In IEEE Conference on Computer Vision and Pattern Recognition, pages 22378–22389, 2024
2024
-
[29]
Xiong, S
H. Xiong, S. Muttukuru, R. Upadhyay, P. Chari, and A. Kadambi. SparseGS: Real-Time 360◦ Sparse View Synthesis using Gaussian Splatting. arXiv preprint arXiv:2312.00206, 2023
2023 arXiv
-
[30]
C. Yang, S. Li, J. Fang, R. Liang, L. Xie, X. Zhang, W. Shen, and Q. Tian. GaussianObject: High-Quality 3D Object Recon- struction from Four Views with Gaussian Splatting. arXiv preprint arXiv:2402.10259, 2024
2024 arXiv
-
[31]
X. Yu, M. Xu, Y . Zhang, H. Liu, C. Ye, Y . Wu, Z. Yan, C. Zhu, Z. Xiong, T. Liang, et al. MVImgNet: A Large-Scale Dataset of Multi- View Images. In IEEE Conference on Computer Vision and Pattern Recognition, pages 9150–9161, 2023
2023
-
[32]
Zhang, P
R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang. The Un- reasonable Effectiveness of Deep Features as a Perceptual Metric. In IEEE Conference on Computer Vision and Pattern Recognition, pages 586–595, 2018
2018
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.