Pith. sign in

REVIEW 1 major objections 4 minor 59 references

Omni-Scene: Omni-Gaussian Representation for Ego-Centric Sparse-View Scene Reconstruction

T0 review · 1 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Hybridizing voxel-anchored Gaussians with per-pixel Gaussians lets a feed-forward network reconstruct ego-centric driving scenes from six sparse surround views, where overlap-based methods fail.

desk verdict The hybrid volume+pixel Gaussian representation is a real step forward for ego-centric sparse-view reconstruction, but the headline geometric claim rests on a monocular-depth eval that is not independent of the method's own depth priors. read the letter →

arxiv 2412.06273 v2 pith:H6Y3EJDC submitted 2024-12-09 cs.CV cs.GR

Dongxu Wei , Zhiqi Li , Peidong Liu This is my paper · ORCID
classification cs.CVcs.GR
keywords Ego-centricreconstructionSparse-view3DGaussiansplattingNovelviewsynthesisAutonomousdrivingscenesVolume-pixelcollaborationTriplanetransformerFeed-forward
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is trying to extend feed-forward sparse-view reconstruction from scene-centric datasets, where input cameras overlap heavily, to ego-centric driving setups where six surround cameras barely overlap and objects are routinely occluded or cut off by the image boundary. Its central proposal is Omni-Gaussian, a hybrid representation that pairs voxel-anchored Gaussians, which supply coarse 3D structure without needing cross-view overlap, with per-pixel Gaussians, which add unbounded distant content and fine detail. The paper reports that this combination reaches 24.27 PSNR on nuScenes against 21.61 for MVSplat and 21.51 for pixelSplat, while staying close to the best prior methods on RealEstate10K. The significance, if the result holds, is that a single forward pass over one frame of surround-view cameras can produce an explorable 3D driving scene.

What carries the argument

Omni-Gaussian representation: a union of volume-based Gaussians anchored on voxels, decoded from a triplane volume whose three planes are enriched by cross-image and cross-plane deformable attentions, and pixel-based Gaussians unprojected along rays with a monocular metric-depth initialization. The two branches are coupled by Projection-Based Feature Fusion, which projects pixel-branch features back onto the triplane queries, and by Depth-Guided Training Decomposition, which uses masked photometric and depth losses so the volume branch specializes inside its bounded range and aligns its scale with the pixel branch. The triplane plus deformable attention is the load-bearing device that lifts multi-view 2D features into 3D without dense voxels and without cross-view overlap.

What would settle it

Render novel-view depth maps from the method's Gaussians on nuScenes and compare them with the dataset's LiDAR point clouds, using a scale-invariant error or per-pixel depth accuracy; if the volume branch's geometry is no more accurate than pixelSplat or MVSplat under that ground-truth comparison, the claimed geometric superiority would be refuted.

Watch

Extended reading notes

Core claim

The central claim is that pixel-based and volume-based Gaussian representations fail in exactly opposite ways, and that a network can be designed so each branch covers the other's blind spots. The paper claims that pixel-based Gaussians require cross-view overlap to estimate depths, so they collapse under minimal overlap and cannot place Gaussians for occluded or truncated content; volume-based Gaussians survive those cases because features are lifted directly into 3D, but their bounded voxel volume misses distant sky and its resolution caps surface detail. Omni-Gaussian therefore keeps both branches, fuses pixel features back onto the triplane so the volume branch knows what is already covered, and uses depth-guided training masks so the volume branch is only penalized inside its range while the two branches are aligned to a common scale. With that design the paper asserts it significantly outperforms the pixel-only baselines in ego-centric reconstruction and matches them in scene-centric reconstruction, and that feeding diffusion-generated surround images through the same model produces feed-forward text- or layout-conditioned 3D driving scenes.

Load-bearing premise

The load-bearing premise is that filling occluded and truncated regions with volume-anchored Gaussians produces true scene geometry rather than plausible-looking content; the paper's evidence for geometry is correlation with monocular depth estimates, not measured ground truth.

Editorial extensions

If this is right

  • Ego-centric sparse-view reconstruction becomes a one-shot operation: six single-frame surround images from a moving car are enough to render novel views, including views displaced 3.2 m or more from the inputs.
  • Occluded and frustum-truncated content, the failure cases of pixel-only splatting, is at least partially recoverable by the volume branch, so reconstruction no longer requires that every novel-view surface be visible in an input image.
  • The same Omni-Gaussian model transfers to scene-centric data with large overlap, giving comparable PSNR/SSIM/LPIPS and higher geometric correlation than the baselines.
  • Initializing the pixel branch with monocular metric depth matters mainly for geometry: the paper's ablation shows pixel-only Pearson correlation drops from 0.780 to 0.572 without it, even though image quality stays similar.
  • Pairing the reconstructor with a 2D diffusion model yields feed-forward 3D driving-scene generation from text, 3D boxes, or BEV maps, avoiding per-scene optimization.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: because the geometric evaluation compares rendered depths with monocular depth estimates, a natural next check is to score the same rendered depths against nuScenes LiDAR; gains in PCC could partly reflect shared priors between the depth-initialization model and the evaluation depth model rather than true recovered geometry.
  • Editorial inference: the volume-pixel recipe suggests a general division of labor for sparse-view reconstruction—a bounded volumetric branch for occlusion-completion plus an unbounded ray branch for distant detail—so the architecture should transfer to other wide-baseline or multi-camera rigs; an easy test is indoor surround-view or multi-camera navigation data with ground-truth scans.
  • Editorial inference: replacing the Metric3D V2 initialization with a different monocular depth model, or with no initialization but an auxiliary depth loss, would separate the contribution of the hybrid collaboration from the contribution of the depth prior.
  • Editorial inference: the diffusion-generation experiment points to a practical pipeline for controllable driving-scene synthesis, but its utility would be better measured by a user study or by downstream planner/segmentation evaluations than by image metrics alone.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

1 major / 4 minor

Summary. The paper proposes Omni-Scene, a feed-forward method for ego-centric sparse-view 3D reconstruction from a single frame of six surrounding cameras. The key idea is an Omni-Gaussian representation that combines volume-anchored Gaussians (predicted from a triplane transformer with deformable cross-image and cross-plane attention) with pixel-based Gaussians (predicted by a U-Net pixel decoder whose per-pixel depth is initialized with Metric3D V2). Two collaboration mechanisms are introduced: projection-based feature fusion and depth-guided training decomposition. Experiments on nuScenes report large improvements over re-implemented pixelSplat and MVSplat baselines (e.g., 24.27 vs. 21.61 PSNR), with comparable results on RealEstate10K (26.19 vs. 26.39 PSNR). The paper also demonstrates a multi-modal text/layout-to-3D generation application by integrating a 2D diffusion model.

Significance. If the reported results hold, the paper makes a useful contribution to feed-forward reconstruction for autonomous driving, where cross-view overlap is minimal and occlusions/truncations are frequent. The representation design is sensible, the task formulation on nuScenes is clearly described, and the supplementary provides detailed architecture and hyperparameter tables for the proposed method. The paper also reports inference speed and model size, showing competitive efficiency. However, the empirical evidence for the claimed geometric superiority is not yet convincing: the geometric metric (PCC) is computed against monocular pseudo-depth, and the method itself uses a monocular depth prior, so the headline PCC gap may partly reflect shared priors rather than genuine geometric accuracy. No ground-truth geometry evaluation is provided despite nuScenes containing LiDAR, and no error bars or significance tests are reported for any of the quantitative claims.

major comments (1)
  1. [Sec. 4.1 (Metrics), Sec. 3.2, Eq. (4), Table 1] Section 4.2 states that pixelSplat and MVSplat are 're-implemented following their official code,' but no training details for these baselines are provided (e.g., number of iterations, learning rate, image resolution, number of GPUs). The supplementary only gives configuration for Omni-Scene. Without these details, the fairness of the comparison is not verifiable, especially because the ego-centric setting (six views with minimal overlap and large 3.2 m view shifts) likely requires adaptation of the official implementations. The authors should report the training setup used for the baselines.
minor comments (4)
  1. [Throughout] There are pervasive formatting artifacts in the manuscript, such as 'V olume Builder', 'Decorater', and inconsistent use of 'V olume-Pixel' (the space after 'V' appears to be a LaTeX rendering issue). These should be corrected in the final version.
  2. [Fig. 3] Figure 3 is dense and the legend for the skip connection and for the different attention arrows is not entirely clear. Adding a concise caption explaining each colored arrow and the fusion path would improve readability.
  3. [Sec. 4.1 / Supp. Sec. 6.1] The bin-based dataset construction uses a 3.2 m interval, but the paper does not state how the 'first and last frames' are selected within each bin when the trajectory length is less than 3.2 m; the supplement says they are used directly, but the exact selection relative to the bin is ambiguous. This should be clarified.
  4. [Sec. 4.1] The resolution is set to 224 x 400 'for compatibility with the 2D diffusion model,' but the rationale for why this resolution is appropriate for the reconstruction evaluation is not discussed. The authors should mention whether this resolution differs from the native nuScenes image resolution and how it affects the comparison with baselines.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: central claims are benchmarked against external ground truth and external monocular depth; the only self-citation (BEVFormer) is non-load-bearing.

full rationale

The paper's derivation chain is not circular. The headline ego-centric result (24.27 vs 21.61 PSNR on nuScenes) is measured against ground-truth novel views rendered from held-out frames, not against any quantity the model produces, so it is externally falsifiable. The geometric superiority claim uses Pearson Correlation Coefficient between rendered depths and Depth Anything V2 predictions (Sec. 4.1, Metrics), an external depth estimator that is not part of the training objective. The pixel branch is initialized with Metric3D V2 depths (Sec. 3.2), a different external model; while this prior influences the rendered depths (as the paper's own ablation in Table 2 shows, PCC drops from 0.780 to 0.572 without depth init), the evaluation is not defined in terms of that prior, so this is a validity limitation rather than a circular step. The Depth-Guided Training Decomposition (Eq. 4) aligns volume- and pixel-rendered depths through L_dpt^V; this is mutual supervision between two branches of the same model, but the final reported metrics are computed on novel-view renderings against ground-truth images and against Depth Anything V2, so no reported number is forced by construction. The only self-citation is BEVFormer [25], by co-author Zhiqi Li, cited as inspiration for deformable attention in the Triplane Transformer; it supplies an architectural idea rather than a load-bearing premise, and no uniqueness theorem is imported from the authors' prior work. The absence of LiDAR-based geometric evaluation on nuScenes is a legitimate correctness and validity concern, but it does not make the derivation self-referential.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim depends on several hand-chosen engineering parameters (volume range, Gaussians per voxel, loss weights, bin size) and on external monocular depth priors for both initialization and evaluation. The method's core modeling hypothesis (triplane deformable attention can substitute for cross-view overlap) is an empirical assumption, not a proven guarantee.

free parameters (4)
  • Volume spatial range and resolution = [-50m, -50m, -3m] to [50m, 50m, 12m]; 192x192x16
    Hand-chosen to cover the vehicle surroundings; defines the boundary between volume-based and pixel-based Gaussians, so it directly affects what each branch reconstructs.
  • Gaussians per voxel V = 3
    Design choice; no sensitivity study is reported in the paper.
  • Loss weights (lambda1, lambda2, lambdaV1, lambdaV2) = 0.05, 1.0, 0.05, 0.01
    Set as hyperparameters; the paper provides no ablation on these weights.
  • Ego-centric bin size = 3.2 m interval between first and last frames (novel views 1.6 m from input)
    Task protocol chosen by the authors; Table 6 shows performance degrades at larger bins without fine-tuning, so the headline result is tied to this choice.
assumptions (5)
  • domain assumption Camera intrinsics and extrinsics for all K views are known and used for 3D-to-2D projection and ray unprojection.
    Required by CIDA in Sec. 3.1 and pixel unprojection in Eq. 3; no self-calibration is considered.
  • domain assumption A pretrained monocular depth model (Metric3D V2) provides reliable initial depth estimates for pixel-based Gaussians.
    Used in Sec. 3.2; ablations show removing it drops pixel-only PCC from 0.780 to 0.572.
  • domain assumption Monocular depth estimates from Depth Anything V2 are a valid proxy for evaluating 3D reconstruction geometry via PCC.
    Used as the geometric ground truth in Sec. 4.1; no LiDAR or other metric 3D ground truth is used.
  • domain assumption Volume features lifted via triplane deformable attention can represent 3D structure without cross-view overlap.
    Core assumption of the Volume Builder in Sec. 3.1; supported only by empirical results, not by an independent geometric guarantee.
  • domain assumption All relevant near-field content lies within the predefined volume range, with distant content assigned to pixel Gaussians.
    Used in Sec. 3.3 depth-guided decomposition; if the range is mis-specified, volume Gaussians cannot cover the scene.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Omni-Scene: Omni-Gaussian Representation for Ego-Centric Sparse-View Scene Reconstruction." pith.science (2026). https://pith.science/paper/H6Y3EJDC

@misc{pith2026241206273,
  author       = {Pith},
  title        = {Pith review of: Omni-Scene: Omni-Gaussian Representation for Ego-Centric Sparse-View Scene Reconstruction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/H6Y3EJDC}},
  note         = {Machine review of arXiv:2412.06273}
}
read the original abstract

Prior works employing pixel-based Gaussian representation have demonstrated efficacy in feed-forward sparse-view reconstruction. However, such representation necessitates cross-view overlap for accurate depth estimation, and is challenged by object occlusions and frustum truncations. As a result, these methods require scene-centric data acquisition to maintain cross-view overlap and complete scene visibility to circumvent occlusions and truncations, which limits their applicability to scene-centric reconstruction. In contrast, in autonomous driving scenarios, a more practical paradigm is ego-centric reconstruction, which is characterized by minimal cross-view overlap and frequent occlusions and truncations. The limitations of pixel-based representation thus hinder the utility of prior works in this task. In light of this, this paper conducts an in-depth analysis of different representations, and introduces Omni-Gaussian representation with tailored network design to complement their strengths and mitigate their drawbacks. Experiments show that our method significantly surpasses state-of-the-art methods, pixelSplat and MVSplat, in ego-centric reconstruction, and achieves comparable performance to prior works in scene-centric reconstruction.

Figures

Figures reproduced from arXiv: 2412.06273 by the authors.

Figure 1
Figure 1. Provided with six surrounding images captured in real world or synthesized by 2D diffusion models, we can generate high-quality [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Illustration of our Omni-Gaussian representation. Our Omni-Gaussian incorporates two representations, pixel-based and volume [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Overview. (a) Obtain images {I i } K i=1 from surrounding cameras with minimal overlap (e.g., adjacent image areas enclosed by green rectangles) in a single frame, and extract 2D features using image backbone. (b) For Volume Builder, we first use Triplane Transformer to lift 2D features {F i } K i=1 to 3D volume space compressed by three orthogonal planes, where we employ cross-image and cross-plane deformable atten… view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: Comparisons on nuScenes [31]. Images of input views (Inputs) and ground-truth novel views (GTs) are outlined by orange and blue rectangles, respectively. The remaining are generated novel views and depth maps (warmer colors denote greater distance while the opposite fo…
Figure 5
Figure 5. Figure 5: Comparisons on RealEstate10K [32]. The red dashed circles denote undesirable artifacts, while the green ones denote plausibly-rendered areas. MuRF [9]. The results of these methods are adopted from their papers directly. Metrics. To measure visual quality, we use three…
Figure 6
Figure 6. Figure 6: Multi-modal 3D scene generation. We accept multi-modal conditions (i.e., 3D boxes, BEV map, textual descriptions) as inputs, [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Ablations on Omni-Gaussian representation. Images [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Data preprocessing of nuScenes [31]. the observed input views, and the first and last frames with 12 surrounding images as the novel views. For videos with a trajectory length less than 3.2 meters, we directly use the first and last frames of the video as the novel vie…
Figure 9
Figure 9. Figure 9: Qualitative ablations on Volume-Pixel Collaboration. Images of input views (Inputs) and ground-truth novel views (GTs) are [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: Qualitative ablations on Depth Initialization. The 1st column present images of input views (Inputs) and ground-truth novel [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]
Figure 11
Figure 11. Figure 11: Qualitative ablations on deformable attentions. The 1st column present images of input views (Inputs) and ground-truth novel [PITH_FULL_IMAGE:figures/full_fig_p016_11.png]
Figure 12
Figure 12. Figure 12: Additional examples of Volume-Pixel Collaboration. The [PITH_FULL_IMAGE:figures/full_fig_p017_12.png]
Figure 13
Figure 13. Figure 13: Additional qualitative results on scene-centric reconstruction performed on RealEstate10K [ [PITH_FULL_IMAGE:figures/full_fig_p018_13.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

59 extracted references · 29 canonical work pages

  1. [1]

    pixelnerf: Neural radiance fields from one or few images

    Alex Yu, Vickie Ye, Matthew Tancik, and Angjoo Kanazawa. pixelnerf: Neural radiance fields from one or few images. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 4578–4587, 2021. 1, 3

    2021

  2. [2]

    Ibr- net: Learning multi-view image-based rendering

    Qianqian Wang, Zhicheng Wang, Kyle Genova, Pratul P Srinivasan, Howard Zhou, Jonathan T Barron, Ricardo Martin-Brualla, Noah Snavely, and Thomas Funkhouser. Ibr- net: Learning multi-view image-based rendering. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4690–4699, 2021

    work page 2021

  3. [3]

    Neural rays for occlusion-aware image-based render- ing

    Yuan Liu, Sida Peng, Lingjie Liu, Qianqian Wang, Peng Wang, Christian Theobalt, Xiaowei Zhou, and Wenping Wang. Neural rays for occlusion-aware image-based render- ing. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 7824–7833,

  4. [4]

    Mvsnerf: Fast general- izable radiance field reconstruction from multi-view stereo

    Anpei Chen, Zexiang Xu, Fuqiang Zhao, Xiaoshuai Zhang, Fanbo Xiang, Jingyi Yu, and Hao Su. Mvsnerf: Fast general- izable radiance field reconstruction from multi-view stereo. In Proceedings of the IEEE/CVF international conference on computer vision, pages 14124–14133, 2021. 3

    2021

  5. [5]

    Geonerf: Generalizing nerf with geometry priors

    Mohammad Mahdi Johari, Yann Lepoittevin, and Franc ¸ois Fleuret. Geonerf: Generalizing nerf with geometry priors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18365–18375, 2022

    work page 2022

  6. [6]

    Efficient neural radiance fields for interactive free-viewpoint video

    Haotong Lin, Sida Peng, Zhen Xu, Yunzhi Yan, Qing Shuai, Hujun Bao, and Xiaowei Zhou. Efficient neural radiance fields for interactive free-viewpoint video. In SIGGRAPH Asia 2022 Conference Papers, pages 1–9, 2022

    work page 2022

  7. [7]

    Nerfusion: Fusing radiance fields for large- scale scene reconstruction

    Xiaoshuai Zhang, Sai Bi, Kalyan Sunkavalli, Hao Su, and Zexiang Xu. Nerfusion: Fusing radiance fields for large- scale scene reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 5449–5458, 2022. 3

    work page 2022

  8. [8]

    Lrm: Large reconstruction model for single image to 3d

    Yicong Hong, Kai Zhang, Jiuxiang Gu, Sai Bi, Yang Zhou, Difan Liu, Feng Liu, Kalyan Sunkavalli, Trung Bui, and Hao Tan. Lrm: Large reconstruction model for single image to 3d. arXiv preprint arXiv:2311.04400, 2023. 3, 4

    arXiv 2023

Show all 59 references
  1. [9]

    Murf: Multi-baseline radiance fields

    Haofei Xu, Anpei Chen, Yuedong Chen, Christos Sakaridis, Yulun Zhang, Marc Pollefeys, Andreas Geiger, and Fisher Yu. Murf: Multi-baseline radiance fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20041–20050, 2024. 1, 3, 6, 7, 2

    2024

  2. [10]

    Light field neural rendering

    Mohammed Suhail, Carlos Esteves, Leonid Sigal, and Ameesh Makadia. Light field neural rendering. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8269–8279, 2022. 1, 3, 2

    2022

  3. [11]

    Generalizable patch-based neural render- ing

    Mohammed Suhail, Carlos Esteves, Leonid Sigal, and Ameesh Makadia. Generalizable patch-based neural render- ing. In European Conference on Computer Vision , pages 156–174. Springer, 2022

    2022

  4. [12]

    Learning to render novel views from wide-baseline stereo pairs

    Yilun Du, Cameron Smith, Ayush Tewari, and Vincent Sitz- mann. Learning to render novel views from wide-baseline stereo pairs. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 4970– 4980, 2023. 1, 3, 6, 2

    2023

  5. [13]

    pixelsplat: 3d gaussian splats from image pairs for scalable generalizable 3d reconstruction

    David Charatan, Sizhe Lester Li, Andrea Tagliasacchi, and Vincent Sitzmann. pixelsplat: 3d gaussian splats from image pairs for scalable generalizable 3d reconstruction. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19457–19467, ...

    2024

  6. [14]

    Mvsplat: Efficient 3d gaussian splatting from sparse multi-view images

    Yuedong Chen, Haofei Xu, Chuanxia Zheng, Bohan Zhuang, Marc Pollefeys, Andreas Geiger, Tat-Jen Cham, and Jianfei Cai. Mvsplat: Efficient 3d gaussian splatting from sparse multi-view images. arXiv preprint arXiv:2403.14627, 2024. 2, 3, 5, 6, 7, 4

    2024 arXiv

  7. [15]

    Flash3d: Feed-forward gener- alisable 3d scene reconstruction from a single image

    Stanislaw Szymanowicz, Eldar Insafutdinov, Chuanxia Zheng, Dylan Campbell, Jo ˜ao F Henriques, Christian Rup- 9 precht, and Andrea Vedaldi. Flash3d: Feed-forward gener- alisable 3d scene reconstruction from a single image. arXiv preprint arXiv:2406.04343, 2024

    2024 arXiv

  8. [16]

    Lgm: Large multi-view gaussian model for high-resolution 3d content creation

    Jiaxiang Tang, Zhaoxi Chen, Xiaokang Chen, Tengfei Wang, Gang Zeng, and Ziwei Liu. Lgm: Large multi-view gaussian model for high-resolution 3d content creation. In European Conference on Computer Vision, pages 1–18. Springer, 2025. 2, 3, 5

    2025

  9. [17]

    Splatter image: Ultra-fast single-view 3d recon- struction

    Stanislaw Szymanowicz, Chrisitian Rupprecht, and Andrea Vedaldi. Splatter image: Ultra-fast single-view 3d recon- struction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 10208– 10217, 2024

    2024

  10. [18]

    Grm: Large gaussian reconstruction model for ef- ficient 3d reconstruction and generation

    Yinghao Xu, Zifan Shi, Wang Yifan, Hansheng Chen, Ceyuan Yang, Sida Peng, Yujun Shen, and Gordon Wet- zstein. Grm: Large gaussian reconstruction model for ef- ficient 3d reconstruction and generation. arXiv preprint arXiv:2403.14621, 2024. 2, 3

    2024 arXiv

  11. [19]

    Geolrm: Geometry-aware large re- construction model for high-quality 3d gaussian generation

    Chubin Zhang, Hongliang Song, Yi Wei, Yu Chen, Jiwen Lu, and Yansong Tang. Geolrm: Geometry-aware large re- construction model for high-quality 3d gaussian generation. arXiv preprint arXiv:2406.15333, 2024

    2024 arXiv

  12. [20]

    Triplane meets gaussian splatting: Fast and generalizable single-view 3d reconstruction with transformers

    Zi-Xin Zou, Zhipeng Yu, Yuan-Chen Guo, Yangguang Li, Ding Liang, Yan-Pei Cao, and Song-Hai Zhang. Triplane meets gaussian splatting: Fast and generalizable single-view 3d reconstruction with transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern...

    2024

  13. [21]

    Fast generalizable gaussian splatting reconstruction from multi-view stereo

    Tianqi Liu, Guangcong Wang, Shoukang Hu, Liao Shen, Xinyi Ye, Yuhang Zang, Zhiguo Cao, Wei Li, and Ziwei Liu. Fast generalizable gaussian splatting reconstruction from multi-view stereo. arXiv preprint arXiv:2405.12218 ,

    arXiv

  14. [22]

    Ggrt: Towards generalizable 3d gaussians without pose priors in real-time

    Hao Li, Yuanyuan Gao, Dingwen Zhang, Chenming Wu, Yalun Dai, Chen Zhao, Haocheng Feng, Errui Ding, Jing- dong Wang, and Junwei Han. Ggrt: Towards generalizable 3d gaussians without pose priors in real-time. arXiv preprint arXiv:2403.10147, 2024. 1, 2, 3

    2024 arXiv

  15. [23]

    Nerf: Representing scenes as neural radiance fields for view syn- thesis

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis. Communications of the ACM, 65(1):99–106, 2021. 1, 3, 2

    2021

  16. [24]

    3d gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42(4):139–1,

  17. [25]

    Bevformer: Learning bird’s-eye-view representation from multi-camera images via spatiotemporal transformers

    Zhiqi Li, Wenhai Wang, Hongyang Li, Enze Xie, Chong- hao Sima, Tong Lu, Yu Qiao, and Jifeng Dai. Bevformer: Learning bird’s-eye-view representation from multi-camera images via spatiotemporal transformers. In European con- ference on computer vision, pages 1–18. Springer, 2022...

    2022

  18. [26]

    Bevfusion: Multi- task multi-sensor fusion with unified bird’s-eye view repre- sentation

    Zhijian Liu, Haotian Tang, Alexander Amini, Xinyu Yang, Huizi Mao, Daniela L Rus, and Song Han. Bevfusion: Multi- task multi-sensor fusion with unified bird’s-eye view repre- sentation. In 2023 IEEE international conference on robotics and automation (ICRA), pages 2774–2781. I...

    2023

  19. [27]

    Petr: Position embedding transformation for multi-view 3d object detection

    Yingfei Liu, Tiancai Wang, Xiangyu Zhang, and Jian Sun. Petr: Position embedding transformation for multi-view 3d object detection. In European Conference on Computer Vi- sion, pages 531–548. Springer, 2022. 2, 3

    2022

  20. [28]

    Surroundocc: Multi-camera 3d occu- pancy prediction for autonomous driving

    Yi Wei, Linqing Zhao, Wenzhao Zheng, Zheng Zhu, Jie Zhou, and Jiwen Lu. Surroundocc: Multi-camera 3d occu- pancy prediction for autonomous driving. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 21729–21740, 2023. 2, 3

    2023

  21. [29]

    Occformer: Dual-path transformer for vision-based 3d semantic occu- pancy prediction

    Yunpeng Zhang, Zheng Zhu, and Dalong Du. Occformer: Dual-path transformer for vision-based 3d semantic occu- pancy prediction. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 9433–9443, 2023

    2023

  22. [30]

    Tri-perspective view for vision- based 3d semantic occupancy prediction

    Yuanhui Huang, Wenzhao Zheng, Yunpeng Zhang, Jie Zhou, and Jiwen Lu. Tri-perspective view for vision- based 3d semantic occupancy prediction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9223–9232, 2023. 2, 3, 4

    2023

  23. [31]

    nuscenes: A multi- modal dataset for autonomous driving

    Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Gi- ancarlo Baldan, and Oscar Beijbom. nuscenes: A multi- modal dataset for autonomous driving. In Proceedings of the IEEE/CVF conference on computer vision and patte...

    2020

  24. [32]

    Stereo magnification: Learning view synthesis using multiplane images

    Tinghui Zhou, Richard Tucker, John Flynn, Graham Fyffe, and Noah Snavely. Stereo magnification: Learning view synthesis using multiplane images. arXiv preprint arXiv:1805.09817, 2018. 3, 6, 7

    2018 arXiv

  25. [33]

    Mip-nerf: A multiscale representation for anti-aliasing neu- ral radiance fields

    Jonathan T Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-Brualla, and Pratul P Srinivasan. Mip-nerf: A multiscale representation for anti-aliasing neu- ral radiance fields. In Proceedings of the IEEE/CVF inter- national conference on computer vision , pa...

  26. [34]

    Zip-nerf: Anti-aliased grid-based neural radiance fields

    Jonathan T Barron, Ben Mildenhall, Dor Verbin, Pratul P Srinivasan, and Peter Hedman. Zip-nerf: Anti-aliased grid-based neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 19697–19705, 2023. 3

    2023

  27. [35]

    Instant neural graphics primitives with a mul- tiresolution hash encoding

    Thomas M ¨uller, Alex Evans, Christoph Schied, and Alexan- der Keller. Instant neural graphics primitives with a mul- tiresolution hash encoding. ACM transactions on graphics (TOG), 41(4):1–15, 2022. 3

    2022

  28. [36]

    Mip-splatting: Alias-free 3d gaussian splat- ting

    Zehao Yu, Anpei Chen, Binbin Huang, Torsten Sattler, and Andreas Geiger. Mip-splatting: Alias-free 3d gaussian splat- ting. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 19447–19456,

  29. [37]

    2d gaussian splatting for geometrically ac- curate radiance fields

    Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, and Shenghua Gao. 2d gaussian splatting for geometrically ac- curate radiance fields. In ACM SIGGRAPH 2024 Conference Papers, pages 1–11, 2024. 3

    2024

  30. [38]

    Emernerf: Emergent spatial-temporal scene decomposition via self-supervision

    Jiawei Yang, Boris Ivanovic, Or Litany, Xinshuo Weng, Se- ung Wook Kim, Boyi Li, Tong Che, Danfei Xu, Sanja Fidler, Marco Pavone, et al. Emernerf: Emergent spatial-temporal scene decomposition via self-supervision. arXiv preprint arXiv:2311.02077, 2023. 3 10

    2023 arXiv

  31. [39]

    Drivinggaussian: Composite gaussian splatting for surrounding dynamic au- tonomous driving scenes

    Xiaoyu Zhou, Zhiwei Lin, Xiaojun Shan, Yongtao Wang, Deqing Sun, and Ming-Hsuan Yang. Drivinggaussian: Composite gaussian splatting for surrounding dynamic au- tonomous driving scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages...

    2024

  32. [40]

    Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly unproject- ing to 3d

    Jonah Philion and Sanja Fidler. Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly unproject- ing to 3d. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceed- ings, Part XIV 16, pages 194–210. Springer, 2020. 3

    2020

  33. [41]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 4

    2016

  34. [42]

    Emerg- ing properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers. In Pro- ceedings of the IEEE/CVF international conference on com- puter vision, pages 9650–9660, 2021. 4, 1

    2021

  35. [43]

    Dmv3d: Denoising multi-view diffu- sion using 3d large reconstruction model

    Yinghao Xu, Hao Tan, Fujun Luan, Sai Bi, Peng Wang, Ji- ahao Li, Zifan Shi, Kalyan Sunkavalli, Gordon Wetzstein, Zexiang Xu, et al. Dmv3d: Denoising multi-view diffu- sion using 3d large reconstruction model. arXiv preprint arXiv:2311.09217, 2023. 4

    2023 arXiv

  36. [44]

    Crm: Single image to 3d textured mesh with convolutional reconstruction model

    Zhengyi Wang, Yikai Wang, Yifei Chen, Chendong Xi- ang, Shuo Chen, Dajiang Yu, Chongxuan Li, Hang Su, and Jun Zhu. Crm: Single image to 3d textured mesh with convolutional reconstruction model. arXiv preprint arXiv:2403.05034, 2024. 4

    2024 arXiv

  37. [45]

    Pixart- \sigma: Weak-to-strong training of diffusion transformer for 4k text-to-image generation

    Junsong Chen, Chongjian Ge, Enze Xie, Yue Wu, Lewei Yao, Xiaozhe Ren, Zhongdao Wang, Ping Luo, Huchuan Lu, and Zhenguo Li. Pixart- \sigma: Weak-to-strong training of diffusion transformer for 4k text-to-image generation. arXiv preprint arXiv:2403.04692, 2024. 5, 1

    2024 arXiv

  38. [46]

    Metric3d v2: A versatile monocular geomet- ric foundation model for zero-shot metric depth and surface normal estimation

    Mu Hu, Wei Yin, Chi Zhang, Zhipeng Cai, Xiaoxiao Long, Hao Chen, Kaixuan Wang, Gang Yu, Chunhua Shen, and Shaojie Shen. Metric3d v2: A versatile monocular geomet- ric foundation model for zero-shot metric depth and surface normal estimation. arXiv preprint arXiv:2404.15506, 2024. 5, 9

    2024 arXiv

  39. [47]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shecht- man, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 586–595, 2018. 6, 7

    2018

  40. [48]

    Magicdrive: Street view generation with diverse 3d geometry control

    Ruiyuan Gao, Kai Chen, Enze Xie, HONG Lanqing, Zhen- guo Li, Dit-Yan Yeung, and Qiang Xu. Magicdrive: Street view generation with diverse 3d geometry control. In The Twelfth International Conference on Learning Representa- tions, 2023. 6, 7

    2023

  41. [49]

    Image quality assessment: from error visibility to structural similarity

    Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Si- moncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4):600–612, 2004. 7

    2004

  42. [50]

    Depth any- thing v2

    Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiao- gang Xu, Jiashi Feng, and Hengshuang Zhao. Depth any- thing v2. arXiv preprint arXiv:2406.09414, 2024. 7

    2024 arXiv

  43. [51]

    Pearson correlation coefficient

    Israel Cohen, Yiteng Huang, Jingdong Chen, Jacob Benesty, Jacob Benesty, Jingdong Chen, Yiteng Huang, and Israel Cohen. Pearson correlation coefficient. Noise reduction in speech processing, pages 1–4, 2009. 7

    2009

  44. [52]

    Adam: A method for stochastic opti- mization

    Diederik P Kingma. Adam: A method for stochastic opti- mization. arXiv preprint arXiv:1412.6980, 2014. 7, 1

    2014 arXiv

  45. [53]

    Magicdrive3d: Controllable 3d genera- tion for any-view rendering in street scenes

    Ruiyuan Gao, Kai Chen, Zhihao Li, Lanqing Hong, Zhenguo Li, and Qiang Xu. Magicdrive3d: Controllable 3d genera- tion for any-view rendering in street scenes. arXiv preprint arXiv:2405.14475, 2024. 8, 2

    2024 arXiv

  46. [54]

    Deformable 3d gaussians for high- fidelity monocular dynamic scene reconstruction

    Ziyi Yang, Xinyu Gao, Wen Zhou, Shaohui Jiao, Yuqing Zhang, and Xiaogang Jin. Deformable 3d gaussians for high- fidelity monocular dynamic scene reconstruction. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20331–20341, 2024. 8

    2024

  47. [55]

    video.mp4

    Tsung-Yi Lin, Piotr Doll ´ar, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. Feature pyra- mid networks for object detection. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 2117–2125, 2017. 1 11 Omni-Scene: Omni-Gau...

    2017

  48. [56]

    Additional Implementation Details 6.1. Data Preprocessing As described in Sec.4.1 of our main manuscript, we par- tition each scene of nuScenes dataset [31] into equally spaced bins, with each bin serving as one data sample. For nuScenes dataset, each video is captured in a si...

  49. [57]

    video.mp4

    Additional Experiments 7.1. Video Results To better demonstrate the quality of 3D reconstruction, we provide exploring video demos in “video.mp4” along with our supplementary material. Specifically, given six surrounding images of a scene, we conduct inference and obtain 3D Ga...

  50. [58]

    cross-img attn

    and light field [10], respectively. They are both lead- ing and representative methods within their respective lines of works, which constitute the mainstream feed-forward methods together with 3DGS-based approaches such as pix- elSplat [13] and MVSplat [14]. As shown in Table...

  51. [59]

    bin size = 6.4m

    into three variants with different bin sizes (i.e., 1.6m, 6.4m, 12.8m) from our original dataset. Here we note that, the larger the bin size, the farther distance between the novel and the input views, which is more challenging for novel view synthesis. Practically, for each d...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.